The existing remote sensing image object detection algorithms have the problem of large parameter quantities and difficulty in deploying on mobile devices. Therefore, this paper proposes a lightweight remote sensing image object detection algorithm without anchor boxes. Firstly, the DWS-Sandglass lightweight module was designed to reduce the model volume, and then the activation function of the model was improved to ensure the detection accuracy. Then, a parameter free attention module SimAM is introduced to enable the network to focus on more important feature information. Finally, prune the redundant channels of the anchor free box algorithm to reduce the number of model parameters, and fine tune to improve accuracy. The experimental results on the HRSC2016 dataset show that compared with the current mainstream anchor free frame detection algorithms, this algorithm has faster detection speed and smaller model size, making it more suitable for deployment on mobile devices with comparable detection accuracy.