The efficiency of models is becoming more and more important in computer vision. In this paper, by studying the neural network structure for flame detection, several key optimization schemes are proposed to improve the efficiency of the model and the detection effect. First, this paper proposes a backbone network (FIRE-Net) constructed from a multi-convolution combined structure, which can efficiently extract rich flame features from multiple scales; second, by using an improved weighted bidirectional feature Pyramid network (BiFPN-mini) to quickly achieve multi-scale feature fusion; third, a new attention mechanism (FIRE-Attention) is proposed to make the detector more sensitive to flame characteristics. based on the above optimization , Developed a new flame detector FIRE-DET, which can always achieve faster detection efficiency than existing deep learning-based detection methods under the condition of limited hardware resources. The FIRE-DET model was trained on a self-built dataset, and the final flame recognition accuracy and frame rate reached 97% and 85FPS, respectively. The experimental results show that this flame detection model detection performance is better than the mainstream algorithm. This article provides a more common solution to solve the problem of flame detection.