Fire disaster can cause great harm to the safety of people and property, and how to detect flame intelligently and efficiently is of great significance. In order to achieve accurate flame recognition under high space conditions, an infrared camera with two degrees of freedom that can detect environmental conditions in all directions is designed, and the target detection algorithm YOLOv5 is improved combined with deep learning. The K-Means clustering algorithm is employed to obtain nine width and height dimensions of clustering center by matching and replace the original network anchor parameters. Considering the relative proportion of the target frame, the loss function is optimized and applied to the Raspberry Pi to achieve flame recognition. The test results show that it takes 2.9 s for the improved YOLOv5 algorithm to detect a single sheet on the Raspberry Pi, which is less than that for the original YOLOv5 algorithm by 78%. The accuracy of the system is 100%, and the confidence of identifying the target frame is above 0.9. The proposed system can identify the flame fast and accurately.