By analyzing the characteristics of the glass insulator and its defects in aerial images of the unmanned aerial vehicle inspection system, a method is presented to extract glass insulator and detect defects. The OTSU's thresholding method is applied to H and S components in Hue,Saturation,Intensity (HSI) color space for image segmentation; and then, the distribution of the contours' angle and area is calculated by using the histogram method to obtain the position of glass insulators accurately. Feature detection algorithm is proposed to detect the glass insulators and mark out the defects. Experimental results indicate that the method can mark the location of the defects accurately in the aerial images with natural background, which has a good practical value.