A Phase Gradient Algorithm(PGA) based algorithm for the automatic recognition of multiple ship wakes in Synthetic Aperture Radar(SAR) image is presented. Firstly, the center point of each ship image is extracted by using a combination algorithm of multiple-structure morphology filtering, the OTSU automatic segmentation and the clustering method. Then, the whole image is divided into sub-images which are chosen to be a square of appropriate size and the centers of sub-images are the detected points. As moving ships in SAR image always result in residual chirps in the azimuthally processed signals, the block-wise PGA is utilized to compensate the position offset and defocusing effect in each sub-image. Finally, an improved local Hough transform is proposed to detect the ship wake based on an inverse analysis of the detected wake's coordinate and angle, and the locations and the trajectories of each moving ship are obtained. The examples demonstrate that this algorithm can accurately recognize and restore the ships' actual trajectories and locations with high detectability and low false detection rate.