The background noise in the echo spectrum of High Frequency Surface Wave Radar(HFSWR) is complex, the clutter accounts for a small proportion and the ionospheric clutter has different forms and positions, therefore, it is difficult to automatically recognize the clutters. Based on DeeplabV3+ deep learning algorithm, an automatic identification method of ionospheric clutter and sea clutter is proposed for HFSWR. Selecting the lightweight MobileNetV2 backbone feature network, adding the channel attention mechanism module SENet, the focused learning of clutter labels is realized, and the loss weight of various labels in the training set is optimized. The model pre-training transfer method is employed to pre-train the network to tackle with the problem of too small sample space. The experimental results on the measured data set show that the proposed method can realize the automatic recognition of ionospheric clutter and sea clutter in HFSWR, and can obtain more accurate and finer clutter recognition results than the original DeeplabV3+ algorithm. The mean Intersection over Union(mIoU) and Accuracy(ACC) of sea clutter recognition results are increased by 2.9% and 5.1% respectively, and the mIoU and ACC of ionospheric clutter recognition results are increased by 3.0% and 4.9% respectively.