Electromagnetic scattering of the metal and dielectric pipeline beneath the rough ground is simulated by using integral equation. Because the soil underground is hierarchical, the integral equations are listed separately according to the surface equivalence principle and boundary conditions, above or below the target in the interface. These integral equations are solved by using the moment method. Utilizing Fast Fourier Transform(FFT) spectral method and Gaussian spectrum to model the rough ground, the simulation of the pipelines with a circular cross section and a rectangular cross section under the rough terrain is performed. The simulation results indicate, the analysis method in this paper is able to calculate the electromagnetic scattering of underground pipelines.