Winograd Fourier Transform Algorithm(WFTA) is a fast algorithm of small points of DFT applying to the non-two integer power, and its computing speed shows a direct impact on system performance. The paper gives the design of hardware structure according to the actual situation of each small point, which adopts shift register and accumulator-based circuit structure in the WFTA design when the point is 3 or 4; when the point increases to 5, 7 and 9, it adopts the method of addition or subtraction matrix factorization to ensure fast computing speed. The simulation results show that the design scheme meets pipeline requirements, and its data processing capability is strong enough.