A load balancing framework for distributed data center network is proposed in order to improve the data center network with high concurrency and low latency performance. Aiming at the distributed load balancing architecture, the whole design of the customized intelligent gateway is realized on the basis of Field Programmable Gate Array(FPGA). Third party testing based on custom gateway can package information based on load balancing, with line speed up to 9.4 Gbps(no packet loss), and the packet loss rate of 10 Gbps is 5%, the port delay is 2 μs. Compared to load balancing scheme(universal software load balancing and hardware load balancing), the distributed load balancing architecture adopts customized intelligent gateway information packet load balancing strategy and intelligent traffic management based on pool, which guarantees the efficient distribution of millions of data flow in server system, and improves the performance of high concurrency applications. In the case of millions of concurrent scenarios, the TCP link response delay is less than 60 ms.