It’s practically complicate to implement the matrix inversion which is widely used in all kinds of real-time circuit calculation. Taking Field Programmable Gate Arrays(FPGA) to implement the operation is able to take advantages of the hardware’s speed and parallelism to accelerate the matrix inversion. Based on improved systolic architecture, a simplification factor algorithm is proposed, in which any n×n upper-matrix is transferred into an upper-matrix whose diagonal value is 1 to split dividing processor with multiplying and adding processor to greatly simplify the matrix inversion. Taking a 4×4 upper-matrix as an example, the algorithm is implemented and executed the functional verification adopting Virtex5 device in Xilinx ISE and utilizing 2 dividers, 3 multipliers and 4 adders in 14 cycles to accomplish all matrix inversion. Compared with the classical systolic architecture, resources are just occupied nearly half, while performance gets improved by 26.43%. Compared with systolic architecture integrated more Processing Elements(PE), the performance does not change and the resources reduce to 1/4, which greatly improves the resources utilization.