Conventional radar generally uses fixed transmit waveform, which is difficult to obtain optimal target detection performance in the presence of interference. To solve this problem, a joint optimization algorithm of waveform and receiving filter for Multiple-Input Multiple-Output(MIMO) radar is proposed. Firstly, output signal to interference and noise ratio maximization is used as objective function. By setting transmit waveform to satisfy constant modulus constraint, the joint optimization model of waveform and receiving filter is established under a similarity constraint involving a reference radar waveform. Then, under the framework of cyclic iterative algorithm, the joint optimization problem is decomposed into two sub-optimization problems, which are solved by using Lagrange multiplier method and semi-definite relaxation technique respectively. The proposed joint optimization algorithm finally results in transmit waveform and receiving filter weights. Simulation results show that the proposed mehod has higher output signal-to-noise ratio than the existing methods, and achieves better interference suppression performance with an eye to pulse compression property of the transmit waveform.