The blind source separation algorithm based on extended informax algorithm in separating the super-Gaussian and sub-Gaussian mixed-signals depends on the signals kurtosis estimates，and it is sensitive to the initial separation matrix and step size. To solve these problems，a new algorithm based on genetic algorithm was proposed. In order to calculate the mutual information between signals the non-linear polynomial was used to approximate the negative entropy. Genetic algorithm instead of the gradient optimization algorithm was used to minimize the mutual information between the separated signals. Simulations show that the proposed algorithm is simple，robust；the Performance Index(PI) value reaches 0.025 5 after 100 times of iteration; and the performance is superior to the Blind Source Separation(BSS) algorithm based on extended infomax algorithm.