Abstract: An approximate linear delay model is proposed to solve the dynamic load balancing problem during parallel computation. Analyzing the stability of the model, the delay-dependent asymptotic stable conditions are obtained, then the theoretical load balancing gain with different time delays and on different system scales is solved approximately by LMI toolkit. The simulation results indicated that the optimal load balancing gain is in inverse proportion to communication/transmission delay and in direct proportion to system scale. Such a conclusion drawn here applies to different communication/transmission delays and system scales, and has guiding effect on designing a more useful load balancing algorithm under time delay conditions.