Abstract: For the more general form of nonlinear discretetime time delays systems with saturating actuators， an infinitetime optimal control scheme was developed by heuristic dynamic programming(HDP) algorithm. In the proposed scheme， the discount factor was added in the value function. Firstly， value function series and control series were given through iterative HDP algorithm， and the convergence analysis was presented to prove that value function series and control series reach the optimal value simultaneously. Secondly， three neural networks(NN)which are model NN， critic NN， action NN were introduced to carry out the HDP algorithm. Model NN was used to approximate system model， critic NN to approximate value function， action NN to approximate control policy. Lastly， the validity of HDP algorithm was illustrated by one simulation example.

Key words: approximate dynamic programming, heuristic dynamic programming, value function, neural networks, optimal control