Abstract:

As the number of electric vehicles （EVs） increases， reinforcement learning （RL） in EV charging scheduling faces challenges， particularly uncertainties and the curse of dimensionality from large‑scale applications. A microgrid model for residential areas， considering the vehicle‑to‑grid （V2G） mode and various nonlinear charging models is developed. The problem is formulated as a constrained Markov decision process （CMDP）， with a model‑free RL framework to handle uncertainties. To address the curse of dimensionality， a strategy is designed where EVs are grouped by states， and agents send control signals to these sets， thus reducing the dimensionality of the action space. A Lagrangian deep deterministic policy gradient （LDDPG） algorithm is employed to solve the charging scheduling problem， with a safety filter ensuring constraint compliance. Numerical simulations validate the strategy’s effectiveness.

Key words: electric vehicle, charging scheduling, safe reinforcement learning, V2G mode, nonlinear charging