The 10th Asian Symposium on Computational Heat Transfer and Fluid Flow ( ASCHT2025)

PID control is widely used across various industrial fields due to its stability and robustness. However, it has inherent limitations, one of which is the overshooting phenomenon that can occur during parameter tuning. Overshooting refers to a situation where the system output temporarily exceeds the target value before settling, which can pose significant challenges in applications requiring precise temperature control.
This study aims to develop a precision temperature control strategy for a thermoelectric-based heat exchanger by replacing conventional PID control with a reinforcement learning (RL)-based approach. Heat exchangers exhibit nonlinear characteristics due to complex interactions among various parameters, making it difficult to apply traditional RL methods that require predefined policies.
To address this, we employ Proximal Policy Optimization (PPO) [1], a deep RL algorithm, to learn effective control polices for nonlinear temperature regulation. However, directly applying PPO to real equipment is both time-consuming and potentially hazardous. To overcome this, experimental data were first collected to construct a deep RL-based outlet temperature prediction model. This model serves as a simulator for the RL environment, enabling safe and efficient policy training. The trained control policies were then evaluated, and optimal parameters were explored. This approach presents a promising alternative to traditional PID control, particularly for applications requiring minimized overshoot and high-precision temperature regulation.
 

PID control, Reinforcement learning, PPO, Temperature control, Thermoelectric module