Heat-pipe reactors (HPRs) coupled with supercritical carbon dioxide (sCO₂) Brayton cycles are compact energy systems whose coupled reactor, heat-pipe, and power-conversion dynamics create a challenging disturbance-rejection problem. This study develops a fixed-bound supervisory residual reinforcement-learning architecture based on twin-delayed deep deterministic policy gradient (TD3) for a RELAP5-based HPR–sCO₂ system. The baseline command trajectory provides the nominal action, while TD3 supplies bounded, slew-limited residual corrections to the precooler cooling-water and hot-bypass channels. Under a hot-bypass disturbance, reactor-power RMSE decreases from 149.30 kW for the baseline to 91.30 kW with residual TD3, corresponding to a 38.8% reduction. The maximum core and heat-pipe temperature ramp rates decrease from 0.03014 to 0.02666 K/s and from 0.04028 to 0.03626 K/s, respectively. Under a persistent cooling-water-flow reduction, reactor-power RMSE decreases from 190.92 to 165.31 kW, corresponding to a 13.4% reduction, while the steady-state power-deficit magnitude is 13.5% lower with residual TD3. The smaller improvement in this case is associated with reduced actuator authority because the hot-bypass channel is already at its lower bound, leaving the cooling-water channel to provide the effective physical correction. The results show that bounded residual TD3 can improve disturbance attenuation while preserving the established baseline-control structure, although the achievable improvement depends on the disturbance and available actuator authority.
 

heat-pipe reactor; sCO2 Brayton cycle; residual reinforcement learning; TD3; disturbance rejection