需量电费影响下的CCHP系统深度强化学习运行优化

doi:10.13304/j.nykjdb.2023.0121

摘要/Abstract

摘要：

在全球能源紧张的趋势下，冷热电三联供（combined cold， hot and power， CCHP）系统因能源可梯级利用和一次能源利用率高的优势日益受到重视。然而，由于影响因素复杂、多变，特别是需量电费的存在，CCHP系统以现有控制手段和实时满足用户侧供能需求的前提下难以以经济性目标运行。为了在考虑需求电费的条件下最大限度地降低运行成本，提出基于TD3算法的CCHP系统控制策略优化方法，对系统的各个设备进行建模，将CCHP系统运行优化问题转化为马尔卡夫决策问题，利用TD3算法求解，并进行实例验证分析。结果表明，考虑需量电费的TD3代理良好地平衡了需量电费和实时运行费用，且具有泛化性；相较于历史运行策略和不考虑需量电费的TD3代理运行策略，总运行成本分别降低了41.5%和8.6%。研究结果为减少农业供能成本、提高经济性提供了新的解决方案。

关键词: 冷热电三联供系统, 需量电费, 深度强化学习, 双延迟深度确定性策略梯度

Abstract:

In the present time of global energy stress， the combined cold， hot and power （CCHP） system is gaining importance due to the advantages of energy cascade utilization and high primary energy utilization. However， due to the complex and variable influencing factors， especially the presence of the demand charge， the CCHP system is difficult to operate in real time with the existing control methods to meet the energy requirements on the customer side under economic objectives. To minimize operating costs with the demand charge， a control strategy optimization of the system based on the TD3 algorithm was proposed. The individual units of the system were modelled and the CCHP system operation optimization problem was transformed into a Markov decision problem， and solved using the TD3 algorithm. Then， the experimental verification analysis was carried out. The results showed that the TD3 agent with demand charge balanced the demand charges and real-time operating costs well， and was generalizable， which reduced the total operating costs by 41.5% and 8.6% compared to the historical operating strategy and the TD3 agent without demand charge， respectively. Above results provided a new solution for reducing the cost and improving the economics of agricultural energy supply.

Key words: combined cold， hot and power system, demand charge, deep reinforcement learning, twin delayed deep deterministic policy gradient

中图分类号:

TK01+9

高文忠, 张毅. 需量电费影响下的CCHP系统深度强化学习运行优化[J]. 中国农业科技导报, 2023, 25(4): 100-109.

Wenzhong GAO, Yi ZHANG. Operational Optimization of CCHP Systems on Deep Reinforcement Learning Under Influence of Demand Charge[J]. Journal of Agricultural Science and Technology, 2023, 25(4): 100-109.

图/表 8

参考文献 25

1	PÉREZ-LOMBARD L, ORTIZ J, POUT C. A review on buildings energy consumption information [J]. Energy Build., 2008, 40(3):394-398.
2	LI Y, TIAN R, WEI M. Operation strategy for interactive CCHP system based on energy complementary characteristics of diverse operation strategies [J/OL]. Appl. Energy, 2022, 310:118415 [2023-03-18]. .
3	CUI Q, MA P, HUANG L, et al.. Effect of device models on the multiobjective optimal operation of CCHP microgrids considering shiftable loads [J/OL]. Appl. Energy, 2020, 275:115369 [2023-03-18]. .
4	LIU L, WANG D, HOU K, et al.. Region model and application of regional integrated energy system security analysis [J/OL]. Appl. Energy, 2020, 260: 114268 [2023-03-18]. .
5	CHENG Z, JIA D, LI Z, et al.. Multi-time scale dynamic robust optimal scheduling of CCHP microgrid based on rolling optimization [J/OL]. Int. J. Electrical Power Energy Syst., 2022, 139:107957 [2023-03-18]. .
6	LIN H, YANG C, XU X. A new optimization model of CCHP system based on genetic algorithm [J/OL]. Sustainable Cities Soc., 2020, 52:101811 [2023-03-18]. .
7	张少明,盛四清.基于改进粒子群算法的微网优化运行[J]. 中国电力, 2020, 53(5): 24-31.
	ZHANG S M, SHENG S Q. Optimal operation of microgrid based on improved particle swarm algorithm [J].China Electric Power, 2020, 53(5):24-31.
8	ZENG R, ZHANG X, DENG Y, et al.. An off-design model to optimize CCHP-GSHP system considering carbon tax [J]. Energy Conv. Manag., 2019, 189(6):105-117.
9	孙轶恺,漆淘懿,张利军,等.市场环境下含冰蓄冷空调的综合能源系统优化运行[J].南方电网技术,2022,16(4):95-104.
	SUN Y K, QI T Y, ZHANG L J, et al.. Optimal operation of integrated energy system including ice-storage air-conditioning in power market [J]. Southern Power Syst. Technol., 2022, 16(4):95-104.
10	HOU J, WANG J, ZHOU Y, et al.. Distributed energy systems: multi-objective optimization and evaluation under different operational strategies [J/OL]. J. Cleaner Prod., 2021, 280:124050 [2023-03-18]. .
11	ZANDI S, GOLBATEN MOFRAD K, SALEHI G, et al.. Multi-objective optimization and thermoeconomic analysis of a novel CCHP with TES and hybrid cooling for residential complex [J/OL]. Thermal Sci. Eng. Progress, 2020, 19:100656 [2023-03-15]. .
12	MAGO P J, CHAMRA L M. Analysis and optimization of CCHP systems based on energy, economical, and environmental considerations [J]. Energy Build., 2009, 41(10):1099-1106.
13	CHEN X, ZHOU H, LI W, et al.. Multi-criteria assessment and optimization study on 5 kW PEMFC based residential CCHP system [J]. Energy Conv. Manag., 2018, 160(3):384-395.
14	CHEN Y, HU X, XU W, et al.. Multi-objective optimization of a solar-driven trigeneration system considering power-to-heat storage and carbon tax [J/OL]. Energy, 2022, 250:123756 [2023-03-18]. .
15	HU M, CHO H. A probability constrained multi-objective optimization model for CCHP system operation decision support [J]. Appl. Energy, 2014, 116(1):230-242.
16	丁雨昊,吕干云,刘永卫,等.考虑碳排放目标约束和需求侧响应的综合能源系统日前优化调度[J].南方电网技术,2022,16(8):1-11.
	DING Y H, LYU G Y, LIU Y W, et al.. Day-ahead optimal scheduling of integrated energy system considering carbon emission target constraints and demand side response [J]. Southern Power Syst. Technol., 2022, 16(8):1-11.
17	JIANG Z, RISBECK M J, RAMAMURTI V, et al.. Building HVAC control with reinforcement learning for reduction of energy cost and demand charge [J/OL]. Energy Build., 2021, 239:110833 [2023-03-18]. .
18	VAN ZOEST V, GOHARY FEL, NGAI E C H, et al.. Demand charges and user flexibility-exploring differences in electricity consumer types and load patterns within the Swedish commercial sector [J/OL]. Appl. Energy, 2021, 302:117543 [2023-03-18]. .
19	XI L, SUN M, ZHOU H, et al.. Multi-agent deep reinforcement learning strategy for distributed energy [J/OL]. Measurement, 2021, 185:109955 [2023-03-18]. .
20	SHEN R, ZHONG S, WEN X, et al.. Multi-agent deep reinforcement learning optimization framework for building energy system with renewable energy [J/OL]. Appl. Energy, 2022, 312:118724 [2023-03-18]. .
21	DU Y, ZANDI H, KOTEVSKA O, et al.. Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning [J/OL]. Appl. Energy, 2021, 281:116117 [2023-03-18]. .
22	董雷,刘雨,乔骥,等.基于多智能体深度强化学习的电热联合系统优化运行[J].电网技术,2021,45(12):4729-4738.
	DONG L, LIU Y, QIAO J, et al.. Optimal dispatch of combined heat and power system based on multi-agent deep reinforcement learning [J]. Power Syst. Technol., 2021, 45(12):4729-4738.
23	ZHANG G, HU W, CAO D, et al.. A multi-agent deep reinforcement learning approach enabled distributed energy management schedule for the coordinate control of multi-energy hub with gas, electricity, and freshwater [J/OL]. Energy Conv. Manag., 2022, 255:116117 [2023-03-18]. .
24	蔺伟山,王小君,孙庆凯,等.不确定性环境下基于深度强化学习的综合能源系统动态调度[J].电力系统保护与控制,2022,50(18):50-60.
	LIN W S, WANG X J, SUN Q K, et al.. Dynamic dispatch of an integrated energy system based on deep reinforcement learning in an uncertain environment [J]. Power Syst. Prot. Control, 2022, 50(18):50-60.
25	阮应君,侯泽群,钱凡悦,等.基于深度强化学习的分布式能源系统运行优化 [J].科学技术与工程,2022,22(17):7021-7030.
	RUAN Y J, HOU Z Q, QIAN F Y, et al.. Optimization of the operation of distributed energy system based on deep reinforcement learning [J]. Sci. Technol. Eng., 2022, 22(17):7021-7030.

机组 Unit	数量 Number	参数 Parameter	值 Value
内燃机ICE	2	额定功率 Power rating/kW	4 282
		效率 Efficiency/%	42.9
		经验系数a₀ Experience factor a₀	0.001 4
		经验系数a₁ Experience factor a₁	1.49
		经验系数a₂ Experience factor a₂	-1.74
		经验系数a₃ Experience factor a₃	0.68
制冷机组AC	2	额定功率Power rating/kW	4 044
制冷机组AC	2	制冷系数COP	1.2
电制冷机组EC	6	额定功率 Power rating/kW	7 000
		制冷系数COP	5.18
		经验系数b₀ Experience factor b₀	-0.005 2
		经验系数b₁ Experience factor b₁	9.41
		经验系数b₂ Experience factor b₂	-7.012
		经验系数b₃ Experience factor b₃	2.79
蓄能罐TET	1	最大蓄冷量 Max storage capacity/kWh	50 000
蓄能罐TET	1	最大每小时最大蓄冷或释冷功率 Max output capacity/kW	14 000

机组 Unit	数量 Number	参数 Parameter	值 Value
内燃机ICE	2	额定功率 Power rating/kW	4 282
		效率 Efficiency/%	42.9
		经验系数a₀ Experience factor a₀	0.001 4
		经验系数a₁ Experience factor a₁	1.49
		经验系数a₂ Experience factor a₂	-1.74
		经验系数a₃ Experience factor a₃	0.68
制冷机组AC	2	额定功率Power rating/kW	4 044
制冷机组AC	2	制冷系数COP	1.2
电制冷机组EC	6	额定功率 Power rating/kW	7 000
		制冷系数COP	5.18
		经验系数b₀ Experience factor b₀	-0.005 2
		经验系数b₁ Experience factor b₁	9.41
		经验系数b₂ Experience factor b₂	-7.012
		经验系数b₃ Experience factor b₃	2.79
蓄能罐TET	1	最大蓄冷量 Max storage capacity/kWh	50 000
蓄能罐TET	1	最大每小时最大蓄冷或释冷功率 Max output capacity/kW	14 000

参数 Parameter	阶梯时段 Stepped tariff	价格 Price
c_Grid_，_buy_，_t	0：00—6：00；22：00—24：00	1.062 yuan·kWh^-1
	6：00—8：00；11：00—13：00；15：00—18：00；21：00—22：00	0.716 yuan·kWh^-1
	8：00—11：00；13：00—15：00；18：00—21：00	0.232 yuan·kWh^-1
c_Grid_，_sell		0.421 4 yuan·kWh^-1
c_NG		2.5 yuan·m^-3
c_DC		42 yuan·kW^-1

参数 Parameter	阶梯时段 Stepped tariff	价格 Price
c_Grid_，_buy_，_t	0：00—6：00；22：00—24：00	1.062 yuan·kWh^-1
	6：00—8：00；11：00—13：00；15：00—18：00；21：00—22：00	0.716 yuan·kWh^-1
	8：00—11：00；13：00—15：00；18：00—21：00	0.232 yuan·kWh^-1
c_Grid_，_sell		0.421 4 yuan·kWh^-1
c_NG		2.5 yuan·m^-3
c_DC		42 yuan·kW^-1

参数 Parameter	TD3算法 TD3 agent
参数 Parameter	Actor网络 Actor network	Critic网络 Critic network
不考虑需量电费输入层结构 Input layer structure without the demand charge	［1，3］	［1，11］
考虑需量电费输入层结构 Input layer structure with the demand charge	［1，4］	［1，12］
隐藏层数量 No. of hidden layers	2	2
隐藏层结构 Size of each hidden layers	［4，128］，［128，64］	［12，128］，［128，64］
输出层结构 Size of output layer	［8］	［1］
隐藏层激活函数 Activation function for the hidden layer	ReLU	ReLU
输出层激活函数 Activation function for the output layer	Sigmoid	ReLU
优化器 Optimizer	Adam	Adam
学习率 Learning rate	0.000 8	0.001 0
折扣因子 Discount factor	0.92
批量大小 Batch size	512