Volume 36 Issue 3
Jul.  2022
Turn off MathJax
Article Contents
YAO Z Y,SHI Z W,DONG Y Z. Deep reinforcement learning for the control of airfoil flow separation[J]. Journal of Experiments in Fluid Mechanics, 2022,36(3):55-64. doi: 10.11729/syltlx20210085
Citation: YAO Z Y,SHI Z W,DONG Y Z. Deep reinforcement learning for the control of airfoil flow separation[J]. Journal of Experiments in Fluid Mechanics, 2022,36(3):55-64. doi: 10.11729/syltlx20210085

Deep reinforcement learning for the control of airfoil flow separation

doi: 10.11729/syltlx20210085
  • Received Date: 2021-08-04
  • Accepted Date: 2022-03-16
  • Rev Recd Date: 2022-03-14
  • Available Online: 2022-07-12
  • Publish Date: 2022-07-04
  • A jet closed-loop control system based on Deep Reinforcement Learning (DRL) was built, and an experimental study was carried out on the separation flow control at high angles of attack on the NACA0012 airfoil. The airfoil chord length is 200 mm and the wind speed was 10 m/s. The Reynolds number was 1.36×105 based on the chord length. The jet actuator was arranged on the upper surface of the airfoil and the solenoid valve was used for stepless control. The pressure coefficient of the airfoil surface and the action output of the agent itself were taken as the observation of the agent. The pressure coefficient of the trailing edge of the airfoil was used as the reward function to train the agent. Our results showed that the trained agent successfully suppresses the flow separation at high angles of attack and the cost-effectiveness ratio is reduced by 50% compared with steady blowing. At the same time, the agent could also stabilize the pressure coefficient of the trailing edge near the target value. The state input and the change of the reward function also have different effects on the final training effect.
  • loading
  • [1]
    SUTTON R S. Learning to predict by the methods of temporal differences[J]. Machine Learning,1988,3(1):9-44. doi: 10.1007/BF00115009
    [2]
    FRANCOIS-LAVET V,HENDERSON P,ISLAM R,et al. An introduction to deep reinforcement learning[J]. Founda-tions and Trends in Machine Learning,2018:219-354. doi: 10.1561/2200000071
    [3]
    GOODFELLOW I, BENGIO Y, COURVILLE A. 深度学习[M]. 赵申剑, 黎彧君, 符天凡, 等译. 北京: 人民邮电出版社, 2017.
    [4]
    PINTO L,ANDRYCHOWICZ M,WELINDER P,et al. Asymmetric actor critic for image-based robot learning[J]. Computer Science,2017:1-8. doi: 10.15607/RSS.2018.XIV.008
    [5]
    BAHDANAU D, BRAKEL P, XU K, et al. An actor-critic algorithm for structured prediction[EB/OL]. [2021-08-24]. https://arxiv.org/abs/1607.07086v2.
    [6]
    MNIH V,KAVUKCUOGLU K,SILVER D,et al. Playing Atari with deep reinforcement learning[J]. Computer Scien-ce,2013:1-9.
    [7]
    SILVER D,SCHRITTWIESER J,SIMONYAN K,et al. Mastering the game of Go without human knowledge[J]. Nature,2017,550(7676):354-359. doi: 10.1038/nature24270
    [8]
    BERNER C, BROCKMAN G, CHAN B, et al. Dota 2 with large scale deep reinforcement learning[EB/OL]. [2021-08-24]. https://arxiv.org/abs/1912.06680v1.
    [9]
    THE ALPHASTAR TEAM. AlphaStar: Mastering the real-time strategy game StarCraft II[EB/OL]. [2021-08-24]. https://deepmind.com/blog/alphastar-mastering-real-time-strategy-game-starcraft-ii.
    [10]
    BROWN N,SANDHOLM T. Superhuman AI for multi-player Poker[J]. Science,2019,365(6456):885-890. doi: 10.1126/science.aay2400
    [11]
    KENDALL A, HAWKE J, JANZ D, et al. Learning to drive in a day[C]//Proc of the 2019 International Conference on Robotics and Automation(ICRA). 2019.
    [12]
    BEWLEY A, RIGLEY J, LIU Y X, et al. Learning to drive from simulation without real world labels[C]//Proc of the 2019 International Conference on Robotics and Automation(ICRA), 2019. doi: 10.1109/ICRA.2019.8793668
    [13]
    KNIGHT W. Google just gave control over data center cooling to an AI [EB/OL]. [2021-08-24]. https://www.technologyreview.com/s/611902/google-just-gave-control-over-data-center-cooling-to-an-ai.
    [14]
    CHNG T L,RACHMAN A,TSAI H M,et al. Flow control of an airfoil via injection and suction[J]. Journal of Aircraft,2009,46(1):291-300. doi: 10.2514/1.38394
    [15]
    COIRO D P,BELLOBUONO E F,NICOLOSI F,et al. Improving aircraft endurance through turbulent separation control by pulsed blowing[J]. Journal of Aircraft,2008,45(3):990-1001. doi: 10.2514/1.33268
    [16]
    VERMA S,NOVATI G,KOUMOUTSAKOS P. Efficient collective swimming by harnessing vortices through deep reinforcement learning[J]. Proceedings of the National Academy of Sciences of the United States of America,2018,115(23):5849-5854. doi: 10.1073/pnas.1800923115
    [17]
    SHIMOMURA S, SEKIMOTO S, FUKUMOTO H, et al. Preliminary experimental study on closed-loop flow separa-tion control utilizing deep Q-network over fixed angle-of-attack airfoil[C]//Proc of the 2018 Flow Control Conference. 2018. doi: 10.2514/6.2018-3522
    [18]
    GUÉNIAT F,MATHELIN L,HUSSAINI M Y. A statistical learning strategy for closed-loop control of fluid flows[J]. Theoretical and Computational Fluid Dynamics,2016,30(6):497-510. doi: 10.1007/s00162-016-0392-y
    [19]
    PIVOT C, CORDIER L, MATHELIN L. A continuous reinforcement learning strategy for closed-loop control in fluid dynamics[C]//Proc of the 35th AIAA Applied Aero-dynamics Conference. 2017. doi: 10.2514/6.2017-3566
    [20]
    XU H,ZHANG W,DENG J,et al. Active flow control with rotating cylinders by an artificial neural network trained by deep reinforcement learning[J]. Journal of Hydrodynamics,2020,32(2):254-258. doi: 10.1007/s42241-020-0027-z
    [21]
    RABAULT J,KUCHTA M,JENSEN A,et al. Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control[J]. Journal of Fluid Mechanics,2019,865:281-302. doi: 10.1017/jfm.2019.62
    [22]
    TANG H W,RABAULT J,KUHNLE A,et al. Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforce-ment learning[J]. Physics of Fluids,2020,32(5):053605. doi: 10.1063/5.0006492
    [23]
    FUJIMOTO S, VAN HOOF H, MEGER D. Addressing function approximation error in actor-critic methods[EB/OL]. [2021-08-24]. https://arxiv.org/abs/1802.09477 2018: arXiv:1802.09477[cs.AI].
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(17)

    Article Metrics

    Article views (1669) PDF downloads(80) Cited by()
    Proportional views
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return