A feedback force controller fusing traditional control and reinforcement learning strategies
Journal
IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM
Journal Volume
2019-July
Pages
259-265
Date Issued
2019
Author(s)
Abstract
We report on a novel force controller that fused a traditional proportional-integral-derivative (PID) controller with a proximal policy optimization (PPO)-based deep reinforcement learning (RL) algorithm. The deep RL algorithm provides long-term predictive force compensation to the real-time PID controller. The controller was applied to a 2D motion task with the goal of maintaining constant normal force in both simulation and experimental environments. The training of the deep RL algorithm used state data including the contact force and the distance between the workpiece and the end-effector of the manipulator. The validations of the proposed controller revealed that the controller with the aided deep RL algorithm could decrease the root mean square error (RMSE) of the measured force by between 11% and 80% in the simulation environment and by between 7% and 45% in the experimental environment, as compared with the controller without the aided deep RL algorithm. In addition, the displacement profile of the end-effector had a better match to that of the workpieces when aided by the deep RL algorithm.
Type
conference paper