Language-Guided Pattern Formation for Swarm Robotics with Multi-Agent Reinforcement Learning
Journal
2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Part Of
IEEE International Conference on Intelligent Robots and Systems
Start Page
8998
End Page
9005
ISSN
21530858
ISBN (of the container)
979-835037770-5
Date Issued
2024-10-14
Author(s)
Abstract
This paper explores leveraging the vast knowledge encoded in Large Language Models (LLMs) to tackle pattern formation challenges for swarm robotics systems. A new framework, named LGPF (Language-Guided Pattern Formation), is proposed to address these challenges. The framework breaks down the pattern formation into two key components: pattern synthesis and swarm robotics control. For the former, this study utilizes the exceptional few-shot generalizability of LLMs to translate high-level natural language descriptions into the desired spatial pattern coordinates. This approach allows for overcoming previous limitations in representing and designing complex patterns. The framework further employs a centralized training with decentralized execution (CTDE) based multiagent reinforcement learning (MARL) approach to control the swarm robots in forming the specified pattern while avoiding collisions. The decentralized policies learned with the CTDE-based MARL algorithm consider coordination between robots without direct communication under a partially observable setup. To validate the effectiveness of our framework, we perform extensive experiments in both simulation and real-world environments. These experiments validate LGPF’s effectiveness in accurately and safely forming diverse user-specified patterns.
Event(s)
2024 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2024
Publisher
IEEE
Type
conference paper