1. To become comfortable with ROS.
2. To write code, tested on turtlebots, that allowed a mobile robot to explore and map new terrain and execute user-specified routes.
This project was my senior engineering project and was motivated largely by an interest in autonomous navigation after a summer spent researching visual SLAM techniques as a part of the Carnegie Mellon Lunar Lab cube rover project IRIS. ROS is a well-known robotics framework, widely used in industry, so this project was also an opportunity I used to broaden my knowledge of robotics past the fundamentals as taught in the Swarthmore mobile robotics course.
To create a goal for my exploration of ROS and autonomous navigation, I framed the project around the development of an autonomous wheelbarrow type robot; my project focused on the design of this hypothetical product’s navigational and user interface systems.
Giving myself a real-world goal helped me focus my project and provided extra motivation because I am motivated by solving real-world problems:
→ real wheelbarrows are heavy and cumbersome
→ moving dirt and construction materials is repetitive manual labor
Solution:
A robotic wheel barrow (RObarrow) in the genre of labor reduction through semi-autonomous robotic partnership; a popular trend in innovation in the manufacturing and agricultural sectors.
Note: If you’re already familiar with ROS you can just skip to the implementation section below
ROS is a flexible framework for writing robot software, initially developed by the Stanford AI Laboratory in 2007 that includes a collection of tools, libraries, and conventions that simplify the task of creating complex and robust robot behavior. The Open Source Robotics Foundation now maintains ROS, so ROS also provides a sort of open source community for robotics development.
Nodes and Messages
ROS is an Operating System in concept, though not an actual independent OS, and provides all the services that any other OS does including a unique message-passing process. This message passing occurs with a loosely coupled node-message structure in which a process is called a node and every node is responsible for one task. Nodes communicate with each other using messages passing via logical channels called topics. Each node can send or get data from the other nodes using the publish/subscribe method -- nodes either publish topics or subscribe to them. In order to manage this loosely-coupled environment, there is a Master in ROS which is responsible for name registration and lookup for the rest of the system.
Transforms and Frames
A robotic system typically has many 3D coordinate frames that change over time, such as a world frame, base frame, gripper frame, etc.
tf (or tf2 for recent versions of ROS) is a package that lets the user keep track of these multiple coordinate frames over time, maintaining the relationship between coordinate frames in a tree structure buffered in time. A user can transform points, vectors, etc between any two coordinate frames at any desired point in time, and use transforms to determine things like where the base frame of a robot is relative to the map frame.
Below is a diagram of a simple transform tree between a robot’s base and laser scanner. It’s important to know this relationship if the laser’s data is going to be used to help localize the robot’s base.
In order to work with transforms, a user must create nodes to either listen for transforms or broadcast transforms. Listening for transforms involves receiving and buffering all coordinate frames that are broadcasted in the system, and querying for specific transforms between frames. A broadcaster meanwhile sends out the relative pose of coordinate frames to the rest of the system. A user can dictate a custom relationship between two coordinate frames using broadcasters; a system can have many broadcasters that each provide information about a different part of the robot.
Example code:
Navigation Stack and move_base
ROS' navigation package contains a 2D Navigation Stack (NavStack) which takes in information from odometry and sensor streams and outputs velocity commands to send to a mobile base. The job of the navigation stack is to produce a safe path for the robot to execute, by processing data from sensors and the environment map. The move_base node within the NavStack provides a more advanced way to control a mobile robot -- it is what does the work of determining the robots position in the environment where it is located (localization) and planning and executing the path to a target location. It also enables autonomous avoidance of both static and dynamic obstacles based on continuous sensor inputs.
The diagram below shows the navigation stack and move_base, and an example of the move_base in use in conjunction with an amcl node. The topic “cmd_vel” contains the published velocity commands.
Mapping and Localization Packages
Simultaneous Localization and Mapping (SLAM) describes a set of techniques used to build maps and localize a robot within a new environment; SLAM algorithms build maps and localize your vehicle in that map at the same time; this allows the vehicle to map out unknown environments. ROS supports a variety of SLAM packages, both standard and open source ones created by the community. The two main packages used for localization in the final iteration of this project’s code were Gmapping and AMCL (adaptive monte-carlo localization), both commonly utilized ROS packages.
Gmapping creates a 2-D occupancy grid map, like a building floor plan, from laser and pose data collected by a mobile robot. Gmapping is an implementation of a SLAM, specifically, this package utilizes a highly efficient Rao-Blackwellized particle filter to learn grid maps from laser range data.
The image below shows the process of mapping the robotics lab with a turtlebot (viewed in RVIZ).
AMCL is a probabilistic localization system for a robot moving in 2D. It implements the adaptive (or KLD-sampling) Monte Carlo localization. This algorithm uses a particle filter to represent the distribution of likely states, with each particle representing a possible state i.e., a hypothesis of where the robot is. The particles become more condensed with increased certainty.
The image below shows a simulated turtlebot navigating a simulated environment -- the green cloud is the particle cloud produced by running AMCL (viewed in RVIZ).
For my code I built and saved maps of environments with gmapping and then used a map server to broadcast the pre-made map so that it could be used as input to the AMCL node for the localization step.
Actionlib and Simple Action Client
The actionlib stack provides a standardized interface for sending preemptable tasks such as moving the base to a target location, performing a laser scan and returning the resulting point cloud, etc.The actionlib allows the user to send a request to a node to perform some task, and also receive a reply to the request. Commands can also be sent using ROS “services” but the actionlib provides tools to create servers that execute long-running goals with greater flexibility such as the ability to cancel the request and get periodic feedback on the request’s status. To use the action lib, one creates an ActionClient and ActionServer which communicate via a "ROS Action Protocol" using ROS messages.
The diagram below demonstrates ActionClient and ActionServer interaction.
In my case I implemented a SimpleActionClient which is an implementation of the ActionInterface which supports only one goal at a time. My simple action client sends the goal information to the action server which is in communication with the move_base node. The move_base does the requisite path planning, while the action server and client send information about the goal position and goal status back and forth.
Simulation Tools Gazebo and Rviz
Gazebo is a 3D simulation tool that is supported by ROS, complete with customizable rendered environments and a large selection of modeled robots including the turtlebot. Gazebo fully simulates robot use including models of sensors that "see" the simulated environment, such as laser range finders, cameras, Kinect style sensors, etc. and subscribes to ROS message topics so that a user can send the simulated root commands just like one would a real-world robot.
Below: a turtle-themed Gazebo environment (clearly the best environment) and a custom Gazebo environment.
Rviz is a 3d visualization tool for ROS. It provides a wide selection of visualization options including a view of your robot model and captured sensor information from robot sensors such as data from camera, lasers, point clouds, and maps such as those created by gmapping. Rviz can also be used for some minimal interfacing with your robot such as sending simple navigation goals.
Below you can see a view of the robot localizing within a mapped environment. From the view in Rviz we can see that the map has not been properly aligned with the physical environment in this example.
The project can be broken down into 4 distinct phases, with increasingly complex goals for what the code allows a "user" to do:
Phase 1: user can map environment and save map
Phase 2: user can navigate to and save locations in the environment map
Phase 3: user can create timed loops between saved locations, and the robot creates routes accordingly
Bonus Phase: same functionality, nicer interface
1 2 3
Before I could begin coding for Phase 1, I needed to establish some basic understanding of ROS.
Step 1: Research
I began by reading ROS documentation and testing my understanding of ROS with coding demo projects. ROS is uniquely structured with a node-messenger framework that takes some getting used to. I approached structuring this project with an emphasis on starting small.
Step 2: Define Program Goals
Building a project from scratch, without the typical structure provided by a class assignment meant step 2 was defining the purpose and scope for myself. Listing the key goals of my code helped me in describing the different pieces of functionality required for my program.
Step 3: Flesh Out the Essential Program Features
With my priorities clear, and with the help of the ROS site demos in understanding the standard building blocks of a ROS navigational program, I was then able to determine the essential features of my ROS program.
The aim of these steps was to maintain project focus by preventing “scope” and “purpose” from evolving too much during early development. For any project, in my experience, it is critical to set small attainable goals, then allow yourself to improve and evolve on these foundations. You accomplish nothing if you work towards abstract, moving goals. In practice, “starting small” can look different depending on the context. Main tools I used to set small attainable goals and stay on schedule:
TOOL BOX
Some of these notes may sound trivial, or obvious, but consciously making these decisions about the design process and building good coding habits was essential to my success in approaching this more open-ended and independent project.
To achieve the main objective of localizing a robot within a new environment, I pursued several of the many different localization packages available in ROS. For this project I tested a few different common ROS-supported packages in simulation and real world environments to choose one best suited for the RObarrow. My main goal was navigating and setting goals in different environments, and so I settled on the joint use of the AMCL (adaptive monte-carlo localization) and gmapping packages, discussed above.
For early tests I built maps of simulated environments in gazebo with gmapping, used a map server to broadcast the pre-made map, and localized the robot in the map with AMCL. My end product uses a similar process with distinct map-building and localization phases.
The early testing phases of this project leveraged the simulation tool Gazebo during a winter spent at home due to the covid-19 pandemic, but I began tests on real turtlebots once I was allowed to return to campus in the Spring of 2021. These later testing phases were slowed somewhat by hardware and technical difficulties in the introduction of this code to real-world testing, however the experience of debugging combined hardware and software issues was invaluable.
Gantt chart:
To structure and visualize the progression of small goals. Also helpful in keeping a schedule.
Version control with github:
Great for preserving old versions of code; especially useful tool for larger projects because even as the project grows you can always revert or roll back the changes you've made with the version control system.
Modular programming:
Essentially modularity is what you achieve when you try to start your project with the most fundamental elements of the program. Writing code modularly means separating the functionality of a program into independent blocks such that each contains everything necessary to execute only one aspect of the desired functionality. The modular design process is already good practice, but additionally integrates nicely within the multi-node structure of ROS which creates an inherent sort of modularity.
Regular testing cycles:
Another important aspect of my design process was the further enforcement of good coding habits. I had never previously approached a project of this scale with such little formal structure, so I focused on working smart in addition to hard. I made use of small testing-cycles as well as “fail fast" coding habits; writing code in chunks that can be tested semi-independently as you work your way through the project, and writing in assertions for unhandled situations that will help code to “fail fast” and catch bugs early.
The result of this project was the creation of a python script that leverages several different ROS packages to localize and direct a robot within a mapped environment. The code is meant to simulate the sort of behavior I’d like to see in a robarrow, though it was mainly run and tested on turtlebots in the robotics lab. The script prints a menu to the terminal and then allows a user to input commands from the command line to add new locations to a stored list of known locations within the map, to navigate to specific locations, or to create loops between specific known locations.
The general script structure uses simultaneous loops that take keyboard input while maintaining the ros subscriber and action client. The keyboard input is managed with the termios module, and is received from the terminal. The script starts by setting up rospy publishers, subscribers and a SimpleActionClient as seen below:
In the early version of this project, the user would then be prompted with a menu in the terminal as can be seen below.
If the user inputs a location name, the location and the robots current pose are stored in a dictionary and then copied over to a json file to save them more permanently. The subscriber seen above is set up to read information from the amcl_pose topic so that my script can assign accurate localization poses to locations when they are designated.
If a user chooses to input a destination or create a loop, the script uses the SimpleActionClient to send goals to the ActionServer which publishes to the move_base node where the robots path is planned. The action client is templated on the action definition, specifying what message types to communicate to the action server with, in this case, a MoveBaseAction. The action client here connects to the specified ‘move_base’ server to send the later specified goal.
The script has to be run in conjunction with the robot’s move-base node, keyboard teleop for manual user control, the amcl node for localization, and a map server to publish the previously saved map for use in the robots navigation and localization. The diagram below shows the structure of the interactions between different ROS nodes and my script.
After successfully writing a program that allows for goal setting in a pre-mapped environment, the main goal of this project, I decided to work towards developing a cleaner user interface since an actual product would ideally not be operated from the command line. I began experimenting with tkinter -- a Python binding to the Tk GUI toolkit, towards the end of the semester. I started with the goal of simply publishing velocity commands, and succeeded in coordinating tk and ROS to create a simple “teleop” gui that publishes to cmd_vel when a direction-button is selected.
Below is an image of my simple teleop GUI and the resulting velocities sent to the turtlebot and displayed in the terminal window.
After this initial test I created the more complex user interface, seen below, that is compatible with my existing navigation script. Formatting the larger GUI was more of a challenge than anticipated, but I’m pretty happy with its aesthetic :)
As an extra-bonus I also mocked up a rough idea of what a physical RObarrow could look like in Autodesk Fusion 360. It’s inspired by trailer tech’s electric (but not autonomous!) hydraulic mini wheelbarrow, and includes the tread design from the rhino track drive module. Should I have spent that extra time writing more complex scripts or perfecting the GUI instead of making fun CAD designs? Perhaps.
