Robots are supposed to do boring or unpleasant jobs for us. However, tedious tasks such as cleaning the bathroom are challenging to automate. How is it possible to calculate the movement of a robot arm so that it can reach every part of a washbasin? What if the basin has unusually curved edges? How much force should be applied at which point?
It would be highly time-consuming to precisely encode all these things in fixed rules and predefined mathematical formulas. A different approach has been taken at TU Wien: a human shows a robot several times what it should do. A specially prepared sponge is used to clean the edge of a sink. By watching the human, the robot learns how cleaning works and can flexibly apply this knowledge to differently shaped objects. The work has now been published at IROS 2024 in Abu Dhabi.
Cleaning, sanding, polishing
Cleaning is just one type of surface treatment. Many other activities that play an essential role in industry are technically very similar — such as sanding or polishing surfaces, painting, or applying adhesives.
“Capturing the geometric shape of a washbasin with cameras is relatively simple,” says Prof Andreas Kugi from the Automation and Control Institute at TU Wien. “But that’s not the crucial step. It is much more difficult to teach the robot: Which type of movement is required for which part of the surface? How fast should the motion be? What’s the appropriate angle? What’s the right amount of force?”
People learn these things through experience and imitation. “In a workshop, someone might look over the apprentice’s shoulder and say: You need to press a little harder on that narrow edge,” says Christian Hartl-Nesic, head of the Industrial Robotics group in Andreas Kugi’s team. “We wanted to find a way to let the robot learn in a very similar way.”
The demo version of a cleaning sponge
A special cleaning tool was developed for this purpose: A cleaning sponge fitted with force sensors and tracking markers was used by humans to repeatedly clean a sink — but only the front edge. “We generate a huge amount of data from a few demonstrations, which is then processed so that the robot learns what proper cleaning means,” explains Christian Hartl-Nesic.
This learning process is made possible by an innovative data processing strategy developed by the research team at TU Wien. It combines several existing techniques from the field of machine learning: The measurement data is first statistically processed, and the results are used to train a neural network to learn predefined movement elements (so-called ‘motion primitives’). The robot arm is then optimally controlled to clean the surface.
This innovative learning algorithm enables the robot to clean the entire sink or other objects with a complex surface after the training, even though it has only been shown how to clean a single edge of the sink. “The robot learns that you have to hold the sponge differently depending on the shape of the surface, that you have to apply a different amount of force on a tightly curved area than on a flat surface,” explains PhD student Christoph Unger from the Industrial Robotics group.
The vision: all workshop robots learn together
The technology presented applies to many processes, whether sanding wooden workpieces in joineries, repairing and polishing paint damage in vehicle bodies, or welding sheet metal parts in metalworking shops. In the future, the robot could be placed on a mobile platform to be used as a useful helper anywhere in a workshop.
Such robots could then even share their knowledge with other robots. “Let’s imagine many workshops use these self-learning robots to sand or paint surfaces. Then, you could let the robots gain experience individually with local data. Still, all the robots could share the parameters they learned with each other,” says Andreas Kugi. Private data — such as the specific shape of a particular workpiece — would remain private, but essential basic principles would be exchanged to further improve the capabilities of all robots. This is referred to as ‘federated learning’.
Numerous tests at TU Wien have proven the sink-cleaning robot’s flexibility. The technology is also already causing a stir internationally: At IROS 2024 (14 to 18 October 2024), a conference with over 3,500 submitted scientific papers, TU Wien’s work was awarded the ‘Best Application Paper Award’ and thus voted one of the top innovations of the year.