Autonomous vehicles will find their way to – and literally in and around – our cities. Considering Amsterdam’s water-rich infrastructure, this provides an opportunity for innovation to relieve the city of traffic. Think about this: instead of a cab, how about ordering an autonomous boat to take you from A to B? Soon this could be our new reality.
“Trusting a technology is important to have inhabitants and other users of the city adopt the new technology, and to grow into a new reality. For trust to develop, one first has to understand how the technology works.”
Stephan van Dijk
Director of Innovation
With Roboat – a joint research project with Massachusetts Institute of Technology (MIT) – we provide self-navigating solutions on water for different use cases, for example, to help with waste management or to transport people and goods.
When new technologies are introduced, like autonomous navigation in this case, questions arise: what does it take for a boat to navigate through a canal without a skipper? What are key ingredients of autonomy, and how do you develop the autonomous technology? In this article, we immerse ourselves in the topic of Roboat’s autonomy.
“The Roboat technology is specifically designed for the urban context. Sailing autonomously in the city requires short-range sensors and more precise localization.”
Daniela Rus
Professor & AMS PI
3 ‘machine room’ components that make Roboat autonomous
Picture being amid the hustle and bustle on the Amsterdam canals – this urban context involves tight space maneuvering, including high complexity and not a lot of structure, caused by a great variety of obstacles that can be encountered.
Therefore, aside from the design specifications that facilitate the autonomy of Roboat – such as having four fixed thrusters to enable movement in all directions – the autonomy of the vessel is at its core connected to the software that runs on the onboard computer. Let’s dive straight into the three key ingredients that allow for autonomy on the Amsterdam canals:
- Navigation;
- Perception, and;
- Control algorithms.
Navigation requires exact localization and heading. The first step in developing autonomy and knowing how to take decisions on how to navigate the canals, is to provide the vessels with an accurate location. For this, Roboat uses RTK GPS.
Based on the current location and the desired endpoint, the boat autonomously determines a free path to sail from point A to B, avoiding all objects encountered on its route. While underway, Roboat constantly checks with the perception layer if any new obstacles might require a change of course.
“A system capable of accurate mapping and robust control is a crucial step towards having fully autonomous boat safely navigating urban waters.”
Wei Wang | Research Scientist | Massachusetts Institute of Technology
The perception allows the boat to map its environment and determine if any obstacles are near the boat. Equipped with a perception kit – consisting of LiDAR (Laser Image Detection and Ranging) and a set of cameras – Roboat perceives its environment. You can consider this to be the eyes and ears of the vessel. Based on the input received from the perception kit, Roboat can currently detect objects and perceive surroundings for a range of up to 100m.
The control algorithms translate the desired path into instructions towards the thrusters. To illustrate, the computer constantly compensates for external factors that influence the path of the boat – like wind, current and waves. All the gathered information is evaluated and integrated by the holistic mission control – the behavioral layer of Roboat – which enables the boat to (re)define its mission, decide where it needs to navigate towards and to translate the information into a specific path to take.
Ultimately, the behavioral layer determines the sequence of actions for the vessel and gives instructions to the engines to maintain a certain path/course.
“Roboat navigates autonomously using algorithms similar to those used by self-driving cars, but now adapted for water. Cooperative transport, using a team of water vehicles, poses distinctive challenges not encountered in aerial or ground vehicles. This is an innovative technical contribution of Roboat.”
Carlo Ratti
Professor at MIT Senseable City Lab & AMS PI
“A unique feature of Roboat is the latching/docking mechanism. This allows the vessel to autonomously connect to a docking station, without the need of a human for tying the boat down. The same mechanism is used to latch itself to other Roboats – for example to tow barges through the city, create a platform or a temporary bridge.”
Rens Doornbusch
Former Mechatronics Engineer Roboat at AMS Institute
Is this a ‘bouy’ the computer asks?
It’s the continuous feedback loops that makes Roboat smarter and as such better equipped for its task. Therefore, the Roboat team deploys algorithms to, among others, categorize specific objects it detects during its pathway. This way, every time the vessel navigates the canals, it gains experience and learns from previous situations and object encounters.
For instance, when the perception kit’s camera – by using object detection – perceives a floating obstacle, such as a buoy, this might be an object the boat has not yet 'seen' before. In that case, the algorithm marks the item as ‘unknown’ and when the team unloads the data, the object is manually selected and tagged as “buoy”.
After doing this manually around 10 times, the algorithm starts making suggestions “is this a buoy”. Sometimes it still marks another object as a buoy, and this is corrected manually, but with each time Roboat collects data, the mechanism is reinforced and the precision and robustness advances.
Roboat autonomously responds to objects on the water - such as boats that pass by - ©Oculus Film
“As a result, eventually, the algorithm outperforms the human eye in detecting a buoy on the water, even in harsh weather conditions.”
Joshua Jordan
Former Software Engineer Roboat Project at AMS Institute
Learning how to respond to wind and waves
Next to developing the algorithm for object-detection, the team also investigates learning mechanisms for the feedback loops that are implemented in the control algorithms. For instance, when a bigger boat passes, and the water gets wavy.
In time, after various learning experiences, the control algorithm will be able to determine the amount of throttle to counteract the effects of wavy water, to make the boat more stable and safer. The boat can even get as smart as to know how to respond to a gust of wind.
When can you say a boat is fully autonomous?
There are many different frameworks to define levels of autonomy. When using the so called DNVGL-CG framework, Roboat aims for the SC level: the self-controlled function. The system will execute an operation, but a human is still able to override the action.
This is sometimes also referred to as ‘human on the loop’. There is one level of autonomy that ranks higher in the definition of a boat being ‘fully autonomous”, but in that case the option for a human to intervene is fully eliminated.
“As Roboat can perform its tasks 24/7, and without a skipper on board, it adds great value for a city. However, for safety reasons it is questionable if reaching level A autonomy is desirable. Just like a bridge keeper, an onshore operator will monitor Roboat remotely from a control center. One operator can monitor over 50 Roboat units, ensuring smooth operations.”
Fábio Duarte
Research Scientist
Direction autonomy. But how to scale up to an urban context?
Roboat started prototyping small vessels late 2015. The first experiments took place a swimming pool at MIT swimming pool with the goal of defining the basic technology to navigate the pool. In the years after a range of progressing prototypes were developed and tested on the Charles River in the USA and later also on the Amsterdam canals.
CSAIl scientists from MIT kickstarted the autonomy, mostly focussing on the perception and control. The work in Amsterdam focused on implementing the functions on the full-scale boat, expand them and make the autonomy more precise and robust. The result today: two full-scale Roboat prototypes cruising the waters of Marineterrein Amsterdam. The team is now preparing a next phase of pilots and commercialization.
“If we would distribute 171 Roboat waste modules throughout the historic city centre of Amsterdam, we could serve 66% of all households. This would reduce the number of heavy trucks, CO2 and congestion substantially. A serious option to consider for cities worldwide!”
Ynse Hendrik Deinema
Former Roboat Project Coordinator at AMS Institute
Roboat can transform the city of Amsterdam by providing new infrastructure, but also in the various applications the boats can operate in. With its modular design the boat allows different configurations for the use-cases like waste management and the transportation of people and goods. And by moving part of the traffic from Amsterdam’s roads to our waterways, these autonomous boats can change urban dynamics and relieve the city of heavy traffic.
