In a paper scheduled for publication in the journal Nature Machine Intelligence on July 18, 2022, researchers at the Max Planck Institute for Intelligent Systems (MPI-IS) present their latest work. They have reportedly built a puppy-sized quadrupedal robot in order to learn how animals learn to walk and, in particular, gain experience from tripping and falling and then climbing back up.
(Photo from: MPI-IS / Dynamic Locomotion Group / Felix Ruppert)
In nature, a newborn fawn or foal must learn to walk as quickly as possible to avoid predators. Even though animals are born with a network of muscle coordination located in the spinal cord, learning the precise coordination of leg muscles and tendons takes some time.
At first, infant animals rely primarily on hard-wired spinal reflexes. Although more primitive, motor control reflexes help them avoid falling and injuring themselves during their first attempts at walking.
It is only through practice over time that they master more advanced and precise muscle control until the nervous system is eventually well adapted to the young animal’s leg muscles and tendons. After crossing the runaway barrier, they can then keep up with the adult animal.
Study illustration – 1: Morti, a quadrupedal robot
Felix Ruppert, a former PhD student in the MPI-IS Dynamic Motion Research Group, states:
“As an engineer and robotics expert, it is building a robot that has reflexes like an animal and learns from its mistakes.
If the animal only trips occasionally, the mistake is not yet clear. But if it stumbles a lot, it can be used to measure the quality of the robot’s walk.”
Study illustration – 2: Schematic diagram of the elastic moldable frame
It is worth noting that after just one hour of learning to walk, the robotic dog Morti has fully mastered its complex leg mechanics.
With Bayesian optimization guiding the learning, the information measured by the foot sensors can be matched with the virtual spinal cord target data modeled by the program.
By continuously comparing the sent and expected sensor information, the robot dog is able to find improvements in cyclic feedback and adjust its motor control pattern to learn to walk.
Study illustration – 3: Comparative reference of simulation versus practical trial and error
The learning algorithm is able to adapt to the control parameters of the Central Pattern Generator (CPG), which in humans and animals is the network of neurons in the spinal cord.
It is capable of generating periodic muscle contractions without relying on brain input, which can help in rhythmic tasks such as walking, blinking or digestion.
As for reflexes, leg sensors and hard-coded neural pathways are used to trigger controlled actions for involuntary movements.
Study illustration – 4: CPG parameters and elastic feedback activity
The CPG is sufficient to control the motor signals from the spinal cord as long as the young animal walks on a perfectly flat surface. However, as soon as there is a small bump in the ground, its walking pattern changes.
To avoid a fall, the reflexes intervene and modulate the motor pattern. These transient changes in the motor signals are reversible or “elastic”. Even if they are disturbed, they can be subsequently restored to their original state.
But if the animal keeps falling over multiple motorcycles – even if the response is active – then it must learn the motor pattern again and make it irreversibly “plastic”.
Study illustration – 5: Results of plastic adaptation
The newborn animal’s initial CPG conditioning is not yet sufficient to cause it to waddle over flat or uneven terrain, as is the case with Morti, the robotic dog presented here.
More importantly, within about an hour, the robot dog had been able to optimize its movement patterns faster than the smaller animal — thanks to Morti’s CPG being simulated on a small, lightweight computer that controls the movement of the machine’s legs.
This virtual spinal cord was placed at the back of the quadruped robot dog’s head, and sensor data from its feet were able to be constantly compared to the predicted effects of the CPG during the hour of smooth walking.
Study illustration – 6: Standardized measurement of torque performance
If the robot stumbles, the learning algorithm changes the distance its legs swing back and forth, the swing speed, and the landing stroke. Also the adjusted motion affects the robot’s ability to flexibly use its leg force.
During the learning process, the CPG sends tuned signals to the motors so that the robot dog can henceforth reduce stumbling and optimize its walking.
However, in this framework, the virtual spinal cord does not have a clear understanding of the robot’s leg movement design, the motors and the physical characteristics of the springs.
(From: Nature Machine Intelligence)
Flix Ruppert explains that the robot dog is actually “born” with some characteristics that allow the built-in CPG to help it walk naturally and intelligently, even if it has no knowledge of its own leg anatomy or how it works.
“The computer generates the signals to control the leg motors, and at first it stumbles a bit. Still, the data that kept flowing back from the sensors could be compared with that generated by the virtual spinal cord CPG.
If the sensor data does not match the expected result, the learning algorithm makes it change the walking behavior until the robot is able to walk well without tripping.
In summary, the learning process of changing the CPG output while keeping the reflexes active, and monitoring the robot dog for trips, is the core part of the process.”
Even better, Morti’s onboard computer will only consume 5 watts of power during the walk.
Finally, while industrial quadruped robots from some well-known manufacturers have learned to operate with the help of sophisticated controllers, they also consume much more power.
