A virtual drone was guided through an obstacle course by a person pretending to move their fingers
Willsey et al.
A paralyzed man with electrodes implanted in his brain can pilot a virtual drone through an obstacle course just by imagining moving his fingers. His brain signals are interpreted by an AI model and then used to control a simulated drone.
Brain-Computer Interface (BCI) research has made great strides in recent years, thanks to people with paralysis. precisely control the mouse cursor and command the computers to speak imagine writing words with a pen. But so far, they haven’t shown much promise in complex applications with multiple inputs.
now, Matthew Willsey The University of Michigan and his colleagues have created an algorithm that allows a user to trigger four discrete signals by simulating moving their fingers and thumb.
The anonymous man who tested the technology is a quadriplegic due to a spinal cord injury. He had already been fitted with a Blackrock Neurotech BCI, consisting of 192 electrodes, implanted in the area of the brain that controls hand movement.
An AI model was used to map the complex neural signals received by the electrodes to the user’s thoughts. The participant learned to think about moving the first two fingers of a hand, creating an electrical signal that can be made stronger or weaker. Another sign was created by the second two fingers and the other two thumbs.
These were enough to allow the user to control a virtual drone through thought alone, and with practice to pilot it skillfully through an obstacle course. Willsey says the experiment could have been done using a real drone, but was kept virtual for ease and safety.
“The goal of making the copter was really shared between our lab and the participant,” says Willsey. “For him, it was the realization of a dream that he thought was lost when he suffered the injury. He had a passion and a dream to fly. He seemed very empowered and qualified; he would ask us to take videos and send them to our friends.”
While the results are impressive, there is still much work to be done before BCIs can be used reliably for complex tasks, says Willsey. First, AI is needed to interpret the electrode signals, and it relies on the individual training of each user. Second, this training must be repeated over time as functionality declines, which may be due to the electrodes moving a little in the brain or changes in the brain itself.
Topics:
