When your mind lets you control a robotic arm, restore the sense of touch

WASHINGTON: Imagine being able to control a robotic arm from a distance, using only your mind. Now imagine being able to feel when its fingers grasp an object, as though it were your own hand.

US researchers published a study in the journal Science on Thursday about the world's first brain-computer interface that allowed a volunteer with paralysis from the chest down to accomplish this very feat.

The team say their work demonstrates that adding a sense of touch drastically improves the functionality of prosthetics for quadriplegics, compared to having them rely on visual cues alone.


"I am the first human in the world to have implants in the sensory cortex that they can use to stimulate my brain directly," Nathan Copeland, 34, told AFP.

"And then I feel as if a sensation is coming from my actual hand."

'Super cool'
In 2004, Copeland was in a car accident that left him with a serious spinal cord injury and without the use of his hands or his legs.

He volunteered to participate in scientific research, and six years ago underwent a major operation to have tiny electrodes implanted in his brain.

Two sets of 88 electrodes the width of a strand of hair are arranged into "arrays" that resemble tiny hairbrushes and penetrate deep into the brain's motor cortex, which directs movement.

Fewer than 30 people in the world have these kinds of implants, the study's co-lead author Rob Gaunt, an assistant professor in the Department of Physical Medicine and Rehabilitation at the University of Pittsburgh, told AFP.

What's unique about Copeland is an additional set of electrodes that are connected to his somatosensory cortex, which receives and processes sensations.

"When we're grabbing objects, we use this sense of touch very naturally to improve our ability to control," explained Gaunt.

The team developed an interface that is "bi-directional" -- meaning not only can it "read" instructions from the brain and send them to the artificial limb, it can also "write" sensations from the device and transmit them back.

The idea of sending tactile feedback to the somatosensory cortex goes back decades, but doing so in a way that is controlled and is understandable by the circuitry of the brain was a challenge.

After operating on Copeland to install the electrodes, the team held their breath.

"No one knew what to expect because this had only been done in monkeys and you can't ask a monkey what something feels like," said Copeland.

Then came the moment of truth, when they tried sending their first touch signal.

"It was really faint," he recalled. He asked them to try again, to be sure it was real.

"Oh yeah, that's what that feels like, and it was just super cool," he said.

Twice as fast
Before the interface could be put to work with the robotic arm, the scientists had to perform a series of tests with Copeland.

First, they needed to learn which electrodes caused what sensation when activated, and which fingers they were associated with, to set up the robotic hand correctly.

They also made him watch videos of the robotic arm moving left or right and recorded the electrodes that lit up when he was asked to "think" it was him controlling it.

Finally it was time to try it out.

Copeland sat next to the metallic black robotic arm and was asked to pick up a series of small objects like rocks, spheres, and place them on a box -- with either the tactile sensors switched on, or off.

He was able to complete each task on average twice as fast when the sensors were enabled, and was even able to achieve more complex tasks like picking up a glass and pouring its contents into another.

"The sensation gave me that assurance and confidence to know that I definitely had a good grab on the object and I could lift it up," said Copeland.

The team wants to further refine the prosthetics because "we don't want to just do science experiments in the lab, we want to actually build devices that are useful to people in their homes," added Gaunt.

Copeland got his brain-computer interface set up at home when the Covid pandemic shut down the university, and has used his downtime to learn how to draw on a tablet and even play video games.

He does this by using his mind to send signals directly to the computer, rather than using the arm to push buttons.

"It is just second nature to me now," he said.