Tactile feedback adds ‘muscle sense’ to prosthetic hand

US, Italian researchers test ‘proprioceptive’ feedback with skin-stretch device

Engineers working to add “muscle sense” to prosthetic limbs found that tactile feedback on the skin allowed blindfolded test subjects to more than double their ability to discern the size of objects grasped with a prosthetic hand. The results will be presented next month in Germany by researchers from Rice University and the Research Center “E.Piaggio” of the University of Pisa and the Italian Institute of Technology (IIT).

“Humans have an innate sense of how the parts of their bodies are positioned, even if they can’t see them,” said Marcia O’Malley, professor of mechanical engineering at Rice. “This ‘muscle sense’ is what allows people to type on a keyboard, hold a cup, throw a ball, use a brake pedal and do countless other daily tasks.”

The scientific term for this muscle sense is proprioception, and O’Malley’s Mechatronics and Haptic Interfaces Lab (MAHI) has worked for years to develop technology that would allow amputees to receive proprioceptive feedback from artificial limbs.

Test subjects were more than twice as likely to correctly discern the size of grasped objects grasped with a prosthetic hand when they received haptic feedback from a simple skin-stretch device on the upper arm. (Photo by Jeff Fitlow/Rice University)

In a new paper to be presented June 7 at the World Haptics 2017 conference in Fürstenfeldbruck, Germany, O’Malley and colleagues demonstrate that 18 able-bodied test subjects performed significantly better on size-discrimination tests with a prosthetic hand when they received haptic feedback from a simple skin-stretch device on the upper arm. The study is the first to test a prosthesis in combination with a skin-stretch rocking device for proprioception, and the work has been recognized as a finalist for best paper award at the conference.

An estimated 1.7 million people in the U.S. live with the loss of a limb. Traditional prostheses restore some day-to-day function, but very few provide sensory feedback. For the most part, an amputee today must see their prosthesis to properly operate it.

Improved computer processors, inexpensive sensors, vibrating motors from cellphones and other electronics have created new possibilities for adding tactile feedback, also known as haptics, to prosthetics, and O’Malley’s lab has done research in this area for more than a decade.

The Rice Haptic Rocker uses a rotating arm to brush a soft rubber pad over the skin of the arm. The more a prosthetic hand closes, the more the pad brushes the skin. (Photo by Brandon Martin/Rice University)

“We’ve been limited to testing haptic feedback with simple grippers or virtual environments that replicate what amputees experience,” she said. “That changed when I was contacted last year by representatives of Antonio Bicchi’s research group at Pisa and IIT who were interested in testing their prosthetic hand with our haptic feedback system.”

In experiments at Rice beginning late last year, Pisan graduate student Edoardo Battaglia and Rice graduate student Janelle Clark tested MAHI’s Rice Haptic Rocker in conjunction with the Pisa/IIT SoftHand. They measured how well blindfolded subjects could distinguish the size of grasped objects both with and without proprioceptive feedback.

While some proprioceptive technologies require surgically implanted electrodes, the Rice Haptic Rocker has a simple, noninvasive user interface — a rotating arm that brushes a soft rubber pad over the skin of the arm. At rest, when the prosthetic hand is fully open, the rocker arm does not stretch the skin. As the hand closes, the arm rotates, and the more the hand closes, the greater the skin is stretched.

“We’re using the tactile sensation on the skin as a replacement for information the brain would normally get from the muscles about hand position,” Clark said. “We’re essentially mapping feedback from one source onto an aspect of the prosthetic hand. In this case, it’s how much the hand is open or closed.”

Edoardo Battaglia (left), Manuel Catalano and Janelle Clark at Rice’s Mechatronics and Haptic Interfaces Lab. (Photo by Jeff Fitlow/Rice University)

Like the Rice Haptic Rocker, the SoftHand uses a simple design. Co-creator Manuel Catalano, a postdoctoral research scientist at IIT/Pisa, said the design inspiration comes from neuroscience.

“Human hands have many joints and articulations, and reproducing and controlling that in a robotic hand is very difficult,” he said. “When you have to grasp something, your brain doesn’t program the movement of each finger. Your brain has patterns, called synergies, that coordinate all the joints (in the hand).”

The Pisa/IIT SoftHand uses a control synergy just like people do in everyday life, Catalano said. “At the same time, thanks to the intrinsic capability of the SoftHand to adapt and deform with the environment, it is robust and able to grasp objects in many different ways.”

Battaglia said neurological studies have identified a set of synergies for the hand. People use these alone or in combination to perform tasks as simple as turning a doorknob and as complex as playing the piano. Grasping an object, like a cup or a coat hanger, is one of the simplest.

“Experiments show that one synergy explains more than 50 percent of all grasps,” he said. “SoftHand is designed to mimic this. It’s very simple. There is just one motor and one control wire to open and close all the fingers at once.”

The latest findings from neuroscience provided the inspiration for the design of the SoftHand, a prosthetic hand from the University of Pisa and the Italian Institute of Technology. (Photo by Jeff Fitlow/Rice University)

In tests, subjects used the SoftHand to grasp objects of varying shapes and sizes, ranging from grapefruit-sized balls to coins (quarters). To close the hand, subjects simply flexed a muscle in their forearm. Electrodes taped to the arm picked up electric signals from the flexing muscle and transmitted those to the motor in the SoftHand.

For the size-discrimination test, subjects were blindfolded and asked to grasp two different objects. They were then asked which of the two was larger. Without haptic feedback, the blindfolded subjects had to base their guesses on intuition. They chose correctly only about 33 percent of the time, which is what one would expect from a random choice. When they performed the same tests with feedback from the Rice Haptic Rocker, the subjects correctly distinguished the larger from smaller objects more than 70 percent of the time.

The researchers are following up to see if amputees get a similar benefit from using the haptic rocker in conjunction with the SoftHand.

“One of the things that makes the research we do in the MAHI lab unique is that we involve end-users from the very beginning, from the design and concept stage all the way to testing and evaluation of our systems,” O’Malley said. “Through our close collaborations in the Texas Medical Center, we are able to have those interactions with the end users — with patients, physical therapists and doctors — all of the way through our design and evaluation process.”

Additional co-authors include Matteo Bianchi of the University of Pisa. The research was supported by the National Science Foundation and the European projects WEARHAP, SOFTPRO and SoftHands. The collaboration will continue to be supported by a recently announced Rice University Award for International Collaboration, allowing Janelle Clark to visit the University of Pisa for an extended period in 2018.