A wristband into a real-time hand reader, and the ultrasound system can make a robot play the piano, shoot mini basketballs, and mimic your fingers with surprising precision

Every day, your hands do something amazing, and most of us don’t take a moment to think about it. Picking up a cup of coffee, threading a needle, typing on a keyboard: feats of biological engineering so complex that machines have struggled to copy them. Now a team at MIT may have found a promising new way forward, and it fits around your wrist.

According to a study, ‘Hand tracking using wearable wrist imaging,’ published in Nature Electronics, researchers at MIT and the University of Southern California have created a fully integrated, wireless ultrasound imaging wristband that, when paired with an artificial intelligence algorithm, can continuously monitor arbitrary hand configurations of the five fingers and the palm in real time, with a delay of less than 120 milliseconds. The paper was led by MIT professor of mechanical engineering Xuanhe Zhao, and co-authored by Gengxi Lu, a former MIT postdoc, among others.

Why your wrist holds the key
The research is based on a beautiful insight. The tendons and muscles in your wrist act as control cables for your fingers. As lead co-author Gengxi Lu explained: “The tendons and muscles in your wrist are like strings pulling on puppets, which are your fingers. So the idea is, each time you take a picture of the state of the strings, you'll know the state of the hand.”

The wristband uses high-frequency sound waves to image the muscles, tendons, and ligaments in the wrist, providing a detailed, real-time view of the mechanisms that drive hand movement. The system tracks the full set of joint motions that allow for the fingers and palm to flex, rotate, and coordinate intricate activity, constantly monitoring all 22 degrees of freedom of the human hand. Then the ultrasound data is fed into an AI model to convert those images into continuous finger positions.

What it can actually do
In testing with eight volunteers of varying hand and wrist sizes, the wristband successfully tracked gestures, including all 26 letters of American Sign Language, as well as handling everyday objects such as a tennis ball, scissors, and a plastic bottle. During practical demonstrations, a person wearing the wristband showed how to guide a robotic hand to play a simple tune on a piano and shoot a miniature basketball into a desktop hoop. The same wristband allowed users to pinch, zoom, and manipulate objects on a computer screen smoothly and continuously.

It’s like a wireless marionette, but the strings are sound waves, and the puppet is a robotic hand.

Why existing tools fall short
The main methods for capturing hand motion today all have real limitations. Camera-based systems can track hands in 3D but require careful setup, can be blocked by other objects or people, and often have difficulty in different lighting conditions. Glove systems embed sensors in cloth, but the hardware can feel bulky, restrict natural movement, and deaden the sense of touch. An ultrasound wristband solves both problems by reading movement from the inside out, under the skin, where the real action is happening.

Training the robots of tomorrow
The implications go far beyond playing the piano or dropping through tiny hoops. According to the MIT News release, the team envisions collecting hand-motion data to build a dataset that could help train humanoid robots for dexterity tasks, such as performing certain surgical procedures. The ultrasound band could also be used to interact with objects in video games, design applications, or other virtual settings.

What comes next
Zhao’s team hopes to shrink the hardware further and train the AI on data from more users. A future version could be lighter, longer-lasting, and easier to wear. The researchers also want the device to work across a wider range of hand sizes, without requiring lengthy personal calibration sessions.

Zhao summed up both the scope and the ambition of the work: “We believe this is the most advanced way to track dexterous hand motion, through wearable imaging of the wrist. We think these wearable ultrasound bands can provide intuitive and versatile controls for virtual reality and robotic hands.”

For now, the tech is living in a lab. But what it hints at, a future where your own hands show robots how to move like humans, is starting to feel very real.