WETrak

Finger Tracking Using Wrist-Worn EMG Sensors

Overview

This paper introduces WETrak, a finger tracking system using wrist-worn electromyography (EMG) sensors. Recent finger tracking methods mainly employ EMGs on armbands. Compared to a range of contactless methods using cameras or wireless, they are not limited by high computational costs, privacy concerns, and mobility, while unlike other wearable-based approaches, they do not require the deployment of sensors on the user's hands. However, users need to wear an additional armband on their forearm each time solely for tracking purpose, which hinders the widespread adoption of finger tracking in practice. This paper investigates the feasibility of moving EMG sensors from the forearm to the wrist for finger tracking. WETrak inherits the advantages of existing EMG-based armband tracking while avoiding the limitation of requiring additional armbands, which brings a strong incentive for integrating EMG sensors into wrist-worn wearables in the future. As sensor placement varies, we find new challenges in determing good locations to place sensors to gather useful information to capture all finger movements and using low-quality signals to still ensure accurate tracking. In this paper, we introduce new, efficient solutions to these problems. We develop a prototype, and the results show that WETrak outperforms the state-of-the-art method and performs consistently well under various settings.

Demo Video

Feasibility of Finger Tracking with Wrist-worn EMGs

  • Observations. Although the amount of muscles in the wrist is smaller, we have carefully studied the function of each muscle on the user's arm and found that the movements of all our fingers are completely covered by five muscles. Fortunately, these muscles can be sensed from the surface of the wrist. They are the extensor digiti minimi (M1), the extensor digitorum (M2), the pollicis muscle group (M3), the flexor pollicis longus (M4), and the flexor digitorum superficialis (M5), as shown in Fig. 4(a-b). On the other hand, we also observed that each finger is mainly associated with two of M1–M5, as follows:
    • Thumb: M3 and M4. Pinky: M1 and M5.
    • Index, middle, and ring fingers: M2 and M5.
    To experimentally understand these observations, we use five EMG sensors, denoted as channels C1 to C5, placed on the muscles of M1 to M5, respectively, and then we perform three feasibility study trials. The following figure shows the results.
  • Results. When the user moves only the thumb, we can see that channels C3 and C4 exhibit dominant responses. Since the stretching and contracting of the muscles are not completely isolated, the movement of each finger also activates other muscles, thus generating certain signals. Likewise, when only the pinky moves, C1 and C5 show the strongest signals, while the other three channels also show some response. Finally, when the index, middle or ring finger (or a combination of them) is moved, the main responses come from channels C2 and C5. In summary, this feasibility experiment provides three important insights to guide our design in the next subsection.
    • Sensor placement: Unlike uniform placement on the armband, EMG sensors should be aligned with the location of the muscles M1–M5 at the wrist. Fig. 1(c) shows our deployment location, where we set C2 as the anchor point. C1 and C3-C5 should be closer to C2, offsets by 10°, 8°, 28° and 36° from a uniformly placed position, respectively.
    • Finger groups: Based on the relationship of each finger to its most relevant muscles, the five fingers can be divided into three groups, with the thumb and pinky each as one group, and the remaining three as a third group. Since the characteristics of EMG signals mapped to corresponding finger movements vary within different groups, we will later leverage this property to guide our tracking design.
    • Sensing signals: Since the stretching or contracting of muscles is not completely isolated, when a finger is moved, the most relevant muscles will have strong responses, and other muscles usually also generate certain signals.We will use this opportunity to further improve performance.

Design

Publication

Finger Tracking Using Wrist-Worn EMG Sensors

Jiani Cao, Yang Liu, Lixiang Han, Zhenjiang Li

IEEE Transactions on Mobile Computing (TMC), 2024

[Paper] [Code] [Slides]

Cite WETrak

@article{cao2024WETrak,
title={Finger Tracking Using Wrist-Worn EMG Sensors},
author={Cao, Jiani and Liu, Yang and Han, Lixiang and Li, Zhenjiang},
journal={IEEE Transactions on Mobile Computing},
year={2024}
}