Hand Gesture Arm Fatigue, Part III
Anatomy and Best Practices
Welcome to the finale of a three-part series on spatial computing ergonomics. Here are the links to the other parts:
Part I: Is Arm Fatigue a Non-Starter for Spatial Computing?
Part II: Why Drumming Tells Us All We Need to Know
Part III: Anatomy and Best Practices (you are here)
In previous parts, we learned the ergonomics of spatial computing are not in great shape yet, but there’s hope! Drumming technique has a lot of lessons to offer, including awareness of a safe ergonomic range for elbow motion.
But we still need to cover WHAT that range is and WHY that safe elbow range exists. To answer those questions, we need to enlist the help of our distant cousin in the primate family: the gibbon.
Look at the coracobrachialis muscle in our pal, the gibbon.
That is no spare bicep. That’s a gibbon-style coracobrachialis. Just like ours? Sort of. It attaches from the clavicle to the upper arm like ours, but in a gibbon, it’s HUGE! And it should be. Gibbons and orangutans swing through trees all day long. They need those giant meat pistons.
Compare that to the human coracobrachialis:
Aw, how cute! It’s itty bitty. We’re a bunch of wimps compared to our primate cousins. We stopped swinging through trees a long time ago. We don’t need that muscle for much of anything anymore. There’s really only one important task for which we continued to utilize it in our most recent evolution.
Try a little experiment:
Pretend you just picked up a morsel of food in your fingertips using your dominant hand. Bring that morsel to your lips. Now pause.
Observe the position of your elbow. That position is important. It’s slightly forward, right? But not by more than a few inches.
You COULD flare the elbow out to the side and still reach your mouth, but that’s a big extra deltoid flex that brings no extra benefit. Good elbows are “in” and “down.”
Let the elbow fall as loosely as possible, but keep your fingers at your lips. If your elbow is just slightly forward from its rest position, then you’re mostly using your coracobrachialis to lift your upper arm. You’re using other muscles too, of course, but the coracobrachialis is one that’s heavily occupied in lifting your elbow through that forward-plus-upward rotation.
We evolved to do that motion all day long, every day, without fatigue, by only that small amount of a few inches. We need that much muscle to feed ourselves, scratch an eyebrow itch, or cover our faces. You use that muscle hundreds or thousands of times per day.
But that’s ALL that muscle is good for anymore. It’s so small, it can’t hang with heavier loads. If you push your elbow farther out than that feeding position, you will start your coracobrachialis’ fatigue clock.
Once you’re past that position, you will fatigue that muscle in about ten minutes. It is too wimpy to hold up the weight of your arm after you cross that threshold.
This is why you can’t zombie-walk, arms straight out, for 20 minutes without your coracobrachialis screaming at you. Zombies must have some seriously swol shoulder caps, just like gibbons and orangutans.
Knowing the position of this elbow threshold is key for drumming, and it’s key for spatial computing. I call this threshold the 5 o’clock position.
If you point your arm straight up and call that 12, then tick downward and forward to 1, 2, and 3, you should be pointing your arm straight forward at 3 o’clock. Your elbow is fully extended into zombie-walk position.
3 o’clock is terrible for ergonomics. It’s a worst-case scenario for that little baby bicep muscle on the front of your shoulder. Keep ticking downward to 4 o’clock. The elbow is still too high. That itty bitty muscle will burn in ten minutes, even at 4 o’clock.
Tick down to 5 o’clock. That’s where your elbow was located when you brought food to your face. 5 o’clock is the threshold. Good hand tracking range for your elbow is 5, 6, or 7 o’clock, where 6 is rest (elbow pointed downward), 5 is one tick forward, and 7 is one tick backward.
Users of hand tracking apps should not have to learn all this elbow clock mumbo jumbo. The anatomy lesson is just inside baseball for us nerds. Users should instead be instructed to generally keep their hands within a certain range near their bodies. If they do so 80% of the time, the elbows will mostly be in a good range, and the shoulders will never fatigue.
I call this hand range HOME ZONE, kind of like home row on a QWERTY keyboard. This is what it would look like if it were visible in the user’s field. Home Zone is the quarter-cylinder depicted in light green.
It looks like the same range in which drumsticks fly, right?
Home Zone begins at elbow height. Lower than that would be too far out of the user’s periphery as they gaze straight ahead. If you place your hands in Home Zone starting position as seen in the graphic, you can barely see your fingertips in your periphery as you look straight. Ergonomics specialists agree the ideal gaze direction is parallel to the ground to avoid neck pain. Neck issues are similar in spatial computing as on PC.
Notice the radial center point of Home Zone. That’s the 5 o’clock elbow position, by no accident.
Home Zone extends as far forward as the user can reach without pushing their elbows past 5 o’clock.
This concept exists solely to draw awareness to your own forearm’s range.
The imaginary top edge of Home Zone can be drawn by starting with elbows at 5 o’clock, and then extending the fingertips forward and rotating the forearm upward until the fingertips touch the eyebrows. Remember not to lift the elbows as you go. They should stay at the 5 o’clock threshold. The fingertips make that half-arch shape along the way. Anyone who reaches beyond that arch is lifting their elbow past 5 o’clock, outside of Home Zone.
It’s the user’s ergo safe space. They can reach outside it, but they should come back ASAP. The moment they leave Home Zone forward and upward, their coracobrachialis fatigue clock starts ticking. The moment they leave it laterally, the deltoid fatigue clock starts ticking.
Some people have a habit of flaring their elbows out to both sides, using our massive deltoid muscles (our main shoulder cap muscles) to help out that wimpy coracobrachialis. This is like slouching or resting your wrists on the base of your QWERTY keyboard. The lateral motion doesn’t assist in alleviating the fatigue. It only burns more energy needlessly, eventually leading to fatigue across the entire shoulder.
For a comfortable hand tracking experience, you want to tuck your elbows in, not by flexing the shoulders, but by completely relaxing them. Sometimes, when people hear the word “tuck,” they overcompensate, squeezing the elbows against the ribs, and that’s just as bad as flaring them out.
Kinda crazy how drum teachers know all of this stuff already, right? These are the exact same descriptions you get in drum lessons.
If you stay in Home Zone, you can do rapid, repetitive actions all day long with no fatigue, just like drummers do. If you need to access things laterally, twist the upper body (or just turn if standing) to square the shoulders up on your target.
You can swing Home Zone toward whatever you need to access. Consider it attached to your upper torso at all times. If you watch a really incredible drummer’s shoulders, you’ll see they twist their chest to face the part of the drum set that’s receiving a majority of their attention. They go wide occasionally with right hand far right and left hand far left, but they know that staying wide is a workout. It’s why they all sit on twisty thrones. They want the hands working together as a compact unit in most cases.
If you exit Home Zone forward and upward, you have about ten minutes. If you exit Home Zone laterally, you have about twenty minutes. The clock varies for everyone. Drummers and athletes can probably go a lot longer, but in spatial computing, building hardware and software that caters only to people with elite shoulder muscle tone is a quick way to sell single-digit quantities of your product.
If a manufacturer designs a new piece of hardware that forces the user to extend their elbow beyond Home Zone in order to use hand tracking, the user is guaranteed to fatigue in every app, no matter what the developer does.
Unfortunately, this has been the fate of most spatial computing hardware (Leap Motion, Hololens 1, and Magic Leap).
I have not tested the hand tracking cone on the newer Ultraleap sensor, Snap Spectacles, or Hololens 2. If you own a newer device, try this test:
Begin in Home Zone starting position. No hands detected? Tick the elbows forward to 5 o’clock. Still no hands? The hardware is an ergonomic non-starter.
Oculus Quest is the only device that has a field of view for hand tracking that JUST BARELY lets the user put hands in Home Zone starting position (elbows at 6, forearms straight forward parallel to the floor, palms facing each other, fingers loosely curled). This is starting position because it minimizes all flex in the upper arm (except for a small squeeze of the bicep to get your hands into a bare-minimum floating spot at the edge of your field of view). It also zeroes out all flex from the elbows to the fingertips. No twisted up wrists, nor pinching or overextended fingers. Everything soft.
At full elbow rest, even Quest frequently loses hand tracking. At 5 o’clock position, the tracking may flutter occasionally, but it’s pretty reliable. So Quest is winning this race due to the super-wide FOV of its inside-out cameras, but even Quest is just barely cutting it with hand ergonomics.
And the problem isn’t just with hardware. Any Oculus Quest software developer can place a UI just out of reach of Home Zone, and then the muscle clock starts ticking again. Even if the hardware makes good ergo possible, the onus flows down to the developer. UI placement should adhere to these same ranges for good comfort.
For example, I love the zombie game Walking Dead: Saints and Sinners, available for Oculus Quest, but every time I played it for a couple hours, my right shoulder showed a lot of fatigue. Why?
Because it made me pull hundreds of pieces of garbage out of a backpack to toss them all into a chest-high trash bin, one at a time. Chest-high means the elbow rises WAY above 5 o’clock position, over and over again.
Ergonomically, yikes. It’s still a great game. You can get away with an ergo faux pas in gaming because VR is supposed to be experiential. Your shoulder WOULD get tired in a zombie apocalypse. Deal with it, you whiny user.
But the developer could easily have placed the trash bin one foot lower. Issues like this can cause long-term injury in productivity apps. If VR and AR are ever to replace the kinds of data-heavy apps relegated to desktop PCs, we must set a higher ergo bar.
It doesn’t mean users can never lift their arms. It just means that they should be encouraged to bring their hands back to Home Zone a vast majority of the time. For elbows and shoulders, 80% relax/20% flex is a good ballpark ratio.
If you observe the gestures in Minority Report closely, you’ll see that none of them NEED full elbow extension.
Tom Cruise could have dropped his elbows and still done all those same hand gestures. It would not have looked as exciting for the camera though.
That’s the difference. As with all things, real life will be less dramatic than cinema, but it will be a whole heck of a lot more comfortable.
Fatigue problem solved! Good luck getting this memo out to the rest of the spatial computing industry before 2030, but if you work in spatial computing, I hope this helps you with your next project.
And remember, the future…
IS NOW!
← If you found this article interesting, please pass it on by clicking “Share on Twitter.”
DISCLAIMER: Every body is different, and I have no background in medicine. If you experience fatigue or consistent pain from use of any spatial computing equipment, consult your physician. Their recommendations trump any advice from my random musings.
Some other spatial computing articles I’ve written in the past:
WTF is a DOF? — noob or pro, it’s more complex than you may think
Unlocking the 3rd D — A spatial design manifesto
Diegesis in XR — in case you haven’t had enough jargon yet
Written By: AJ Campbell, guy who builds stuff https://twitter.com/texasgreentea
