Follow my Hands

  • Category: Web Design
  • Tools: HTML, CSS, Javascript, Bootstrap 5
  • Project date: June - July, 2021

Can I simulate a robot following my hand movements with Hand Tracking?

Some of my favorite idea's pop into my head in the most random ways. One day, I saw an ad for a robot hand that could be controlled by a specialized glove. My first thought was: "Why bother with an expensive gloe when you can buy an Oculus Quest for $300". And perfectly at that same time, I had conviniently bought Unity's Lunar Year Bundle, which included an asset called Bio IK. This asset allows you to simulate Inverse Kinematic movements by giving the plugin an transform (point in space) to follow.

My goal was to simulate the ability to have a robotic arm follow your hand movements. To do this, I set up Bio IK on a rigged character mesh, and after a few days of tweaking it worked much better than I expected! In the video and pictures above, my hands are shown as see-though and the rigged character moves its joints to match/follow the positioning of the virtual bone components in my hands.

This was a fun one, and in the future I'd love to expand on it. I see a lot of potential with this idea!