How to Track Deficient or Spastic Hands in Virtual Reality

Playing VR without controllers for disabled people with spastic hands

For people with disabilities who have deficient or spastic hands, we prepared a solution that will allow them to use VR without using controllers.

This is especially important for workout in VR and meetings with physiotherapists when every minute is precious. Installing controllers for spastic hands can take a long time. Eliminating the need to use the controllers will allow players to effectively use the workout time.

This VR adaptation can be used for games that do not require the use of buttons on controllers or in conjunction with the Gameplay in Virtual Reality with Assistance.

How to use hand tracking with Kinect

Prerequisites

For optimal setup and usage prepare following elements:

Kinect 2.0 and Kinect Adapter to PC. Connect Kinect using Kincet Adapter to PC with USB Cable. It will also work with Kinect 360 however Kinect 2.0 is recommended.

Xbox One Controller is optional, but very practical to adjust player’s position and emulate button pressing. Check details on Gameplay with Assistant page.

Install Kinect 2.0 Runtime. Click here to download.

Using hand tracking with Kinect

  1. Place Kinect in front of your gameplay area. Ensure it is facing the default direction you face in VR.
  2. Start VR and ensure it is up and running. Power on also VR Controllers and ensure they are operating.
  3. Navigate to Hand tracking tab in WalkinVR and enable checkbox Use Kinect for Hand Tracking. This will start the Kinect. You will also note that your VR Controllers are not tracked any more but your hands even though you don’t hold anything.
  4. If you haven’t performed calibration, they will be in totally different location where you would expect. Calibration is described below.
  5. Now you can move your hands and they should be visible to you as controllers. In order to press buttons, use Xbox Controller or the original VR controllers.

Parameters to Adjust Hand Tracking

Depending on game, you might consider adjusting the parameters of hand tracking.

Responsiveness

First thing is to determine Orientation of the controller. If you select hand tracking, then Kienct will determine orientation based on how hand is rotated. This is very good method for games where player can operate with objects. Like for example in Job Simulator, where you can pet the dog.

Hand absolute position

Another option is Hand absolute position. In this case the orientation is based solely on the position of hand. It is practical for players who can’t control the angle of hand. As result of spastics for example. It is very practical setting for games like Beat Saber, Box VR or Audio Shield.

Calibration

This is the very first step when setting up Kinect. This step is required to align tracking from your VR Setup and from Kinect.

  1. First navigate to Settings tab in WalkinVR. Press Calibrate VR Headset and Kinect. When calibration window opens, press Calibrate.
  2. Put headset on, take controller in each hand and stand in front of the Kinect. You can enable Desktop preview in SteamVR dashboard to see WalkinVR Window. There are two similar steps. Put your hands a little to the side and wait until calibration step is over.
  3. Take step back and wait in same pose for until calibration is complete. After each step you will hear applause sound that will make whole process easier when wearing the headset.