Within an university programming project we tried out several methods to control the mouse cursor with Kinect for Windows. One of the results is a small application, which lets you control the Windows mouse cursor with your hands. Generally the cursor moves like the Kinect cursor on Xbox, but in our opinion there are several disadvantages on the Xbox Kinect mouse control:
- On Xbox you need to hold the cursor for about a second over a tile if you want to click on it. This is an unnecessary delay. So we were looking for a clicking method, which is as fast as a click on a PC mouse.
- With the Xbox Kinect control you only have the opportunity to make a standard click. There is neither a possibility for a right click nor a possibility for a double click and also no possibilty for Drag & Drop. But you need all of these special mouse actions, when you are working with your PC.
How does our application work?
Just move your hand for controlling the cursor. The recognition of your hand is a bit insensitive when you are too close to the sensor. If the cursor is too much jumping, just go one or two steps back. When you start the application it takes a moment for the sensor to recognize your hand. Make sure that Kinect can see your face as well as your chest and both of your hands. Sometimes the inital recognition is better when you are waving.
As Kinect recognizes the distance from your hand to the sensor, clicking is possible by moving your hand just a bit forward and then directly backwards (see picture below). Just make a little motion, the sensor should recognize that and executes the click on the position where the mouse cursor was, when you started the motion.
Double clicking works the same way like normal clicking. The difference to a normal click is that you have to move your hand just a bit more forward and backward.
For a right click the motion is quite similar to the motion for the left click. Use your left hand and move it just a bit backward and then directly forward (instead forward and backward as at the left click).
It’s important that you are working with your left hand when using right click, as there is an incompatibility with the distinction between right click and Drag & Drop.
Drag & Drop
Just move your right hand a bit back. This motion will execute a Mouse Down Event. Then you can move the cursor. When you move your hand back forward, the Mouse Up Event is executed. Drag & Drop ends at this point.