Leap Motion: 移动光标
4.91/5 (20投票s)
如何使用 Leap 控制器通过手指移动光标。
引言
Leap 是一款3D运动传感设备，可以检测和跟踪手、手指和类似手指的工具。在本文中，我将介绍如何使用Leap 和Leap SDK创建一个.NET控制台应用程序，让您能够用手指移动光标。
要求
要运行示例项目，您需要以下条件：
- Leap（开发者单元），
- Leap SDK，
- VS2012
如果您没有Leap控制器，可以申请加入开发者计划，并希望免费获得一个开发者单元。加入开发者计划还可以让您访问Leap SDK。
注意：本文介绍的示例项目是使用开发者单元v.06.5和Leap SDK版本0.7.3开发的。
运行项目
要运行示例项目，您必须将Leap.dll和_LeapCSharp.dll添加到项目中。（这两个文件是Leap SDK的一部分）。确保这两个文件的复制到输出目录属性设置为始终复制。
确保您也添加了对LeapCSharp.NET4.0.dll的引用，该文件也是Leap SDK的一部分，其中包含使您能够访问Leap控制器数据的类。
Leap 坐标系
Leap使用右手笛卡尔坐标系，原点位于控制器中心，X轴和Z轴位于水平面。Y轴是垂直的。
移动光标
为了移动光标，我们需要调用win32 API中的一个函数。这可以通过MouseCursor类中的一个方法来完成。
Public Class MouseCursor
Private Declare Function SetCursorPos Lib "user32" (x As Integer, y As Integer) As Boolean
Public Shared Sub MoveCursor(x As Integer, y As Integer)
SetCursorPos(x, y)
End Sub
End Class
using System;
using System.Runtime.InteropServices;
using System.Windows;
using System.Windows.Interop;
namespace ConsoleLeapMouse
{
class MouseCursor
{
[DllImport("user32.dll")]
private static extern bool SetCursorPos(int x, int y);
public static void MoveCursor(int x, int y)
{
SetCursorPos(x, y);
}
}
}
MoveCursor()接受两个参数，用于定义光标应移动到的屏幕位置。要获取这些参数的值，我们需要处理来自Leap控制器的数据。
要从Leap获取数据，我们需要创建一个Controller对象。Controller对象将控制台应用程序与Leap软件连接，并通过Frame对象提供手和手指的跟踪数据。
Imports Leap
Module Module1
Sub Main()
Dim cntrl As New Controller
Dim listener As New LeapListener
cntrl.AddListener(listener)
Console.WriteLine("Press Enter to quit...")
Console.ReadLine()
cntrl.RemoveListener(listener)
cntrl.Dispose()
End Sub
End Module
using System;
using Leap;
namespace ConsoleLeapMouse
{
class Program
{
static void Main(string[] args)
{
Controller cntrl = new Controller();
LeapListener listener = new LeapListener();
cntrl.AddListener(listener);
Console.WriteLine("Press Enter to quit...");
Console.ReadLine();
cntrl.RemoveListener(listener);
cntrl.Dispose();
}
}
}
在上面的代码片段中，将类型为LeapListener的对象添加到Controller对象中。LeapListener是Listener类的子类，该类定义了一组回调方法，可以通过重写这些方法来响应Leap分派的事件。
Imports Leap
Public Class LeapListener
Inherits Listener
Public Overrides Sub OnInit(cntrlr As Controller)
Console.WriteLine("Initialized")
End Sub
Public Overrides Sub OnConnect(cntrlr As Controller)
Console.WriteLine("Connected")
End Sub
Public Overrides Sub OnDisconnect(cntrlr As Controller)
Console.WriteLine("Disconnected")
End Sub
Public Overrides Sub OnExit(cntrlr As Controller)
Console.WriteLine("Exited")
End Sub
Private currentTime As Long
Private previousTime As Long
Private timeChange As Long
Public Overrides Sub OnFrame(cntrlr As Controller)
' Get the current frame.
Dim currentFrame As Frame = cntrlr.Frame
currentTime = currentFrame.Timestamp
timeChange = currentTime - previousTime
If (timeChange > 10000) Then
If (Not currentFrame.Hands.Empty) Then
' Get the first finger in the list of fingers.
Dim finger = currentFrame.Fingers(0)
' Gets the closest screen intercepting a ray projecting from the finger.
Dim screen = cntrlr.CalibratedScreens.ClosestScreenHit(finger)
If (screen IsNot Nothing And screen.IsValid) Then
' Get the velocity of the finger tip.
Dim tipVelocity = finger.TipVelocity.Magnitude
' Use tipVelocity to reduce jitters when attempting
' to hold the cursor steady.
If (tipVelocity > 25) Then
Dim xScreenIntersect = screen.Intersect(finger, True).x
Dim yScreenIntersect = screen.Intersect(finger, True).y
If (xScreenIntersect.ToString <> "NaN") Then
Dim x = CInt(xScreenIntersect * screen.WidthPixels)
Dim y = screen.HeightPixels - CInt(yScreenIntersect * screen.HeightPixels)
Console.WriteLine("Screen intersect X: " & xScreenIntersect.ToString)
Console.WriteLine("Screen intersect Y: " & yScreenIntersect.ToString)
Console.WriteLine("Width pixels: " & screen.WidthPixels.ToString)
Console.WriteLine("Height pixels: " & screen.HeightPixels.ToString)
Console.WriteLine(vbCrLf)
Console.WriteLine("x: " & x.ToString)
Console.WriteLine("y: " & y.ToString)
Console.WriteLine(vbCrLf)
Console.WriteLine("Tip velocity: " & tipVelocity)
' Move the cursor
MouseCursor.MoveCursor(x, y)
Console.WriteLine(vbCrLf & New String("=", 40) & vbCrLf)
End If
End If
End If
End If
previousTime = currentTime
End If
End Sub
End Class
using System;
using Leap;
namespace ConsoleLeapMouse
{
class LeapListener: Listener
{
public override void OnInit(Controller cntrlr)
{
Console.WriteLine("Initialized");
}
public override void OnConnect(Controller cntrlr)
{
Console.WriteLine("Connected");
}
public override void OnDisconnect(Controller cntrlr)
{
Console.WriteLine("Disconnected");
}
public override void OnExit(Controller cntrlr)
{
Console.WriteLine("Exited");
}
private long currentTime;
private long previousTime;
private long timeChange;
public override void OnFrame(Controller cntrlr)
{
// Get the current frame.
Frame currentFrame = cntrlr.Frame();
currentTime = currentFrame.Timestamp;
timeChange = currentTime - previousTime;
if (timeChange > 10000)
{
if (!currentFrame.Hands.Empty)
{
// Get the first finger in the list of fingers
Finger finger = currentFrame.Fingers[0];
// Get the closest screen intercepting a ray projecting from the finger
Screen screen = cntrlr.CalibratedScreens.ClosestScreenHit(finger);
if (screen != null && screen.IsValid)
{
// Get the velocity of the finger tip
var tipVelocity = (int)finger.TipVelocity.Magnitude;
// Use tipVelocity to reduce jitters when attempting to hold
// the cursor steady
if (tipVelocity > 25)
{
var xScreenIntersect = screen.Intersect(finger, true).x;
var yScreenIntersect = screen.Intersect(finger, true).y;
if (xScreenIntersect.ToString() != "NaN")
{
var x = (int)(xScreenIntersect * screen.WidthPixels);
var y = (int)(screen.HeightPixels - (yScreenIntersect * screen.HeightPixels));
Console.WriteLine("Screen intersect X: " + xScreenIntersect.ToString());
Console.WriteLine("Screen intersect Y: " + yScreenIntersect.ToString());
Console.WriteLine("Width pixels: " + screen.WidthPixels.ToString());
Console.WriteLine("Height pixels: " + screen.HeightPixels.ToString());
Console.WriteLine("\n");
Console.WriteLine("x: " + x.ToString());
Console.WriteLine("y: " + y.ToString());
Console.WriteLine("\n");
Console.WriteLine("Tip velocity: " + tipVelocity.ToString());
// Move the cursor
MouseCursor.MoveCursor(x, y);
Console.WriteLine("\n" + new String('=', 40) + "\n");
}
}
}
}
previousTime = currentTime;
}
}
}
}
上面代码片段中最重要的一个方法是OnFrame()方法。当可用的手和手指跟踪数据的新帧时，Controller对象会调用OnFrame()方法。可以通过调用Controller对象的Frame()方法来访问数据。由于Leap的帧率非常高，我猜测最高大约为115fps，尝试在每次调用OnFrame()方法时移动光标将无法获得预期结果。因此，我指定了处理跟踪数据帧之间 >10ms的时间间隔。
注意：Frame.Timestamp()是自Leap启动以来以微秒为单位的帧捕获时间。
要获取手指指向的屏幕，我们使用Controller.CalibratedScreens.ClosestScreenHit()方法，并传递一个Finger对象，在本例中是Finger对象列表中的第一个Finger对象。指定的屏幕将是拦截从Finger对象投射的射线的第一块屏幕。投射的射线从Finger的TipPosition沿其Direction向量发出。
要获取手指指向屏幕的像素位置，我们将Screen.WidthPixels()和Screen.HeightPixels()返回的值与Screen.Intersect()返回的归一化坐标相乘。归一化坐标表示屏幕与从Finger对象投射的射线之间的交点，定义为屏幕宽度和高度的百分比。这些坐标的形式将是(0.1, 0.1, 0)。
...
Dim xScreenIntersect = screen.Intersect(finger, True).x
Dim yScreenIntersect = screen.Intersect(finger, True).y
If (xScreenIntersect.ToString <> "NaN") Then
Dim x = CInt(xScreenIntersect * screen.WidthPixels)
Dim y = screen.HeightPixels - CInt(yScreenIntersect * screen.HeightPixels)
...
End If
...
var xScreenIntersect = screen.Intersect(finger, true).x;
var yScreenIntersect = screen.Intersect(finger, true).y;
if (xScreenIntersect.ToString() != "NaN")
{
var x = (int)(xScreenIntersect * screen.WidthPixels);
var y = (int)(screen.HeightPixels - (yScreenIntersect * screen.HeightPixels));
...
}
如果您想知道为什么我通过screen.HeightPixels - (yScreenIntersect * screen.HeightPixels)来设置y变量的值，那是因为我的计算机图形坐标系将屏幕的原点放在左下角。
注意：如果Finger对象平行于屏幕指向或背离屏幕，则Screen.Intersect()返回的向量的分量将全部设置为NaN（非数字）。
结论
在本文中，我没有解释如何实现鼠标点击。为此，您需要自定义手势以及一些win32 API调用。尽管如此，我希望本文中的信息对您有所帮助。
历史
- 2013年2月21日：初版