Tutorial 3:

There is now a service implemented into MyRobotLab, (Thanks to Alessandro Didonna) Which performs the hand and the LeapMotion in the matter as the video below. The great thing about this is that it’s compatible on Mac, Windows, Linux and possibly Androids!

Here is the script link to automaticaly run the hand and the Leap in MyRobotLab:

https://github.com/MyRobotLab/pyrobotlab/blob/master/home/hairygael/InMoov4.LeapMotion.py

  • Connect your Leap to the PC.
  • Start the Leap visualiser with the Leap Api.(see below in the tutorial2/Step4/section 1,2,3)
  • Start MyRobotLab (version above 1.0.2000), add/paste the script in the python tab.
  • Make sure to adapt the min and max values for each of your servos.
  • Change the COM port to your Arduino port. (see below in the tutorial2/Step4/section 4)
  • Run the script.
  • Voilà!!

Tutorial 2:

Another way is to run Leap Motion on your Mac without using MyRobotLab.

Please note: This tutorial assumes you are using a Mac OS. In this tutorial when there is code or text referenced, do not include the “ ”. For example, if you see: Search “Leap Motion” only enter Leap Motion without the quotations. Lastly, once terminal is open keep it open as we’ll be working through several steps in order

Step 1: Components:

  1. (1)Leap Motion
  2. (1)InMoov Hand and Forearm
  3. (5) HK15298 Servo
  4. (1) Arduino Uno or Mega
  5. (1) NervoBoard or a servo shield
  6. (1) Meter USB Cable
  7. (1) Power Supply 5-6V with a minimum of 3Amps

Step 2: Build InMoov Hand and Forearm

Step 3: Connect the Wires

For this part you’ll need the following components/tools/supplies:

1.(1)An Arduino Uno (or Arduino Mega)

2.(1)NervoBoard (nervo-board-tutorial) or a servo shield

3.(5) x HK15298 Servos (These servos are nice. I don’t recommend MG995 or servo with plastic gears!)

4.Separate power supply

                  Option 1: variable power supply

                  Option 2: Old power supply with the following specs: DC Between 5V and 7V Greater than 3A to sufficiently power all servos

5.The diagram attached, is to show you the basic connection of the servos to the Arduino.You can see that there are two wires (black and red) extending to the right from the prototyping PCB. Please note that these wires are where you should connect your separate power supply for the servos.

Step 4: Install Software

 

 

 

 

 

 

 

 

 

 

1.Install Leap Motion Desktop Software: Download Desktop App

2.See: System requirements

3.Verify Leap Motion settings:

                   1-Open Leap Motion (Cmd + SPACE > search “Leap Motion”)

                   2-Click on Leap Motion icon at top of page > click on settings

                   3-Verify that your settings match those in the pictures included:

                   4-Calibrate if needed. If you do, ideally you should have the Leap Motion surface very clean and you should also have a clean mirror available. I used the mirror in the bathroom with the Leap Motion connected to my laptop.

                   5-Once done, click “OK” to close.

4.Install Arduino IDE: Click here to download

5.Install Node.js: Click here to download

6.Verify Node.js and npm are installed:

                  1-Open terminal (Cmd + SPACE > “terminal” > press enter)

                  2-Verify Node.js:

                             1-In terminal type “node -v” > press enter

                             2-If you see something like “v8.9.3” then you are good to go

                             3-If you see something like “-bash: npp: command not found” then you need to install Node.js

                   3-Verify npm:

                             1-In terminal type “npm -v” > press enter

                             2-If you see something like “5.5.1” then you are good to go

3-If you see something like “-bash: npp: command not found” then you need to install npm. Note: npm is installed along with Node.js

7.Set up folders:

                  1-In terminal:

                             1-Return to home folder: “cd” > press enter

                             2-Go to Documents folder: “cd Documents” > press enter

                             3-Create new folder then go to new folder: “mkdir roboHand; cd roboHand” > press enter

8-Install needed Node.js modules:

                   1-In terminal:

                              1-“npm install -g gl-matrix” > press enter

                              2-“npm install -g johnny-five” > press enter

3-“npm install -g underscore” > press enter

4-“npm install -g ws” > press enter

                    2-Verify modules are installed (optional):

                              1- “npm list MODULENAME” > press enter

9.Install Leap Motion JavaScript SDK:

                    1-In terminal:

                              1-“npm install leapjs” > press enter

CONGRATULATIONS! IF YOU’VE FOLLOWED THE STEPS SUCCESSFULLY, YOU SHOULD NOW HAVE ALL OF THE SOFTWARE NEEDED FOR THIS PROJECT INSTALLED ON YOUR COMPUTER.

Step 5: Prepare/Run the Code

1.Prepare Hardware:

                   1-Make sure Leap Motion is plugged into your computer

                   2-Get ready to turn power on to power your servos after JavaScript code is running

2.Prepare Arduino:

                   1-We’ll be uploading code included with the Arduino IDE called StandardFirmata

                   2-Open Arduino IDE > Plug in Arduino

                   3-Verify port: Tools > Port > select appropriate port

                   4-Verify correct board: Tools > Board > select appropriate board

                   5-Upload StandardFirmata code: File > Examples > Firmata > StandardFirmata > upload to Arduino

3.Prepare JavaScript Code:

                   1.In terminal:

                              1-“git clone git://github.com/SJCCRAC/leapApp” > press enter

4.Run JavaScript Code (leapApp.js):

                   1-In terminal:

                               1-“cd leapApp/script” > press enter

                               2-“node leapApp.js” > press enter

                   2-Your code should now have started…turn power on to your servos!

5.Play with camera controlled robot hand:

1-If everything went smoothly, you should now be able to place your hand above the Leap Motion and control the robot hand.

6.Terminal/JavaScript Code Tips:

                   1-To stop JavaScript code:

                             1-In terminal type: CONTROL + C… CONTROL + C

                   2-To restart JavaScript code:

1-In terminal, press up arrow on the keyboard to cycle through most recent entries > press enter when you’ve found the one you want to enter

 

 

Tutorial 1:

This is my original version of the tutorial:

Thanks to Seokju Lee, and Doctor Shim. KAIST
Unmanned System Research Group (USRG) & Field Robotics Center (C-FRIEND)
Flight Dynamics and Control Laboratory (FDCL)
Dept. of Robotics Program

This is exactly what I received for to make the Leap control the InMoov Hand, I have spent many hours pulling of my hair because things are not always obvious for everybody.

Windows is not working correctly with Leap and this script, Linux will work but you won’t have the fancy Visualiser that I showed in the video.

So your best option is to use a Apple Computer, Leap requires a lot of video resource, so don’t expect to do this on a old machine. I used a MacBookPro.

To install the files and node.js you will need to use the terminal, so it is not that easy if you’ve never done that before.

The [directory] tree you will create should be like this:

———————————-

|-[Project folder (you’ve made)]
|—-[node_modules]
|——–[johnny-five]
|——–[ws]
|—-[LeapHand (unzip file)]
|——–[src]
|————[lib]
|—————-joint.js
|—————-leapFrame.js
|————app.js
|——–Makefile
|——–README.md

If you need to adjust the degree rotation of your servos, edit app.js and modify the values.

Comments 32

  1. Alejandro Guerrero

    Hi! I’m working in this project but i have some problems in the moment to make run the program, i don’t know what to do, i install the Johny five thing, but the next i stop it,, the js files i have to move it in some folder, or what? please i will thank you if you can send me a guide o some information more specific,
    greetings from Mexico!

  2. Pingback: InMoov at the Paris 3D PrintShow. | InMoov
  3. OF

    I got this :
    node src/app.js
    module.js:338
    throw err;
    ^

    Error: Cannot find module ‘serialport’
    at Function.Module._resolveFilename (module.js:336:15)
    at Function.Module._load (module.js:286:25)
    at Module.require (module.js:365:17)
    at require (module.js:384:17)
    at Board.Serial.detect (/Users/orenforkosh/node_modules/johnny-five/lib/board.js:70:7)
    at new Board (/Users/orenforkosh/node_modules/johnny-five/lib/board.js:328:23)
    at Object. (/Users/orenforkosh/LeapInmoov/LeapHand/src/app.js:11:13)
    at Module._compile (module.js:434:26)
    at Object.Module._extensions..js (module.js:452:10)
    at Module.load (module.js:355:32)
    make: *** [run] Error 1

    any idea why it can’t find the port ? ( leap motion is working well and Arduino working well too)

    Thanks

        • Gael Langevin

          Hi,
          You need to use MyRobotLab version 1.1.107 to get this script working. That version is available on the InMoov site tutorial about MyRobotLab.
          Run the scriptin the python window with the Leap connected and correctly set in your PC and get the right working with it.

          inmoov = Runtime.createAndStart(“inmoov”,”InMoov”)
          inmoov.startRightHand(“COM7”)
          inmoov.rightHand.index.setMinMax(0,160)
          inmoov.rightHand.thumb.setMinMax(55,135)
          inmoov.rightHand.majeure.setMinMax(50,170)
          inmoov.rightHand.ringFinger.setMinMax(48,145)
          inmoov.rightHand.pinky.setMinMax(30,168)
          sleep(1)
          inmoov.rightHand.startLeapTracking()
          # inmoov.rightHand.stopLeapTracking()

  5. Brett

    Hello Gael,

    I have the Leap motion communicating with the hand, but the script that you provide makes the hand move exactly opposite to the position of my hand (I open my hand, InMoov makes a fist). Is a there a quick solution to this problem?

  6. Rahul Kumar

    Hello Gael,
    I have built your model and it is successfully working with leap motion controller. I wonder how can i take values of the z axis of the hand so that i can provide that value to the biceps i built. Your code limits the value to the fingers and palm rotation.

    • Gael Langevin

      Indeed, currently the codes only take the hand as value. Normaly with my first code, the z axis is actuating the wrist.
      To modify that, in order to adapt it for the bicep, you will be on your own. Because it’s been a while back since I digged into these codes, and currently I really don’t have the time appropriate to get back on that.
      I think you should share your thought on the forum, certainly some people will come along to help modifying the code.

  7. Maarten Keus

    Dear Gael,

    i followed tutorial 2! Amazing btw! But I stumbled upon two things, one i found a little mistake, at point 8 : install needed node.js you are talking about 2-“npm install -g johnn-five” > press enter. but there is a Y missing at johnny

    second i cant seem to get the last step working. everything went fine and i verified all my installed nodes, but when i run leapApp.js its give me this error.

    Error: Cannot find module ‘johnny-five’
    at Function.Module._resolveFilename (module.js:538:15)
    at Function.Module._load (module.js:468:25)
    at Module.require (module.js:587:17)
    at require (internal/module.js:11:18)
    at Object. (/Users/user/Documents/roboHand/leapApp/script/leapApp.js:284:12)
    at Module._compile (module.js:643:30)
    at Object.Module._extensions..js (module.js:654:10)
    at Module.load (module.js:556:32)
    at tryModuleLoad (module.js:499:12)
    at Function.Module._load (module.js:491:3)

    i hope you can help me out

    • Gael Langevin

      If you follow the first tutorial, you should be able to get the wrist working.
      I haven’t used it since a few years now, and I must say, I really forgot how I did it.
      I am glad I posted those tutorials because it is also a good reminder for me.

  10. Towfeeq Gasant

    Hi could someone help me getting the leap motion to control my servo motors. Whenever I run the script outlined in tutorial 3 myrobotlab crashes (freezes). I am using Manticore. I can get it to run if I comment out the line inmoov.rightHand.startLeapTracking(). Then the program runs and if I uncomment this afterwards and run it again it doesn’t crash but the servos are not moving when I give the leap motion sensor an input. I can move the servos manually with the slider.

  14. Izzy

    Hi everyone, just wondering if anybody knows where to find the image for tutorial 2 step 3 in connecting the wires. i tried everything to get it up but it doesn’t seem to want load an image
    Thanks and hello from Australia
    Izzy (:

  15. DHINESH D

    hello please help me its a urgent i need to make inmoov hand working with leap motion i done using nodejs method it shows error:

    C:\Windows\System32\leapApp\script>node leapApp.js
    node:internal/modules/cjs/loader:1078
    throw err;
    ^

    Error: Cannot find module ‘underscore’
    Require stack:
    – C:\Windows\System32\leapApp\lib\hand.js
    – C:\Windows\System32\leapApp\lib\frame.js
    – C:\Windows\System32\leapApp\lib\controller.js
    – C:\Windows\System32\leapApp\lib\index.js
    – C:\Windows\System32\leapApp\script\leapApp.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1075:15)
    at Module._load (node:internal/modules/cjs/loader:920:27)
    at Module.require (node:internal/modules/cjs/loader:1141:19)
    at require (node:internal/modules/cjs/helpers:110:18)
    at Object. (C:\Windows\System32\leapApp\lib\hand.js:5:9)
    at Module._compile (node:internal/modules/cjs/loader:1254:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1308:10)
    at Module.load (node:internal/modules/cjs/loader:1117:32)
    at Module._load (node:internal/modules/cjs/loader:958:12)
    at Module.require (node:internal/modules/cjs/loader:1141:19) {
    code: ‘MODULE_NOT_FOUND’,
    requireStack: [
    ‘C:\\Windows\\System32\\leapApp\\lib\\hand.js’,
    ‘C:\\Windows\\System32\\leapApp\\lib\\frame.js’,
    ‘C:\\Windows\\System32\\leapApp\\lib\\controller.js’,
    ‘C:\\Windows\\System32\\leapApp\\lib\\index.js’,
    ‘C:\\Windows\\System32\\leapApp\\script\\leapApp.js’
    ]
    }

    Node.js v18.14.2

    C:\Windows\System32\leapApp\script>

    secondly i tried with the mrl the arduino is connecting and i can manually run the servo using the slider but when uploading the script:

    inmoov = Runtime.createAndStart(“inmoov”,”InMoov”)
    inmoov.startRightHand(“COM7”)
    inmoov.rightHand.index.setMinMax(0,160)
    inmoov.rightHand.thumb.setMinMax(55,135)
    inmoov.rightHand.majeure.setMinMax(50,170)
    inmoov.rightHand.ringFinger.setMinMax(48,145)
    inmoov.rightHand.pinky.setMinMax(30,168)
    sleep(1)
    inmoov.rightHand.startLeapTracking()
    # inmoov.rightHand.stopLeapTracking()

    im getting so many errors releated to org python and null pointer seems the script wont run in the version im using 1.1.106 of mrl the current one
    i seen about the 1.0.107 make it working but i cant find the link anywhere and its broken here saying the zip is invalid
    third method i tried is using unity i found in a youtube video using leap api and some scripts in unity which too shows error like please clear the errors and load the c# scripts again this was in vain too. ( i tried 2021 2020 vers of unity too)

    please help me someone what am i missing? if somone know how to use leap with the lastest nixxie 1.1.106 ver of mrl please guide me or if someone having the 1.0.107 ver please provide me the file… or help with the nodejs method or anyother methods please help… thank you

Leave a Reply