MICROSOFT ROBOTICS DEVELOPER STUDIO + OPENCV

In a previous post I explained how to add a simulated stereo camera to your simulated robot. OK! Having a stereo camera is fun... but, process the images and getting some interesting results is even funnier!!

Now, you could implement several Computer Vision techniques by yourself or take advantage of existing libraries.

For that matter, one of the most used libraries for computer vision is OpenCV. There is only a small problem... OpenCV is intended to be used under C/C++ and Microsoft Robotics Developer Studio is programmed in C#.

The solution to that problem is called EmguCV. Emgu CV is a cross platform .Net wrapper to the Intel OpenCV image processing library. Allowing OpenCV functions to be called from .NET compatible languages such as C#, VB, VC++, IronPython etc. The wrapper can be compiled in Mono and run on Linux / Mac OS X.

In order to use it in your MRDS projects you should:

1. Install EmguCV

    http://www.emgu.com/wiki/index.php/Download_And_Installation

2. Add a reference to Emgu.CV and Emgu.Util to your project in Visual Studio.

3. Add the needed "using" statements at the beginning of your source code

using Emgu.CV;
using Emgu.Util;

And that is it. Now you can use all the functionality of OpenCV in your robots (real or simulated).

MRDS: MOBILE BASE + ROBOT ARM

Today I would like to explain how to attach a robot arm to a mobile base using the simulation environment of Microsoft Robotics Developer Studio. It is a very simple task but it can get a bit tricky.

I have been following the examples and explanations on the book Professional Microsoft Robotics Developer Studio (you can find the source code of the examples at http://www.promrds.com ). In this post I will explain how to attach a Simulated LynxL6Arm to a simulated Corobot. 

First of all, you should open in Visual Studio the project where the Corobot Entity is defined and add a reference to SimulatedLynxL6Arm.Y2007.M07. 

Then edit the file Corobot.cs and locate the definition of the constructor public CorobotEntity(string name, Vector3 initialPos). At the end of the constructor insert an instance of the robotic arm like this:

//Insert the LynxL6Arm
LynxL6Arm.SimulatedLynxL6Arm l6arm = new LynxL6Arm.SimulatedLynxL6Arm(name + "_arm", new Vector3(0,0,0));
l6arm.State.Pose = new Pose(
        new Vector3(0, 0.23f, 0),
        Quaternion.FromAxisAngle(0, 1, 0, (float)(-Math.PI / 2)));


InsertEntityGlobal(l6arm);

Now, compile the project and run it. You should see something like this:

That was easy, wasn't it? But now, if you try to drive the Corobot around you will find that IT WONT MOVE! You will see the wheels spinning, but the robot doesn't move an inch from its position. WTF!

Now is when one becomes crazy trying to find out what is going on. You start reading MRDS's forums and find answers as crazy as "copy the source code of the definition of the robot arm inside the definition of the mobile base"... WHAT?? Come on! There must be a more elegant solution!!

And, indeed, there is. It took me one whole day to find out what was happening but, if you take a deep breath and focus, you eventually find the solution.

If you look to the picture above (the one with the Yellow Lynx L6 Arm on top of the Corobot) you will notice that it has a cylindric base. Well, that base is, by default, defined to be Kinematic, that means that it will not follow the physic laws and will stay like sticked to the ground. That is why the robot will not move!

Now, the solution to this problem is to remove the kinematic flag from the arm instance that we just created. So the correct source code looks like this:

//Insert the LynxL6Arm
LynxL6Arm.SimulatedLynxL6Arm l6arm = new LynxL6Arm.SimulatedLynxL6Arm(name + "_arm", new Vector3(0,0,0));
l6arm.State.Pose = new Pose(
        new Vector3(0, 0.23f, 0),
        Quaternion.FromAxisAngle(0, 1, 0, (float)(-Math.PI / 2)));

//Remove the Kinematic flag!! Otherwise the mobile base gets stuck!
l6arm.State.Flags = l6arm.State.Flags & ~(EntitySimulationModifiers.Kinematic);
InsertEntityGlobal(l6arm);

KINECT: HELLO WORLD

The first shot taken with Microsoft Kinect. Thanks a lot to Fukui-sensei for providing such a nice tool.

MRDS: SIMULATED STEREO CAMERA

During the last few weeks I have been working with Microsoft Robotics Developer Studio, it incorporates a very powerful simulation environment that enables you to visualize your robot and even program it even before actually building it.

By default a lot of simulated entities, basic for robotics, are provided: Simulated IR Distance Sensor, Simulated Sonar, Simulated Webcam, Simulated GPS Sensor, Simulated Laser Range Finder, etc...

But I was missing another basic entity often used by robots: A Stereo Camera. After reading some posts at MRDS forum I could not find a suitable solution (easy and fast to implement), so I just built my own solution.

If you want to simulate a stereo camera on Microsoft Robotics Developer Studio, you can follow this 3 steps:

Step 1- Add this class to your project:

public class StereoCameraEntity
    {
        public CameraEntity leftCam;
        public CameraEntity rightCam;
 
        public StereoCameraEntity()
        {
        }
 
        public StereoCameraEntity(
            String parentEntityName,// The name of the parent entity (used to generate a unique name) 
            int viewSizeX, //Image width  
            int viewSizeY, //Image height 
            float viewAngle,//View angle of the camera in degrees  
            Vector3 position,//Position of the center of the stereo camera 
            float baseLine,//Distance between the center of the cameras in centimeters 
            bool isRealTime)//Renders every frame 
        {
            //Initialize left camera 
            leftCam = new CameraEntity(
                viewSizeX,
                viewSizeY,
                (float)(viewAngle * Math.PI / 180.0));
            leftCam.State.Name = parentEntityName + "_LeftCam";
            leftCam.IsRealTimeCamera = isRealTime;
            leftCam.State.Pose.Position = new Vector3(
                position.X - (baseLine / 100.0f) / 2.0f ,
                position.Y,
                position.Z);
 
            //Initialize right camera 
            rightCam = new CameraEntity(
                viewSizeX,
                viewSizeY,
                (float)(viewAngle * Math.PI / 180.0));
            rightCam.State.Name = parentEntityName + "_RightCam";
            rightCam.IsRealTimeCamera = isRealTime;
            rightCam.State.Pose.Position = new Vector3(
                position.X + (baseLine / 100.0f) / 2.0f ,
                position.Y,
                position.Z);
        }
    }

This class is just a wrapper for CameraEntity. When calling the constructor for the new class that you just created you need to provide:

• parentEntityName: A string containing the name of the parent entity (It will be used to create a unique name for the cameras)
• viewSizeX: An integer for the horizontal resolution of the cameras (In pixels).
• viewSizeY: An integer for the vertical resolution of the cameras (In pixels).
• viewAngle: A float for the view angle of the cameras (in degrees).
• position: A Vector3 containing the position of the stereo camera (in meters).
• baseLine: The separation between the cameras (in centimeters).
• isRealTime: True to render every frame.

Step 2- Inside the constructor of your entity create a new StereoCameraEntity and insert it as a child:

StereoCameraEntity stereoCam = new StereoCameraEntity(
        name,
        320,
        240,
        30.0f,
        new Vector3(
                xLocation,
                yLocation,
                zLocation),
        10.0f,
        true);
InsertEntityGlobal(stereoCam.leftCam);
InsertEntityGlobal(stereoCam.rightCam);

This creates a new stereo camera with a resolution of 320x240 pixels, a field of view of 30 degrees, a base line of 10 centimeters, located at the point (xLocation, yLocation, zLocation) and rendering every frame.

Step 3- Modify the manifest of your project to include this:

<servicerecordtype>
        <dssp:contract>http://schemas.microsoft.com/2006/09/simulatedwebcam.html</dssp:contract>
        <dssp:service>http://localhost/MyRobot/LeftCam</dssp:service>
        <dssp:partnerlist>
          <dssp:partner>
            <dssp:service>http://localhost/MyRobot_LeftCam</dssp:service>
            <dssp:name>simcommon:Entity</dssp:name>
          </dssp:partner>
        </dssp:partnerlist>
      </servicerecordtype>

     <servicerecordtype>
        <dssp:contract>http://schemas.microsoft.com/2006/09/simulatedwebcam.html</dssp:contract>
        <dssp:service>http://localhost/MyRobot/RightCam</dssp:service>
        <dssp:partnerlist>
          <dssp:partner>
            <dssp:service>http://localhost/MyRobot_RightCam</dssp:service>
            <dssp:name>simcommon:Entity</dssp:name>
          </dssp:partner>
        </dssp:partnerlist>
      </servicerecordtype>

And that's it. This is the result of the simulation of a modified Corobot with stereo camera: