Export (0) Print
Expand All
1 out of 7 rated this helpful - Rate this topic

Simple Vision Sample

Microsoft Robotics

Glossary Item Box

Microsoft Robotics Developer StudioSend feedback on this topic

Simple Vision Sample

The SimpleVision sample service shows you how to write a service that implements image processing functions using a webcam. This service performs a color object, a simplified face and hand gestures detections. Other services can get the detection results by subscribing to a SimpleVision service.

This sample is provided in the C# language. You can find the project files for this sample at the following location under the Microsoft Robotics Developer Studio installation folder:

Sample location
Samples\Technologies\Vision\SimpleVision

This sample shows you how to:

Prerequisites

Hardware

The SimpleVision sample service supports a real webcam and a simulated webcam. You can also test the service using the simulation environment by simply changing manifest.xml file.

Software

This sample is designed for use with Microsoft Visual C#. You can use:

  • Microsoft Visual C# Express Edition
  • Microsoft Visual Studio Standard, Professional, or Team Edition.

You will also need Microsoft Internet Explorer or another conventional web browser.

Step 1: Set the Initial State

The SimpleVision service supports the Initial State partner. Initial state is used to configure:

The default configuration file for this service is called "simplevision.config.xml", and it specifies the webcam polling interval, the normalized RGB color components of a specfied target oject, a color similarity threshold value and area threshold values that will be used in image processing.

The configuration file would be as follows:

Console
<?xml version="1.0" encoding="utf-8"?>
<SimpleVisionState xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:d="http://schemas.microsoft.com/xw/2004/10/dssp.html" xmlns="http://schemas.microsoft.com/robotics/2007/05/simplevision.html">
  <WebCamPollingIntervalInMs>100</WebCamPollingIntervalInMs>
  <TrackingObjectColor>
    <Red>0.23485383420420847</Red>
    <Green>0.2969919502895072</Green>
    <Blue>0.4681542155062845</Blue>
    <SimilarityMeasure>0.999</SimilarityMeasure>
  </TrackingObjectColor>
  <ColorAreaThreshold>200</ColorAreaThreshold>
  <SkinAreaThreshold>250</SkinAreaThreshold>
  <HeadAreaThreshold>250</HeadAreaThreshold>
</SimpleVisionState>

Step 2: Start and Run the Sample

Start the DSS Command Prompt from the Start > All Programs menu.

To run SimpleVision with a real webcam:

Start a DssHost node and create an instance of the service by typing the following command:

Console
dsshost /port:50000 /t:50001 /manifest:"samples\config\SimpleVision.manifest.xml"

This starts the service and you can see a response like the following:

Console
*   Service uri:  [05/31/2007 14:06:55][http://p3dx:50000/directory]
*   Service uri:  [05/31/2007 14:06:57][http://p3dx:50000/constructor/15974110-4982-4f44-8a66-82bd990ef270]
*   Starting manifest load: file:///c:/msrs/SimpleVision.manifest.xml [05/31/2007 14:06:59]
      [http://p3dx:50000/manifestloaderclient]
*   Manifest load complete [05/31/2007 14:07:03][http://p3dx:50000/manifestloaderclient]
*   Service uri:  [05/31/2007 14:07:05][http://p3dx:50000/simplevision]

The window form of SimpleVision will appear like this figure:

Figure 1

Figure 1 - Figure of window form

To run SimpleVision with a simulated webcam:

Start a DssHost node and create an instance of the service by typing the following command:

Console
dsshost /port:50000 /t:50001 /manifest:"samples\config\SimpleVisionSim.manifest.xml"

The Microsoft Visual Simulation Environment and the window form of SimpleVision will appear like these figures: You can also use SimpleDashboard service to control the pioneer robot in simulation.

Figure 2

Figure 2 - A Simulated Pioneer in Microsoft Visual Simulation Environment

Figure 3

Figure 3 - Figure of window form

Step 3: Understand the Service

While the service is running, it can detect a color object, a face and hand gestures. The service provides four types of notifications. One for a color registration, the others, for sending detection results.

In the service, a color object is considered as a user's colored shirt. This assumption is also used to detect a face and hand gestures. To detect a specified color object, the service uses a normalized color and a similarity measure between current pixel vectors and a registered color vectors and segmentations.

To detect a face, the service uses a skin image filtered by a predefined skin color model. After attaining a skin image, the service performs segmentation over the skin image. To find a face region among the segmented face candidate regions, the service uses an ellipse validation and two simple geometric constrains. The first constrain is a face on a color object and the second constrain is a face that should be in the upper image plane.

To detect hand gestures, the service uses geometric information over a skin image and foreground image. It also uses results of a detected color object and a face. The foreground image is calculated by subtracting the current image from the background image. The background image is grabbed when a motion is not detected between the current camera frame and the previous camera frame for a specified time in a different image.

The update operation and notifications provided are:

  • SetObjectTrackingColor.
  • NotifyObjectDetection.
  • NotifyFaceDetection.
  • NotifyHandGestureDetection.

 

Step 4: Additional Information

Window form interface and method to register color

The service provides the window form to register a specified color and to show the image processing outputs. You can register a color using mouse. First, make a specified circle by pressing the left mouse button in a given position and dragging to enlarge the size in the 'Camera' image on the form. Finally, press 'TrainColor' button to register an average color in a circle region. The registered color is saved automatically in the configuration file.

  • "simplevision.config.xml"

 

The service also provides following method to register object color from other services:

  • SetObjectTrackingColor

 

You can send the message by posting or using the helper method after partnering with the SimpleVision service like following:

Console
vision.ColorVector MyColor = new vision.ColorVector();

MyColor.Red = 0.51;
MyColor.Green = 0.26;
MyColor.Blue = 0.23;

MyColor.SimilarityMeasure = 0.995;

//_visionPort.Post(new vision.SetObjectTrackingColor(MyColor));
_visionPort.SetObjectTrackingColor(MyColor);

Subscriptions

The service allows subscriptions from other services and sends three kinds of detection result notifications to other services.

Output

In addition to the notifications, the service makes calls to LogInfo() and LogError(). By default, these methods send output to the Debug output. These may be viewed using the control panel and the appropriate development tools.

 

Summary

 

 

© 2012 Microsoft Corporation. All Rights Reserved.

Did you find this helpful?
(1500 characters remaining)
Thank you for your feedback
Show:
© 2014 Microsoft. All rights reserved.