Image marker Tracking

Keywords:
vision, tracking, marker, image marker, augmented reality
Author(s): Timo Engelke
Date: 2008-12-16

Summary: This tutorial shows you how to use image markers instead of normal markers.

Introduction

This tutorial will show how to setup instantVision in order to track image markers. We assume, that you have already understood the usage of code based markers as described in the marker tracker tutorial. The implementation is very similar to a code based approach, but instead of a marker code we will provide an image to the vision system. Image markers might be a good replacement for code based markers, as they appear more natural.

This tutorial will describe the following aspects:

You will need the following:

Preparing the image

First, we have to tell you some general aspects of the image marker tracker. Like in the code based marker tracker, the vision system generally searches for a frame inside the accuired camera image. This usually is a bright rectangular field on a black ground or reversed. Based on this frame a new captured image will be cut out and compared to an existing one. Once found, using the specified dimensions and including the camera parameters (intrinsics) a camera pose (extrinsics) will be calculated. So when preparing an image you might need to generate two images. One for tracking, having a frame and for printing out, and one for matching, without a frame.

Image: Example of 2 prepared image markers

Generally the image marker tracker is based on grey pictures. If you like, you can also create a black image on a white ground. In the scope of this tutorial, you can either create your own image, or just use the supplied ones, that can be downloaded at the bottom of this page. Anyway some points will maybe make the image marker work better:

Defining the image marker in instantVision

Now, as you have your images lets have a look at how they are integrated into an instantVision file. The printed example below shows just the definition of an image marker inside a .pm file, that can be loaded by the vision system inside instantVision or instantReality. The full example can be found at the bottom of the page.

Code: Part of image marker definition inside the pm file

            
        <World ....
        <TrackedObject key="instantLogo">
        <ExtrinsicData calibrated="0">
         ....
        </ExtrinsicData>
        <ImageMarker fileName="project_dir:\InstantReality.jpg" key="myImageMarker" markerBorder="0.01">
          <Points3D nb="4">
            <HomgPoint3d w="1" x="0" y="2" z="0"/>
            <HomgPoint3d w="1" x="7" y="2" z="0"/>
            <HomgPoint3d w="1" x="7" y="0" z="0"/>
            <HomgPoint3d w="1" x="0" y="0" z="0"/>
          </Points3D>
        </ImageMarker>
        ...
      </TrackedObject>
     
    

As you can see, instead of a Marker element an ImageMarker-Element is used. The fileName-Attribute represents the image file to be loaded on initialization. You can either provide a full path name or place a "project_dir:\", which referres to the exact same path where the loaded .pm file resides.

The markerBorder-attribute describes the border to be ignored when comparing the cut out image with the saved one. The cut out image sometimes has a small border of the surrounding left and this should be ignored. Changing this value sometimes helps increasing the matching quality and thus can be changed. The value is given in percent divided by 100.

Testing the image marker in instantVision

We assume that you have already tried out the simple marker tutorial and maybe the marker tutorial for two objects. Within this tutorial we will not focus so much on the X3D integration as this already has been described inside the other marker tracker tutorials. We will rather show you some hints on using instantVision for testing your marker definitions and enhancing tracking quality.

Best is to download the files at the bottom and put them in one directory. You can also download the zip archive and unpack it into one folder.

Then the following steps have to be done:

Now put the printed out image marker in front of the camera. The data set should hold a key called MarkerTrackerInternalContour. This is generated by the ImageMarkerTracker and represents the detected contours inside the live camera image. By dragging this into the viewer, you will be able to see the detected contours. At least one contour has to cover the white instantReality logo in order to make the image marker tracker working. You can change the behaviour through the ImageMarkerTracker properties, which can be opened by double clicking on the ImageMarkerTracker inside the ActionManager on ther right hand side of the instantVision window.

Image: Showing the detected contours and the properties dialog of the ImageMarkerDetectAction

The action properties are divided into keys and attributes. Where keys refer to data set objects and attributes configure the behaviour of the action inside the action pipe itself. By double clicking on the value inside the value column of the attributes section you can directly modify the behaviour usually in runtime.

It follows a short explanation of the parameters that may be changed in order to adapt the image marker tracking for your needs.

Once you have configured your contour detection parameters to fit your image in the situations of your needs it is the time to test your config by augmenting the image with a 3D model. You can load the CoordFrame.wrl by pressing the add data button (). InstantVision also needs to know which perspective it has to render the model. So simply drag the InstantCamera object from the data set into the viewer. The viewer will configure itself for displaying the model and the camera on top of the picture. You can verify or modify this information by pressing the small button on the lower right corner of the viewer window and toggeling the status bar. An additional dialog will be appended for the viewer, allowing to show the propriate configuration of the viewer.

Image: The configuration of the viewer window

At this point you may wonder why the viewer gets totally black sometimes when the marker gets out of scope. This happens usually when the estimated pose is not valid, due to a not recognized contour or image. Then the viewer does not know, from which perspective the configured model should be rendered. When interfacing with instantReality, this sometimes might be a problem, as you maybe always want a valid camera pose.

A solution to this problem can be using the last valid used pose instead. So flickering can be prevented. This is done by a by default not activated action called "IfAction". This action allows checking if the current pose is valid. If it is not, it just copies the last valid one to the current one. And if it is valid, the last valid pose will be overwritten by the current one. You can try it out by enabling the action. The ActionManager is a tree view and you can open it up like in a file explorer. You will notice 2 action pipes, which itself can hold a series of actions, as you can see them in the root of the tree view. The first action pipe will always be executed if the condition specified inside the "IfAction" is true. Else the other one will executed.

In this case the condition looks like this:

Code: IfCondition for checking the invalidity of an estimated camera pose of the tracked object

[World.instantLogo.ExtrinsicData:PoseValid]!=1 

You can find the condition inside the condition attribute of the "IfAction". It refers to the image marker, called "instantLogo" inside the World object. Usually every "TrackedObject" gets an "ExtrinsicData" object inside, which represents the camera. The "PoseValid" attribute can be compared to a value and thus be checked if the condition is valid or not.

If everything worked fine, you just configured a nice image marker tracker for yourself. You can also use this one inside instantReality. Beware that you might have to prepare the .pm file by saving it to a new name. You will have to reopen it with a text editor in order to remove the "MegaWidgetActionPipe" element which represents the viewer and can not be loaded by the instantReality system. The rest can remain the same.

Basically the InstantCamera object will be transfered to instantReality in order to be routed to a ViewFrustum or inverted to an object to be represented. If you want you can download the ImageMarkerTrackerTest.x3d at the bottom of the page and try it out. You maybe have to change it and replace the given .pm file with your newly created one.

Image: Augmented flipper in instantReality (ImageMarkerTrackerTest.x3d)
Files: