Institute of Computer Science Pattern Recognition and Bioinformatics

AViSMo is an integrated system for memorizing visual data. Its primary goal is to provide some kind of visual memory, e.g. to interactive systems that use active cameras for exploration tasks and gather large amounts of visual data that cannot or shall not be processed immediately. AViSMo is based on mosaic images. A mosaic image is generated by stitching together all images of a sequence, thus eliminating redundancies and actually reducing the amount of data to be processed. AViSMo supports online- as well as offline-mosaicing. Although the main focus of AViSMo is placed on representing iconic data, additionally symbolic data (e.g object recognition results) can be attached to the mosaics. Furtheron, since robust mosaicing of arbitrary sequences requires some sort of motion detection, AViSMo provides information about moving objects (e.g. trajectories) and as well some kind of low-level interpretation of detected movements. Altogether this yields a complete scene representation.
The system itself consists of different modules mostly independent of each other. Single algorithms can easily be replaced and extensions added comfortably. Since AViSMo has been developed using a client-server approach, it can run stand-alone or easily be integrated into existing frameworks.

System Overview:
The system consists of several modules mostly independent of each other.

1. Parameter Estimation:
   Calculates a parametric transformation between the current image and the mosaic. At present we have implemented two hierarchical feature-less approaches with robustness additions. These take the recently detected motion of step 2 into account.
2. Motion Detection:
   Detects independently moving image regions based on intensity residuals. This step is crucial to make the parameter estimation step possible in dynamic scenes.
3. Tracking:
   Finds and records groups of regions with common motion. A graph is build that tracks all regions' positions over time and makes further activity analysis possible. The tracking can also influence the motion detection results.
4. Activity Detection:
   Analyses the tracking graph based on splits and merges and so finds activities like insertion or removal of an object in the scene.
5. Image Integration:
   Integrates the current view into the mosaic. Only those image parts that do not belong to a moving region (found in step 2 and 3) are used to avoid blurring.
6. Symbolic Information:
   Attachement of external information to the mosaic puts both into a common reference frame and facilitates communication between different components of the system utilizing AViSMo.

Communication: Client-Server-Architecture
AViSMo uses a client-server architecture that is able to run in a variety of circumstances. Clients can be e.g. a mobile robot, a graphical user interface or an interactive system. Communication with the Mosaic Server happens through the Communication Interface and is quite flexible. The system can run in its own thread or embedded, waiting to receive and transmit all results or unsynchronized. Currently we use a CORBA interface or direct C++ function calls but any other method can easily be added. AViSMo has been implemented on a linux system using C++ and sometimes utilizes the gnu scientific library for mathematical calculations.

Current Work and Future Extensions:

• More robust and flexible parameter estimation:
  Since correct sets of transformation parameters are essential for image integration, motion analysis and an appropriate choice of projection planes (especially when mosaicing is done online), one of our goals is to develope evaluation methods for estimated parameter sets and error handling routines, e.g. online correction of erroneous parameters or re-initialization of the mosaicing process.
• Multi-Mosaics:
  Mosaicing image sequences of a stationary but rotating and zooming camera that covers a large viewing angle yields perspective distortions if all images are projected onto one single image plane. We aim at reducing such distortions (but at the same time keeping rectangular image coordinates) by projecting the images onto a multi-resolution polytope tiling a sphere.
• Manifold-Mosaics:
  Mosaicing based on homographies as motion model restricts the freedom of movement of the camera to rotations and zoom. To be able to cope with translations as well a different mosaicing paradigm has to be used. Manifold-Mosaics approximate the image of a slit-camera that sees only one line of the world at once. The mosaic is then build up accumulating these lines over time.
• Mosaic-World:
  Mobile robots that perform exploration tasks have to remember large amounts of visual data. Combining multi-mosaics computed for interesting places in a scene and manifold-mosaics generated during travelling from one such a place to another, a visual representation of the environment is build. This iconic (and partly symbolic) memory may guide the robot in solving search and recognition tasks and help to recover paths through the environment.

Related Publications:

	B. Möller, Denis Williams and S. Posch. "Robust Image Sequence Mosaicing". In B. Michaelis, editor, Pattern Recognition, Proc. of 25th DAGM Symposium, LNCS 2781, pp. 386-393, Magdeburg, Germany, September 2003.
	D. Williams, B. Möller and S. Posch. "Integrated System for a Visual Memory based on Mosaics". In Proc. of 2003 International Conference on Imaging Science, Systems, and Technology (CISST'03), pp. 633-639, Las Vegas, Nevada, USA, June 2003.
	B. Möller and S. Posch. "Analysis of Object Interactions in Dynamic Scenes". In L. van Gool, editor, Pattern Recognition, Proc. of 24th DAGM Symposium, LNCS 2449, pp. 361-369, Zurich, Switzerland, September 2002. [PDF]
	B. Möller and S. Posch. "Detection and Tracking of Moving Objects for Mosaic Image Generation". In B. Radig and S. Florczyk, editors, Pattern Recognition, Proc. of 23rd DAGM Symposium, LNCS 2191, pp. 208-215, Munich, Germany, September 2001. [PDF]

Contact:
For any questions, comments or further suggestions, or if you would like to get an up-to-date version of our software, please sent an email:

Birgit Möller (moeller@informatik.uni-halle.de),  Denis Williams (williams@informatik.uni-halle.de),   Stefan Posch (posch@informatik.uni-halle.de)