NEUROSYSTEMS: Augmented Reality

As a sub-domain of Computer Vision, Augmented Reality is a term for a live direct or indirect view of a physical, real-world environment whose elements are augmented by computer-generated sensory input, such as sound or graphics. It combines real and virtual, is interactive in real time and is registered in 3D.

Let's take a simple example to briefly explain how NeuroSystems' realtime 3D Augmented Reality system works.

1 - THE MARKER

LEFT image: it represents the marker, which is the reference symbol or image the algorithms will try to find in the source image in order to figure out its perspective and then calculate the right camera position and orientation.

RIGHT image: this is how the system "sees" the corresponding marker.

2 - INPUT SOURCE

LEFT image: this is the source image that generally comes from a video input.

RIGHT image: this is how the algorithms "see" this source. The green stroke represents the typical and efficient edge detection for computer vision.

3 - TRACKING PROCESS

The system analyzes each video source image up to 30 times per second in order to recognize the marker it already knows. Once found, the algorithm concludes its projective geometry in order to calculate the real camera position and orientation relative to this rectangular physical marker.

4 - 3D MODEL

LEFT image: a 3D computer graphics object with its position and orientation matching the initial known marker, generally top-side view (this image is for illustration purpose only).

RIGHT image: Once the real camera lens, position and direction is known, a virtual camera can be positioned at the same point with the same direction and focal, and the 3D model rendered exactly overlaid on the real marker.

5 - IMAGE PROCESSING

The final process consists of compositing the source image and the photorealistic rendering we just got through the virtual camera.

Here are two short experimental videos from NeuroSystems showing realtime 3D photorealistic Augmented Reality.