The VisionKit camera framework

Specific for embedded Linux systems, we have developed a machine vision proof (since year 2006) camera framework which is now going public, as it is considered one of the most versatile Linux powered frameworks around. It consists of the following modules:

1. ppivideo: High performance buffer queue based kernel driver for Blackfin/uClinux
2. videoserver: Multi threaded camera server: Pipelined, quasi-simultaneous acquisition, image processing and data transmission via netpp or standard UDP/TCP video streaming. The videoserver is portable to other architectures (i86, ARM, ...)
3. Sensor access library, supporting full register access, see supported sensor list on the bottom
4. display: Remote display server with various video display options
5. The Camasutra remote control user interface. Originally developed for cameras (hinting the origin of its name) it can be customized to support any type of device

The important differences to the existing SoC v4l2 framework are, that all configuration and control of sensor devices is running in userspace via the netpp framework. Advantages:

• No requirement to opensource your specs or drivers!
• No bloated kernell
• No/Very little driver maintenance required when upgrading to newer kernels

The image processing capabilities of the videoserver are basically featured by a FIFO buffer queue. The user tells the kernel (preferrably via the v4l2 USERPTR method) where the video buffers are stored. A DMA process automatically fills the buffers with incoming video data and notifies the user when a buffer is ready. Then, a raw buffer is taken off the FIFO queue, data is processed and is optionally stuffed back onto the FIFO queue for transmission or further processing.

The entire architecture was designed such that no data loss can occur, which allows to employ the system for line scan applications. The used three stage FIFO scheme ('fresh', 'process', 'deliver') provides maximum performance while being most flexible. For extra performance or low latency applications, Realtime driver models and extensions are supported as well.

Reference applications:

• MJPEG streaming via http
• MPEG streaming using ffmpeg libraries
• 2D barcode reader, 3-5 frames per second
• Line scan application
• High reliability point tracking and blob detection

The VisionKit is not a plug'n'play camera from the box. It is meant for specific system developers who wish to combine the (possibly wireless) networking functionality of Linux systems with typical high performance image processing functionality on their own camera (or on section5-designed) hardware. It is based on a (uC)Linux kernel version 2.6.34.

Since based on the netpp framework, the VisionKit provides

• Platform independent scripting support: Remote control your entire camera pool using Python scripts
• Fast design of camera type specific user interfaces in XML which can be loaded by the Camasutra GUI control tool - at runtime

Screen shots

Example application of Camasutra remote control tool and remote display server. The processing is entirely done on the camera.

Further documentation:

• Preliminary API documentation
• Example HTML property reference (This is a reference of all properties and registers supported by the standard Visionkit supporting a few Aptina sensors)

The VisionKit supports the following commercially available cameras and modules:

Support for other, custom sensor modules can be implemented easily using XML descriptions provided by the netpp framework