 | Referee - The football game players motion tracking and presentation system. |
The Referee software was invented in 2004-th for providing the techniques for real time multiple objects identification and tracking in sport games. The position data about all objects on the playground is extracted from multiple cameras video streams in real time. Object's identification and tracking provided by special algorithms and devices, which are doing this task automatically.
The system was built on several computer units (CU), each handles its own task, for example: some CUs process video streams from cameras, recognize objects (players, ball etc.), other integrate this information, resolves collisions, build the entire 3D scene of the game. There are some CUs that present resulting information, perform continuous diagnostic of the system etc.
The sketch of the system is presented below:
The Referee is the camera and primary processing module. The number of the Referee modules is equal to the number of cameras in the system. Usually, the number of the modules is from 4 to 6. This is crucial module in the system. It receives input data with the rate about 2 MB/s, and passes after processing just about 200 B/s -2 kB/s for final processing. The module can be divided on two parts:
Capture sub-module is implemented as the separate plug-in module. That allows attaching any new camera without recompilation of entire module. Capture sub-module works in the synchronous mode: the capture module starts capture of each new frame on the external software trigger pulse.
Primary processing module works under full control of Final Processing module (FSolver application on a sketch). The main processing thread of the module works on the principles of stage processing, i.e., all process is divided on a sequence of stages, and each stage decreases the size of data to be processed. This is illustrated below.
First stage is looking-up for small objects. That task is performing by convolution with most probable mask (player):
The results of this first stage are presented to the motion detector module. The using of the processed data, instead of original frame, as source data for motion detector, decreases significantly the influence of the intensity illumination variations on results of processing.
At this, as a result of motion detector stage, we have mask for moving objects, we use it jointly low performance edge-detection algorithm. The mask using at that stage solves two tasks - we will have at output just moved objects, and decrease time of processing due to reduction of the overall area of processing.
As result of last processing, we have binary images in which all moved objects are well confined. The results are stored in results queue and passed to Final processing module.
The Final processing module (FSolver) is quite complex application. It is related with necessity both powerful user interface and efficient processing module. Both modules are implemented in one program module. The reason of that is in the fact that both modules are close related and have to exchange by fair amount of information. Another important part of the Final Processing Module is the consolidation of all modules in one processing system and the system management/control.
The main part of the module is the geometry conversion thread, which collect and transforms data from cameras and transform data to the field physical coordinates. The process is simplistic shown below:
The matching thread - joins data from different cameras with same time-mark in one data block and looks matching between positions of objects taken from different cameras, the result of the processing is physical coordinates of the objects on the field:
The tracing and assignments thread. If previous thread joins data in a space domain, this thread joins data in time domain. Using history data, it calculates the probability zone of new objects position by interpolation of previous data (position, speed) for each object:
Program interface is presented below:
Game description sub-module. This module manages by all collected during the game information. The data is stored in a database.
Puppeteer: The camera emulation module emulates real game with full set of players and ball. One of the presentation modes of this module is presented below:
