Video Browser Showdown

The Video Browser Showdown (VBS) is a live video browsing competition where international researchers, working in the field of interactive video search, evaluate and demonstrate the efficiency of their tools in presence of the audience. The aim of the VBS is to evaluate video browsing tools for efficiency at known-item search (KIS) tasks with a well-defined data set in direct comparison to other tools. For each task the moderator presents a target clip on a shared screen that is visible to all participants. The participants use their own systems to perform an interactive search in the specified video file taken from a common data set and try to find the desired segment as fast as possible. Found segments are submitted to a server that is responsible for three tasks: (1) checking whether the submitted segment is correct, (2) measuring the task solve time and (3) computing scores for all teams and tasks. The performance of participating tools is evaluated in terms of successful submissions and search time.

The first VBS was held at the 18th International Conference on MultiMedia Modeling (MMM 2012) in Klagenfurt, Austria, where eight international teams participated. The competition consisted of 14 tasks. The first eight tasks were performed by experts (the developers of the tools) and the last six tasks were performed by eight volunteers randomly selected from the audience and assigned to one of the tools. For each task the corresponding video containing a randomly selected target clip was mentioned first, in order to allow the participants to set up their systems for that video. After that, the 20 seconds long target clip was projected on a wall and the tone was played on loudspeakers. After this short presentation, the participants were given a maximum of two minutes to interactively find the target clip. The collection from which videos for the tasks were randomly selected consisted of 30 videos with an average length of 77 minutes (min: 31, max: 139).

Participants of VBS 2012 used very diverse strategies and implementations to support interactive search for the competition. While some participants used rather simple content analysis (e.g., colors and motion) but enhanced visualization, others used a combination of visual, textual and conceptual features for fast shot filtering, improved interaction means for fast navigation, and some participants used no content analysis at all. Interestingly, in the expert run a tool without any content analysis but parallel playback windows and hierarchical navigation/refinement features (the AAU Video Browser) clearly outperformed all other systems.

The next Video Browser Showdown (VBS 2013) will take place at the 19th International Conference on MultiMedia Modeling (MMM2013) in Huangshan, China.