direkt zum Inhalt springen

direkt zum Hauptnavigationsmenü

Sie sind hier

TU Berlin

Inhalt des Dokuments

Download Evaluation Script for C2LM (Consistent Two-Level Metric)

Why a new Metric?

Since the commonly used benchmarks for abandoned object detection (AOD) only have few abandoned objects and a non-standardized evaluation procedure, an objective performance comparison between different methods is hard. Therefore, we propose a new evaluation metric which is focused on an end-user application case and an evaluation protocol which eliminates uncertainties in previous performance assessments.

Code Guidelines

This script example was written for the evaluation of AOD methods using the proposed evaluation metrics. It contains the Ground Truth for all used datasets and a examplary file structure for the evaluation of detection results. To use the script python 2.7 is recommended, but 3. or higher can also be used.

The dataset contains the evaluation script as well as three extra folders, which contain:

  • gt_files: contains the Ground Truth files for each sequence and a folder with files about the respective video lenghts
  • sod_results: contains a folder for each sequence, which contain a bounding_box.txt file (which contains a list of bounding boxes for each object at each frame) as well as so_*method*.txt files, which contain the respective object IDs
  • output: contains the results corresponding to each dataset and *method* (e.g. statistic_aboda_baseline)

When renaming or adding *method*s, the corresponding part in the script (line 316) must be renamed as well!

The script also creates some latex files (result tables) and the corresponding 3D plots as seen below for each dataset and method.

Exemplary Data

Please feel free to contact us ({krusch, bochinski}@nue.tu-berlin.de) if you have problems using the script.

[List of people who downloaded the C2LM Script]

To receive the script, please fill out the form.

Download

Terms of use

All code is provided for research purposes only and without any warranty. Any commercial use is prohibited. By using the code in your research work, you should cite the respective paper.

A Consistent Two-Level Metric for Evaluation of Automated Abandoned Object Detection Methods [BibTex]

Zusatzinformationen / Extras

Direktzugang

Schnellnavigation zur Seite über Nummerneingabe