Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

TransNet V2: An effective deep network architecture for fast shot transition detection

About

Although automatic shot transition detection approaches are already investigated for more than two decades, an effective universal human-level model was not proposed yet. Even for common shot transitions like hard cuts or simple gradual changes, the potential diversity of analyzed video contents may still lead to both false hits and false dismissals. Recently, deep learning-based approaches significantly improved the accuracy of shot transition detection using 3D convolutional architectures and artificially created training data. Nevertheless, one hundred percent accuracy is still an unreachable ideal. In this paper, we share the current version of our deep network TransNet V2 that reaches state-of-the-art performance on respected benchmarks. A trained instance of the model is provided so it can be instantly utilized by the community for a highly efficient analysis of large video archives. Furthermore, the network architecture, as well as our experience with the training process, are detailed, including simple code snippets for convenient usage of the proposed model and visualization of results.

Tom\'a\v{s} Sou\v{c}ek, Jakub Loko\v{c}• 2020

Related benchmarks

TaskDatasetResultRank
Video UnderstandingVideo-MME
Overall Score63.2
96
Shot Boundary DetectionRAI dataset
F1 Score93.9
14
Shot Boundary DetectionRAI (test)
F1 Score0.939
10
Frame-level RecallSynFlash
Camouflage Recall95.55
8
Shot Boundary DetectionClipShots
F1 Score77.6
5
Shot Boundary DetectionBBC
F1 Score96.2
5
Real-World Anomaly DetectionUCF-Crime HIVAU-70k
Error Rate40.97
4
Real-World Anomaly DetectionXD-Violence HIVAU-70k
Error Rate37.16
4
Real-World Anomaly DetectionHIVAU-70k Weighted Average
Error Rate37.56
4
Shot Transition DetectionClipShots (test)
F1 Score77.9
4
Showing 10 of 13 rows

Other info

Code

Follow for update