A naukri.com initiative
Medium
4d
329
Image Credit: Medium
“Lights, Camera, AI”: Video Trailer Generation using Multimodal Data Analysis
- Researchers developed a novel approach for video trailer generation focusing on auditory features, crucial for the horror genre.
- A dataset of 311 short films and their official trailers was compiled from various sources like YouTube, Vimeo, and others.
- Audio and video files were extracted using the Python library youtube-dl for feature extraction.
- Audio cues were found to be predominant for horror-thriller genres, leading to an audio-guided design.
- Trailer features were clustered using k-means to identify 'trailer-worthy' audio moments during inference.
- Evaluation metrics included Hamming Score (HS), Intersection Over Union (IOU), and Task Accuracy (TA) to assess machine learning models' effectiveness.
- Average Hamming Score, indicating trailer-worthiness, was found to be 0.6930 over 20% of the dataset.
- Average IOU, representing overlap with actual trailer segments, was computed as 0.3455, showcasing room for additional trailer-worthy segments.
- Average Task Accuracy, measuring segment prediction accuracy, stood at 0.5625 during testing.
- An audio-guided visual framework automated trailer generation for horror films, achieving notable Hamming Score, IOU, and Task Accuracy values.
- The framework predicted highly trailer-worthy segments effectively, with potential for further research to enhance its robustness.
Read Full Article
19 Likes
For uninterrupted reading, download the app