CNN and HEVC Video Coding Features for Static Video Summarization
Date
2022Author
Issa, Obada
Shanableh, Tamer
Advisor(s)
Unknown advisorType
Article
Peer-Reviewed
Published version
Metadata
Show full item recordAbstract
This study proposes a novel solution for the detection of keyframes for static video summarization. We preprocessed the well-known video datasets by coding them using the HEVC video coding standard. During coding, 64 proposed features were generated from the coder for each frame. Additionally, we converted the original YUVs of the raw videos into RGB images and fed them into pretrained CNN networks for feature extraction. These include GoogleNet, AlexNet, Inception-ResNet-v2, and VGG16. The modified datasets are made publicly available to the research community. Before detecting keyframes in a video, it is important to identify and eliminate duplicate or similar video frames. A subset of the proposed HEVC feature set was used to identify these frames and eliminate them from the video. We also propose an elimination solution based on the sum of the absolute differences between a frame and its motion-compensated predecessor. The proposed solutions are compared with existing works based on an SIFT flow algorithm that uses CNN features. Subsequently, an optional dimensionality reduction based on stepwise regression was applied to the feature vectors prior to detecting key frames. The proposed solution is compared with existing studies that use sparse autoencoders with CNN features for dimensionality reduction. The accuracy of the proposed key-frame detection system was assessed using the positive predictive values, sensitivity, and F-scores. Combining the proposed solution with Multi-CNN features and using a random forest classifier, it was shown that the proposed solution achieved an average F-score of 0.98.DSpace URI
http://hdl.handle.net/11073/24062External URI
https://doi.org/10.1109/ACCESS.2022.3188638Collections
Related items
Showing items related by title, author, creator and subject.
-
Static Video Summarization Using Video Coding Features with Frame-level Temporal Sub-Sampling and Deep Learning
Issa, Obada; Shanableh, Tamer (MDPI, 2023)There is an abundance of digital video content due to the cloud’s phenomenal growth and security footage, it is therefore essential to summarize these videos in data centers. This paper offers innovative approaches to the ... -
Online Video-Based Handwriting Recognition System
Hajjaj, Husamudin (2007-06)In the recent years, handwriting recognition systems started to have a major role in the new technology such as computers, mobile phones and hand-held devices to enhance the interaction between humans and these systems. ... -
Multi-Level Adaptive Video Streaming Over Wireless Channels
Mukhtar, Husameldin (2010-05)Delivery of digital video over wireless networks offers added functionalities and advantages over traditional wired video systems. However, reliable video streaming over wireless channels is fraught with several challenges. ...