Video Highlight Creation Of massive video feeds

You see videos in daily life . In this article we will talk about how to create video highlight .

Highlight means focusing on main events in the Video . As in sports videos , highlight means to points out distilling the most key , salient and interesting parts from video . As in 50-50 over cricket  match , highlight means to generate highlight of events as wickets falling , boundaries , catches , run-outs , umpire decision etc .

Highlight Generation is the process of extracting the most interesting clips from a video .

Basic Idea : Whenever an interesting events occurs , there is an increase in the voice as well as the spectators .  

There are many approaches to generate Video highlight . It will depend on the problem domain , which technique we will use .

1. Short Time Energy

The best thing about this approach is you don't need training data for your model . 

Question : What is short time speech ?

Answer : The short time energy is the energy of the short speech segment .

The energy or power of an audio signal refers to the loudness of the sound . It is computed by the sum of the square of the amplitude of an audio signal in the time domain . When energy is computed for a chunk of an entire audio signal , then it is known as Short Time Energy .

Image Courtesy : https://www.analyticsvidhya.com

 Step By Step Process

1. Input the Video
2. Extract the audio
3. Break the audio into chunks
4. Compute short-time energy of every chunk
5. Classify every chunk as excitement or not(based on a threshold value)
6. Merge all the excitement-clips to form the video highlights

This approach is best for sports videos .

2. Using the Open CV

Suppose we have videos in which no sound is their , like CCTV surveillance camera . Than Above approach will fail . Open CV is used to detect and track the objects .
Suppose we want to make a highlight video from ATM CCTV camera . As in 24 hours videos only some hours transaction happened , first we need to extract those clips from main video .


3.  Using the NLP (Natural Language Processing)

In this approach we convert sounds into text and if we find the important text than we extract that clip .

Here is a step-by-step procedure:

1. Extract the audio from an input video
2. Transcribe the audio to text
3. Apply Extractive based Summarization techniques on text to identify the most important phrases
4. Extract the clips of corresponding important phrases to generate highlights

References :
1. https://ieeexplore.ieee.org/search/searchresult.jsp?searchWithin=%22Authors%22:%22Xiaoquan%20Yi%22&newsearch=true