Highlight Creation with using OpenCv For ATM Videos

Our this article is successive of   Our Previous Article . In previous article we have talked about generating the video highlight using short term energy approach . But for every video we cannot generate the highlight  using that approach because in short term energy approach , we need  audio in a video . Many videos like ATM videos or CCTV footage does not have audio .

By using OpenCV  we can generate highlight of those videos . Here our objective is to detect human from web camera and make video highlight .

We are using following :

1. Python Programming Language

2. Open CV Library

3. Spyder IDE

4. Inbuilt web camera

Our Approach 

1. We are using Haar Cascade Classifier to detect human face from web camera   
2. We are writing the camera feed into small clips in which human faces are detecting and ignoring other frames. We are saving those clips into a folder . 
3. At last we are merging all our clips to generate highlight .

Our Program is running on following system configuration

1. Intel i7 Processor
2. 8GB Ram
3. Window 8.1
4. OpenCV 4.4.1
5. Spyder 3.3.6

The Code with explanation is Here

References

1. OpenCV

2.https://realpython.com/face-detection-in-python-using-a-webcam/

3.https://towardsdatascience.com/https-medium-com-dilan-jay-face-detection-model-on-webcam-using-python-72b382699ee9?gi=bc4727910c28

Human Detection with Open CV

Human Detection is type of Object Detection in Computer Vision .

Image credit : Google.com

What is Object Detection ?

Object Detection is a computer technology related to computer vision and image processing that deals with detecting instances of semantics objects of a certain . (Wikipedia Definition)

 This task involves both identification of the presence of the objects and identification of the rectangular boundary surrounding each object (i.e. Object Localisation).

 An object detection system which can detect the class “Human” can work as a Human Detection System .

We can detect human using following algorithm 

1. Haar cascade  ( Research paper Haar Cascade )
2. HOG based approaches 


1. Haar Cascade Approach : 

This is proposed by Paul Viola and Michael Jones in their paper “Rapid Object Detection using a Boosted Cascade of Simple Features” published in 2001. This approach is widely used for Face Detection .

More About Haar Cascade

2. Histograms of Oriented Gradients for Human Detection

This is proposed by N. Dalal and B. Triggs in their paper “Histograms of oriented gradients for human detection” published in 2005.

Video Highlight Creation Of massive video feeds

You see videos in daily life . In this article we will talk about how to create video highlight .

Highlight means focusing on main events in the Video . As in sports videos , highlight means to points out distilling the most key , salient and interesting parts from video . As in 50-50 over cricket  match , highlight means to generate highlight of events as wickets falling , boundaries , catches , run-outs , umpire decision etc .

Highlight Generation is the process of extracting the most interesting clips from a video .

Basic Idea : Whenever an interesting events occurs , there is an increase in the voice as well as the spectators .  

There are many approaches to generate Video highlight . It will depend on the problem domain , which technique we will use .

1. Short Time Energy

The best thing about this approach is you don't need training data for your model . 

Question : What is short time speech ?

Answer : The short time energy is the energy of the short speech segment .

The energy or power of an audio signal refers to the loudness of the sound . It is computed by the sum of the square of the amplitude of an audio signal in the time domain . When energy is computed for a chunk of an entire audio signal , then it is known as Short Time Energy .

Image Courtesy : https://www.analyticsvidhya.com

 Step By Step Process

1. Input the Video
2. Extract the audio
3. Break the audio into chunks
4. Compute short-time energy of every chunk
5. Classify every chunk as excitement or not(based on a threshold value)
6. Merge all the excitement-clips to form the video highlights

This approach is best for sports videos .

2. Using the Open CV

Suppose we have videos in which no sound is their , like CCTV surveillance camera . Than Above approach will fail . Open CV is used to detect and track the objects .
Suppose we want to make a highlight video from ATM CCTV camera . As in 24 hours videos only some hours transaction happened , first we need to extract those clips from main video .


3.  Using the NLP (Natural Language Processing)

In this approach we convert sounds into text and if we find the important text than we extract that clip .

Here is a step-by-step procedure:

1. Extract the audio from an input video
2. Transcribe the audio to text
3. Apply Extractive based Summarization techniques on text to identify the most important phrases
4. Extract the clips of corresponding important phrases to generate highlights

References :
1. https://ieeexplore.ieee.org/search/searchresult.jsp?searchWithin=%22Authors%22:%22Xiaoquan%20Yi%22&newsearch=true

Computer Vision :" See with the eye of Computers "

Computer Vision gives eyes to Computers . By using the computer vision  computer can detect , track the object in image and video .

Definition from Wikipedia

"Computer vision is an interdisciplinary scientific field that deals with how computers can be made to gain high-level understanding from digital images or videos."

Computer Vision is also composed of various aspects such as image recognition , object detection , image generation , image super-resolution and many other things .

Image Recognition :  is the detail an image holds. The term applies to raster digital images , film images, and other types of images. Higher resolution means more image detail . Image resolution can be measured in various ways. Resolution quantifies how close lines can be to each other and still  be visibly resolved. 

Object Detection : It is a computer technology related to computer vision and image processing that deals with detecting instances of semantics objects of a certain class ( i.e. humans , buildings or cars ) in digital images and videos .

Image Generation : It is the task of generating new images .

Image Super - resolution : It is class of techniques that enhance (increase) the resolution of an imaging system . In some SR techniques - termed optical SR - the diffraction limit of systems is transcended , while in others - geometrical SR - the resolution of digital imaging sensors is enhanced .

Dataset / Interesting Blogs Links

1. IBM Research Releases ‘Diversity in Faces’ Dataset to Advance Study of Fairness in Facial Recognition Systems

https://www.ibm.com/blogs/research/2019/01/diversity-in-faces/

2. Abu Dhabi National Oil Company (ADNOC): Enhancing accuracy, consistency and speed of rock analysis to support better decisions

https://www.ibm.com/case-studies/abu-dhabi-national-oil-company-adnoc

Video Highlight Creation using Short Time Energy

In this article we will talk about how to generate video highlight using the short time energy. 

The best thing about this approach is you don't need training data for your model .

Question : What is short time energy ?
Answer : The short time energy is the energy of the short speech segment .

The energy or power of an audio signal refers to the loudness of the sound . It is computed by the sum of the square of the amplitude of an audio signal in the time domain . When energy is computed for a chunk of an entire audio signal , then it is known as Short Time Energy .

Image Courtesy : https://www.analyticsvidhya.com

Basic Idea : Whenever an interesting events occurs , there is an increase in the voice as well as the spectators . 

 Step By Step Process

1. Input the Video
2. Extract the audio
3. Break the audio into chunks
4. Compute short-time energy of every chunk
5. Classify every chunk as excitement or not(based on a threshold value)
6. Merge all the excitement-clips to form the video highlights