Highlight Creation with using OpenCv For ATM Videos

Our this article is successive of   Our Previous Article . In previous article we have talked about generating the video highlight using short term energy approach . But for every video we cannot generate the highlight  using that approach because in short term energy approach , we need  audio in a video . Many videos like ATM videos or CCTV footage does not have audio .

By using OpenCV  we can generate highlight of those videos . Here our objective is to detect human from web camera and make video highlight .

We are using following :

1. Python Programming Language

2. Open CV Library

3. Spyder IDE

4. Inbuilt web camera

Our Approach 

1. We are using Haar Cascade Classifier to detect human face from web camera   
2. We are writing the camera feed into small clips in which human faces are detecting and ignoring other frames. We are saving those clips into a folder . 
3. At last we are merging all our clips to generate highlight .

Our Program is running on following system configuration

1. Intel i7 Processor
2. 8GB Ram
3. Window 8.1
4. OpenCV 4.4.1
5. Spyder 3.3.6

The Code with explanation is Here

References

1. OpenCV

2.https://realpython.com/face-detection-in-python-using-a-webcam/

3.https://towardsdatascience.com/https-medium-com-dilan-jay-face-detection-model-on-webcam-using-python-72b382699ee9?gi=bc4727910c28

Human Detection with Open CV

Human Detection is type of Object Detection in Computer Vision .

Image credit : Google.com

What is Object Detection ?

Object Detection is a computer technology related to computer vision and image processing that deals with detecting instances of semantics objects of a certain . (Wikipedia Definition)

 This task involves both identification of the presence of the objects and identification of the rectangular boundary surrounding each object (i.e. Object Localisation).

 An object detection system which can detect the class “Human” can work as a Human Detection System .

We can detect human using following algorithm 

1. Haar cascade  ( Research paper Haar Cascade )
2. HOG based approaches 


1. Haar Cascade Approach : 

This is proposed by Paul Viola and Michael Jones in their paper “Rapid Object Detection using a Boosted Cascade of Simple Features” published in 2001. This approach is widely used for Face Detection .

More About Haar Cascade

2. Histograms of Oriented Gradients for Human Detection

This is proposed by N. Dalal and B. Triggs in their paper “Histograms of oriented gradients for human detection” published in 2005.

Video Highlight Creation Of massive video feeds

You see videos in daily life . In this article we will talk about how to create video highlight .

Highlight means focusing on main events in the Video . As in sports videos , highlight means to points out distilling the most key , salient and interesting parts from video . As in 50-50 over cricket  match , highlight means to generate highlight of events as wickets falling , boundaries , catches , run-outs , umpire decision etc .

Highlight Generation is the process of extracting the most interesting clips from a video .

Basic Idea : Whenever an interesting events occurs , there is an increase in the voice as well as the spectators .  

There are many approaches to generate Video highlight . It will depend on the problem domain , which technique we will use .

1. Short Time Energy

The best thing about this approach is you don't need training data for your model . 

Question : What is short time speech ?

Answer : The short time energy is the energy of the short speech segment .

The energy or power of an audio signal refers to the loudness of the sound . It is computed by the sum of the square of the amplitude of an audio signal in the time domain . When energy is computed for a chunk of an entire audio signal , then it is known as Short Time Energy .

Image Courtesy : https://www.analyticsvidhya.com

 Step By Step Process

1. Input the Video
2. Extract the audio
3. Break the audio into chunks
4. Compute short-time energy of every chunk
5. Classify every chunk as excitement or not(based on a threshold value)
6. Merge all the excitement-clips to form the video highlights

This approach is best for sports videos .

2. Using the Open CV

Suppose we have videos in which no sound is their , like CCTV surveillance camera . Than Above approach will fail . Open CV is used to detect and track the objects .
Suppose we want to make a highlight video from ATM CCTV camera . As in 24 hours videos only some hours transaction happened , first we need to extract those clips from main video .


3.  Using the NLP (Natural Language Processing)

In this approach we convert sounds into text and if we find the important text than we extract that clip .

Here is a step-by-step procedure:

1. Extract the audio from an input video
2. Transcribe the audio to text
3. Apply Extractive based Summarization techniques on text to identify the most important phrases
4. Extract the clips of corresponding important phrases to generate highlights

References :
1. https://ieeexplore.ieee.org/search/searchresult.jsp?searchWithin=%22Authors%22:%22Xiaoquan%20Yi%22&newsearch=true

Computer Vision :" See with the eye of Computers "

Computer Vision gives eyes to Computers . By using the computer vision  computer can detect , track the object in image and video .

Definition from Wikipedia

"Computer vision is an interdisciplinary scientific field that deals with how computers can be made to gain high-level understanding from digital images or videos."

Computer Vision is also composed of various aspects such as image recognition , object detection , image generation , image super-resolution and many other things .

Image Recognition :  is the detail an image holds. The term applies to raster digital images , film images, and other types of images. Higher resolution means more image detail . Image resolution can be measured in various ways. Resolution quantifies how close lines can be to each other and still  be visibly resolved. 

Object Detection : It is a computer technology related to computer vision and image processing that deals with detecting instances of semantics objects of a certain class ( i.e. humans , buildings or cars ) in digital images and videos .

Image Generation : It is the task of generating new images .

Image Super - resolution : It is class of techniques that enhance (increase) the resolution of an imaging system . In some SR techniques - termed optical SR - the diffraction limit of systems is transcended , while in others - geometrical SR - the resolution of digital imaging sensors is enhanced .

Dataset / Interesting Blogs Links

1. IBM Research Releases ‘Diversity in Faces’ Dataset to Advance Study of Fairness in Facial Recognition Systems

https://www.ibm.com/blogs/research/2019/01/diversity-in-faces/

2. Abu Dhabi National Oil Company (ADNOC): Enhancing accuracy, consistency and speed of rock analysis to support better decisions

https://www.ibm.com/case-studies/abu-dhabi-national-oil-company-adnoc

Video Highlight Creation using Short Time Energy

In this article we will talk about how to generate video highlight using the short time energy. 

The best thing about this approach is you don't need training data for your model .

Question : What is short time energy ?
Answer : The short time energy is the energy of the short speech segment .

The energy or power of an audio signal refers to the loudness of the sound . It is computed by the sum of the square of the amplitude of an audio signal in the time domain . When energy is computed for a chunk of an entire audio signal , then it is known as Short Time Energy .

Image Courtesy : https://www.analyticsvidhya.com

Basic Idea : Whenever an interesting events occurs , there is an increase in the voice as well as the spectators . 

 Step By Step Process

1. Input the Video
2. Extract the audio
3. Break the audio into chunks
4. Compute short-time energy of every chunk
5. Classify every chunk as excitement or not(based on a threshold value)
6. Merge all the excitement-clips to form the video highlights

TLD Implementation in Python With Explanation

#importing cv2 and system package
import cv2
import sys
#So how do you ensure that your code will work no matter which version of OpenCV your production environment is using
# Extract major, minor, and subminor version numbers
(major_ver, minor_ver, subminor_ver) = (cv2.__version__).split('.')
#Every Python module has it’s __name__ defined and if this is ‘__main__’, it implies that the module is being run standalone
#by the user and we can do corresponding appropriate actions.
if __name__ == '__main__' :
    # Set up tracker.
    tracker_type = 'TLD'
    if int(minor_ver) < 3:
        tracker = cv2.Tracker_create(tracker_type)
    else:
        if tracker_type == 'TLD':
            tracker = cv2.TrackerTLD_create()
    # Read video
    video = cv2.VideoCapture("./videos/chaplin.mp4")
    # Exit if video not opened.
    if not video.isOpened():
        print("Could not open video")
        sys.exit()
    # Read first frame.
    ok, frame = video.read()
    if not ok:
        print('Cannot read video file')
        sys.exit()
    #rectangular region of interest (ROI)
    #Let’s start with a sample code. It allows you to select a rectangle in an image,
    #crop the rectangular region and finally display the cropped image.
    bbox = cv2.selectROI(frame, False)
    # Initialize tracker with first frame and bounding box
    ok = tracker.init(frame, bbox)
    file=open("Coordinate.txt","w")
    while True:
        # Read a new frame
        ok, frame = video.read()
        if not ok:
            break
   
        # Start timer
        timer = cv2.getTickCount()
        # Update tracker
        ok, bbox = tracker.update(frame)
        # Calculate Frames per second (FPS)
        fps = cv2.getTickFrequency() / (cv2.getTickCount() - timer);
        frame_count = int(video.get(cv2.CAP_PROP_FRAME_COUNT))
 
        # Draw bounding box
        if ok:
            # Tracking success
            p1 = (int(bbox[0]), int(bbox[1]))
            p2 = (int(bbox[0] + bbox[2]), int(bbox[1] + bbox[3]))
     
            cv2.rectangle(frame, p1, p2, (255,0,0), 2, 1)
        else :
            # Tracking failure
            cv2.putText(frame, "Tracking failure detected", (100,80), cv2.FONT_HERSHEY_SIMPLEX, 0.75,(0,0,255),2)
        #cv2.putText(img, text, position, font, fontScale, color, thickness, lineType, bottomLeftOrigin)
 
        # Display tracker type on frame
        cv2.putText(frame, tracker_type + " Tracker", (100,20), cv2.FONT_HERSHEY_SIMPLEX, 0.75, (50,170,50),2);
 
        timer=(cv2.getTickCount()-timer)/cv2.getTickFrequency()
 
        #Display X and Y Coordinate
        cv2.putText(frame,"X and Y Coordinate "+str(p1)+" and "+str(p2), (100,70), cv2.FONT_HERSHEY_SIMPLEX, 0.75, (50,170,50),2);
 
        file.write(str(timer)+" :: UpperLeft(x,y) and BottomRight(x,y) "+str(p1)+" and "+str(p2)+"\n")
     
        # Display FPS on frame
        cv2.putText(frame, "FPS : " + str(int(fps)), (100,50), cv2.FONT_HERSHEY_SIMPLEX, 0.75, (50,170,50), 2);
        # Display result
        cv2.imshow("Tracking", frame)
        # Exit if ESC pressed
        k = cv2.waitKey(1) & 0xff
        if k == 27 :
            file.close()
            break

Credit : Github
Note : Code has taken from Github and modified according to need . 

FAQ about UOH MTech Admission

Can MTech CS or IT or IS student learn AI ?

Definitely you can learn anything you want .

Can MTech CS or IT or IS guys take AI electives ? 

Yes ,  but not all some electives they can take .

How many electives student can take in first and second semester ?

It depends on the course , generally 3 electives per semester and 2 mandatory core subjects.

Number of Food canteens  in UOH ?

Let divide UOH in two parts :- North campus and south campus .
In North Campus following canteens are their

1.Student canteen
2.Goaps
3.F canteen
4.North Shop Complex
5.Chemistry canteen
6. Night Canteen (Its Timing is 10pm - Morning)



In South campus following canteens are their

1. South Shop Complex
2. Hotel , Dhabas are available outside the South Campus.

Visiting places inside campus ?
Number of places are their to visit . Below playlist contains more then 15+ videos about UOH campus .

From where to purchase bed sheets , bucket , mug etc .
These things are not available  inside the Campus . Just outside the south campus there are shops by where you can purchase all these things .

What is level of Assignment here ?
It depends on the faculty . Generally you need to do it by own . If you found copy by others , marks will distribute equally .

What about labs here ?
Beside the 5 subjects , two labs are in first semester and one lab in second second .
Two labs in first semester are
1. DSP Lab (Data Structure and Programming Lab)
2. Algorithm Lab

From where to purchase cycles?
For new cycle , you need to go outside campus .
For old cycle , contact seniors .

Sports Facility in UOH
Sports Complex is in North Campus .

What is timing of classes ?
General it is from 9:30 AM to 6:30 PM . But You will get break between classes . 
They are not for whole a day .

What about holidays ?
 Saturday and Sunday no classes . Other holidays are mentioned in Academic Calendar .

What about the middle terms exam here ?
Generally two minors are here , some faculty takes three minors (Out of  three , best two they consider) . Total minor marks are 40 and External exam mark is 60 in each subject .

From Which month I will get Gate Stipend ?
From December or January , you will receive your  first gate stipend .

Is 75%  attendance mandatory  for gate stipend ?
Strictly , it is mandatory .
Note: 75% is overall in all subjects .

From where to learn AI ?
Andrew Ng ML course is available on  Coursera and YouTube
Geoffrey Hinton  course is also available on YouTube.
You can purchase other AI Course like Applied AI etc.

If you have any query please write to us at
hemjoshi745@gmail.com  or do whatsapp 9675467414

If you like this article , subscribe our youtube channel .