In this code pattern, learn how to extract speaker diarized notes and meaningful insights reports using IBM® Watson™ Speech To Text, Watson Natural Language Processing, and Watson Tone Analysis when given any video.
In a virtually connected world, staying focused on work or education is very important. Studies suggest that many people lose their focus in live virtual meetings or virtual classroom sessions after approximately 20 minutes. Therefore, many meetings and virtual classrooms are recorded so that an individual can watch it later.
It might help if these recordings could be analyzed, and a detailed report of the meeting or class is generated by using artificial intelligence (AI). This code pattern explains how to do that. Given a video recording of the virtual meeting or virtual classroom, it explains how to extract audio from a video file using the FFmpeg open source library, transcribe the audio to get speaker-diarized notes with custom-trained language and acoustic speech to text models, and generate a natural language understanding report that consists of the category, concepts, emotion, entities, keywords, sentiment, top positive sentences, and word clouds using a Python Flask runtime.
After completing the code pattern, you understand how to:
- Use the Watson Speech to Text service to convert the human voice into the written word
- Use advanced natural language processing to analyze text and extract metadata from content such as concepts, entities, keywords, categories, sentiment, and emotion
- Leverage Watson Tone Analyzer cognitive linguistic analysis to identify a variety of tones at both the sentence and document level
-
User uploads recorded video file of the virtual meeting or a virtual classroom in the application.
-
FFMPG Library extracts audio from the video file.
-
Watson Speech To Text transcribes the audio to give a diarized textual output.
-
Watson Language Translator (Optionally) translates other languages into English transcript.
-
Watson Tone Analyzer analyses the transcript and picks up top positive statements form the transcript.
-
Watson Natural Language Understanding reads the transcript to identify key pointers from the transcript and get the sentiments and emotions.
-
The key pointers and summary of the video is then presented to the user in the application.
-
The user can then download the textual insights.
This is an Interactive instructions, follow the steps to setup a working instance of the code pattern in the Developer Playground.
Open Terminal
Get the Code Pattern on Playground
You need to be logged in to your IBM Cloud account in the Developer Playground to create and configure services.
Login to IBM Cloud
Do not have an IBM Cloud Account? click here to create one for free.
This code pattern uses the following IBM Watson Services:
-
Natural Language Understanding: Use advanced NLP to analyze text and extract meta-data from content such as concepts, entities, keywords, categories, sentiment, emotion, relations, and semantic roles.
-
Tone Analyzer: Tone Analyzer leverages cognitive linguistic analysis to identify a variety of tones at both the sentence and document level.
-
Speech to Text: The Speech to Text service converts the human voice into the written word.
Create these services and configure the credentials in the code pattern with just a click of button.
Create IBM Watson Services
You can build and run the Application within the Developer Playground, click on Build and Run to start the application.
Build and Run
You can make use of any meeting video or classroom video that you have or you can download the video that we have used for the demonstration purpose.
Stop Running Application
Completed the code pattern? Click on Clean up to delete the IBM Cloud services that were created.
Clean up
You can also manage the created services in IBM Cloud Dashboard.
You can find the detailed steps in the README file.