Programmatically generates a series of highlight clips from a given stream VOD link.
Why don't you demo me?
Programmatically generates a series of highlight clips from a given stream VOD link. We realized that chat frequency at any time step could give a good estimate about the quality of the clip, which boiled our problem down into detecting peaks in time series data.
Algorithm: We implemented a peak signal detection algorithm based on how many standard deviations the chat frequency at a time T is away from the running mean to figure out spikes in chat frequencies.
Back End: Set up a server on Google Cloud that retrieves timestamp data from Twitch.tv and also implemented a cache feature that stores computed timestamps for videos.
Front End: Implemented UI inspired by material-io and used the Twitch.tv API (Javascript) to auto seek in the video.
Google Cloud is super hard to work with. Some versions support Python 3.7 natively (which we needed to download the Twitch chat log), but most do not. We were also bottlenecked by download speed as streams have huge chat logs.
We got a lot of moving parts and pieces working! We also heard a goat sing off-key for the past 10 hours while doing it.
Come to a hackathon with friends!
We could modify our algorithm to figure out which clips are highlights more accurately.