New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: transmux before append (TBA) and low latency http streaming (LHLS) #494

Merged

gkatsev merged 1 commit into master from feat/tba-lhls

May 20, 2019

Contributor

brandonocasey commented May 3, 2019 •

edited by gkatsev

Loading

Description

This takes most of the changes from #466, gesinger#10, and changes that have occurred in master since both of those and packages them up.

I still have the following to do:

Verify that all fixes and changes from master have been brought into this branch
Make the src diff easier to read where possible, or add comments on code.
Get all the tests working and make sure that all the tests still exist
what is the difference between mux.js segmentTimingInfo/videoSegmentTimingInfo and timingInfo/videoTimingInfo.
Generate an encrypted ts segment with a key for unit tests

This is based on the following, which will have to be merged first:

brandonocasey commented

View reviewed changes

src/master-playlist-controller.js

                     });
                     this.triggerPresenceUsage_(this.master(), media);
-                    try {

Contributor Author

brandonocasey May 3, 2019

this is handled by a sourceopen on the media source in handleSourceOpen_()

brandonocasey commented

View reviewed changes

src/master-playlist-controller.js

                         this.audioSegmentLoader_.load();
                       }
                     }
-                    if (!updatedPlaylist.endList) {

Contributor Author

brandonocasey May 3, 2019

Presumably we no longer have to update seekable or deal with durationchange since we are using a native media source. This code should have lived in our non-native media source I think.

brandonocasey commented

View reviewed changes

src/master-playlist-controller.js

                     this.tech_.trigger({type: 'usage', name: 'hls-aes'});
                   }
-                  if (Hls.Playlist.isFmp4(media)) {
-                    this.tech_.trigger({type: 'usage', name: 'hls-fmp4'});

Contributor Author

brandonocasey May 3, 2019

This code was moved to setupSegmentLoaderListeners_() and we trigger the usage event when a segmentLoader triggers an fmp4 event.

brandonocasey commented

View reviewed changes

src/master-playlist-controller.js

                     }, ABORT_EARLY_BLACKLIST_SECONDS);
                   });
-                  this.mainSegmentLoader_.on('reseteverything', () => {
-                    // If playing an MTS stream, a videojs.MediaSource is listening for

Contributor Author

brandonocasey May 3, 2019 •

edited

Loading

we use a native media source so reseteverything and segmenttimemapping are no longer fired or needed.

brandonocasey force-pushed the feat/tba-lhls branch 2 times, most recently from 043f330 to db5f24f Compare

May 3, 2019 17:21

brandonocasey commented

View reviewed changes

src/master-playlist-controller.js

+                 * @private
+                 */
+                handleSourceEnded_() {
+                  if (!this.inbandTextTracks_.metadataTrack_) {

Contributor Author

brandonocasey May 3, 2019

mostly taken and modified from the sourceended handler in html-media-source

brandonocasey force-pushed the feat/tba-lhls branch 5 times, most recently from 98ec660 to fdfd3a3 Compare

May 3, 2019 17:40

brandonocasey commented

View reviewed changes

src/master-playlist-controller.js

-                    // even if the media source is open and source buffers are not
-                    // updating, something about the media source being in an invalid state.
-                    this.logger_(`Setting duration from ${this.mediaSource.duration} => ${newDuration}`);
-                    try {

Contributor Author

brandonocasey May 3, 2019

The try catch here was moved into sourceUpdaters actions.duration handler. Most of the rest of this logic is reformatted but unchanged.

brandonocasey force-pushed the feat/tba-lhls branch 3 times, most recently from 068da21 to 9aec73c Compare

May 3, 2019 17:53

brandonocasey commented

View reviewed changes

src/media-groups.js

-                  // loader and not restart the audio loaders
-                  mainSegmentLoader.resetEverything();
-                  return;
+                if (type === 'AUDIO') {

Contributor Author

brandonocasey May 3, 2019

I think this just handles moving from demuxed audio/video to muxed audio/video in a better way.

brandonocasey commented

View reviewed changes

src/playlist.js

-               *
-               * @return {Boolean} true if the playlist contains fMP4
-               */
-              export const isFmp4 = function(media) {

Contributor Author

brandonocasey May 3, 2019

We now use util/codecs.js#isLikelyFmp4Data() to determine if a segment is fmp4 during media segment requests. Then we add segment.isFmp4 to the segment object.

brandonocasey force-pushed the feat/tba-lhls branch 4 times, most recently from f21c2d2 to 373e4ea Compare

May 3, 2019 19:56

brandonocasey commented

View reviewed changes

src/util/text-tracks.js

-                  for (let trackId in segment.captionStreams) {
-                    if (!sourceBuffer.inbandTextTracks_[trackId]) {
-                      player.tech_.trigger({type: 'usage', name: 'hls-608'});

Contributor Author

brandonocasey May 3, 2019

all of this logic was the same as createCaptionsTrackIfNotExists.

brandonocasey commented

View reviewed changes

src/util/text-tracks.js

-              export const addTextTrackData = function(sourceHandler, captionArray, metadataArray) {
-                let Cue = window.WebKitDataCue || window.VTTCue;
-                if (captionArray) {

Contributor Author

brandonocasey May 3, 2019

This is/was addCaptionData, not sure why we repeated the logic here, but it was removed and is now done in segment-loader.

brandonocasey force-pushed the feat/tba-lhls branch 2 times, most recently from 74b70cc to f757166 Compare

May 3, 2019 20:33

brandonocasey commented

View reviewed changes

src/source-updater.js

-                    this.pendingCallback_ = null;
-                    this.sourceBuffer_.removing = false;
-                    this.logger_(`buffered [${printableRange(this.buffered())}]`);

Contributor Author

brandonocasey May 3, 2019

this logic is now in the onUpdateend function right above the class for this file.

brandonocasey commented

View reviewed changes

src/source-updater.js

+                      sourceUpdater: this,
+                      action: actions.callback(callback),
+                      name: 'callback'
+                    });
                   }
                 }

Contributor Author

brandonocasey May 3, 2019

I think this file will eventually need some updates. Some places we have function that pass a type and are generic for audio and video. Then others functions are prepended with audio or video (IE audioTimestampOffset vs videoTimestampOffset) and they do nearly the same thing.

brandonocasey commented

View reviewed changes

src/sync-controller.js

-                 *
-                 * @param {SegmentInfo} segmentInfo - The current active request information
-                 */
-                probeSegmentInfo(segmentInfo) {

Contributor Author

brandonocasey May 3, 2019 •

edited

Loading

All of the probe logic seems to have been moved into src/util/segment and I think we store syncinfo on the playlist now.

brandonocasey commented

View reviewed changes

src/transmuxer-worker.js

-                }
+                });
+                transmuxer.on('caption', function(caption) {

Contributor Author

brandonocasey May 3, 2019

I think caption and id3Frame takes the place of transmuxer.captionStream?

brandonocasey force-pushed the feat/tba-lhls branch 2 times, most recently from 5dac930 to 2590492 Compare

May 7, 2019 20:34

gkatsev added this to the TBA/LHLS milestone

brandonocasey force-pushed the feat/tba-lhls branch from 12f62c3 to cabf2c1 Compare

May 9, 2019 19:49

brandonocasey force-pushed the tba-master branch from 9b60a2c to f74f571 Compare

May 9, 2019 19:52

brandonocasey force-pushed the feat/tba-lhls branch from cabf2c1 to 3781b26 Compare

May 9, 2019 19:56

brandonocasey force-pushed the tba-master branch from f74f571 to eec284f Compare

May 9, 2019 19:56

brandonocasey force-pushed the feat/tba-lhls branch 6 times, most recently from c632c4c to adfb2cc Compare

May 10, 2019 19:33

brandonocasey force-pushed the tba-master branch from eec284f to 869bbda Compare

May 13, 2019 20:09

brandonocasey force-pushed the feat/tba-lhls branch from 1d5a23b to ee5bcbb Compare

May 13, 2019 20:10

brandonocasey mentioned this pull request

feat: Transmux Before Append and LHLS #466

Closed

34 tasks

brandonocasey commented

View reviewed changes

package.json Outdated Show resolved Hide resolved

brandonocasey force-pushed the feat/tba-lhls branch 6 times, most recently from af9912b to cf2030e Compare

May 20, 2019 18:18


          feat: transmux before append (TBA) and low latency http streaming (LHLS)

41aadb3

brandonocasey force-pushed the feat/tba-lhls branch from cf2030e to 41aadb3 Compare

May 20, 2019 18:20

brandonocasey changed the base branch from tba-master to master

May 20, 2019 18:21

gkatsev approved these changes

View reviewed changes

gkatsev merged commit f077d1e into master

brandonocasey deleted the feat/tba-lhls branch

June 25, 2019 15:35

gesinger mentioned this pull request

fix: handle rollover and don't set wrong timing info for segments with high PTS/DTS values #1040

Merged

6 tasks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet