Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Live Stream to HTML5 Video Tag #598

Closed
gehatg opened this issue Jan 7, 2017 · 14 comments
Closed

Live Stream to HTML5 Video Tag #598

gehatg opened this issue Jan 7, 2017 · 14 comments

Comments

@gehatg
Copy link

gehatg commented Jan 7, 2017

Hallo,

Is it possible to use JavaCV to live-stream a sequence of images from a FrameGrabber into an HTML5 Canvas as an html5 video stream (currently OGV, MP4, WEBM are supported from browsers). I can see that FFMPEG supports http streaming using an expiramental http server: https://ffmpeg.org/ffmpeg-protocols.html#http but how can that be possible using javacv?

Identically could it be possible to use the framegrabber on a servlet and serve Range packets using the FFMPEGRecorder?

@saudet saudet added the question label Jan 7, 2017
@saudet
Copy link
Member

saudet commented Jan 7, 2017

FFmpeg supports the HTTP protocol, sure: https://ffmpeg.org/ffmpeg-protocols.html#http

@saudet saudet closed this as completed Jan 7, 2017
@saudet
Copy link
Member

saudet commented Jan 7, 2017

I'm not sure what you mean by "range packets" though. Could you elaborate on why this isn't possible with now with FFmpegFrameRecorder?

@saudet
Copy link
Member

saudet commented Jan 7, 2017

FYI, there is some sample code here:
https://github.com/bytedeco/javacv/blob/master/platform/src/test/java/org/bytedeco/javacv/FrameGrabberTest.java
Just replace the InputStream by the URL you want to publish from, and setOption("listen", "1") should work to set the option.

@gehatg
Copy link
Author

gehatg commented Jan 7, 2017

Thanks a lot I will try and let you know, this should be able to start an HTTP Server running inside FFMPEG and do the streaming.

My second question was whether I can use my own http server or a web container like tomcat. To do that I will have to implement a Servlet that will respond to Range requests that Chrome and some other browsers will send to download video see example range serlvet here: https://gist.github.com/jsfeng/2629858 .

The Range request will usually have an "Accept-Ranges: bytes" header and it will split the video downloading in many byte range requests. So how can I get bytes from the FrameGrabber or the Recorder and get an encoded n amount of bytes?

What I am actually trying to achieve is to stream a sequence of images that I receive from a socket in JPEG format into an HTML Canvas element as a live video stream.

@saudet
Copy link
Member

saudet commented Jan 7, 2017

I see. You can't expect FFmpeg to know magically how many bytes you're going to have before encoding happens...

@gehatg
Copy link
Author

gehatg commented Jan 7, 2017

I tried your suggestion and I can't seem to start the FFMPEG HTTP Server. I am getting the following exception:

Output #0, webm, to 'http://localhost:9090':
Stream #0:0: Unknown: none (libvpx)
[libvpx @ 000000001b6b41e0] v1.6.0
org.bytedeco.javacv.FrameRecorder$Exception: avio_open error() error -138: Could not open 'http://localhost:9090'
at org.bytedeco.javacv.FFmpegFrameRecorder.startUnsafe(FFmpegFrameRecorder.java:709)
at org.bytedeco.javacv.FFmpegFrameRecorder.start(FFmpegFrameRecorder.java:294)
at TestVideoServer.start(TestVideoServer.java:67)
....

This is how I am trying to initialize the WEBM HTTP Streaming Server:

	recorder = new FFmpegFrameRecorder("http://localhost:9090", imageWidth, imageHeight);
	recorder.setVideoCodec(avcodec.AV_CODEC_ID_VP8);
	recorder.setAudioCodec(0);
	recorder.setPixelFormat(avutil.AV_PIX_FMT_YUV420P);
	recorder.setFormat("webm");
	recorder.setGopSize(10);
	recorder.setFrameRate(20);
	recorder.setVideoBitrate(5000);
	recorder.setOption("content_type","video/webm");
	recorder.setOption("listen", "1");
        recorder.start()

@gehatg
Copy link
Author

gehatg commented Jan 7, 2017

regarding second question the HTML5 tag performs a progressive download so if there is nothing encoded yet we will simply wait until there is and in the range requested, meaning that recording will be done in a separate thread somehow. From my understanding It is not actually streaming but endless progressive downloading of byte array pieces. So is there a way to get an encoded packet based on a range?

@saudet
Copy link
Member

saudet commented Jan 8, 2017

For some reason, it seems like the URLs need to start with "tcp" instead of "http". This works fine here:

FFmpegFrameRecorder recorder = new FFmpegFrameRecorder("tcp://localhost:9090?listen", imageWidth, imageHeight);
recorder.setFormat("webm");
recorder.start();

For your questions concerning FFmpeg specifically, you should ask on FFmpeg's mailing list, not here.

@saudet
Copy link
Member

saudet commented Jan 8, 2017

Actually, FFmpegFrameRecorder wasn't passing the options to the protocol. With the commit above, it now works with the http protocol as well when calling setOption("listen", "1") before start().

@gehatg
Copy link
Author

gehatg commented Jan 8, 2017

Hallo,

I can not build the javacv-master using mvn install:

[ERROR] Failed to execute goal on project javacv: Could not resolve dependencies for project org.bytedeco:javacv:jar:1.3.2-SNAPSHOT: The following artifacts could not be resolved: org.bytedeco:javacpp:jar:1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:opencv:jar:3.1.0-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:ffmpeg:jar:3.2.1-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:flycapture:jar:2.9.3.43-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:libdc1394:jar:2.2.4-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:libfreenect:jar:0.5.3-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:libfreenect2:jar:0.2.0-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:librealsense:jar:1.9.6-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:videoinput:jar:0.200-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:artoolkitplus:jar:2.3.1-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:flandmark:jar:1.07-1.3.2-SNAPSHOT: Could not find artifact org.bytedeco:javacpp:jar:1.3.2-SNAPSHOT -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal on project javacv: Could not resolve dependencies for project org.bytedeco:javacv:jar:1.3.2-SNAPSHOT: The following artifacts could not be resolved: org.bytedeco:javacpp:jar:1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:opencv:jar:3.1.0-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:ffmpeg:jar:3.2.1-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:flycapture:jar:2.9.3.43-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:libdc1394:jar:2.2.4-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:libfreenect:jar:0.5.3-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:libfreenect2:jar:0.2.0-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:librealsense:jar:1.9.6-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:videoinput:jar:0.200-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:artoolkitplus:jar:2.3.1-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:flandmark:jar:1.07-1.3.2-SNAPSHOT: Could not find artifact org.bytedeco:javacpp:jar:1.3.2-SNAPSHOT
at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies(LifecycleDependencyResolver.java:221)
at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies(LifecycleDependencyResolver.java:127)
at org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved(MojoExecutor.java:245)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:199)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.project.DependencyResolutionException: Could not resolve dependencies for project org.bytedeco:javacv:jar:1.3.2-SNAPSHOT: The following artifacts could not be resolved: org.bytedeco:javacpp:jar:1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:opencv:jar:3.1.0-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:ffmpeg:jar:3.2.1-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:flycapture:jar:2.9.3.43-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:libdc1394:jar:2.2.4-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:libfreenect:jar:0.5.3-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:libfreenect2:jar:0.2.0-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:librealsense:jar:1.9.6-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:videoinput:jar:0.200-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:artoolkitplus:jar:2.3.1-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:flandmark:jar:1.07-1.3.2-SNAPSHOT: Could not find artifact org.bytedeco:javacpp:jar:1.3.2-SNAPSHOT
at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve(DefaultProjectDependenciesResolver.java:211)
at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies(LifecycleDependencyResolver.java:195)
... 23 more
Caused by: org.eclipse.aether.resolution.DependencyResolutionException: The following artifacts could not be resolved: org.bytedeco:javacpp:jar:1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:opencv:jar:3.1.0-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:ffmpeg:jar:3.2.1-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:flycapture:jar:2.9.3.43-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:libdc1394:jar:2.2.4-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:libfreenect:jar:0.5.3-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:libfreenect2:jar:0.2.0-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:librealsense:jar:1.9.6-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:videoinput:jar:0.200-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:artoolkitplus:jar:2.3.1-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:flandmark:jar:1.07-1.3.2-SNAPSHOT: Could not find artifact org.bytedeco:javacpp:jar:1.3.2-SNAPSHOT
at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:384)
at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve(DefaultProjectDependenciesResolver.java:205)
... 24 more
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: The following artifacts could not be resolved: org.bytedeco:javacpp:jar:1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:opencv:jar:3.1.0-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:ffmpeg:jar:3.2.1-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:flycapture:jar:2.9.3.43-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:libdc1394:jar:2.2.4-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:libfreenect:jar:0.5.3-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:libfreenect2:jar:0.2.0-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:librealsense:jar:1.9.6-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:videoinput:jar:0.200-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:artoolkitplus:jar:2.3.1-1.3.2-SNAPSHOT, org.bytedeco.javacpp-presets:flandmark:jar:1.07-1.3.2-SNAPSHOT: Could not find artifact org.bytedeco:javacpp:jar:1.3.2-SNAPSHOT
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:444)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:246)
at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:367)
... 25 more
Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact org.bytedeco:javacpp:jar:1.3.2-SNAPSHOT
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:434)
... 27 more
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following arti

@saudet
Copy link
Member

saudet commented Jan 8, 2017

Reset the version to 1.3 and try again.

@gehatg
Copy link
Author

gehatg commented Jan 8, 2017

Changed pom.xml to 1.3

Now getting same for 1.3:

[ERROR] Failed to execute goal on project javacv: Could not resolve dependencies for project org.bytedeco:javacv:jar:1.3: Failure to find org.bytedeco.javacpp-presets:libfreenect2:jar:0.2.0-1.3 in https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal on project javacv: Could not resolve dependencies for project org.bytedeco:javacv:jar:1.3: Failure to find org.bytedeco.javacpp-presets:libfreenect2:jar:0.2.0-1.3 in https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until th

@saudet
Copy link
Member

saudet commented Jan 8, 2017

Right, you can remove that since you don't need it.

@saudet
Copy link
Member

saudet commented Jan 15, 2017

I've just released version 1.3.1 with the fix, so no need to build from source. Enjoy!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants