Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Video Streaming framerate infinity #165

Open
frapit opened this issue Nov 25, 2021 · 2 comments
Open

Video Streaming framerate infinity #165

frapit opened this issue Nov 25, 2021 · 2 comments

Comments

@frapit
Copy link

frapit commented Nov 25, 2021

Hi,

the video stream in some bag files appears to be broken, I think the calculation of the frame rate in

may need some additional checking.

bagdb_1     | 2021-11-25 08:54:09.273 [mvcTaskExecutor-18] INFO  c.g.swrirobotics.bags.BagService - Starting video stream.
bagdb_1     | 2021-11-25 08:54:09.328 [mvcTaskExecutor-18] DEBUG c.g.swrirobotics.bags.BagService - Image format: bgr24 / 1536x1536 / 0.0s / Infinity Hz
bagdb_1     | 2021-11-25 08:54:09.331 [mvcTaskExecutor-18] INFO  c.g.swrirobotics.bags.BagService - Beginning to stream image data to ffmpeg.
bagdb_1     | 2021-11-25 08:54:09.331 [Thread-127] DEBUG c.g.swrirobotics.bags.BagService - Piping data from ffmpeg to the client.
bagdb_1     | 2021-11-25 08:54:09.398 [Thread-127] DEBUG c.g.swrirobotics.bags.BagService - Finished processing output from ffmpeg.
bagdb_1     | 2021-11-25 08:54:09.398 [mvcTaskExecutor-18] ERROR c.g.swrirobotics.bags.BagService - Error encoding video:
bagdb_1     | java.io.IOException: Broken pipe
bagdb_1     | 	at java.base/java.io.FileOutputStream.writeBytes(Native Method)
bagdb_1     | 	at java.base/java.io.FileOutputStream.write(FileOutputStream.java:354)
bagdb_1     | 	at java.base/java.io.BufferedOutputStream.write(BufferedOutputStream.java:123)
bagdb_1     | 	at java.base/java.io.FilterOutputStream.write(FilterOutputStream.java:108)
bagdb_1     | 	at org.apache.commons.io.IOUtils.write(IOUtils.java:2987)
bagdb_1     | 	at com.github.swrirobotics.bags.BagService$FfmpegImageHandler.process(BagService.java:535)
bagdb_1     | 	at com.github.swrirobotics.bags.reader.BagFile.forMessagesOnTopic(BagFile.java:395)
bagdb_1     | 	at com.github.swrirobotics.bags.BagService.writeVideoStream(BagService.java:806)
bagdb_1     | 	at jdk.internal.reflect.GeneratedMethodAccessor479.invoke(Unknown Source)
bagdb_1     | 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
bagdb_1     | 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
bagdb_1     | 	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
bagdb_1     | 	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197)
bagdb_1     | 	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749)
bagdb_1     | 	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
bagdb_1     | 	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688)
bagdb_1     | 	at com.github.swrirobotics.bags.BagService$$EnhancerBySpringCGLIB$$93f073e2.writeVideoStream(<generated>)
bagdb_1     | 	at com.github.swrirobotics.bags.BagController.lambda$getVideo$0(BagController.java:189)
bagdb_1     | 	at org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBodyReturnValueHandler$StreamingResponseBodyTask.call(StreamingResponseBodyReturnValueHandler.java:110)
bagdb_1     | 	at org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBodyReturnValueHandler$StreamingResponseBodyTask.call(StreamingResponseBodyReturnValueHandler.java:97)
bagdb_1     | 	at org.springframework.web.context.request.async.WebAsyncManager.lambda$startCallableProcessing$4(WebAsyncManager.java:326)
bagdb_1     | 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
bagdb_1     | 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
bagdb_1     | 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
bagdb_1     | 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
bagdb_1     | 	at java.base/java.lang.Thread.run(Thread.java:834)
bagdb_1     | 2021-11-25 08:54:09.399 [mvcTaskExecutor-18] ERROR c.g.swrirobotics.bags.BagService - ffmpeg output:
bagdb_1     | [rawvideo demuxer @ 0x5652427c55c0] Unable to parse option value "Infinity" as video rate
bagdb_1     | [rawvideo demuxer @ 0x5652427c55c0] Error setting option framerate to value Infinity.
bagdb_1     | pipe:0: Invalid argument
bagdb_1     | 2021-11-25 08:54:09.399 [mvcTaskExecutor-18] INFO  c.g.swrirobotics.bags.BagService - Done streaming video.
@pjreed
Copy link
Contributor

pjreed commented Nov 29, 2021

Hmm, good catch. Looking at the code, while that's possible, I think it would be pretty weird for that situation to happen. I think you would need to have a bag file with an image topic that has at least two frames, but every frame's header would have to have an identical timestamp, causing it to think that that the average period between frames is 0. Do you know if that's the case here?

If a topic doesn't have reasonable timestamps on its image messages, it might not be possible to determine the frame rate for the topic, but I suppose we could just make something up or ask the user for a default frame rate.

Edit: I also think it'd be possible to calculate the frame rate based on the time messages were recorded in the bag file, although it'll take a little work on the bag reader library to make that accessible at this place in the code.

@frapit
Copy link
Author

frapit commented Dec 1, 2021

Thanks for the quick answer. Yes you are right, I just checked some of the bags for which the video stream is broken and they all have all timestamps zero on the image topics. That probably causes the issue. I am wondering why rtq tools, etc. have no problem with such messages. Anyway, I think a reasonable default framerate in that case should be sufficient.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants