-
Notifications
You must be signed in to change notification settings - Fork 10k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[YouTube] [SOLVED] 3D/2D 360° videos - now encoded in a new, proprietary & non-standard format #15267
Comments
Is there an existing tool for such conversion? I guess |
No existing tools that I'm aware of. As I explained, this proprietary format is brand-spankin' new. I found an article on Google Blog from March 2017 (https://blog.google/products/google-vr/bringing-pixels-front-and-center-vr-video/) where they discuss this new format (called the Equi-Angular Cubemap, or EAC). Essentially, it takes the 360° dome panorama (that would have originally been uploaded via the equirectangular format in my first picture), and reencodes it so that every side of the dome that is projected is now the face of a cube. Which, if you look at the second picture I posted, is exactly what is happening. The screen is split in half: left half for left eye, right half for right eye. Then each half is split into six quadrants, each representing one face of a cube:
So, even though Google has been working on this new format for a while, it has only gone live on YouTube as of approx. a week ago. So we can do one of three things now:
|
The new format still doesn't look right on a cube. It is more of a curved cube FYI. Still trying to figure this one out =/ What they documented in March 2017 doesn't match the format of the videos we see today. |
Guy Godin! Developer of Virtual Desktop! I had no idea! I saw your post today on the Oculus Rift Reddit forums about Virtual Desktop 1.9 and how it was compatible with YouTube's cube format, so I immediately went out and bought Virtual Desktop on Steam, and installed the beta! You're right, YouTube's current implementation must not be 100% accurate to their documentation. I used Virtual Desktop tonight to watch some of the EAC videos I downloaded using youtube-dl, and yeah, you're 99% there, but the seams between the cube faces are slightly warped. Not too noticeable, until you notice them, and then they stick out like a sore thumb. However, the EAC videos are at least in a watchable state outside of YouTube and that's great. Anyways, awesome work you're doing. I love my Oculus Rift for all the cool VR experiences like VR Chat and all, but I definitely enjoy watching some awesome VR videos. Your player is definitely the best I've used for the Rift, so thank you for your hard work!! |
Thanks! Yeah the implementation I have right now is not 100% perfect as I had to fiddle with the UVs manually until it looked right so it’s really just an approximation of what Google engineered. Hopefully they’ll release more details about the projection they are using. I’m not a fan of the new format, the quality is inferior to the old one (most likely because they need to re-project from an equirectangular source which makes the image more blurry). So until they have tools to generate their Cube at the source, it will look worse. |
That is true about the re-encoding worsening the quality. For some of the videos I've produced, I render at 3840x4320, and upload it to YouTube at 120Mbps, so by the time YouTube does it's encoding for streaming, the quality hit when played at 2160s60 won't look too bad. But yeah, anytime an encoding happens, quality is going to be lost (unless you're encoding losslessly, but that's waaaay too large of a bitrate to be viable). Yeah, hopefully the EAC format either a) goes away or b) becomes so big that it becomes a standard for all software for panoramic videos. |
Lol... looks like YouTube's experiment with EAC might be ending. I just downloaded a brand new 3D 360° video today and it was in the old, standard equirectangular panorama format. I'll post again in a week or so to confirm or deny today's findings. |
Cubemap 360 video is an open standard too, just not as common. Cubemaps' advantage vs. equirectangular projection is more resolution in the middle of the frame, where you're likely to be looking. The disadvantage is that the stored image no longer makes intuitive visual sense to a human. |
Oh yeah, I fully realize and can appreciate the advantages that cubemap projection has over equirectangular. The problem lies in the following: The desired projection standard needs to be universally accepted and utilized at all steps of the video production/distribution pipeline. If YouTube wants their superior EAC format to work, they need to make sure that 360° cameras can capture the footage in EAC, and (for animation) that all rendering software can render the image in EAC. Then, YouTube needs to allow videos that are in EAC format to be able to be uploaded to YouTube at a good high bitrate for YouTube to re-encode into the different quality levels that they do. Right now, for users who wish to upload a 360° video to YouTube, there are two choices: a) You upload the video (injected with YouTube's metadata injector) in the equirectangular projection format. This defeats the whole purpose of the EAC format to begin with, because you're starting with a potentially lower quality source and encoding it into a higher grade format. I like the metaphor someone used on the Oculus Rift Reddit forums comparing it to someone who re-encodes a 128kbps mp3 file to 320kbps... The quality isn't going to get any better! And every re-encode just makes the quality worse, so these EAC re-encodes just end up making the video blurrier than if it had just remained in its equirectangular original. b) You get one of the few (if any) devices that support YouTube's new "Jump" platform: https://vr.google.com/jump/ But yeah, I think YouTube's realizing that the EAC format was pushed out too soon and doesn't improve the quality of videos uploaded in equirectangular to begin with, because every new 360° video that has been uploaded in the past few days (since my last post) that I have downloaded with youtube-dl has been in equirectangular panorama. |
I have tried to download few 360 videos over the last week and I've always found them to use the new EAC cubemap format. |
@PriteshDesai Check the video upload date. If the 360° video was uploaded prior to January 27th, 2018 (approximate), it will likely have been encoded (or re-encoded) to the EAC format by YouTube. However, it appears that they stopped doing so for any 360° videos uploaded since January 27th. All new 360° videos that I've downloaded after that date (that were uploaded after January 27th) have been in the old standard equirectangular format. |
@Cyroxchan Thanks for the clarification. I was able to pass a blank "browser user-agent" via a custom C# script. I was able to download videos in Equirectangular format for - 2K resolution MP4 and 2K and 4K resolution MKV. However when I pass a browser user-agent, Youtube serves 4K MP4 and 4 K MKV video but switches to an EAC format. I am looking for 4K resolution MP4. |
this is plain stupid. |
Solve it, in Unity and a little bit of Video editing. |
@adrianposteuca Sorry for disturbing, but did you solve that problem ? |
@Uncleout Yes, but is not a stable solution... it works only in Unity. Need a conversion from unity again to Equilateral. |
this is very interesting, I wonder if they tag the files correctly with their mesh standard (https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md) |
@adrianposteuca Can you explain how you solved it in unity ? Thanks |
Found a solution, thanks to @PriteshDesai
Using Virtual Desktop, I could play the videos and they don't wrap or chop incorrectly, seems these programs are downloading older versions of the YouTube video and they aren't formatted the new way. |
thanks @PriteshDesai, with |
Hi everyone, Thanks |
@j0hj0h & @PriteshDesai i got 4K equirectangular in |
The Unity 360 player, only displays regular 6 cubemap videos properly. The YT cubemap isn't natively supported. Has anyone been able to display the Youtube cubemap properly in Unity? |
I'm on this path at the moment, trying to get it to work in Unity myself. I managed to get the transformations to get the right faces into the right locations, but now it's the warping that I have to fix. |
Hello Guys, what I have achieved so far is to create a 3D rig made up of two parts: In a video editing software, I cut the video in half on horizontal, so I got two videos for A and B. It works on some videos, but still has a problem with lens since is cube seen with a curved lens |
That sounds like what I have so far, except I've done it with a single shader on a single cube (or, as of right now, the skybox). The problem is, it's still distorted because I'm not applying the mapping required to undo the EAC mapping. |
@trejkaz @adrianposteuca How about creating an open source skybox for Youtube? We can all contribute to it and make it available for all. |
Now my skybox code has the correct transform to undo the EAC mapping, but the faces don't quite align correctly. I'm assuming YouTube adds some bleed somewhere and that this bleed isn't documented in anything that I've read so far so I don't know how to avoid it when transforming. The current release is here. The code in there is written for CG but may apply for any C-like language which wants to do similar calculations. The other option was to use the new mesh standard. I had two problems with using this for my own project:
Not using the mesh standard may even be why there is a minor issue with alignment still. I'm hoping that getting this initially version out there will encourage some sharing of solutions. :) |
So, I'm probably a little late to the game here, but thanks to the work and explanations of @PriteshDesai and @j0hj0h the solution has been discovered! In order to download the originally uploaded equirectangular panorama format version of the 360° video using youtube-dl, include the following option in your command-line: --user-agent "" Returning to my original example video, https://www.youtube.com/watch?v=8DqSh_T20WY Which, when played, shows the correct equirectangular format once again: YouTube's EAC re-encoding has been thwarted! Thank you everyone for your hard work! Now, on to another YouTube VR-video-related issue! The VR180 format has some hidden streams that are not being picked up by youtube-dl. Link to issue here: #16647 |
The trap there is that the equirectangular copy can be a lot fuzzier than the EAC copy. (This is why I spent time to make a player specifically for the layout I was getting.) Example:
Best copies are 313 and 266. 313 is reasonably sharp, but is EAC. 266 is equirectangular, but too fuzzy to enjoy. And then if you add
All it seems to do is remove the best quality equirectangular option from the list. :) |
@trejkaz In your second stream list (where you added --user-agent ''), where 266 has been removed, did you try downloading stream #313? I think you'll be pleasantly surprised to find that by passing --user-agent '', the stream is no longer EAC, but Equirectangular! 😄 FILESIZE NOTE: In your first list, stream #313 is 363.84MiB (EAC version). In the second list, stream #313 is 363.21MiB (Equirectangular version). And yeah, the #313 (for 30fps) or #315 (for 60fps) stream is in almost all cases the best quality version to download -- the vp9 codec is vastly superior in quality compared to the avc1 codec. Stream #313 is as sharp in Equirectangular as it is in EAC. Screenshot below taken from stream #313 video downloaded using: |
Yeah, OK. It does send a different video after all. I didn't expect to get a different video when I provided the exact same format ID, because I assumed something called an ID would behave like an ID. :/ So my only concern left is the quality, right? Both videos are packing the information into 3480x1920, but the EAC one should be fitting better quality in the same size. (Assuming that they introduced it for this reason, which seems to be what the blog posts are claiming.) |
For sure, the EAC one should, in theory be better quality. It really all depends on the size and quality of the original upload. If, for example, the author uploaded their video at 3840x2160 resolution in equirectangular format, then there is essentially no way the EAC version of the video at 3840x2160 is going to be better quality. If the author uploaded the video at a higher resolution, say 7680x4320 resolution in equirectangular format, then the EAC version of the video at 3840x2160 could end up better quality than an equirectangular one at the same resolution. EAC has its merits for sure, but it's not universally accepted. Until it is, I'll prefer my locally saved files in equirectangular. |
Guys on YouTube, the EAC is implemented only on 1080s quality and above. At 720s and below, the old equirectangular format only is used. So you can download it in 720s (It looks like 240p quality but it's atleast viewable !) |
@shreytyagi Good to know, but the solution has already been discovered and described above. You can download 360° 4K streams in equirectangular, just by passing |
(accidentally closed - i am noob) |
@Jithug6 Potplayer didn't work with my video |
Hi, Cyroxchan. I used your command: youtube-dl -f 313+171 --merge-output-format mkv [URL]--user-agent '' . For some 360° videos ,I can download the 360° 4K streams in equirectangular. but for some 360° videos, with the same command, the downloaded 360° 4K streams is in EAC. |
How to watch YouTube's ridiculous new 360 3D video format on your computer:
|
Is There really any solution for this problem ????? |
Goddamn GitHub made me delete my comment ... Has anyone tried VLC ? Or any other player for that matter ? PotPlayer is a Window$-exclusive thing, so that's kinda outta the window. The Firefox extension uses FFmpeg to do the conversions. If FFmpeg has somehow reverse-engineered the new format, that's the place to look at. |
If targeted on Web Player. egjs-view360 can be solution for viewing EAC Video. It supports various types of 360 contents. For example Equirectangular, Cubemap and EAC(Youtube) |
Yest this works!
Done!!!!! |
We have some internal tools at Pixvana (that I wrote) to perform these conversions, and it's all based on the publicly-available YouTube information. So while I probably can't share the internal code, I will say that such a tool is certainly buildable using available information. Wish I could help more directly. |
Sorry I did not read this long topic, I wait for a desktop, with a phone
|
Hi to all, finally I was able to scroll the whole 3d :) Problem with https://www.youtube.com/watch?v=KBQ3ICqpyIs Instead of the right view as you can see here (lowest YT resolution due to following your receipt I got an unacceptable view: Questions: thanks a lot for your opinion. Here what I did: Edit: resolved due to my intuition. |
THX A LOT! it worked for me :) thx so much! |
It's rly works! Thanks! |
I don't know how relevant my question in this thread is but I feel this is the only thread on this repo where I can ask this. I have been looking for this on Oculus forums and am surprised that how/ if noone has encountered this issue before. Here is the issue:
So my question is, is it happening because the video I'm running is downloaded from YouTube and YouTube now has their own proprietary encoding for 360 videos, which cannot be rendered anywhere outside YouTube or its something else? Any help/reference links would be highly appreciated. |
Before submitting an issue make sure you have:
What is the purpose of your issue?
Description of your issue, suggested solution and other information
It appears that approx. one week ago, YouTube has changed its encoding algorithms for panoramic 360° 2D/3D videos.
As a producer of such videos, I am familiar with the standard format used by most 360° video players to convert the flat video image into a 360° panorama (in either 2D or 3D). The standard video format is in "equirectangular panorama" projection for 2D, and "over/under equirectangular panorama" projection for 3D 360° videos. A 360° video producer would format their video to one of the above projections, and then use YouTube's "Spatial Media Metadata Injector" tool (found here: https://support.google.com/youtube/answer/6178631?hl=en&ref_topic=2888648) to inject the code needed to inform YouTube that the video being uploaded is a 360° panoramic video.
Approx. one week ago, YouTube has changed their internal encoding processes to convert uploaded 360° videos to a custom proprietary format (no longer equirectangular panorama). This format now makes it completely useless to download the desired 360° for viewing in best possible quality & on offline video players, as no viewer outside of YouTube would recognize the proprietary format. YouTube has also been re-encoding previously uploaded videos to their new proprietary format.
For example, here is a screenshot from a video (found on YouTube at: https://www.youtube.com/watch?v=8DqSh_T20WY -- I downloaded it on 2017-11-07):
The above image shows the correct over/under equirectangular panorama projection for a 3D 360° video.
However, as of right now, using youtube-dl to download the video ("youtube-dl https://www.youtube.com/watch?v=8DqSh_T20WY -f 315+171 --merge-output-format mkv"), a screenshot of the video at the same timestamp as above looks like this:
I can see three possible reasons for encoding their videos in this new proprietary format:
They don't want people downloading these videos for offline viewing -- they want people to view the video on YouTube. This is a pretty obvious reason, however if true, why haven't they encoded all the types of videos available on YouTube into an unviewable mess? Why only 360° videos? Which leads me to reason 2.
This new format may be faster for their players to process into a 360° video. This would be a fair reason if true, but it makes life miserable for those of us who enjoy downloading the clips for offline viewing in the player of their choice. However, reason 3 is the most likely:
The re-encoded video is designed to put as much resolution as possible in the areas of the video that people are most likely to look at. Some research I read shows that 75% of the time, viewers of 360° videos look in the forward 180° or so of the video and don't really look behind them much. In addition, the equirectangular panorama format gives far too much resolution to the top and bottom of the dome projection, and nowhere near enough to the center. YouTube's new encoding format definitely improves on the standard format -- more real estate given to the front half of the video, and the back and bottom are given significantly less. However, it still doesn't help those of us who want to watch it outside YouTube. To be acceptable, it would essentially require the VR/360° industry to accept YouTube's new format as a standard for watching 360° videos and incorporate it into their players & software. For this to happen, of course, YouTube will need to release the specifications for their new encoding format.
Currently, it appears that there's nothing youtube-dl could do to fix this situation. Thus, this "issue" is a request for new functionality: being able to re-encode the 360° videos from YouTube's new proprietary format back into the standard equirectangular panorama projection that most 360° video players recognize.
Thank you for taking the time to review this issue. Hopefully it leads to improvements in youtube-dl in the future!
The text was updated successfully, but these errors were encountered: