State of the library in comparison to OpenAL Soft? #3
Comments
|
Thanks! The first thing to keep in mind is that mini_al is not a 3D audio API, so if that's something you need then it won't be suitable (or you'll need to do your own mixing). Besides that, there's still a bit of work to do before it's on par with OpenAL-Soft:
I took a quick look at your code. You'd need to do a bit of work to implement your own mixer. mini_al does not do any sound/source/buffer management like OpenAL, so all of the logic for mixing the audio data of currently playing sounds and music will need to be done manually. Is volume and pitch the only effects you apply to sounds and music? From what I can tell, it looks like you've actually got fairly simple audio requirements so your own mixer wouldn't be too difficult. If you like, I can probably get something quick and dirty up and running for you as a proof of concept? If I can get that working, and then have somebody test/fix the OpenAL back end on Mac, it may actually work... I also noticed you're using my dr_flac library. Nice :) 👍 |
|
Hi @mackron! Thank you very much for the quick answer! :D
That's not a big issue, I can keep my OpenAL Soft backend on those platforms... I neither have a Mac to test it, all raylib testing on macOS has been done by users.
No problem.
That's great, actually, I also tried to implement some functions to format wave data... but very bad quality...
Yeah, that's what I saw... probably that's the difficult part... dealing with sounds/source/buffers was quite easy...
Yes, actually I exposed that for raylib users because it came with OpenAL Soft...
Wow! That would be great! :D
Yeah, it's great! Been following your libs for some time now. raylib is a simple and easy-to-use library primary intended for education, prototyping and small games so, no need for advanced audio features, there are bigger libraries out there intended for bigger projects. :) |
|
I've done some work on this and it seems to work fine so far. You can follow my progress here: https://github.com/mackron/raylib/tree/dr/mini_al (the dr/mini_al branch). I've got Sound, AudioStream and Music working, but I've not tested it thoroughly. It's very quick and dirty and probably has a few bugs, but it shows that mini_al can work. I can polish it up a little bit if you decide mini_al integration might be worth pursuing. A few things:
I also have some suggestions for improving your API design in a few places, but I think I'll raise a separate issue about that one... Have fun playing around with this. Let me know if you have any issues! |
|
Hi @mackron! Amazing work! Thank you very much for your time and your effort on this feature! About your comments:
Undoubtly it worths it! Replacing such a big dependency as is OpenAL Soft by a more lightweight, portable and faster alternative is a goal!
No worries, it's not crucial... I think it's not that easy, it involves some Fourier Transformation... Here I found a good reference and some code.
No big issue, it can be adjusted to a lower base value.
Yeah, I know... the infamous
That's great, allowing the user to choose between both as a transition step is the best idea!
Ok.
Ok.
That's great!
That's really great! :D
Amazing!
Don't hessitate to open an issue or just send the improvements with a pull request.
I tried compiling it with MinGW32 (GCC 5.3.0) and I got some errors: You can send me a pull request to my develop branch if you want, I will need some time to review all the changes and understand how everything works but after a quick look it seems quite clear and comprehensive. |
|
Thanks for that MinGW compiler error - I'll get that fixed up. I'll do a bit of polish and more testing before I doing a pull request. Not sure when that'll be, but I won't forget :) The complexity with pitch shifting is keeping the original sound the same length. I can easily change the pitch by simply running every frame through a sample rate converter, but that lengthens or shortens the sound, depending on the direction of the shift. I'll take a look at those references you provided. My original intent for the USE_MINI_AL macro was mainly just for testing and safety in case mini_al doesn't work out too well. In the future when the mini_al version matures I would just remove the OpenAL version entirely just to keep it simpler. |
Great! No hurries, take your time! :) |
|
I've updated the dr/mini_al branch with a fix for the first compiler error, but regarding this error:
That's one of the headers for the WASAPI backend. You need to update your Win32 API. Look at the "w32api" download on this page to get you started. In the short term, add "#define MAL_NO_WASAPI" before the implementation of mini_al (in mini_al.c). Let me know how that goes! As a side note, I'm going to start using the "__has_include" statement with GCC (and any other compiler that supports it) to cleanly exclude backends when the compiler does not have the necessary development packages installed. |
|
Hi @mackron, thanks for the quick solution! I'm updating my MinGW headers and libs, it seems some packages were missing. |
|
I just pushed some changes to branch dr/mini_al which adds support for pitch shifting. The mini_al backend should now be at feature parity. Were you ever able to get it working on your end? Pull request coming soon. |
|
Ok, managed to compile it and it works flawlessly. 😄 Just tried with mingw32 updated to latest available version (GCC 6.3.0); neither MinGW32 or TDM-GCC-32 toolchains include the required libraries for #define MAL_NO_WASAPI
#define MAL_NO_DSOUND
#include "external/mini_al.h"but it didn't work, When compiling Again, thank you very much for all the hard work! It's trully amazing! |
|
I'll get those issues sorted out. Thanks. Strange that they don't come with dsound.h and audioclient.h... I've never known a distribution of MinGW to not include headers for DirectSound... |
|
I'm just looking at these compiler issues you're having, and indeed it looks like the old 32-bit versions of MinGW do not include dsound.h nor audioclient.h (why?!). The 64-bit version of TDM-GCC definitely includes them, though, because that's what I use for GCC on Windows. Also, you shouldn't comment out those MAL_SUPPORTS_DSOUND and MAL_SUPPORTS_WASAPI lines. I also just wanted to clarify something. When you say #define-ing MAL_NO_DSOUND and MAL_NO_WASAPI didn't work - what exactly didn't work? What errors were you getting? I just tried on that same version of MinGW you're building against and it works fine (except for those other warnings you mentioned). |
|
Sorry, my fault, double checked it... I was defining on #define MAL_NO_WASAPI
#define MAL_NO_DSOUND
#include "external/mini_al.h"but not on Yeah, I also checked MinGW-w64 and it comes with all required libraries and headers, actually, the different MinGW32 versions also include the Probably I should use MinGW-w64 32bit version of the package, more updated and complete than the old MinGW32 project... |
|
I can confirm that years ago, my mingw setup did not include any DirectX header/library at all. |
|
I've updated the dr/mini_al branch with fixes for those warnings.
I had been using old MinGW (not MinGW-w64) for ages and never realized it didn't have dsound.h! |
|
I've been trying raylib with mini_al in multiple platforms and those are the results: PLATFORM_DESKTOP (Windows 10 64bit) Everything works great. PLATFORM_RPI (Raspbian Stretch) Compilation worked ok, just required Unfortunately, when running audio examples I got a PLATFORM_ANDROID (Android NDK API 16) Everything compiles fine, APK is generated fine but when running no audio is played. No crash or weird behaviour, just no audio. Compilation output: PLATFORM_WEB (emscripten SDK 1.37.21) Fails on compilation, tries to compile with OSS backend (Linux). Compilation output: On PLATFORM_WEB it could probably be compiled for OpenAL backend (supported with no additional linkage). Is there any method to force a specific audio backend? Something like Also, is there any method to retrieve Audio Device information? It would be great to get some information similar to Graphic Device info: |
There is currently no way to retrieve the name of the device from a mal_device object. I'll consider this... I'll some tracing to help with debugging these issues. Some of those warnings have been fixed by the way - synchronize your local copy of the dr/mini_al branch :) |
|
Wow! Super fast response! Thanks for the answer! :D
Here the backtrace I got with GDB: |
|
OK, I've pushed some changes which includes some logging of some device information. Are you able to update to this new version and show your output for the Raspberry and Android builds? Regarding Emscripten, unfortunately I don't think it's going to work because mini_al is an asynchronous API and uses a background thread, which if my understanding is correct, Emscripten does not support. I also use dlopen/dlsym which I understand is also not supported (though I can easily enough work around that by linking at compile time for Emscripten builds). I'm not sure what to do with Emscripten... Still working on the Rasberry and Android issues. I may need some help with those ones, though... Another question: What model of Raspberry Pi do you have? UPDATE: I've pushed a potential fix for the Android build. It was trying to dynamically link against pthread at run-time which fails. Tested against API level 16 on an emulator. Could you try that one again? Regarding Emscripten, I'm going to do some research into an SDL backend and see how that works. UPDATE 2: I've pushed a very experimental change that adds an SDL backend in an attempt to get Emscripten working. I've not had a chance to test and debug it properly, but could you give that version a go? You'll need to do |
|
Hi @mackron, thank you very much for the update! Don't know if I understand it correctly... about emscripten, it support pthreads, actually, physic examples run on web using multiple threads. Current OpenAL Soft backend works ok in web. Would it be possible to just force OpenAL mini_al backend on emscripten? emscripten supports OpenAL out of the box without any special linking flags or similar (emscripten-core/emscripten#666). About Android and Raspberry Pi, you have all my help, I can not test it this weekend but on Monday morning I can work on that. I'm usually testing on RPi 1 B+ with Raspbian Jessie and RPi 3 with Raspbian Stretch. I think using emscripten SDL backend implies linking agains all SDL library, that's a bit overloading... Actually, raylib is kind of SDL alternative. |
|
When researching I saw that link, and in particular, I was looking at this:
mini_al should Just Work, and not having pthreads enabled by default (and working reliably) makes things too hard and too annoying.
Nope. Take a look a look at the latest update on dr/mini_al :-) I can get a slightly modified version of my simple_playback example (just to get it working with Emscripten) working with this:
The necessary packages come with Emscripten and there's no need for any special compile time flags. On non-Emscripten platforms the SDL backend requires no linking and no headers. It Just Works without any need for installing any SDL development packages. It's also the lowest priority backend so it won't be picked by default in most cases (only on Emscripten). It links to SDL2.dll at run-time and if it doesn't find it, silently ignores SDL and tries the next backend. |
raylib physics examples work with multithreading with no extra parameter or enabling anything in particular, just linking with
Yes, emscripten comes with a bunch of libraries already compiled to asm.js, in most cases there is no need to link explicitly with those libraries, emscripten detects automatically its use and links, that happens for library_openal.js and also for library_sdl.js. So I proposed just falling to OpenAL backend, it should work out-of-the-box. Tomorrow I'll take a closer look to all platforms implementations (Android, RPI, HTML5). |
|
How is browser support for pthreads, do you know? It makes me nervous to have the Emscripten build be completely dependent on a system where the official documentation says it's still in the prototyping stage. In my mind it makes so much more sense to use SDL because it's backend is way simpler than OpenAL and it avoids the threading issue entirely. It doesn't add any complexity to the build system either. I just don't see the downside... I might try pthreads+OpenAL at some point as an experiment, but I doubt it'll be as good as the SDL solution. Also, I ordered a Raspberry Pi 3b :-) |
|
Hi @mackron, just tested current mini_al build on RaspberryPi 3, Android and HTML5. On Raspberry Pi 3 got same result as before, here it's the output: On Android device just got no audio, actually, it fails even before showing the audio devide information, adding some // Keep the device running the whole time. May want to consider doing something a bit smarter and only have the device running
// while there's at least one sound being played.
result = mal_device_start(&device);
if (result != MAL_SUCCESS)
{
TraceLog(LOG_WARNING, "[MINI_AL] Could not start audio device");
mal_device_uninit(&device);
mal_context_uninit(&context);
return;
}On HTML5 now it compiles perfectly but when running I got the following Javascript output: About your questions:
Actually, I don't know exactly how it works but here I found some related information: emscripten-core/emscripten#5533 Checking your current implementation for SDL backend, I think it could be very similar with OpenAL backend, just |
|
So the Android build is failing when it tries to start the device? Interesting... Are you trying that on real hardware or an emulator? If an emulator, can you give details so I can try the same one? The HTML5 issue is due to me being stupid - I have an SDL 1.2 fallback (which is the default for Emscripten builds) in case SDL 2 isn't supported, but 1.2 doesn't support floating point. Will get that fixed up for you so you can try again. Regarding the OpenAL stuff, if you want to do that, check for MAL_NO_RUNTIME_LINKING, and if defined, just #include the standard headers rather than inlining the function declarations. The reason is that it's most likely the compiler will have the AL headers if they are linking at compile time. This makes inlining the function declarations unnecessary, and in turn reduces maintenance costs for mini_al. UPDATE: I just pushed a potential fix for the Emscripten build to dr/mini_al for you to try. Will look at the Raspberry and Android stuff later... I've tested the Android build on emulators, but I don't have an easy way to test on real hardware... Do you have a log I can look at for the Android build? |
|
Hi @mackron! Wow, what a fast response! :D
I'm trying it on a device, a Motorola MotoG (XT1032) with Android 5.0.
Hehehe, no worries, it happens to me all the time! :P I'll try it and let you know... I will also try to update the OpenAL backend with
I can extract the full running log but unfortunately there is not much info about audio there... in any case, I'll port it here ASAP. |
Tried it and got this message (included by me on audio device start): Here the Android logcat output showing warnings and errors: |
|
Thanks for that! I've got a fix in now for both Android and SDL. The problem was that mal_device_start() was simply returning the wrong value for those backends (they run on a slightly different code path to the other backends and I didn't notice till now). Evidently I need to update my examples and tests... Try that one out and see how you go! |
|
Hi @mackron! Tested new changes! Unfortunately, not working yet... it crashes on Android and HTML5. Android logcat output (relevant parts): HTML5 console output (relevant parts): |
|
Thanks! How frustrating... Potential fix is in. Assuming that was it, that was actually a bug in the new mixing code I did for raylib, not mini_al. Once you get a chance to test that out and you're happy with the results I'll bump the version of mini_al and do a pull request. |
|
Hey @mackron! GREAT NEWS! It works! :D On Android platform it works perfectly. On HTML5 platform it works ok but there seems to be some problem with buffer initialization and there is some delay in sound... you can check a music example (weird init behaviour) and sound example (long delay before playing). On Raspberry Pi 3 platform it works but sound seems to be reproduced at wrong sample-rate, extremely slow playing. Many thanks for your patience and your hardwork on this issue! |
|
Well at least Android is figured out... I'm not sure what's causing the issues with Emscripten and Raspberry Pi. I'll need to look at that in more detail later. Do you hear any glitching with Raspberry Pi, or is it just slow like a pitch shift? |
|
Tried Raspberry Pi more carefully and I'm afraid I can not explain in detail how it sounds... it seems kind of pitch shift with some audio saturation noise... but what I did was measuring RPI music playing time in comparison to original song played in Windows version and, in Raspberry Pi, it took approximately twice the time to play the same song. EDIT: Just in case it could be useful: Output Windows: Output Raspberry Pi: |
|
OK, I've pushed some potential fixes for RPI and SDL, but there's a few things I need you to do for me. Raspberry Pi
When I use a USB output device everything works perfectly fine for me on RPI. It's only the built-in analogue output that has issues for me (though, I can't test HDMI output because I don't have a device to test with). Emscripten Regarding the issue with what sounds like a buffer initialization glitch, assuming we're talking about the same thing, I think that issue is not so much a bug with the audio system, but rather the way in which the music stream is updated. It looks to me in my testing that the first call to Could you also show your output again like you did in your previous message? I've update it with some more detailed information. |
|
Hi @mackron, thanks for the quick update! Not sure if I could look at this during the weekend, I've got my RPIs at work... maybe I could try the Emscripten version... in any case, I'll let you know my results ASAP. |
|
I'm testing everything right now, just a detail:
I use HDMI output by default, I can also try analogue output... More info in a while! |
|
Ok, everything tested, here my results: Raspberry Pi 3 First I tried all proposed solutions with no luck, also tried analogue jack output vs HDMI output, jack output has some interferences/noise, HDMI output was clear. After some tests, found the solution to get it working: That was the minimum value to get it working in par with Windows desktop. Output info: HTML5 (emscripten) Tried modified code and the delay is way shorter but you can hear some glitches/noise on music and sounds. After that tried with line commented and Output info: The only problem I find is that linking against full SDL2 generates a ~9MB file instead of the ~900KB file generated when using OpenAL Soft backend... |
|
Thanks for your feedback on this! A buffer size of 32768 is huge - what is the latency like with that? In any case, I'm suspecting it's just because the RPI has crappy built-in audio which isn't suited for low-latency applications. I'm just not sure how I could detect the RPI at run-time and automatically tune it. I really don't want applications to require a compile time macro like "PLATFORM_RPI" - I want mini_al to Just Work. If you have any ideas, let me know! A 9MB output for HTML5 is no good - I'll look into that. Does emcc have a "-s" option like GCC? Does that do anything? What is the output size with "-O2"? I will look into the SDL issues and try getting OpenAL working with Emscripten, but I'll be out of action on this project for the next few days so I don't know when I'll get a chance to get to it. |
|
@mackron https://raspberrypi.stackexchange.com/questions/754/how-can-i-detect-that-im-compiling-for-raspberry-pi ?
|
Yes, it's quite big... but actually latency is not that bad, when playing a sound there is a delay between key pressed and playing of around 0.5 seconds.
Actually not many ideas but I remember a similar issue with OpenAL related to // The size of a streaming buffer must be at least double the size of a period.
unsigned int periodSize = device.bufferSizeInFrames / device.periods;
unsigned int subBufferSize = AUDIO_BUFFER_SIZE;
if (subBufferSize < periodSize) {
subBufferSize = periodSize;
}I saw
There is no
No worries @mackron, I'm also quite busy lately on my current job... EDIT: @r-lyeh, I think @mackron means not requiring any platform dependent check to get it working correctly, just get a generic solution. |
|
I can't reproduce the crackling issue on my machine for SDL. Could you perhaps try playing around with that |
|
I've pushed a potential fix for RPI, but I was hoping you could test something out for me. I have some logic in mini_al where it scales the default buffer size depending on known audio devices (currently only the RPI audio devices are used). I've currently got this set the 32, but I'm not sure if that's over the top. Would you be able to play around with those numbers to see what values work for you? The lower the better... Said code looks like the following, at about line 5484. "bcm2835 ALSA" is the important one. |
|
Hi @mackron, just tested new code. Tried with values: Using HDMI audio output. With Sorry for not being able to provide more detailed information. On loading, output info is similar in all the modes (just changes the I know it's a bit difficult to track those small issues working remotely... but now it sounds pretty good to me, maybe we can integrate those changes and keep working from there? |
|
Pull request submitted. Probably best to move this discussion to over there. |
|
Hi @mackron, do you think we can close this issue for now? |
|
Yep, closing. |
Hi @mackron, congrats for such an amazing lib, I've been looking for a library like this for long time now.
I'm thinking about replacing current raylib audio module backend (OpenAL Soft) by mini_al and I would like to know how mini_al compares to OpenAL Soft in features... Actually, I'm mostly just using audio playback for complete loaded sounds and music streaming.
My final goal would be to remove any external library dependency for raylib, I mean, include with raylib basecode all required dependencies.
The text was updated successfully, but these errors were encountered: