Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

media/media_codec: Replace construct_never_null and infallible unwrap() pattern with something more sensible #248

Merged
merged 1 commit into from
Mar 26, 2022

Conversation

MarijnS95
Copy link
Member

These functions return a pointer directly instead of an error code with the pointer as argument, making them unsuitable for use with construct_never_null. Because there is no error code, returning a manual success value still requires an infallible unwrap() call to unpack the Result.

Replace this with a new helper function that does not unnecessarily wrap the resulting pointer in a Result when there is no result code to base this off, and match the signature to a function that returns the pointer directly.

At the same time reduce the size of some unsafe {} blocks to wrap just what is unsafe.

…rap()` pattern with something more sensible

These functions return a pointer directly instead of an error code with
the pointer as argument, making them unsuitable for use with
`construct_never_null`.  Because there is no error code, returning a
manual success value still requires an infallible `unwrap()` call to
unpack the `Result`.

Replace this with a new helper function that does not unnecessarily wrap
the resulting pointer in a `Result` when there is no result code to base
this off, and match the signature to a function that returns the pointer
directly.

At the same time reduce the size of some `unsafe {}` blocks to wrap just
what is unsafe.
@MarijnS95
Copy link
Member Author

@zarik5 I trust you have checked all these calls wrapped in *_never_null to never return null for the given input arguments (ie. they are always valid)? In release NonNull::new_unchecked is used, which is UB if there are valid cases where these functions return null (ie. I hope you've not picked this function for convenience only).

@zarik5
Copy link
Contributor

zarik5 commented Mar 22, 2022

@MarijnS95 Well no, I didn't check all functions. In general the MediaCodec API requires the user to know what they're doing, because if you call some functions in the wrong context it will probably lead to a crash. For example it's invalid to call AMediaCodec_getOutputBuffer() in encoder mode and AMediaCodec_getInputBuffer() in decoder mode, unless maybe the surface supports host memory transfer. It's too hard to check all valid usages.

@MarijnS95
Copy link
Member Author

@zarik5 Meaning that none or most of these should not call *_never_null(), since they can return null on very real (misuse) scenarios?

In that case these should never be substituted with new_unchecked I think, perhaps even return an Option/Result to the caller...

@MarijnS95
Copy link
Member Author

For example it's invalid to call AMediaCodec_getOutputBuffer() in encoder mode and AMediaCodec_getInputBuffer() in decoder mode

If you want the caller to uphold "undocumented" (I don't see it anywhere in media_codec.rs...) constraints such public should be marked unsafe, fwiw.

@zarik5
Copy link
Contributor

zarik5 commented Mar 22, 2022

For when the function returns a pointer this makes sense. But specifically for AMediaCodec_getOutputBuffer and AMediaCodec_getInputBuffer I don't think we can intercept the invalid usage before a crash. More testing is definitely needed.

@zarik5
Copy link
Contributor

zarik5 commented Mar 22, 2022

If you want the caller to uphold "undocumented" (I don't see it anywhere in media_codec.rs...) constraints such public should be marked unsafe, fwiw.

I'm fine with this.

@MarijnS95
Copy link
Member Author

For when the function returns a pointer this makes sense. But specifically for AMediaCodec_getOutputBuffer and AMediaCodec_getInputBuffer I don't think we can intercept the invalid usage before a crash. More testing is definitely needed.

Would it crash in the native function or just correctly return a null value that we should subsequently propagate to the caller with an Option?

If you want the caller to uphold "undocumented" (I don't see it anywhere in media_codec.rs...) constraints such public should be marked unsafe, fwiw.

I'm fine with this.

Note again that this should only be necessary if we can't do anything else about it. Ie. if the function just returns null on invalid input/state we're better off returning an Option/Result.

@zarik5
Copy link
Contributor

zarik5 commented Mar 22, 2022

Would it crash in the native function or just correctly return a null value that we should subsequently propagate to the caller with an Option?

The the thing is I don't know. I'm afraid it might crash inside the native function. I tested only the decoder path, with the proper usage, and at the moment I put that code pause to work on other things. I might try to do more testing, but it will be still be limited. This is more a problem with the (loack of) official documentation.

@MarijnS95
Copy link
Member Author

MarijnS95 commented Mar 22, 2022

This is more a problem with the (loack of) official documentation.

Yeah indeed.

Let me know if you get to testing, otherwise I might try at some point. We can still merge this PR even though there's not much sense if we end up replacing all *_never_null with something that actually deals with nulls.

I do like to get this done before the next release though, so that these are no breaking changes after the initial API for media_codec.rs has been published :)

@MarijnS95 MarijnS95 merged commit 22a11ae into master Mar 26, 2022
@MarijnS95 MarijnS95 deleted the remove-infallible-unwrap branch March 26, 2022 22:39
@MarijnS95
Copy link
Member Author

Merged this PR for its initial cleanup, @zarik5 one of us should really see about fallibility so that we can fix up the API before the next release.

@zarik5
Copy link
Contributor

zarik5 commented Mar 27, 2022

Unfortunately I can't allocate more time on this. The only help i can provide is this code, which has been tested to some degree but not fully. I started working (for a job) on other fronts.

@MarijnS95
Copy link
Member Author

As such I'm not sure if we should back out of this change entirely, or mark the module private until safety concerns are resolved. Alternatively one could preemptively change the necessary interfaces to return Option, or "hide" those interfaces behind a "possibly-unsafe" feature flag until we're sure whether the interfaces can return null. For example:

  • All construct_never_null usages seem fine: those functions return an error code and Android better not returns success without assigning to the pointer;
  • AMediaCodec_getOutputBuffer and AMediaCodec_getInputBuffer also seem fine: your Rust wrapper enforces them to only be callable with an argument from, and after a successful result from AMediaCodec_dequeueOutputBuffer / AMediaCodec_dequeueInputBuffer which I presume is the main reason why those functions don't otherwise have a valid return code.
    It is however unknown when these indices "expire" or how/when the user has to recycle them? I doubt Android keeps buffers for dequeued indices around, that'd be a huge memory leak;
  • Same concerns for AMediaCodec_getBufferFormat regarding expiry of the index, but otherwise seems like there's no chance to have a buffer without associated format?
  • Then AMediaCodec_getInputFormat and AMediaCodec_getOutputFormat remain, not sure what to do with those.

FWIW I think we're also fine removing NonNull::new_unchecked from get_never_null and keeping this a panic under !cfg!(debug_assertions) conditions.

MarijnS95 added a commit that referenced this pull request Mar 28, 2022
The Android documentation does not comment on when functions could
possibly return `null`, and while discussion in [#248] finds these
unlikely enough to be communicated back to the user there is no 100%
guarantee and the resulting pointers should as such always be checked
(either at all, or also when `cfg!(debug_assertions)` happens to be
disabled).

Note that this `NonNull::new(..).unwrap()` pattern is found in many
other places in the `ndk`.

[#248]: #248
@MarijnS95
Copy link
Member Author

@zarik5 Perhaps if you do manage to find time at some point, would you mind porting that code to a minimal example for inclusion in ndk-examples? That ought to help users and maintainers out, thanks!

@zarik5
Copy link
Contributor

zarik5 commented Mar 28, 2022

A "possibly-unsafe" feature is ugly but I don't have any nicer alternative. Removing the whole module would be a loss since it would work fine if used correctly.

I found this crate which wraps the mediacodec API, which seems to have much more extensive checking. Maybe a collaboration is possible? Currently it's not nice to use with ndk (the crate in this repo) since it reimplements all wrappers for pointers like ANativeWindow etc.

@zarik5
Copy link
Contributor

zarik5 commented Mar 28, 2022

I'll open an issue on their repo

@MarijnS95
Copy link
Member Author

MarijnS95 commented Mar 28, 2022

@zarik5 Good call. Sharing their media_codec code is one thing, but it's also a little unfortunate to have a duplicate wrapper around other NDK bits. Unifying it would definitely be good, though that repo seems more like someone publishing their personal project than a crate intended for upstream use (regardless, their media_codec implementation seems more elaborate and would be worth learning/sourcing from).

MarijnS95 added a commit that referenced this pull request Apr 19, 2022
…256)

The Android documentation does not comment on when functions could
possibly return `null`, and while discussion in [#248] finds these
unlikely enough to be communicated back to the user there is no 100%
guarantee and the resulting pointers should as such always be checked
(either at all, or also when `cfg!(debug_assertions)` happens to be
disabled).

Note that this `NonNull::new(..).unwrap()` pattern is found in many
other places in the `ndk`.

[#248]: #248
MarijnS95 added a commit that referenced this pull request Jun 15, 2023
Back when I refactored these in #248 and #256 the closure argument
should have been removed entirely as it is only called immediately
without any arguments, so its return value could have been passed
instead.  At that point the function becomes simple enough to warrant
removal entirely, inlining the unwrap and even allowing a more natural
string literal error message in `.expect()`.
MarijnS95 added a commit that referenced this pull request Jun 15, 2023
Back when I refactored these in #248 and #256 the closure argument
should have been removed entirely as it is only called immediately
without any arguments, so its return value could have been passed
instead.  At that point the function becomes simple enough to warrant
removal entirely, inlining the unwrap and even allowing a more natural
string literal error message in `.expect()`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants