Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary of Changes
fp16
Closes #201
Motivation
Sending float16 data to the server from Python is important to maintain parity with C++.
Implementation
There was an issue raised in Pybind11 about this issue and a suggested workaround, which I've duplicated here.
Prior to this PR, the MIGraphX worker would crash the server if the worker threw an exception during batch processing. The cause was that that the
new_batch
was null. Now, this condition is checked for and it returns early. In the base class, I also allowed the return value from the worker to be null. This indicates that there is nothing to pass on to a subsequent worker even if one is configured. This scenario will occur in general when the worker encounters an error though the other workers aren't actively handling this well at the moment.There are missing Python preprocessing methods for different types as they're being added on an as-needed basis. This also adds fp16 variants to them.
Notes
The preprocessing code was using functions like
imagePreprocessFloat()
when FP32 was the only type bound. This function has been deprecated. Instead, new code should useimagePreprocessFp32()
.