-
-
Notifications
You must be signed in to change notification settings - Fork 669
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow creating images from raw image sources #2263
Conversation
@@ -99,6 +99,9 @@ def __init__( | |||
src.save(buffer, format="png", compress_level=0) | |||
self._impl = self.factory.Image(interface=self, data=buffer.getvalue()) | |||
|
|||
elif isinstance(src, self.factory.Image.RAW_TYPE): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@rmartin16 If you're looking for a hard-mode typing challenge: How should the type annotation on src
be modified to allow for this? It's a type on a class that, by definition, is platform specific, and won't exist in the environment where toga-core is being built, tested, or documented.
The only solution I can think of is to add UIImage | NSImage | GdkBitmap | ...
but that won't be correct - you can't use a UIImage unless you're on iOS.
Any ideas?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this might be possible....but it, perhaps, starts to verge on the ridiculous....
For instance, here's where I got to before I, more or less, gave up:
if TYPE_CHECKING:
BaseImageSrcT: TypeAlias = Union[
Path,
bytes,
bytearray,
memoryview,
"Image",
PIL.Image.Image,
None,
]
if sys.platform == "win32":
from toga_winforms.images import WinImage
ImageSrcT: TypeAlias = Union[BaseImageSrcT, WinImage]
if sys.platform.startswith("linux"):
from toga_gtk.images import GdkPixbuf
ImageSrcT: TypeAlias = Union[BaseImageSrcT, GdkPixbuf.Pixbuf]
if sys.platform == "darwin":
from toga_cocoa.images import NSImage
ImageSrcT: TypeAlias = Union[BaseImageSrcT, NSImage]
class Image:
def __init__(self, src: ImageSrcT): ...
I'm not sure if mypy technically considers this strategy above part of their larger type narrowing support, but it helps accommodate the runtime environment.
That said, for this to really make mypy happy, it'll require other changes in Toga...notably, replacing the star imports in the platform-specific Toga code to explicitly import these native library classes....and we'll also need typing stubs for, at least, PyGObject.
BUT...this has major consequences for building docs. On my Linux machine, it tries to interpret the definition of ImageSrcT
for Linux but cannot find toga_gtk
; I can install toga_gtk
to build the docs....but then the build blows up with RuntimeError
s about DISPLAY
env var. Maybe Sphinx has some buried support for effectively ignoring certain symbols...and that could serve as some type of resolution here.
At any rate, the src
argument feels like a good candidate for Any
where the documentation fills in the details.
P.S. - type checkers and IDEs are still likely to resolve the type of src
to Any
even with all these accommodations if Pillow isn't installed. And if we just add these platform-specific image classes, such as GdkPixbuf.Pixbuf
, then the type will always be Any
because it would be impossible to resolve all the types in the Union
on any one system.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this might be possible....but it, perhaps, starts to verge on the ridiculous....
For instance, here's where I got to before I, more or less, gave up:
That was broadly my initial thought as well - except that it falls down on iOS/Android (which will never be installed in the local environment where sys.platform will find it), and on custom backends, and in the Sphinx builds you mention later.
That said, for this to really make mypy happy, it'll require other changes in Toga...notably, replacing the star imports in the platform-specific Toga code to explicitly import these native library classes....and we'll also need typing stubs for, at least, PyGObject.
Woof... good luck with that endeavour... I like yaks as much as the next person, but autogeneration of types from GI bindings is a yak to far for me :-)
At any rate, the
src
argument feels like a good candidate forAny
where the documentation fills in the details.
Is there any advantage to specifying the type at all in this case? What (if any) benefit exists to an explicit Any
over "no type declaration"? Cosmetically, "Any" reads to me as "You can use literally anything", which isn't true (at least, not in spirit). This is a case where I think I'd prefer to say nothing, and force the reader to actually read words, rather than provide something explicitly too broad as an automated description.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there any advantage to specifying the type at all in this case? What (if any) benefit exists to an explicit
Any
over "no type declaration"? Cosmetically, "Any" reads to me as "You can use literally anything", which isn't true (at least, not in spirit). This is a case where I think I'd prefer to say nothing, and force the reader to actually read words, rather than provide something explicitly too broad as an automated description.
I think it depends on your goals.
As far as typing is concerned, an untyped argument will be resolved as Any
...so, in that regard, adding Any
or not in this case is functionally the same.
However, the absence of a type can mean different things. It could mean this function hasn't been evaluated for typing; this would be useful when gradually typing a code base and using mypy's functionality to ignore untyped functions and methods. Although, as you mention, if someone takes Any
seriously, then they will be especially misled on what this argument can accept.
That said, given this is part of the API surface, I think it will be useful to ensure it has a type assigned....so, we can know the type has been assessed.
So, thinking a bit out of the box perhaps, you could create a NewType
called ImageDataT
that's just assigned to Any
. In that way, it's an indication a specific type is being requested....it just isn't expressed in Python types.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Replacing all the specific Image classes with an ImageT
that maps to Any
seems like a decent option that is going to render well as documentation, and won't be intractable from a mypy perspective. I'll go with that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Additionally, fwiw, using NewType
wouldn't really work either....since it would, in fact, require users to explicitly cast the image data as ImageType
which wouldn't be a great experience at all.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ugh... fine, I guess we'll just go with Any
in the union then... le sigh...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The docs still look quite cluttered with everything in the union. Can we just make ImageType an alias for Any, and then describe all the options in its docstring?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See the discussion above - that's where I started, and it looks awful; it renders in Sphinx as NewType("Image", Any)
, with the sphinx link to "NewType" being to the Python docs. AFAIK, it's not possible to provide documentation for the type itself (which would be the ideal situation).
I'm open to other suggestions, but putting Any
in the union is the only version I could find that mollifies mypy/PyCharm and doesn't read like a train wreck.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since a union that contains Any no longer provides any type-checking safety, and there are so many things in the union that it isn't very good documentation either, my suggestion was to just replace the whole union with Any, and rely on the documentation text to explain what's accepted.
Ideally we would do this with an alias, but if Sphinx doesn't work with that, we can use Any directly, and put the documentation in its own section on the Image page with incoming links from all the relevant places.
f20b060
to
b0600a6
Compare
core/src/toga/widgets/imageview.py
Outdated
:param image: The image to display. This can take all the same formats | ||
as the `src` parameter to :class:`toga.Image` -- namely, a file path | ||
(as string or :any:`pathlib.Path`), bytes data in a supported image | ||
format, an instance of the platform's native Image type, or |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Decapitalize, since the type probably isn't literally called Image
.
format, an instance of the platform's native Image type, or | |
format, an instance of the platform's native image type, or |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Although, if this is now part of the public API, we should probably list the native types explicitly, rather than making the reader guess.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, I see them now, they're on the Image
page, but not referenced from the constructor docstring. If we can have an ImageType
alias linked from both Image
and ImageView
, that would be a good place to move that list.
The list of file formats (PNG, JPG, etc.) should probably stay where it is, because they apply to saving images as well as loading them. But there should be a link to that list from each relevant place.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was on the fence about documenting the internal types at all. As an end user, unless you're reaching into the native layer to implement a feature, you shouldn't need to know the internal types. On that basis, I'm OK with this being a little obscure.
As mentioned in the other comment about types, I'm not aware of any way that we can structure this in Sphinx that is readable and mollifies type checkers; including not being able to document the actual type. I'm open to any suggestions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
unless you're reaching into the native layer to implement a feature, you shouldn't need to know the internal types
We recommend people to reach into the native layer on a fairly regular basis, so we should document this properly. Saying "Can also accept the platform's native image format" without clearly indicating what that means is just going to annoy the reader. A link to the list would be enough to fix this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Following the type documentation wrangling I did for #2259, I've adopted the same typing and documentation pattern for Image. This means there's some overlap in the patch (e.g., the updates to conf.py).
d50fdd3
to
d606b95
Compare
# Define a type variable for generics where an Image type is required. | ||
ImageT = TypeVar("ImageT") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is imported in one place, and added to the spelling wordlist, but It doesn't looks like it's actually being used.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It was pre-existing; it's used in all the as_image()
(canvas, image view) and as_format()
(Image) calls.
It's in the dictionary now because the default sphinx handling of type annotations apparently considers the type to be a word that needs to be spell checked. The previous plugin didn't do this.
If you're calling a native platform API, you'll get back an image in native format (e.g., UIImage on iOS, NSImage on macOS, etc). At present, there's no easy way to create a toga.Image from that raw representation - you have to convert it to bytes, then create a toga.Image from the raw byte data... which will create a UIImage/NSImage etc.
This PR allows the platform's native image format as a valid source type when creating an Image; if provided, it's essentially a shortcut to "use this object as the native representation of the image."
Testing this required a couple of tweaks to the dummy platform's Image representation to give it a unique data type.
PR Checklist: