-
-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use the Windows method to get TCL functions on Cygwin #5807
Conversation
This is related to linking semantics, so Cygwin should follow the Windows codepath.
for more information, see https://pre-commit.ci
#undef _WIN32 | ||
#undef __WIN32__ | ||
#undef WIN32 | ||
#endif |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder whether it might be better to keep these defined to fix other potential issues?
It would be helpful to get the output of a full test run on Cygwin, or even adding to the CI (I don't have the time to try this at the moment).
I'm pretty sure a similar issue might happen around the FriBiDi shim (although it is disabled by default). There are also a few functions that are only enabled on Windows platforms, and I don't think there's anything stopping them working on Cygwin.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
python selftest.py
reports no errors before or after this change, if that's what you're asking.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I mean python3 -m pytest -v Tests
in the root of the git checkout.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Running it in the root of the modified sdist shows only one error, a PermissionError when attempting to overwrite a test image in a temporary directory. I haven't figured out how to get that passing, but it seems unrelated to this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
a PermissionError when attempting to overwrite a test image in a temporary directory
Which test is this? But yes, probably unrelated.
Edit: Looks like it might be Tests/test_image.py::TestImage::test_readonly_save
shows only one error
I'm also interested in the skips and xfails, but I can take a look at it myself now, as I had a bit of free time just now and I got Cygwin working on GHA (surprisingly easy compared to some other platforms): https://github.com/nulano/Pillow/runs/4105517818?check_suite_focus=true
There are some failing tests (I haven't looked into them yet), but all dependencies seem to be detected correctly at least.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
a PermissionError when attempting to overwrite a test image in a temporary directory
Which test is this? But yes, probably unrelated. Edit: Looks like it might be
Tests/test_image.py::TestImage::test_readonly_save
Yes, that one. I have a reproducer (create a file named "a.txt" before running):
>>> import mmap
>>> with open("a.txt", "r") as in_file:
... mapped = mmap.mmap(in_file.fileno(), 0, access=mmap.ACCESS_READ)
... with open("a.txt", "w") as out_file:
... out_file.write(line)
...
Traceback (most recent call last):
File "<stdin>", line 3, in <module>
PermissionError: [Errno 13] Permission denied: 'a.txt'
shows only one error
I'm also interested in the skips and xfails, but I can take a look at it myself now, as I had a bit of free time just now and I got Cygwin working on GHA (surprisingly easy compared to some other platforms): https://github.com/nulano/Pillow/runs/4105517818?check_suite_focus=true There are some failing tests (I haven't looked into them yet), but all dependencies seem to be detected correctly at least.
Skipped and expected failing tests as of my most recent run:
====================== short test summary info =======================
SKIPPED [1] Tests/test_file_jpeg.py:877: Windows only
SKIPPED [1] Tests/test_file_tiff.py:732: Windows only
SKIPPED [1] Tests/test_image.py:196: Test requires opening an mmapped file for writing
SKIPPED [1] Tests/test_image_access.py:398: Failing on AppVeyor / GitHub Actions when run from subprocess, not from shell
SKIPPED [1] Tests/test_image_fromqimage.py:38: Qt bindings are not installed
SKIPPED [1] Tests/test_image_fromqimage.py:43: Qt bindings are not installed
SKIPPED [1] Tests/test_image_fromqimage.py:48: Qt bindings are not installed
SKIPPED [1] Tests/test_image_fromqimage.py:53: Qt bindings are not installed
SKIPPED [1] Tests/test_image_fromqimage.py:58: Qt bindings are not installed
SKIPPED [1] Tests/test_imagedraw.py:974: failing
SKIPPED [2] Tests/test_imagefontctl.py:233: fails with this font
SKIPPED [1] Tests/test_imagegrab.py:13: requires Windows or macOS
SKIPPED [1] Tests/test_imagegrab.py:37: X connection failed: error 1
SKIPPED [1] Tests/test_imagegrab.py:39: tests missing XCB
SKIPPED [1] Tests/test_imagegrab.py:77: Windows only
SKIPPED [1] Tests/test_imagegrab.py:87: Windows only
SKIPPED [1] Tests/test_imageqt.py:15: Qt bindings are not installed
SKIPPED [1] Tests/test_imageqt.py:44: Qt bindings are not installed
SKIPPED [1] Tests/test_imageqt.py:49: Qt bindings are not installed
SKIPPED [1] Tests/test_imageshow.py:44: Only run on CIs; hangs on Windows CIs
SKIPPED [4] Tests/test_imagetk.py:30: TCL Error: couldn't connect to display ":0.0"
SKIPPED [1] Tests/test_imagewin.py:37: Windows only
SKIPPED [1] Tests/test_imagewin.py:47: Windows only
SKIPPED [1] Tests/test_imagewin.py:58: Windows only
SKIPPED [1] Tests/test_imagewin.py:72: Windows only
SKIPPED [1] Tests/test_imagewin.py:87: Windows only
SKIPPED [1] Tests/test_qt_image_qapplication.py:50: Qt bindings are not installed
SKIPPED [1] Tests/test_qt_image_toqimage.py:15: Qt bindings are not installed
XFAIL Tests/test_decompression_bomb.py::TestDecompressionBomb::test_exception_ico
different exception
XFAIL Tests/test_file_palm.py::test_p_mode
Palm P image is wrong
XFAIL Tests/test_image_resample.py::TestCoreResampleAlphaCorrect::test_levels_rgba
Current implementation isn't precise enough
XFAIL Tests/test_image_resample.py::TestCoreResampleAlphaCorrect::test_levels_la
Current implementation isn't precise enough
========== 2867 passed, 32 skipped, 4 xfailed, 4 warnings in 231.29s (0:03:51) ==========
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TCL Error: couldn't connect to display ":0.0"
I'm guessing you ran this on a system without an X11 desktop environment (I'm not sure how this works on Cygwin...)? This means that TCL was not actually tested at all. If this test passed (and failed without this PR) instead I would say this probably fixes #5795, but with a skip it is unclear.
It may be possible to test this using xvfb-run
as is done on some Ubuntu CIs (and what I plan to add to the Cygwin job)
Pillow/.github/workflows/test.yml
Line 85 in fcb87ec
xvfb-run -s '-screen 0 1024x768x24' .ci/test.sh |
I would be interested in whether the test_imagegrab and test_imagewin "Windows only" skips can be made to pass (I'm not sure to what extent Cygwin allows access to WinAPI), but perhaps that is beyond the scope of this PR. There is also an #ifdef _WIN32
in _imagingcms.c
which is also related to the Windows display, but I guess it doesn't have a test or it would show up in the above.
The FriBiDi code that I mentioned in my first comment has to be specifically requested at compilation and is intended for published wheels, so perhaps it is not very important, but if you really feel like checking it, see #5062. However, I certainly intend to test it before submitting the PR to add a Cygwin run on GHA, probably some time before next release (I don't know how much time I will have in the near future).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That was through tox. I'm not sure what all I need to do to convince X to work through tox, besides passing the DISPLAY
variable through. Running pytest myself (not through tox) with PYTHONPATH
set to include the changed version of Pillow runs those tests, producing 27 skips and four expected failures.
There are xorg-server-common
and xinit
packages on Cygwin, which I use on a regular basis locally and through ssh forwarding. xvfb-run
seems to be in xorg-server-extra
(cygcheck -p xcfb-run
).
I did check that the changes in this PR allowed import PIL._imagingtk
to work, and the person who posted the original issue said this change fixed their STC as well as the bug they narrowed down to that STC.
The test seems to require opening a file for reading, mmapping it, then opening that file for writing. Windows doesn't allow this.
@pytest.mark.skipif( | ||
sys.platform == "cygwin", | ||
reason="Test requires opening an mmaped file for writing", | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cygwin can't use Python's mmap? Could you provide some more information about this, or link to a discussion about it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
$ echo "Lorem itsum dolor sit amet" > a.txt
$ python
>>> import mmap
>>> with open("a.txt", "r") as in_file:
... with mmap.mmap(in_file.fileno(), 30, access=mmap.ACCESS_COPY) as mapped:
... with open("a.txt", "w") as out_file:
... out_file.write(mapped)
...
Traceback (most recent call last):
File "<stdin>", line 3, in <module>
PermissionError: [Errno 13] Permission denied: 'a.txt'
My initial comment was based on the above test case failing. In a fresh python session, the below case works fine:
>>> with open("a.txt", "r") as in_file:
... with open("a.txt", "w") as out_file:
... out_file.write(in_file.read())
...
0
so it looks like having the mmap open while opening the file for writing is a problem, but opening the mmap seems to be fine. Since the test case is nearly this exact sequence of operations, I marked it as XFAIL.
I get a different error using Windows-native python from anaconda:
>>> import mmap
>>> with open("a.txt", "r") as in_file:
... with mmap.mmap(in_file.fileno(), 30, access=mmap.ACCESS_COPY) as mapped:
... with open("a.txt", "w") as out_file:
... out_file.write(mapped)
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
OSError: [WinError 8] Not enough memory resources are available to process this command
I have no idea why that error is happening; I have gigabytes of memory free and requested 30 bytes.
This is related to linking semantics, so Cygwin should follow the Windows codepath.
Fixes #5795 (and the bug that prompted that issue).
Changes proposed in this pull request: