Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

gh-106368: Increase Argument Clinic CLI test coverage #107156

Merged
merged 11 commits into from
Jul 24, 2023
Merged

gh-106368: Increase Argument Clinic CLI test coverage #107156

merged 11 commits into from
Jul 24, 2023

Conversation

erlend-aasland
Copy link
Contributor

@erlend-aasland erlend-aasland commented Jul 23, 2023 •
edited
Loading

Instead of hacking into the Clinic class, use the Argument Clinic tool
to run the ClinicWholeFileTest ClinicExternalTest test suite.

TODO:

  • Remove tmp stuff from test_external, use -o to redirect the output, and simply compare the redirected output with clinic.test.c
  • A new test that verifies that the file is not touched if no changes are made (currently a part of test_external)
  • A test that verifies that an incorrect output checksum (and the generated output) is regenerated if -f is given
  • Test verbose mode

@erlend-aasland
pythongh-106368: Increase Argument Clinic CLI test coverage
12c0dc7 
Instead of hacking into the Clinic class, use the Argument Clinic tool
to run the ClinicWholeFileTest test suite.
@bedevere-bot bedevere-bot mentioned this pull request Jul 23, 2023
Closed
@bedevere-bot bedevere-bot added the tests Tests in the Lib/test dir label Jul 23, 2023
@erlend-aasland
Add test for the --converters opt
e09e416
@erlend-aasland
Fix converters test
f9d637b
@erlend-aasland
Split test_external in two
9f617de 
test_external: run clinic on clinic.test.c

test_no_change: check that dest is left unchanged if content does not change
@erlend-aasland
Test force/no-force runs
39ee0eb
@erlend-aasland
Test verbose mode
a3f505e
@erlend-aasland erlend-aasland marked this pull request as ready for review July 23, 2023 22:36
@erlend-aasland
Copy link
Contributor Author

FWIW; this messes up test coverage locally; that's solvable (IIRC), though.

@erlend-aasland
Copy link
Contributor Author

The coveragerc file that's included in the sources disables coverage for if __name__ == "__main__", which is unfortunate; we should change that.

sobolevn
sobolevn reviewed Jul 24, 2023
View reviewed changes
Copy link
Member

@sobolevn sobolevn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better error messages :)

Lib/test/test_clinic.py Outdated Show resolved Hide resolved
Lib/test/test_clinic.py Outdated Show resolved Hide resolved
erlend-aasland and others added 2 commits July 24, 2023 10:12
@erlend-aasland @sobolevn
Update Lib/test/test_clinic.py
e97414c 
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
@erlend-aasland @sobolevn
Update Lib/test/test_clinic.py
e048663 
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
@erlend-aasland
Copy link
Contributor Author

Thanks for the review, Nikita!

sobolevn
sobolevn approved these changes Jul 24, 2023
View reviewed changes
Copy link
Member

@sobolevn sobolevn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

sobolevn
sobolevn approved these changes Jul 24, 2023
View reviewed changes
Copy link
Member

@sobolevn sobolevn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! 👍

@erlend-aasland erlend-aasland merged commit 83a2837 into python:main Jul 24, 2023
15 checks passed
@erlend-aasland erlend-aasland deleted the clinic/coverage-cli branch July 24, 2023 18:49
@erlend-aasland erlend-aasland added the needs backport to 3.11 only security fixes label Jul 24, 2023
@erlend-aasland erlend-aasland added the needs backport to 3.12 bug and security fixes label Jul 24, 2023
@miss-islington
Copy link
Contributor

Thanks @erlend-aasland for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

@miss-islington
Copy link
Contributor

Thanks @erlend-aasland for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jul 24, 2023
@erlend-aasland @sobolevn @miss-islington
pythongh-106368: Increase Argument Clinic CLI test coverage (pythonGH…
d867999 
…-107156)

Instead of hacking into the Clinic class, use the Argument Clinic tool
to run the ClinicExternalTest test suite.

(cherry picked from commit 83a2837)

Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
@bedevere-bot
Copy link

GH-107189 is a backport of this pull request to the 3.12 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.12 bug and security fixes label Jul 24, 2023
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jul 24, 2023
@erlend-aasland @sobolevn @miss-islington
pythongh-106368: Increase Argument Clinic CLI test coverage (pythonGH…
bd0c8b6 
…-107156)

Instead of hacking into the Clinic class, use the Argument Clinic tool
to run the ClinicExternalTest test suite.

(cherry picked from commit 83a2837)

Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
@bedevere-bot
Copy link

GH-107190 is a backport of this pull request to the 3.11 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.11 only security fixes label Jul 24, 2023
erlend-aasland added a commit that referenced this pull request Jul 24, 2023
@miss-islington @erlend-aasland @sobolevn
[3.12] gh-106368: Increase Argument Clinic CLI test coverage (GH-107156…
69d4e8c 
…) (#107189)

Instead of hacking into the Clinic class, use the Argument Clinic tool
to run the ClinicExternalTest test suite.

(cherry picked from commit 83a2837)

Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
erlend-aasland added a commit that referenced this pull request Jul 26, 2023
@miss-islington @erlend-aasland @sobolevn
[3.11] gh-106368: Increase Argument Clinic CLI test coverage (GH-107156…
6bf2c29 
…) (#107190)

Instead of hacking into the Clinic class, use the Argument Clinic tool
to run the ClinicExternalTest test suite.

(cherry picked from commit 83a2837)

Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
erlend-aasland added a commit to erlend-aasland/cpython that referenced this pull request Jul 26, 2023
@miss-islington @erlend-aasland @sobolevn
[3.11] pythongh-106368: Increase Argument Clinic CLI test coverage (p…
cebe60b 
…ythonGH-107156) (python#107190)

Instead of hacking into the Clinic class, use the Argument Clinic tool
to run the ClinicExternalTest test suite.

(cherry picked from commit 83a2837)

Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
jtcave pushed a commit to jtcave/cpython that referenced this pull request Jul 27, 2023
@erlend-aasland @sobolevn @jtcave
pythongh-106368: Increase Argument Clinic CLI test coverage (python#1…
1da869f 
…07156)

Instead of hacking into the Clinic class, use the Argument Clinic tool
to run the ClinicExternalTest test suite.

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sobolevn sobolevn sobolevn approved these changes

@AlexWaygood AlexWaygood Awaiting requested review from AlexWaygood

Assignees
No one assigned
Labels
skip news tests Tests in the Lib/test dir
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@erlend-aasland @miss-islington @bedevere-bot @sobolevn