Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add end-of-file-fixer to pre-commit #106065

Merged
merged 4 commits into from
Jun 25, 2023

Conversation

hugovk
Copy link
Member

@hugovk hugovk commented Jun 24, 2023

Follow on from #104275.

This adds a check that is part of make patchcheck's normalize_whitespace(python_files) to pre-commit, which showed up in PR #105984:

Getting base branch for PR ... origin/main
Getting the list of files that have been added/changed ... 3 files
Fixing Python file whitespace ... 1 file:
  Lib/test/test_webbrowser.py
Fixing C file whitespace ... 0 files
Fixing docs whitespace ... 0 files
Please fix the 1 file(s) with whitespace issues
(on UNIX you can run `make patchcheck` to make the fixes)
##[error]Bash exited with code '1'.

https://dev.azure.com/Python/cpython/_build/results?buildId=130810&view=logs&j=256d7e09-002a-52d7-8661-29ee3960640e&t=3d7276d3-4e8d-5309-55ad-fb0b172d9925

Adding here will make the failure more visible, and show the actual change needed.

I'll push another temporary commit to demonstrate.

@hugovk
Copy link
Member Author

hugovk commented Jun 24, 2023

Temporary commit not needed, as there's a file that needs fixing:

check yaml...............................................................Passed
fix end of files.........................................................Failed
- hook id: end-of-file-fixer
- exit code: 1
- files were modified by this hook

Fixing Lib/test/coding20731.py

trim trailing whitespace.................................................Passed
Sphinx lint..............................................................Passed
pre-commit hook(s) made changes.
If you are seeing this message in CI, reproduce locally with: `pre-commit run --all-files`.
To run `pre-commit` as part of git workflow, use `pre-commit install`.
All changes made by hooks:
diff --git a/Lib/test/coding20731.py b/Lib/test/coding20731.py
index b0e227a..7b61917 [100](https://github.com/python/cpython/actions/runs/5366030779/jobs/9735224499?pr=106065#step:4:105)644
--- a/Lib/test/coding20731.py
+++ b/Lib/test/coding20731.py
@@ -1,4 +1 @@
 #coding:latin1
-
-
-
Error: Process completed with exit code 1.

https://github.com/python/cpython/actions/runs/5366030779/jobs/9735224499?pr=106065

@hugovk
Copy link
Member Author

hugovk commented Jun 24, 2023

Like trailing-whitespace, should we run on C and RST in addition to Python files?

diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index e21feee3ed..78627f9839 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -4,7 +4,7 @@ repos:
     hooks:
       - id: check-yaml
       - id: end-of-file-fixer
-        types_or: [python]
+        types_or: [c, python, rst]
       - id: trailing-whitespace
         types_or: [c, python, rst]

If so, it would do an initial tidy-up of these files:

Files
#       modified:  [1] .pre-commit-config.yaml
#       modified:  [2] Doc/c-api/allocation.rst
#       modified:  [3] Doc/c-api/frame.rst
#       modified:  [4] Doc/c-api/gcsupport.rst
#       modified:  [5] Doc/c-api/init.rst
#       modified:  [6] Doc/c-api/intro.rst
#       modified:  [7] Doc/c-api/long.rst
#       modified:  [8] Doc/c-api/marshal.rst
#       modified:  [9] Doc/copyright.rst
#       modified: [10] Doc/faq/installed.rst
#       modified: [11] Doc/howto/index.rst
#       modified: [12] Doc/howto/instrumentation.rst
#       modified: [13] Doc/howto/ipaddress.rst
#       modified: [14] Doc/howto/sockets.rst
#       modified: [15] Doc/howto/urllib2.rst
#       modified: [16] Doc/library/_thread.rst
#       modified: [17] Doc/library/array.rst
#       modified: [18] Doc/library/base64.rst
#       modified: [19] Doc/library/binary.rst
#       modified: [20] Doc/library/copy.rst
#       modified: [21] Doc/library/curses.ascii.rst
#       modified: [22] Doc/library/curses.panel.rst
#       modified: [23] Doc/library/dbm.rst
#       modified: [24] Doc/library/email.charset.rst
#       modified: [25] Doc/library/email.encoders.rst
#       modified: [26] Doc/library/email.header.rst
#       modified: [27] Doc/library/filecmp.rst
#       modified: [28] Doc/library/getopt.rst
#       modified: [29] Doc/library/gzip.rst
#       modified: [30] Doc/library/heapq.rst
#       modified: [31] Doc/library/imaplib.rst
#       modified: [32] Doc/library/logging.handlers.rst
#       modified: [33] Doc/library/mailbox.rst
#       modified: [34] Doc/library/marshal.rst
#       modified: [35] Doc/library/modulefinder.rst
#       modified: [36] Doc/library/pyexpat.rst
#       modified: [37] Doc/library/rlcompleter.rst
#       modified: [38] Doc/library/shelve.rst
#       modified: [39] Doc/library/site.rst
#       modified: [40] Doc/library/socketserver.rst
#       modified: [41] Doc/library/stringprep.rst
#       modified: [42] Doc/library/termios.rst
#       modified: [43] Doc/library/text.rst
#       modified: [44] Doc/library/trace.rst
#       modified: [45] Doc/library/tty.rst
#       modified: [46] Doc/library/urllib.error.rst
#       modified: [47] Doc/library/uuid.rst
#       modified: [48] Doc/library/winreg.rst
#       modified: [49] Doc/library/winsound.rst
#       modified: [50] Doc/library/wsgiref.rst
#       modified: [51] Doc/library/xml.dom.rst
#       modified: [52] Doc/library/xml.sax.reader.rst
#       modified: [53] Doc/library/xml.sax.rst
#       modified: [54] Doc/library/xml.sax.utils.rst
#       modified: [55] Doc/library/zipapp.rst
#       modified: [56] Doc/reference/introduction.rst
#       modified: [57] Doc/tutorial/appetite.rst
#       modified: [58] Doc/tutorial/stdlib2.rst
#       modified: [59] Doc/tutorial/whatnow.rst
#       modified: [60] Doc/using/mac.rst
#       modified: [61] Doc/whatsnew/2.0.rst
#       modified: [62] Doc/whatsnew/2.1.rst
#       modified: [63] Doc/whatsnew/2.2.rst
#       modified: [64] Doc/whatsnew/2.4.rst
#       modified: [65] Doc/whatsnew/2.5.rst
#       modified: [66] Doc/whatsnew/2.6.rst
#       modified: [67] Doc/whatsnew/3.5.rst
#       modified: [68] Include/cpython/longobject.h
#       modified: [69] Include/internal/pycore_ast_state.h
#       modified: [70] Include/internal/pycore_blocks_output_buffer.h
#       modified: [71] Include/internal/pycore_weakref.h
#       modified: [72] Include/iterobject.h
#       modified: [73] Include/py_curses.h
#       modified: [74] Include/pyexpat.h
#       modified: [75] Include/warnings.h
#       modified: [76] Mac/BuildScript/README.rst
#       modified: [77] Misc/NEWS.d/next/Core and Builtins/2023-06-11-09-14-30.gh-issue-105331.nlZvoW.rst
#       modified: [78] Misc/coverity_model.c
#       modified: [79] Modules/_blake2/impl/blake2-config.h
#       modified: [80] Modules/_blake2/impl/blake2-impl.h
#       modified: [81] Modules/_blake2/impl/blake2.h
#       modified: [82] Modules/_blake2/impl/blake2b-load-sse2.h
#       modified: [83] Modules/_blake2/impl/blake2b-load-sse41.h
#       modified: [84] Modules/_blake2/impl/blake2b-ref.c
#       modified: [85] Modules/_blake2/impl/blake2b-round.h
#       modified: [86] Modules/_blake2/impl/blake2s-load-sse41.h
#       modified: [87] Modules/_blake2/impl/blake2s-load-xop.h
#       modified: [88] Modules/_blake2/impl/blake2s-ref.c
#       modified: [89] Modules/_blake2/impl/blake2s-round.h
#       modified: [90] Modules/_blake2/impl/blake2s.c
#       modified: [91] Modules/_decimal/docstrings.h
#       modified: [92] Modules/_decimal/libmpdec/examples/compare.c
#       modified: [93] Modules/_decimal/libmpdec/examples/div.c
#       modified: [94] Modules/_decimal/libmpdec/examples/divmod.c
#       modified: [95] Modules/_decimal/libmpdec/examples/multiply.c
#       modified: [96] Modules/_decimal/libmpdec/examples/pow.c
#       modified: [97] Modules/_decimal/libmpdec/examples/powmod.c
#       modified: [98] Modules/_decimal/libmpdec/examples/shift.c
#       modified: [99] Modules/_decimal/libmpdec/examples/sqrt.c
#       modified: [100] Modules/_hacl/Hacl_Hash_MD5.c
#       modified: [101] Modules/_hacl/Hacl_Hash_SHA1.c
#       modified: [102] Modules/_hacl/Hacl_Hash_SHA2.c
#       modified: [103] Modules/_hacl/Hacl_Hash_SHA3.c
#       modified: [104] Modules/_ssl_data_111.h
#       modified: [105] Modules/_ssl_data_300.h
#       modified: [106] Modules/_ssl_data_31.h
#       modified: [107] Modules/_testbuffer.c
#       modified: [108] Modules/_testimportmultiple.c
#       modified: [109] Modules/cjkcodecs/emu_jisx0213_2000.h
#       modified: [110] Modules/cjkcodecs/mappings_hk.h
#       modified: [111] Modules/cjkcodecs/mappings_jp.h
#       modified: [112] Modules/cjkcodecs/mappings_kr.h
#       modified: [113] Modules/cjkcodecs/mappings_tw.h
#       modified: [114] Modules/expat/pyexpatns.h
#       modified: [115] Modules/getpath.c
#       modified: [116] Modules/hashlib.h
#       modified: [117] Modules/unicodedata_db.h
#       modified: [118] Objects/bytesobject.c
#       modified: [119] Objects/capsule.c
#       modified: [120] Objects/exceptions.c
#       modified: [121] Objects/stringlib/fastsearch.h
#       modified: [122] Objects/stringlib/find_max_char.h
#       modified: [123] Objects/stringlib/partition.h
#       modified: [124] Objects/stringlib/split.h
#       modified: [125] Objects/stringlib/ucs4lib.h
#       modified: [126] Objects/unicodectype.c
#       modified: [127] Objects/unicodetype_db.h
#       modified: [128] PC/frozen_dllmain.c
#       modified: [129] Programs/_bootstrap_python.c
#       modified: [130] Programs/_freeze_module.c
#       modified: [131] Python/Python-ast.c
#       modified: [132] Python/ceval_gil.c
#       modified: [133] Python/getopt.c
#       modified: [134] Python/pyctype.c
#       modified: [135] Python/suggestions.c
#       modified: [136] Python/traceback.c
#       modified: [137] Tools/msi/bundle/bootstrap/pch.h

If not, we could simplify a bit:

diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index e21feee3ed..1197ef745e 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -4,7 +4,7 @@ repos:
     hooks:
       - id: check-yaml
       - id: end-of-file-fixer
-        types_or: [python]
+        types: [python]
       - id: trailing-whitespace
         types_or: [c, python, rst]

@hugovk hugovk marked this pull request as ready for review June 24, 2023 20:26
@hugovk hugovk requested a review from AlexWaygood as a code owner June 24, 2023 20:26
@AlexWaygood
Copy link
Member

AlexWaygood commented Jun 25, 2023

Like trailing-whitespace, should we run on C and RST in addition to Python files?

Let's do the hopefully-uncontroversial bit (adding it for .py files) first, and we can consider other file extensions separately in followup PRs, since they'd lead to much bigger diffs

@hugovk hugovk added the needs backport to 3.12 bug and security fixes label Jun 25, 2023
Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@hugovk hugovk merged commit 8c24a83 into python:main Jun 25, 2023
@hugovk hugovk deleted the patchcheck-end-of-file-fixer branch June 25, 2023 10:44
@miss-islington
Copy link
Contributor

Thanks @hugovk for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

@bedevere-bot
Copy link

GH-106080 is a backport of this pull request to the 3.12 branch.

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jun 25, 2023
(cherry picked from commit 8c24a83)

Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
@bedevere-bot bedevere-bot removed the needs backport to 3.12 bug and security fixes label Jun 25, 2023
AlexWaygood pushed a commit that referenced this pull request Jun 25, 2023
Add end-of-file-fixer to pre-commit (GH-106065)
(cherry picked from commit 8c24a83)

Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants