Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable isort in the pre-commit tool settings #672

Closed

Conversation

webknjaz
Copy link
Contributor

@webknjaz webknjaz commented Nov 5, 2021

No description provided.

# https://pycqa.github.io/isort/docs/configuration/multi_line_output_modes.html
# NOTE: Another mode could be "5" for grouping multiple "import from" under
# NOTE: a single instruction.
multi_line_output = 9
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@abhinavsingh this one corresponds best to the style you already use in the repo but I usually prefer 5: https://pycqa.github.io/isort/docs/configuration/multi_line_output_modes.html

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM. We can use 5 which is reasonable limit.

I am unsure what will be the implications. I guess need to split multiple grouped imports or may be longer import lines? Irrespective, happy to get this in. Let's keep it consistent with aio-libs which is already accepted widely in the developer community :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not a limit, it's a style (or "mode") of formatting, follow the lint and you'll see. aiohttp actually uses 3 for some reason but I didn't set that up and try not to change the formatting settings in that project.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"isort your imports, so you don't have to."

I see what you mean now. It's part of a fixture config :).

Happy to see this integration, indeed following a standard is the way to go, I was sorting them based upon a random in-my-brain-only logic 🗡️

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, also one of the most common import orders is to have stdlib at the top, followed by third-party imports and finally project-local. But with isort you can define extra sections if some things have additional semantical meaning within the project or explicitly declare that some things belong to certain sections.

@webknjaz
Copy link
Contributor Author

webknjaz commented Nov 5, 2021

Wow, I see that the failing tests reveal some import loops present in the project. Not sure how hard would it be to solve them.

@abhinavsingh
Copy link
Owner

Wow, I see that the failing tests reveal some import loops present in the project. Not sure how hard would it be to solve them.

Code has gone through a lot of changes between versions (from a single file to module based), so I am not surprised this can pop up after import order change :).

Another reason for circular import errors could be type checking requirements. I do remember importing classes from other modules, just to define a typed variable. This can cause issue. From top-of-the-head, flags could be another culprit here, as it is imported by literally every file in the repo. But will need to dig deep to find the root cause(s).

Ideally most of the sub-modules are written in isolations. But they may have reminiscence from v1 code.

@webknjaz webknjaz marked this pull request as draft November 5, 2021 11:07
@webknjaz
Copy link
Contributor Author

webknjaz commented Nov 5, 2021

I'll probably try adding a pytest test that I picked up from the pytest project itself, for catching circular imports.

@webknjaz
Copy link
Contributor Author

webknjaz commented Nov 5, 2021

I'll probably try adding a pytest test that I picked up from the pytest project itself, for catching circular imports.

Here you go: #678

@abhinavsingh abhinavsingh marked this pull request as ready for review November 15, 2021 20:40
@abhinavsingh abhinavsingh marked this pull request as draft November 15, 2021 20:41
@abhinavsingh
Copy link
Owner

@webknjaz I added isort support recently in another private project of mine. Wondering if we could still enable it for proxy.py. For now I am closing this, as most likely we might need to re-sync the branch and resolve conflicts. A lot has changed since this PR. Let's re-open or create a new one once ready. I'll also spend sometime and see how can we get isort in.

@webknjaz
Copy link
Contributor Author

In order to enable isort, you'd need to solve the import loops it causes but I imagine you could try enabling it for a part of the project and not all of the files and then gradually include more modules as you sort out the problems.

@abhinavsingh
Copy link
Owner

Sounds good, let me take a look. Currently unsure what are these circular import issues. The circular import test we added seems to catch no issues. Will report back my findings.

abhinavsingh added a commit that referenced this pull request Jan 10, 2022
abhinavsingh added a commit that referenced this pull request Jan 10, 2022
* isort the tests folder

* Carry over changes from #672

* Disable pre-commit

* Revert flake8 config change

* isort examples too
abhinavsingh added a commit that referenced this pull request Jan 14, 2022
* Remove menubar (#930)

Remove `menubar`

* Use `128 KB` as default value for `DEFAULT_BUFFER_SIZE` (#926)

* Add `TlsInterception` acceptance test

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Generate CA cert if not available

* Fix lint and tests

* Fix args

* Remove acceptance tests for now

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* pip prod(deps): bump wheel from 0.37.0 to 0.37.1 (#934)

Bumps [wheel](https://github.com/pypa/wheel) from 0.37.0 to 0.37.1.
- [Release notes](https://github.com/pypa/wheel/releases)
- [Changelog](https://github.com/pypa/wheel/blob/main/docs/news.rst)
- [Commits](pypa/wheel@0.37.0...0.37.1)

---
updated-dependencies:
- dependency-name: wheel
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* npm: bump jasmine from 3.10.0 to 4.0.0 in /dashboard (#933)

Bumps [jasmine](https://github.com/jasmine/jasmine-npm) from 3.10.0 to 4.0.0.
- [Release notes](https://github.com/jasmine/jasmine-npm/releases)
- [Commits](jasmine/jasmine-npm@v3.10.0...v4.0.0)

---
updated-dependencies:
- dependency-name: jasmine
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* npm: bump ws from 8.3.0 to 8.4.0 in /dashboard (#936)

Bumps [ws](https://github.com/websockets/ws) from 8.3.0 to 8.4.0.
- [Release notes](https://github.com/websockets/ws/releases)
- [Commits](websockets/ws@8.3.0...8.4.0)

---
updated-dependencies:
- dependency-name: ws
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* pip prod(deps): bump types-paramiko from 2.8.4 to 2.8.6 (#937)

Bumps [types-paramiko](https://github.com/python/typeshed) from 2.8.4 to 2.8.6.
- [Release notes](https://github.com/python/typeshed/releases)
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-paramiko
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com>

* `DescriptorsHandlerMixin` and `Descriptors`, `SelectableEvents` types (#938)

* Add `Descriptors` type

* Add a `DescriptorsHandlerMixin` class used throughout the http framework

* Remove dependency upon `HasFileno` ie `typing_extension` too

* Define `SelectableEvents` type

* Fix doc

* Blank line

* Remove dep on `typing-extensions`

* Discover base plugin class

* await on now async handlers

* Fix doc spell

* Add `--port-file` flag (#942)

* Add `--port-file` flag

* Use `--port-file` flag for integration tests using `get_available_port`

* Use temp dir

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix `base_klass` variable related lint issues

* Fix main tests

* Fix integration

* Use timeout when terminating proc

* Skip integration on win instead of xmark

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Expose `UpstreamConnectionPool` to web & proxy plugins (#946)

* Expose conn pool to plugins

* Fix reusable state handling

* Separate `release` and `retain` methods

* Fix conn pool tests

* Fix tests

* npm: bump chrome-devtools-frontend in /dashboard (#949)

Bumps [chrome-devtools-frontend](https://github.com/ChromeDevTools/devtools-frontend) from 1.0.952865 to 1.0.956881.
- [Release notes](https://github.com/ChromeDevTools/devtools-frontend/releases)
- [Changelog](https://github.com/ChromeDevTools/devtools-frontend/blob/main/docs/release_management.md)
- [Commits](https://github.com/ChromeDevTools/devtools-frontend/commits)

---
updated-dependencies:
- dependency-name: chrome-devtools-frontend
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* pip prod(deps): bump sphinxcontrib-towncrier from 0.2.0a0 to 0.2.1a0 (#941)

Bumps [sphinxcontrib-towncrier](https://github.com/sphinx-contrib/sphinxcontrib-towncrier) from 0.2.0a0 to 0.2.1a0.
- [Release notes](https://github.com/sphinx-contrib/sphinxcontrib-towncrier/releases)
- [Commits](sphinx-contrib/sphinxcontrib-towncrier@v0.2.0a0...v0.2.1a0)

---
updated-dependencies:
- dependency-name: sphinxcontrib-towncrier
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com>

* `isort` everything except lib (for now) (#952)

* isort the tests folder

* Carry over changes from #672

* Disable pre-commit

* Revert flake8 config change

* isort examples too

* pip prod(deps): bump httpx from 0.20.0 to 0.21.3 (#951)

Bumps [httpx](https://github.com/encode/httpx) from 0.20.0 to 0.21.3.
- [Release notes](https://github.com/encode/httpx/releases)
- [Changelog](https://github.com/encode/httpx/blob/master/CHANGELOG.md)
- [Commits](encode/httpx@0.20.0...0.21.3)

---
updated-dependencies:
- dependency-name: httpx
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com>

* Decouple transport framework from dashboard plugin (#953)

* Decouple transport framework from dashboard plugin

* Move `InspectTrafficPlugin` within `http.inspector` module

* Avoid exporting plugins within `__init__.py` files

* Use `/transport/` prefix to avoid #945 conflict issue

* Add todo

* Build containers in parallel and pre-check (#954)

* Build containers in parallel and pre-check

* Fix loop in workflow

* Test container needs build

* Invoke `WebSocketTransportBasePlugin` connected and disconnected callbacks (#956)

* [HttpProtocolHandler] Handle invalid request parsing exceptions (#957)

* Handle invalid request parsing exception when raised, log the bytes for later inspection

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* npm: bump jasmine-ts from 0.3.0 to 0.4.0 in /dashboard (#958)

Bumps [jasmine-ts](https://github.com/svi3c/jasmine-ts) from 0.3.0 to 0.4.0.
- [Release notes](https://github.com/svi3c/jasmine-ts/releases)
- [Changelog](https://github.com/svi3c/jasmine-ts/blob/master/CHANGELOG.md)
- [Commits](https://github.com/svi3c/jasmine-ts/commits/v0.4.0)

---
updated-dependencies:
- dependency-name: jasmine-ts
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Ignore `utf-8` decoding errors within event core (#961)

* [WebServer] Refactor routing to allow same path for websocket and web requests (#962)

* Switch to WS

* Refactor

* Ignore utf-8 decode error during logging (#966)

* On-demand `TlsInterception` capability, driven by plugins (#965)

* Add `TlsInterceptionPropertyMixin`

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add `do_intercept` hook

* call super init

* No super from mixin as it is followed by abc?

* type ignore

* spell

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* [WebServer] Fix routing (#968)

* Fix web server routing

* Fix mypy

* pip prod(deps): bump furo from 2021.11.23 to 2022.1.2 (#959)

Bumps [furo](https://github.com/pradyunsg/furo) from 2021.11.23 to 2022.1.2.
- [Release notes](https://github.com/pradyunsg/furo/releases)
- [Changelog](https://github.com/pradyunsg/furo/blob/main/docs/changelog.md)
- [Commits](pradyunsg/furo@2021.11.23...2022.01.02)

---
updated-dependencies:
- dependency-name: furo
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com>

* pip prod(deps): bump paramiko from 2.9.1 to 2.9.2 (#970)

Bumps [paramiko](https://github.com/paramiko/paramiko) from 2.9.1 to 2.9.2.
- [Release notes](https://github.com/paramiko/paramiko/releases)
- [Changelog](https://github.com/paramiko/paramiko/blob/main/NEWS)
- [Commits](paramiko/paramiko@2.9.1...2.9.2)

---
updated-dependencies:
- dependency-name: paramiko
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com>

* Test submodule and refactor (#971)

* Refactor tests into submodules

* isort tests

* Add malicious request headers test

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Handle `OSError` on shutdown & `TimeoutError` on recv (#974)

* Expect `TimeoutError` during `recv`

* Expect `OSError` during socket shutdown, can happen if other end has already closed the socket

* pip prod(deps): bump mypy from 0.920 to 0.931 (#976)

Bumps [mypy](https://github.com/python/mypy) from 0.920 to 0.931.
- [Release notes](https://github.com/python/mypy/releases)
- [Commits](python/mypy@v0.920...v0.931)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* npm: bump @types/jquery from 3.5.4 to 3.5.13 in /dashboard (#975)

Bumps [@types/jquery](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/jquery) from 3.5.4 to 3.5.13.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/jquery)

---
updated-dependencies:
- dependency-name: "@types/jquery"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Abhinav Singh <126065+abhinavsingh@users.noreply.github.com>

* Refactor into separate `Work` module (#977)

* work module

* Fix imports

* String based typing for multiprocessing.synchronize

* Fix `test_accepts_client_from_server_socket`

* Move staticmethod outside of threadless pool class

* Fix doc build

* Fix test mock

* mp grouped together

* pylint happy

* import only for type checking

* doc build

* wrong import order

* Type fixes (#979)

* Type fixes

* Type fix

* Py class role

* unused any import

* [TlsInterception] GHA integration tests (#981)

* Add TLS interception integration tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fixture to gen certificate once for the `test_integration` module

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Use https endpoints in tls interception tests

* Fix modify post data integration test

* Only start 3 acceptor & 3 workers during integration run

* disable chunk response

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* npm: bump typescript from 3.9.7 to 4.5.4 in /dashboard (#982)

Bumps [typescript](https://github.com/Microsoft/TypeScript) from 3.9.7 to 4.5.4.
- [Release notes](https://github.com/Microsoft/TypeScript/releases)
- [Commits](microsoft/TypeScript@v3.9.7...v4.5.4)

---
updated-dependencies:
- dependency-name: typescript
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* pip prod(deps): bump types-paramiko from 2.8.6 to 2.8.9 (#983)

Bumps [types-paramiko](https://github.com/python/typeshed) from 2.8.6 to 2.8.9.
- [Release notes](https://github.com/python/typeshed/releases)
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-paramiko
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Pass separate `--ca-cert-dir` flag for parallel TLS interception tests (#984)

* Pass separate `--ca-cert-dir` flag for parallel TLS interception tests

* Temp disable `test_modify_post_response_integration`

* mock ca cert dir

* Is threaded an issue with TLS interception?

* Disable modify chunk response for python < 3.10

* Disable modify chunk response for python < 3.10

* [TlsInterception] Fix broken `ChunkedResponsePlugin` for `Python < 3.10` (#986)

* Add TLS interception integration tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix CI

* unused

* Well 3.9 just worked locally

* Dispatching empty byte to client results in OSError for Python < 3.10

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Uncomment old integration tests

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Add `# pragma: no cover` for unnecessary code (#987)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants