Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drop loguru and use builtin logging #1133

Merged
merged 3 commits into from
Oct 29, 2023
Merged

Conversation

lopagela
Copy link
Contributor

  • Changed the 2 existing print in the private_gpt code base into actual python logging, stop using loguru (dependency will be dropped in a later commit).
  • Try to use the key=value logging convention in logs (to indicate what dynamic values represents, and what is dynamic vs not).
  • Using %s log style, so that the string formatting is pushed inside the logger, giving the ability to the logger to determine if the string need to be formatted or not (i.e. strings from debug logs might not be formatted if the log level is not debug)
  • The (basic) builtin log configuration have been placed in private_gpt/__init__.py in order to initialize the logging system even before we start to launch any python code in private_gpt package (ensuring we get any initialization log formatted as we want to)
  • Disabled uvicorn custom logging format, resulting in having uvicorn logs being outputted in our formatted.

Some more concise format could be used if we want to, especially:

COMPACT_LOG_FORMAT = '%(asctime)s.%(msecs)03d [%(levelname)s] %(name)s - %(message)s'

Python documentation and cookbook on logging for reference:

@lopagela
Copy link
Contributor Author

Results after this PR:
Screenshot 2023-10-29 at 17 07 34

When enabling DEBUG level:
Screenshot 2023-10-28 at 17 14 17

@@ -5154,20 +5136,6 @@ files = [
[package.extras]
test = ["pytest (>=6.0.0)", "setuptools (>=65)"]

[[package]]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Highlighting this one -- It has been removed as a side effect of poetry remove loguru


How should poetry.lock be managed? Should it be generated by some CICD tool? Or maintained by core dev of this project?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should ignore for the most part, it's just for consistent builds

pabloogc
pabloogc previously approved these changes Oct 29, 2023
@@ -26,6 +27,8 @@
CompletionResponseGen,
)

log = logging.getLogger(__name__)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tiny NIT but could you rename this to logger (and in the other file too)?

Will be more consistent naming wise for the future.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure sorry - I'm used to Java's Lombok @Log4j2 that injects log variable into classes 😅

I personally preferring log as it's shorter than logger (it makes lines shorter and seems more readable to me), but no problem, changing to logger

@pabloogc
Copy link
Collaborator

Great changes, thanks for addressing the old prints we left around.

louis-jaris and others added 3 commits October 29, 2023 18:58
Changed the 2 existing `print` in the `private_gpt` code base into actual python logging, stop using loguru (dependency will be dropped in a later commit).
Try to use the `key=value` logging convention in logs (to indicate what dynamic values represents, and what is dynamic vs not).
Using `%s` log style, so that the string formatting is pushed inside the logger, giving the ability to the logger to determine if the string need to be formatted or not (i.e. strings from debug logs might not be formatted if the log level is not debug)
The (basic) builtin log configuration have been placed in `private_gpt/__init__.py` in order to initialize the logging system even before we start to launch any python code in `private_gpt` package (ensuring we get any initialization log formatted as we want to)
Disabled `uvicorn` custom logging format, resulting in having uvicorn logs being outputted in our formatted.

Some more concise format could be used if we want to, especially:
```
COMPACT_LOG_FORMAT = '%(asctime)s.%(msecs)03d [%(levelname)s] %(name)s - %(message)s'
```

Python documentation and cookbook on logging for reference:
* https://docs.python.org/3/library/logging.html
* https://docs.python.org/3/howto/logging.html
Result of `poetry remove loguru`
@pabloogc pabloogc merged commit 64c5ae2 into zylon-ai:main Oct 29, 2023
6 of 7 checks passed
NetroScript added a commit to NetroScript/privateGPT that referenced this pull request Oct 29, 2023
@lopagela lopagela deleted the better-logging branch October 29, 2023 22:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants