Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gutenberg logs are all under the same name #206

Open
benoit74 opened this issue Aug 21, 2023 · 3 comments
Open

Gutenberg logs are all under the same name #206

benoit74 opened this issue Aug 21, 2023 · 3 comments

Comments

@benoit74
Copy link
Collaborator

In Gutenberg logs, only one logger name (gutenberg2zim.constants) is used making it pretty useless.

[gutenberg2zim.constants::2023-08-19 11:40:30,563] INFO:	Parsing file cache/epub/99/pg99.rdf for book id 99
[gutenberg2zim.constants::2023-08-19 11:40:31,442] INFO:	Parsing file cache/epub/9/pg9.rdf for book id 9
[gutenberg2zim.constants::2023-08-19 11:40:32,515] INFO:Add possible url to db
[gutenberg2zim.constants::2023-08-19 11:40:32,517] DEBUG:bash -c rsync -a --list-only rsync://aleph.pglaf.org/gutenberg/ > tmp/file_on_aleph_pglaf_org

We should not log the name anymore and instead log the filename with %(filename)s or module with %(module)s

@rgaudin
Copy link
Member

rgaudin commented Aug 21, 2023

We've found that a single name is enough in most scrapers so we use the name to distinguish our logs from the other dependencies. Here it should use gutenberg2zim instead of the module name.
We could use different name base on file or module but it brings little value and make the logs very difficult to read because lines are not aligned (prefix size changes)

@elfkuzco
Copy link
Contributor

@benoit74 , I would like to implement this. Should I stick with keeping the module names or just use gutenberg2zim as @rgaudin suggested?

@benoit74
Copy link
Collaborator Author

Just use one name, gutenber2zim as suggested by @rgaudin

And please adapt the code to create the logger with scraperlib getLogger function like we try to harmonize among our codebase.

One good example of this approach is in offspot/demo:

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants