Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: pdf argument #4

Merged
merged 1 commit into from
Sep 12, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .github/workflows/pdf.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: pdf render
on:
push:
branches:
- master
pull_request:
branches:
- master

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Install Python 3
uses: actions/setup-python@v1
with:
python-version: 3.8
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python setup.py install
- name: Download book with pdf render
run: make pdf
3 changes: 3 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,9 @@ asmhentai:

bulk:
tomoe --bulk doujin.json

pdf:
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see a new target testing here, but no new workflows was added, If you want to add testing for pdf could you add the new .github/workflows ci too?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

workflow added

tomoe --nhentai 255369 --pdf

api-mock: # check api if something down
python -m unittest test.test_api
Expand Down
74 changes: 39 additions & 35 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,12 @@
<a href="https://codeclimate.com/github/sinkaroid/tomoe/maintainability"><img src="https://api.codeclimate.com/v1/badges/a729e38da1fe1ee520b1/maintainability" /></a>
</p>

Tomoe is a CLI tool for downloading doujinshi from various doujinboards. It's also has built-in modular bulk downloads, and has auto render into PDF (**Portable Document Format**) for it's utility, hopefully will be reusable.
Tomoe is a CLI tool for downloading doujinshi from various doujinboards. It's also has built-in modular bulk download and auto render into PDF (**Portable Document Format**) for it's utility, hopefully will be reusable.

<a href="#options">🚀 Commands</a> •
<a href="https://github.com/sinkaroid/tomoe/wiki">Documentation</a> •
<a href="https://github.com/sinkaroid/tomoe/issues/new/choose">Report Issues</a>

</div>

- [Tomoe](#)
Expand All @@ -30,37 +31,37 @@ Tomoe is a CLI tool for downloading doujinshi from various doujinboards. It's al
- [Acknowledgements](./CLOSING_REMARKS.md)
- [Legal](#legal)
- [EoF](#eof)

---

## Tomoe vs. the doujinboards
Tomoe consumes [Jandapress](https://github.com/sinkaroid/jandapress) and uses [janda](https://pypi.org/project/janda/) for the doujin Python library.

**Features availability** Speed or perfomace may not accurate because internet connection or API response. Some tests has high resolve time and rendering a bit longer, because some source does not providing real extension of a images, tomoe should check and guessing it's format.
Tomoe consumes [Jandapress](https://github.com/sinkaroid/jandapress) and uses [janda](https://pypi.org/project/janda/) for the doujin Python library.

| Site | Status | Bulk download | Average response | Downloaded |
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ | ------------- | ---------------------- |
| `nhentai` | [![Nhentai download](https://github.com/sinkaroid/tomoe/workflows/nhentai/badge.svg)](https://github.com/sinkaroid/tomoe/actions/workflows/nhentai.yml) | `Yes` | ~0.52 minutes | ~10.39 MB (26 images) |
| `pururin` | [![Pururin download](https://github.com/sinkaroid/tomoe/workflows/pururin/badge.svg)](https://github.com/sinkaroid/tomoe/actions/workflows/pururin.yml) | `Yes` |~0.63 minutes | ~15.55 MB (20 images) |
| `hentaifox` | [![Hentaifox download](https://github.com/sinkaroid/tomoe/workflows/hentaifox/badge.svg)](https://github.com/sinkaroid/tomoe/actions/workflows/hentaifox.yml) | `Yes` |~0.33 minutes | ~8.18 MB (26 images) |
| `hentai2read` | [![Hentai2read download](https://github.com/sinkaroid/tomoe/workflows/hentai2read/badge.svg)](https://github.com/sinkaroid/tomoe/actions/workflows/hentai2read.yml) | `Yes` |~0.10 minutes | ~7.78 MB (26 images) |
| `simply-hentai` | [![Simply-hentai download](https://github.com/sinkaroid/tomoe/workflows/simplyh/badge.svg)](https://github.com/sinkaroid/tomoe/actions/workflows/simply.yml) | `Yes` |~0.29 minutes | ~42.61 MB (19 images) |
| `asmhentai` | [![Asmhentai download](https://github.com/sinkaroid/tomoe/workflows/asmhentai/badge.svg)](https://github.com/sinkaroid/tomoe/actions/workflows/asmhentai.yml) | `Yes` |~0.23 minutes | ~4.96 MB (23 images) |
**Features availability** Speed or performance may not be accurate because internet connection or API response. Some tests have high resolve time and render a bit longer, because some sources do not provide real extension of an images, tomoe should check and guess it's format.

| Site | Status | Bulk download | Average response | Downloaded |
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | ---------------- | --------------------- |
| `nhentai` | [![Nhentai download](https://github.com/sinkaroid/tomoe/workflows/nhentai/badge.svg)](https://github.com/sinkaroid/tomoe/actions/workflows/nhentai.yml) | `Yes` | ~0.52 minutes | ~10.39 MB (26 images) |
| `pururin` | [![Pururin download](https://github.com/sinkaroid/tomoe/workflows/pururin/badge.svg)](https://github.com/sinkaroid/tomoe/actions/workflows/pururin.yml) | `Yes` | ~0.63 minutes | ~15.55 MB (20 images) |
| `hentaifox` | [![Hentaifox download](https://github.com/sinkaroid/tomoe/workflows/hentaifox/badge.svg)](https://github.com/sinkaroid/tomoe/actions/workflows/hentaifox.yml) | `Yes` | ~0.33 minutes | ~8.18 MB (26 images) |
| `hentai2read` | [![Hentai2read download](https://github.com/sinkaroid/tomoe/workflows/hentai2read/badge.svg)](https://github.com/sinkaroid/tomoe/actions/workflows/hentai2read.yml) | `Yes` | ~0.10 minutes | ~7.78 MB (26 images) |
| `simply-hentai` | [![Simply-hentai download](https://github.com/sinkaroid/tomoe/workflows/simplyh/badge.svg)](https://github.com/sinkaroid/tomoe/actions/workflows/simply.yml) | `Yes` | ~0.29 minutes | ~42.61 MB (19 images) |
| `asmhentai` | [![Asmhentai download](https://github.com/sinkaroid/tomoe/workflows/asmhentai/badge.svg)](https://github.com/sinkaroid/tomoe/actions/workflows/asmhentai.yml) | `Yes` | ~0.23 minutes | ~4.96 MB (23 images) |

## Features

- Plenty of sites coverage
- Built-in auto render into PDF
- Modular bulk downloads
- Modular bulk download
- Minimalist dependencies
- Download with ease, doesn't require you a lot of arguments

<img src="https://cdn.discordapp.com/attachments/997107089921028136/1014351915578040380/tomoe_.png" width="600" alt="tomoe">

## Site support

Currently tomoe support the following websites:
Currently tomoe supports the following websites:

- [nhentai.net](https://nhentai.net/)
- [pururin.to](https://pururin.to/)
Expand All @@ -69,19 +70,17 @@ Currently tomoe support the following websites:
- [simply-hentai.com](https://simply-hentai.com/)
- [asmhentai.com](https://asmhentai.com/)


## Prerequisites
## Prequisites

<table>
<td><b>NOTE:</b> Python 3.7 or above</td>
</table>

## Installation

`pip install tomoe` / `pipenv install tomoe`

- Or manual build by cloning the repository and running `python setup.py install`
`pip install tomoe` / `pipenv install tomoe`

- Or manual build by cloning the repository and run `python setup.py install`

## Command usage

Expand All @@ -97,18 +96,20 @@ After that, you could see the download results or throw you an error if somethin

It's fairly simple to use tomoe

| **Argument** | **Description** | **Example** |
| --------------------------------- | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| --nhentai, int | nhentai | [`tomoe --nhentai 255369`](https://nhentai.net/g/255369/) |
| --pururin, int | pururin | [`tomoe --pururin 47226`](https://pururin.to/gallery/47226/crot-sampe-lumpuh) |
| --hentaifox, int | hentaifox | [`tomoe --hentaifox 59026`](https://hentaifox.com/gallery/59026/) |
| --hentai2read, str chapter:number | hentai2read | [`tomoe --hentai2read chaldea_life:1`](https://hentai2read.com/chaldea_life/) |
| --simply, str | simply-hentai | [`tomoe --simply "fate-grand-order/fgo-no-ashibon-fgo-foot-book/all-pages"`](https://www.simply-hentai.com/fate-grand-order/fgo-no-ashibon-fgo-foot-book/all-pages) |
| --asmhentai, int | asmhentai | [`tomoe --asmhentai 311851`](https://asmhentai.com/g/311851/) |
| --bulk, str | custom bulk download | [`tomoe --bulk doujin.json`](/doujin.json) |
| **Argument** | **Description** | **Example** |
| --------------------------------- | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| --nhentai, int | nhentai | [`tomoe --nhentai 255369`](https://nhentai.net/g/255369/) |
| --pururin, int | pururin | [`tomoe --pururin 47226`](https://pururin.to/gallery/47226/crot-sampe-lumpuh) |
| --hentaifox, int | hentaifox | [`tomoe --hentaifox 59026`](https://hentaifox.com/gallery/59026/) |
| --hentai2read, str chapter:number | hentai2read | [`tomoe --hentai2read chaldea_life:1`](https://hentai2read.com/chaldea_life/) |
| --simply, str | simply-hentai | [`tomoe --simply "fate-grand-order/fgo-no-ashibon-fgo-foot-book/all-pages"`](https://www.simply-hentai.com/fate-grand-order/fgo-no-ashibon-fgo-foot-book/all-pages) |
| --asmhentai, int | asmhentai | [`tomoe --asmhentai 311851`](https://asmhentai.com/g/311851/) |
| --bulk, str | custom bulk download | [`tomoe --bulk doujin.json`](/doujin.json) |
| --pdf, str | render pdf for each title | [`tomoe --nhentai 255369 --pdf`](https://nhentai.net/g/255369/) |

## Bulk download
You can passing multiple id to requests bulk download

You can pass multiple id to request bulk download

| **Sites** | **Description** | **Example** |
| ----------- | ------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
Expand All @@ -120,12 +121,14 @@ You can passing multiple id to requests bulk download
| asmhentai | place multiple id | `tomoe --asmhentai 311851 210135 309068` |

## Bulk download using nested list
`tomoe --bulk doujin.json`

Since this tool covers plenty of website, If You need to download from each multiple different website You can make bulk download with arbitrary methods, Meant you can mix and wrap the whole book id into nested array in a JSON file. Read more about [Understanding Nested Arrays.](https://javascript.plainenglish.io/understanding-the-nested-arrays-fbf3ab13c2b4#:~:text=An%20array%20is%20an%20ordered,the%20element%20of%20an%20array)
`tomoe --bulk doujin.json`

If you need to download multiple doujins from each different website, you can do a bulk download using an arbitrary method. Meaning that you can mix and wrap the whole book ids into nested array in a JSON file. Read more about [Understanding Nested Arrays.](https://javascript.plainenglish.io/understanding-the-nested-arrays-fbf3ab13c2b4#:~:text=An%20array%20is%20an%20ordered,the%20element%20of%20an%20array)

Create `doujin.json` in the same directory where you want to run tomoe and follow this structure:
**Note** _You should not change the "book" property_

Create `doujin.json` in the same directory You want to run tomoe and follow this structure:
**Note** *You should not change the "book" property*
```js
{
"book": [
Expand Down Expand Up @@ -163,13 +166,14 @@ Create `doujin.json` in the same directory You want to run tomoe and follow this
}
```

Then tells tomoe to resolve all the book id from the JSON file.
Then tell tomoe to resolve all the book ids from the JSON file.
`tomoe --bulk doujin.json`

## Legal

This tool can be freely copied, modified, altered, distributed without any attribution whatsoever. However, if you feel
like this tool deserves an attribution, mention it. It won't hurt anybody.

> Licence: WTF.

## Pronounciation
Expand All @@ -178,4 +182,4 @@ like this tool deserves an attribution, mention it. It won't hurt anybody.

## EoF

All books from those third-party doujinboards are definitely ilegal from original authors. Support the authors by buying the original book.
All books from those third-party doujinboards are definitely illegal from original authors. Support the authors by buying the original book.
2 changes: 1 addition & 1 deletion tomoe/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
__version__ = "3.0.13"
__version__ = "3.0.14"
from .pururin import get_pur
from .nhentai import get_nh
from .hentaifox import get_hfox
Expand Down
49 changes: 11 additions & 38 deletions tomoe/asmhentai.py
Original file line number Diff line number Diff line change
@@ -1,25 +1,27 @@
import asyncio
import janda
import requests
import os
import re
import time

import janda
import requests
from inputimeout import TimeoutOccurred

from .pdf import process_pdf
from .utils.const import TOMOE_HTML
from .utils.misc import (
choose,
split_name,
get_size,
convert_html_to_pdf,
project,
get_size_folder,
log_data,
log_file,
log_final,
log_warn,
project,
split_name,
)
from inputimeout import inputimeout, TimeoutOccurred

asmh = janda.Asmhentai()
t: str = "tomoe.html"


async def get_asm(ids=choose().asmhentai):
Expand Down Expand Up @@ -101,7 +103,7 @@ async def process_asmhentai(id: int):
f"{(time.time() - initial) / 60:.2f}", get_size_folder(neat_dir)
)

with open(neat_dir + "/" + t, "x", encoding="utf-8") as f:
with open(neat_dir + "/" + TOMOE_HTML, "x", encoding="utf-8") as f:
f.write("<html><center><body>")
f.write(f"<h1>{parser['data']['id']}</h1>")

Expand All @@ -128,35 +130,6 @@ async def process_asmhentai(id: int):
print(f"Directory: {os.path.abspath(neat_dir)}")

try:
desired = inputimeout(
prompt="Do you want to render it all to .pdf? (y/n) ",
timeout=10,
)
to_pdf = desired

if to_pdf == "y":
try:
source = open(f"{neat_dir}/{t}")
output = f"{neat_dir}/{parser['data']['id']}.pdf"
filepdf = output.rsplit("/", 1)[-1]

convert_html_to_pdf(source, output)
print(
f"Successfully rendered to {filepdf} | {get_size(output)} MB"
)

except Exception as e:
print(f"Something went wrong while converting to pdf: {e}")

elif to_pdf == "n":
print("Okay")
os.remove(neat_dir + "/" + t)
return

else:
print("Invalid input")
os.remove(neat_dir + "/" + t)
return

process_pdf(neat_dir, parser["data"]["id"])
except TimeoutOccurred:
print(f"Timeout occurred, not rendering pdf {id}")
49 changes: 11 additions & 38 deletions tomoe/hentai2read.py
Original file line number Diff line number Diff line change
@@ -1,25 +1,27 @@
import asyncio
import janda
import requests
import os
import re
import time

import janda
import requests
from inputimeout import TimeoutOccurred

from .pdf import process_pdf
from .utils.const import TOMOE_HTML
from .utils.misc import (
choose,
split_name,
get_size,
convert_html_to_pdf,
project,
get_size_folder,
log_data,
log_file,
log_final,
log_warn,
project,
split_name,
)
from inputimeout import inputimeout, TimeoutOccurred

h2r = janda.Hentai2read()
t: str = "tomoe.html"


async def get_h2r(ids=choose().hentai2read):
Expand Down Expand Up @@ -86,7 +88,7 @@ async def process_h2r(id: str, chapter: int):
)
print(f"Directory: {os.path.abspath(neat_dir)}")

with open(neat_dir + "/" + t, "x", encoding="utf-8") as f:
with open(neat_dir + "/" + TOMOE_HTML, "x", encoding="utf-8") as f:
f.write("<html><center><body>")
f.write(f"<h1>{parser['data']['title']}</h1>")

Expand All @@ -99,36 +101,7 @@ async def process_h2r(id: str, chapter: int):
f.close()

try:
desired = inputimeout(
prompt="Do you want to render it all to .pdf? (y/n) ",
timeout=10,
)
to_pdf = desired

if to_pdf == "y":
try:
source = open(f"{neat_dir}/{t}")
output = f"{neat_dir}/{parser['data']['title']}.pdf"
filepdf = output.rsplit("/", 1)[-1]

convert_html_to_pdf(source, output)
print(
f"Successfully rendered to {filepdf} | {get_size(output)} MB"
)

except Exception as e:
print(f"Something went wrong while converting to pdf: {e}")

elif to_pdf == "n":
print("Okay")
os.remove(neat_dir + "/" + t)
return

else:
print("Invalid input")
os.remove(neat_dir + "/" + t)
return

process_pdf(neat_dir, parser["data"]["title"])
except TimeoutOccurred:
print(
f"Timeout occurred, not rendering pdf {id}, chapter: {chapter}"
Expand Down
Loading