-
Notifications
You must be signed in to change notification settings - Fork 0
/
linksUteis.txt
35 lines (29 loc) · 1.58 KB
/
linksUteis.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
---------ROBOTS----------
http://www.robotstxt.org/orig.html
https://docs.python.org/3/library/urllib.robotparser.html
https://en.wikipedia.org/wiki/Robots_exclusion_standard
https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html?highlight=parse#urllib3.util.parse_url
https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html?highlight=parse#urllib3.util.Url
https://morioh.com/p/b4ea3a076f3a
https://developers.google.com/search/docs/advanced/robots/robots_txt
----------UNIXTIME---------
https://www.codegrepper.com/code-examples/python/how+to+get+unix+timestamp+in+python
----------WARC-------------
https://pypi.org/project/warcio/
https://github.com/webrecorder/warcio
---------URLLIB3---------
https://urllib3.readthedocs.io/en/stable/index.html
--------PYTHON-----------
https://realpython.com/python-property/#providing-write-only-attributes
https://dbader.org/blog/priority-queues-in-python
https://dbader.org/blog/queues-in-python
https://docs.python.org/3/library/queue.html
https://github.com/seomoz/reppy
-----Multiprocessing----
https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Pipe
https://docs.python.org/3/library/multiprocessing.html#exchanging-objects-between-processes
https://docs.python.org/3/library/concurrent.futures.html#processpoolexecutor
http://pythonclub.com.br/paralelismo-em-python-usando-concurrent.futures.html
----------EXTRAIR LINKS----------------
https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html#urllib3.util.parse_url
https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html#urllib3.util.Url