Maybe just some Warning!

[Indeserpen]

Hello!

First amazing script, lately got more familiar with it, and now use almost in every security assesment! It helps me to get very clear insight of web application and makes job more easy! Have some ideas and functions what maybe could be added! When ready gonna let you know you!

Second thing i noticed that on large scale crawl, I,m getting this error: Database karta.db initialized, crawl started with 10 threads
[==== ] 966 of 7928 pages processed in 67 minutes Exception in thread Thread-9:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/share/htcap/core/crawl/crawler_thread.py", line 62, in run
self.crawl()
File "/usr/share/htcap/core/crawl/crawler_thread.py", line 212, in crawl
probe = self.send_probe(request, errors)
File "/usr/share/htcap/core/crawl/crawler_thread.py", line 161, in send_probe
probeArray = self.load_probe_json(jsn)
File "/usr/share/htcap/core/crawl/crawler_thread.py", line 99, in load_probe_json
return json.loads(jsn)
File "/usr/lib/python2.7/json/init.py", line 339, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 367, in decode
raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 4 column 1 - line 4 column 249 (char 58 - 306)
Maybe its just my system warning! I,m Using Kali Linux but its highly modified and build up for me and maybe because of this!

It seems that there is no impact on work flow, it continues crawling and after when working on database there is no error!

Anyways good script and good luck developing it!

[segment-srl]

Hello!
Many thanks for your feedback! I'm always looking for good advices from users, so please feel free to contact me for any feature request, advice ecc.. !

About the warning, I believe that the json returned from phantomjs contains some garbage so the parser just rises an alert. I'm trying to figure out why it happens.

Btw I just pushed some changes so maybe the problem got fixed...

[GuilloOme]

It seems to be the same error as the one in #11

[GuilloOme]

@Indeserpen, what version of PhantomJS do you run in your Kali? Is it the one from the repo?

[segment-srl]

This issue is related to phantomjs build on some linux distros. Using the binary from the officail website should fix the problem.
Since phantomjs is no more supported, htcap is now moving to headless chrome so issue similar to this one won't be fixed.

[Tren]

Ok wait for htcap update!