Proxy not used when crawling on localhost network

[GuilloOme]

When launching a crawl, it seems that only the start url and robots.txt are requested through the proxy (during the validation process).

way to reproduce:

• start a crawl with:
  $ ./htcap.py crawl -v -p http:127.0.0.1:8080 http://localhost/index.html test.db
  you get:

Initializing . . done
Database test.db initialized, crawl started with 10 threads
crawl result for: link GET http://localhost/index.html  
  new request found link GET http://localhost/test1.html 
crawl result for: link GET http://localhost/test1.html  
  new request found link GET http://localhost/test2.html 
  new request found link GET http://localhost/index.html 
crawl result for: link GET http://localhost/test2.html  
  new request found link GET http://localhost/test1.html 
  new request found link GET http://localhost/index.html 

Crawl finished, 3 pages analyzed in 0 minutes

• I only got 2 hits in the proxy log:
  • http://…/index.html
  • http://…/robots.txt

[GuilloOme]

After investigating, it seems that phantomjs is not happy with playing on the localhost network…
I filled a bug #14808 to them.

[GuilloOme]

It's seems a behavior of QT (the lib used by phantomJS) ; unfortunately, it's not be "fixable"… (see this response)