Bounty: Investigate solution how to solve blocked Tor connection in China

[ManfredKarrer]

Bitsquare cannot be used inside China mainland because Tor is blocked by the (not so) great Firewall.

We want to set up a bounty for finding the best technical solution to overcome that problem.
It might be possible to solve by using a VPN, though it would still be a big hurdle for many users (https://forum.bitsquare.io/t/using-bitsquare-in-mainland-china). If we find a way to integrate a VPN inside Bitsquare so the user has no setup costs, that might be an interesting approach.
If VPN does not solve that issue, Tor Bridges or Tors pluggable transports might be solutions.
Please note that we use Tor Hidden Services, so Bitsquare Tor traffic is never leaving the Tor network to the clearnet (via exit nodes), so it is a different use case as usual.

[meapistol]

I add 1 BTC to this bounty. I was in fact considering this bounty myself but you beat me to it. VPN might not be necessary if Bitsquare is hosted outside China. Tor Bridges or Tor pluggable transport will most likely not work. I tried all these things without success in Wuhan. VPN is throttled back by the Chinese authorities as well.

[ManfredKarrer]

Got first input from a friend that it should work by extending the config file with Bridges. If that does not work then pluggable Transports should work but that needs inclusion of extra binaries.

[agardener]

@ManfredKarrer I have been in the process of testing modifications of the torr config as well as testing over hyperboria for most of the day. What is required to take this bounty?

[ManfredKarrer]

@agardener: A solution which works for mainland China. We need some testers there to verify that it works.

[agardener]

If someone in China wants to test bridges in Bitsquare you can use these steps:

• run Bitsquare (Bitsquare should be running during this entire process)

• Goto https://bridges.torproject.org and get some fresh bridges

• choose 'none' on pluggable transports option (for now)

• open ~/.local/share/Bitsquare/mainnet/tor/torrc in a text edditor and add the bridges that you got from bridges.torproject.org at the bottom of the file, it should look something like:

  AvoidDiskWrites 1
  ControlPort auto
  CookieAuthentication 1
  CookieAuthFile /home/bob/.local/share/Bitsquare/mainnet/tor/.tor/control_auth_cookie
  DataDirectory /home/bob/.local/share/Bitsquare/mainnet/tor
  GeoIPFile geoip
  GeoIPv6File geoip6
  HiddenServiceDir /home/bob/.local/share/Bitsquare/mainnet/tor/hiddenservice
  HiddenServicePort 9999 127.0.0.1:57169
  PidFile pid
  RunAsDaemon 1
  SafeSocks 1
  SocksPort auto

  UseBridges 1
  bridge 63.142.157.162:9001 9C27FABC13D868F94F7377D17C22998683AFCCAD
  bridge 107.23.146.44:8443 08D75007901C33900733E2D0CF51B6F196DA2824
  bridge 185.22.52.223:443 BA3AACD41D8F7EE69350CC3769BC0C9730049AEB

• Now we need to restart tor, find the PID by running something like ps -aux|grep torrc or top. You will see a line that looks like (the PID below is 1361):

  bob 1361 2.0 3.7 20384 19016 ? S Jun11 0:35 /home/bob/.local/share/Bitsquare/mainnet/tor/tor -f /home/bob/.local/share/Bitsquare/mainnet/tor/torrc __OwningControllerProcess 1319

• Now we send the SIGHUP signal to Tor in order to reload the config

  kill -SIGHUP 1361

• In order to see if a connection has been made run netstat using one of your bridge IPs

  netstat -an|grep 63.142.157.162

• or view all of your tcp connections

  netstat -an|grep tcp

• Check status of BitSquare window

This is a quick way to get up and testing Bitsquare with bridges, I am happy to provide more support to any users willing to test. Also if we discover that basic bridges do not function I can provide additional configuration information for pluggable transports, but as ManfredKarrer pointed out it will be as simple as including additional binaries. It is not currently state aware as it appears tor is creating a new config every time Bitsquare restarts - I believe this could be fixed by creating a patch that reads torrc and runs controlConnection.setConf.

I can also help build/document other layers(like cjdns or vpn) if needed however if bridges and transports are not working in China any longer I suspect most public(free) networks will also be caught in the same web.

[ManfredKarrer]

@agardener Thanks a lot!

I think I will do the following to make it as easy for users as possible:
Bitsquare starts up. If Tor connection does not work a screen gets displayed to ask user if he wants to use Bridges. Unfortunately the Brigdes webpage use terrible captchas, maybe i find a way to provide the user directly current addresses, otherwise the user has to do that manually.
After the Bridge addresses are added, Bitsquare restarts and use those addresses in the torrc at the next start. Alternatively a internal restart of tor might be even better.
Do you have an idea how to get the bridge addresses automatically (avodi user to enter captcha and go to webpage but load them form bitsquare)? Probably not possible as it is designed to avoid that, so they cannot get blocked easily....

[ManfredKarrer]

@agardener Do you have any technical contact inside China who might be available for testing it?

[MarcSte]

@ManfredKarrer oakpacific?

[agardener]

No problem! Open communications ftw!

@ManfredKarrer I think you have the right idea with allowing the user to set their bridges or alternative connection methods at startup. Socks5 proxies might be another helpful option for example (a user may wish to hide their tor traffic) - this is also defined in the torrc.

Do you have an idea how to get the bridge addresses automatically (avoid user to enter captcha and go to webpage but load them form Bitsquare)? Probably not possible as it is designed to avoid that, so they cannot get blocked easily....

Its intentionally designed to not be easily scraped. I also think collecting a list of bridges and hardcoding them would be counter productive, unless perhaps they were private bridges run by the Bitsquare team as a regular bridge might be stable today but 6 months down the road some of the users could be left in the dark . Having a solution where the user sets their own bridges and transports may provide more stability/decentralization.

I don't know anyone off hand. If no one can test I can try and see if there is a VPS available... not sure how easy that is to find though I have seen availability in hong-kong.

[ManfredKarrer]

@agardener To run our own bridges might be an interesting option indeed. I could ping them and if they are reachable will use those, otherwise ask the user. I want to make it as easy as possible for users (they are lazy...). Users who want/are willing to run their own bridges are even better, but I assume not many will do that, though I could offer that option as well (plan that also for a private full bitcoin node to connect to instead of seednodes).
I am not sure how quickly I can work on that, have a long list of other stuff, but hope the next 1-2 weeks.

Would you be available to help to set up a bridge?

[meapistol]

Having been in China and seen the efficiency of blocking tor also when bridges and pluggable transport is used I have my doubts that any method involving tor inside China will work. Hong Kong is very different and I guess tor is allowed there. A few years ago bridges were supplied to trusted people in China by e-mail who then distributed the adresses. Hopefully tor has been able to solve these issues lately, but China has more than 30 000 people employed to harden their Firewall so it will not be easy. I can mention that I did manage to get Bitsquare to start and find seed nodes over VPN, very very slowly, when I was in Wuhan this March so Chinas deep packet inspection is not perfect. I never got it to run fully though. I don't want to be negative here and I think you should try since this issue is so important. It will be necessary to have a skilled tester in the mainland.

[agardener]

@ManfredKarrer Definitely. I could write the server configuration into some basic anisble playbooks. Not now, but later this week

@meapistol What do you think about private bridges? Think they are likely stand more of a chance?

[meapistol]

@agardener Maybe private bridges work. Maybe tor can run over "high quality" VPN. I used 12VPN. I also tried www.vpngate.net which is dedicated to China but their servers did not work at all. They did work a year ago though. Somebody from tor should be able to answer these questions. Unfortunately all my friends in China are not good at this so I cannot ask them. In fact most Chinese are satisfied with their local alternatives, Baidu, Wechat etc.

[ManfredKarrer]

I think to use private bridges might be a good start. As long Bitsquare is not relevant in China we will probably be under the radar. Later we can improve. Multipe bridges and a more dynamic address allocation might be the next step. More input from Tor devs would be definitely helpful.

[agardener]

@meapistol I think before VPN I would suggest using something like the hyperboria network, setup with internal bridges to the Tor network. This mesh format creates more decentralization. But it maybe hard to integrate into Bitsquare.

@ManfredKarrer I think that relevancy is the biggest thing to note. If Bitsquare becomes 'noticed' I believe it will have to be just as agile as Tor in allowing users to set their connections.

@ManfredKarrer @meapistol I have been involved generally with Tor/mesh R+D and QA for around 10 years. While Im not a programmer per say I may be able to answer questions, or at least research an answer.

[ManfredKarrer]

@agardener I was assuming you have deep background with Tor/networking stuff :-) Great to have you here!
I have no experience with hyperboria, but as you said that might be an extra challenge to integrate into Bitsquare. Usability (no setup effort for users) is a must-have for Bitsquare if there are any alternative solutions.
And yes I don't assume Bitsquare will get too soon into their radar, so we will have time. Also Bitsquare is not that directly political as other Tor usage, so even when they discover it they might tolerate it.

So my plan for the first version:
Setup a private bridge (maybe @ivilata can help here), use that hardcoded for torrc in case normal tor does not succeed.
Find someone in mainland China to test.
If that succeeds the bounty is fulfilled for the first goal.

Later we can improve.

[agardener]

@ManfredKarrer Thanks! I don't mind setting up and running a VPS for a bridge if the costs could be covered(I could share login credentials somewhere for someone to foot the bill). Alternatively if someone can provide ssh access I can start there. Something to note, as I have not done a ton of development in relation to the not-so-GFW it maybe true that it has a live black list. So if our initial bridges are noticed and blocked this may mean we need to move to a new IP before testing pluggable transports. I don't want to loose the bounty, but the sun is shining here and Im afraid most people are probably sleeping in china.

How can @ivilata help?

[agardener]

To answer HostFat's question obfs4 can be tested in much the same way as regular bridges.

I used the information located at the obsf4 github, I tested on debian 8

• First install go for debian do

aptitude install golang

AFAIK everything else should not be debian specific

• Now create the $GOPATH

  export GOPATH=$HOME/go
  export PATH=$PATH:$GOROOT/bin:$GOPATH/bin

• Then run the go get command to install obsf4, this also install the needed depends

  go get git.torproject.org/pluggable-transports/obfs4.git/obfs4proxy

• After a minute the binary should be ready to use, copy obfs4proxy from ~/go/bin to /usr/local/bin

  cp /home/bob/go/bin/obfs4proxy /usr/local/bin/.

• After this the setup is much the same as normal bridges as posted above, your torrc should look something like (make sure to select obsf4 bridges on bridges.torproject.org):

  AvoidDiskWrites 1
  ControlPort auto
  CookieAuthentication 1
  CookieAuthFile /home/bob/.local/share/Bitsquare/mainnet/tor/.tor/control_auth_cookie
  DataDirectory /home/bob/.local/share/Bitsquare/mainnet/tor
  GeoIPFile geoip
  GeoIPv6File geoip6
  HiddenServiceDir /home/bob/.local/share/Bitsquare/mainnet/tor/hiddenservice
  HiddenServicePort 9999 127.0.0.1:57169
  PidFile pid
  RunAsDaemon 1
  SafeSocks 1
  SocksPort auto

  ClientTransportPlugin obfs4 exec /usr/local/bin/obfs4proxy
  UseBridges 1

  bridge obfs4 195.132.208.182:42679 69C... cert=pGG... iat-mode=0
  bridge obfs4 195.132.208.105:43666 108E... cert=EXz8... iat-mode=0
  bridge obfs4 212.218.216.127:10575 D2A.... cert=fVvl... iat-mode=0

• Now we need to restart tor, find the PID by running something like ps -aux|grep torrc or top. You will see a line that looks like (the PID below is 1361):

  bob 1361 2.0 3.7 20384 19016 ? S Jun11 0:35 /home/bob/.local/share/Bitsquare/mainnet/tor/tor -f /home/bob/.local/share/Bitsquare/mainnet/tor/torrc __OwningControllerProcess 1319

• Now we send the SIGHUP signal to Tor in order to reload the config

  kill -SIGHUP 1361

• In order to see if a connection has been made run netstat using one of your bridge IPs

  netstat -an|grep 195.132.208.182

• or view all of your tcp connections

  netstat -an|grep tcp

• Check status of BitSquare window

[ManfredKarrer]

@agardener Adding obfs4proxy would require that we add that binary, so that would be more effort. Lets start first with the easiest solution and see if we succeed.

I talked to @ivilata (the dev who joined to work on the Bitsquare network module, he is more experienced with sys admin and network stuff than me...) and he agreed to handle the bridge installation with you. I leave it up to you how to proceed. I assume the VPS costs are < 20 EUR for the period we need it for testing, so no worry I will cover that. After successful test I will run my own bridges (with @ivilata's help).

I need to get a few other things done first, but then I will try to build a version with bridge support. Hope in about 1 week I have that. I might start with some stable bridge addresses and later use our private bridge.

What are the system requirements for such a VPS? I assume they are very low (1GB ram). Do u have a recommendation for a hosting?

[agardener]

@ManfredKarrer OK, no problem.

Sounds good, I would suggest going with a host that provides per minute pricing and multiple regions, instead of getting a single VPS for the month. This way the account balance can be loaded up and if we want to switch hosting region or re-deploy for a new IP there wont be any issues. @ivilata let me know where to start.

OK, in app support will be cool.

Requirements are generally bound by RAM, which is tied to transfer/users/etc. For testing I think all of this usage will be really low and being a private bridge helps. I think we would be safe with starting in the low range(~512mb) working our way up or adding other nodes as required to do so.

I don't have a specific host in mind but there are a few hosts here that provide per minute hosting and with multiple regions: http://www.cryto.net/~joepie91/bitcoinvps.html .

Any testers in China yet?

[ManfredKarrer]

@agardener Cool!
No I have no testers yet, but at least 2 or 3 contacts who I will ask as soon we have a binary ready for testing. I will release a new Bitsquare version soon and maybe I can directly add it to that version already.

[ManfredKarrer]

@agardener Is it ok for you that you setup your own VPS and hand over the setup scripts/configuration after testing succeeded so @ivilata can install it on my VPS. If you want I can pay you in advance the estimated costs for VPS, lets say for 1 month.

[agardener]

@ManfredKarrer Sure.

You can use this address 19q8sKjicr4SKzB5FbK85HiqqJoRWvEUiS. What ever you send I will forward to the hosting services, I think it will cost about 0.0057 per hour.

[ManfredKarrer]

It was part of the master branch until 0.4.9 I guess. But it did not help so we removed it. It needs Meek/Obfusc4 but we have not start working on that. If you are experienced in that get in touch.

[ABISprotocol]

This is the thing I gather? @ahf

[CR0CKER]

The only reliable way to connect to Tor without VPN from China is the meek-amazon pluggable transport. This has been the case for a long time. Bridges don't work for longer than a couple weeks before they are blocked, and are difficult for the average user to obtain without VPN. Happy to test if someone integrates meek-amazon and meek-azure into Bitsquare.

[ManfredKarrer]

@CR0CKER: Thanks a lot for the info and your offer to help testing. As soon we have resources to work on it we will come back to you.

[ghost]

New to Bisq by way of Dash. I'm based in Mainland China with software testing experience, willing to help testing new builds to get Bisq working in China. I agree that domain fronting using the meek pluggable transport is the most promising approach here.

[meapistol]

Just some information. I got Bisq to work in Wuhan, China, using 12VPN (https://12vpn.net/) first and then starting another VPN to my job inside 12VPN followed by Bisq. It worked for at least a week until I left China. Using one VPN did not work.

[hanzac]

I've been able to connect to the network by changing the below files:
C:\Users\NT_USER_NAME\AppData\Local\Bisq\app\Bisq.jar\torrc:
ControlPort auto
CookieAuthentication 1
DisableNetwork 1
AvoidDiskWrites 1
PidFile pid
RunAsDaemon 1
SafeSocks 0
SOCKSPort 9050
Socks5Proxy 127.0.0.1:1080

C:\Users\NT_USER_NAME\AppData\Local\Bisq\app\Bisq.jar\socks.properties:
port = 9050
proxy = 127.0.0.1:9050

The main idea here is using a socks5 or http or https proxy provided by VPN or ShadowSocks. I'm running on the Windows.
Hope these configuration can be moved to outside or there will be some UI or documentation.

[ManfredKarrer]

@hanzac: Why you could not connect by default? Is Tor blocked in your country? If not maybe a firewall or blocking ISP was the reason?

[hanzac]

@ManfredKarrer Yes. Tor is blocked and I need to use proxy to connect to Tor network.

[kdaye]

Thanks @hanzac
I have add Socks5Proxy 127.0.0.1:1080 to C:\Users\NT_USER_NAME\AppData\Local\Bisq\app\Bisq.jar\torrc:,and it is working with shaodowsocksR.
can not connect tor network resolved .

@ManfredKarrer
My country blocked Tor network ,but Tor bridge obfs4 is working.
and the bisq not working default mode.

[taylorshuang]

@hanzac, Tor is blocked in my country, i follow kdaye's method , just add Socks5Proxy 127.0.0.1:1080 to C:\Users\NT_USER_NAME\AppData\Local\Bisq\app\Bisq.jar\torrc, and i can't connect to Tor network . Do I need to modify SOCKSPort 9050 in C:\Users\NT_USER_NAME\AppData\Local\Bisq\app\Bisq.jar\torrc and C:\Users\NT_USER_NAME\AppData\Local\Bisq\app\Bisq.jar\socks.properties:?

[ManfredKarrer]

If anyone wants to work on pluggable transports integration for Bisq, here is the lib from @JesusMcCloud : https://github.com/JesusMcCloud/netlayer/
That would need to be integrated in bisq and replace the current tor libs. And would need a lot of testing of course...

[seanlis]

FYI.
I could use bisq in China with a VPN set up on my US based VPS.

[taylorshuang]

@seanlis, Hi, dude, can you tell me how to connect Bisq use US vps. I have a virtua machine on amazon cloud, and use shodowsocks as a VPS, but i can't to connect Bisq in China.

[taylorshuang]

I have solved my problem with a VPN , thanks!

[kdaye]

@taylorshuang you forgot open the local port(1080) for Bisq ,yep?_

[hanzac]

@taylorshuang I modified C:\Users\NT_USER_NAME\AppData\Local\Bisq\app\Bisq.jar\socks.properties, because I make tor to serve at 9050 port.

[taylorshuang]

@hanzac, OK, Thanks. It is good idea to directly buy a VPN and no need modify any.

[raidsan]

each time I launch bisq-desktop.bat, the following file will be re-create
C:\Users\username\AppData\Roaming\Bisq\btc_mainnet\tor\torrc
C:\Documents and Settings\username\AppData\Roaming\Bisq\btc_mainnet\tor\torrc

It prevent I add proxy setting in the torrc file, how to make it re-use exist torrc file and not refresh it?

[ManfredKarrer]

@raidsan It is not supported yet but planned.
See JesusMcCloud/netlayer#7

[JesusMcCloud]

@raidsan @ManfredKarrer torrc overrides are supported, using an existing tor is also (basically) working (see JesusMcCloud/netlayer#7).
If you just need to add something to your torrc, you can user torrc overrides are the way to go (though I'm not sure how Bisq handles it).
If you want to use an external tor instance, check JesusMcCloud/netlayer#7

[estoniah]

bisq on Linux:
I can't connect bisq network from china, Tor bundle work well through meek-amazon,
But meek-amazon of bisq can't work?

[joncamfield]

I came across this thread in a random search, you might be interested in the various options of using different transports (and some self-managed bridge options) over at https://www.pluggabletransports.info/

[ManfredKarrer]

We support pluggable transports. You can enable it either in the settings/network or when the app cannot connect for 4 min. it will show up a popup.

[msq6323013]

MacOS:
cp Bisq-x-x-x-.jar /yourDirectory
jar xvf Bisq-x-xx-.jar
mv torrc
// add Socks5Proxy 127.0.0.1:1080
jar uf Bisq-x-xx-.jar torrc