Use Net::HTTP::Persistent in Diego client

[philippthun]

This replaces HTTPClient with Net::HTTP::Persistent in the Diego::Client class.

Net::HTTP::Persistent is based on Net::HTTP which automatically triggers a failover to another IP address after reaching the given connect timeout.

Furthermore Net::HTTP::Persistent adds persistent HTTP connections and thread safety. It is configured to use a pool of up to 50 connections. This equals the size of WorkPools used by various background processes (e.g. deployment updater).

Background information

ℹ️ Use case: Mitigates the risk of Runner is unavailable errors during a cf push when the client is unable to reach a bbs instance.

❌ First, discarded solution proposal: #3002

✔️ Second, merged solution proposal: #3048

💥 Issues caused by #3048

• After recent changes in bbs selection process, passive nodes randomly causing errors #3109
• /v2/spaces/<spaceguid>/summary produces "-1" running_instances for all apps, instead of actual # instances #3112
• sporadically failing sync-integration-tests

🔄 Revert PR: #3113

• I have reviewed the contributing guide

• I have viewed, signed, and submitted the Contributor License Agreement

• I have made this pull request to the main branch

• I have run all the unit tests using bundle exec rake

• I have run CF Acceptance Tests

[philippthun]

@johha and me prepared a local test environment to simulate an unresponsive bbs server and ran tests with different HTTP clients:

HTTP client	10 sequential pings	50 parallel threads	Notes
httpclient	❌ fails	n/a	current implementation
httpclient + monkey patch	✅ succeeds	✅ succeeds	set connect_timeout on TCPSocket
net_http	✅ succeeds	❌ fails
net_http_persistent	✅ succeeds	✅ succeeds	pool_size: 50
http.rb	❌ fails	n/a