Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Diego client tries all bbs domain IPs #3048

Merged
merged 4 commits into from
Nov 11, 2022

Commits on Nov 4, 2022

  1. Diego client tries all bbs domain IPs

    Mitigates the risk of 'Runner is unavailable' errors when an outage makes some (but not all) bbs instances unreachable, by trying all IPs before re-resolving.
    
    bbs operates as a cluster, with only one instance at a time 'active' and able to respond to API requests. At present, whenever the Diego::Client needs to communicate with bbs it hits the URL passed in from config.diego.bbs.url. The default value passed in for this from the capi-release is https://bbs.service.cf.internal:8889. This domain is defined by cf-deployment as a bosh-dns alias for the diego-api instance group with query q-s4 (i.e. resolving to all instances, whether healthy or unhealthy, in a random order).
    
    The diego client previously made three attempts to reach bbs. Upon a network timeout its HTTP client would bail and a retry would be attempted with a fresh DNS resolution. This returned a list of bbs IPs in a random order, and with two instances there was a 1/8 chance to get the same unreachable instance's IP first in the list on all three attempts.
    will-gant committed Nov 4, 2022
    Configuration menu
    Copy the full SHA
    c50d74f View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2022

  1. Configuration menu
    Copy the full SHA
    1af3604 View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2022

  1. Configuration menu
    Copy the full SHA
    a21a76b View commit details
    Browse the repository at this point in the history

Commits on Nov 10, 2022

  1. Address PR feedback

    will-gant committed Nov 10, 2022
    Configuration menu
    Copy the full SHA
    dce76a9 View commit details
    Browse the repository at this point in the history