Handle intermediary certificate failures

[pjoulot]

I use this link checker on multiple websites and I found those two false positives. I don't know why they fail yet. The second one has a redirection though.

https://www.tc.gc.ca/eng/civilaviation/standards/commerce-menu.htm/
https://www.cmhc-schl.gc.ca/odpub/pdf/60946.pdf

[JessSaavedra]

The same happens to me with:
https://sa.www4.irs.gov/irfof-wmsp/notice

[JustinBeckwith]

Thanks for the report! A few observations.

https://www.tc.gc.ca/eng/civilaviation/standards/commerce-menu.htm/

This one is super fascinating! When I try to make a plan request to that endpoint with node.js, I get this error:

FetchError: request to https://www.tc.gc.ca/eng/civilaviation/standards/commerce-menu.htm/ failed, reason: unable to verify the first certificate
    at ClientRequest.<anonymous> (/Users/beckwith/Code/linkinator/node_modules/node-fetch/lib/index.js:1455:11)
    at ClientRequest.emit (events.js:315:20)
    at TLSSocket.socketErrorListener (_http_client.js:461:9)
    at TLSSocket.emit (events.js:315:20)
    at emitErrorNT (internal/streams/destroy.js:100:8)
    at emitErrorCloseNT (internal/streams/destroy.js:68:3)
    at processTicksAndRejections (internal/process/task_queues.js:84:21) {
  type: 'system',
  errno: 'UNABLE_TO_VERIFY_LEAF_SIGNATURE',
  code: 'UNABLE_TO_VERIFY_LEAF_SIGNATURE',
  config: {
    method: 'HEAD',
    url: 'https://www.tc.gc.ca/eng/civilaviation/standards/commerce-menu.htm/',
    responseType: 'stream',
    validateStatus: [Function: validateStatus],
    params: [Object: null prototype] {},
    paramsSerializer: [Function: paramsSerializer],
    headers: {}
  }
}

It looks like that TLS Certificate was signed with an intermediary authority that isn't by default trusted by node.js. I am not entirely sure what the right answer is going to be, but that's fascinating.

https://www.cmhc-schl.gc.ca/odpub/pdf/60946.pdf

After several scans, this one works for me 🤷 Is it still a problem for you?

https://sa.www4.irs.gov/irfof-wmsp/notice

This one issues an HTTP 403, but also returns a response body? Linkinator will only respond success for 2xx response codes, so I don't think I'm going to do anything here.