is and has support

[cfvescovo]

Closes #169

[cfvescovo]

@adamreichold You mentioned you have a diverse set of scrapers a while ago so I was wondering if you could test this

[adamreichold]

@adamreichold You mentioned you have a diverse set of scrapers a while ago so I was wondering if you could test this

Yes, gladly. It might take a few days until I get this into our staging environment for one of our overnight runs and the results back here, but I will give it a spin.

[adamreichold]

I has seen one overnight run and I have not seen any error rate increases. I'll give it another round and browse the logs a bit tomorrow...

[adamreichold]

Sorry for not coming back earlier, but things did not go as planned. We now have unexplained hangs. I do think this is actually tokio-rs/tokio#6682 but I will have to bump dependencies and wait for runs (without hangs) to be sure.

[cfvescovo]

Sorry for now coming back earlier, but things did not go as planned. We now have unexplained hangs. I do think this is actually tokio-rs/tokio#6682 but I will have to bump dependencies and wait for runs (without hangs) to be sure.

Maybe you could retry using this Tokio branch instead of the latest release or main

[cfvescovo]

nvm, they should have fixed it in 1.38.1

[adamreichold]

Yes, I have redeployed using this branch and Tokio 1.38.1 (instead of 1.38.0). First run was successful, but funnily enough daily and nightly runs do not appear comparable as the servers we are scraping appear to produce more timeouts/retries over night due various maintenance processes...

[adamreichold]

So after deploying Tokio 1.38.1, this have been going smoothly and I think this branch was never at fault. So this good to go from my side.