Google Drive robots.txt broken

[nightpool]

Robots txt parsing is broken for google drive. Here's the directives:

User-agent: *
Crawl-delay: 1
Allow: /$
Allow: /?hl=
Disallow: /?hl=*&
Allow: /support/
Allow: /a/
Allow: /Doc
Allow: /View
Allow: /ViewDoc
Allow: /present
Allow: /Present
Allow: /TeamPresent
Allow: /EmbedSlideshow
Allow: /presentation
Allow: /templates
Allow: /previewtemplate
Allow: /fileview
Allow: /gview
Allow: /viewer
Allow: /leaf
Allow: /file
Allow: /open
Allow: /document
Allow: /drawings
Allow: /demo
Allow: /folder
Allow: /start
Allow: /spreadsheet
Allow: /forms
Allow: /macros
Allow: /keep
Allow: /static
Allow: /drive/
Disallow: /templateabuse
Disallow: /

showing a clear intent to allow "/file".

and here's the error I get when I try to save this page: https://drive.google.com/file/d/1SDlPNIl02BH-q-itDAR1nMNJxGOB_iCz/view

[anjackson]

That's a Internet Archive playback issue rather than anything to do with Heritrix. I recommend you drop them a line via https://archive.org/about/contact.php

[nlevitt]

Doesn't seem to be an issue anymore, was able to save the url just now.
https://web.archive.org/web/20190314182509/https://drive.google.com/file/d/1SDlPNIl02BH-q-itDAR1nMNJxGOB_iCz/view
Sorry for not responding sooner :)

[nightpool]

thanks for the reply!

…

On Thu, Mar 14, 2019, 2:26 PM Noah Levitt ***@***.***> wrote: Doesn't seem to be an issue anymore, was able to save the url just now. https://web.archive.org/web/20190314182509/https://drive.google.com/file/d/1SDlPNIl02BH-q-itDAR1nMNJxGOB_iCz/view Sorry for not responding sooner :) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#193 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAORV2x_1dV_xz7rwwmzBh-obM50eQO1ks5vWpRxgaJpZM4QbQE3> .