You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am running a tileserver-gl instance on the internet and have recently noticed that Google has started indexing multiple files from my tileserver. This is not my intention, and it seems like it's not super easy to configure a robots.txt with this project. I would have to run a second Docker container which runs nginx or some other file-server and configure my reverse proxy to serve it when tileserver.my.tld/robots.txt is requested. This is quite cumbersome.
The simplest fix would be to serve a robots.txt when it's available somewhere (configurable) in the file tree, and return a 404 on the robots.txt route if it's not found. This way, you could easily mount a read-only robots.txt into the container.
The text was updated successfully, but these errors were encountered:
I wonder if anyone considers google indexing their tile server a good thing, should we just put a default robots.txt in "public/resources" which gets served as root of the site right now?
As for a configurable option, i'm not really sure the best way to approach it.
@acalcutt Completely blocking indexing is also fine in my opinion, but I am not sure if there are cases where people would want their tile server to be indexed. But we could just serve a noindex robots.txt and have people override that using their reverse proxy setup should indexing be wanted.
I am running a tileserver-gl instance on the internet and have recently noticed that Google has started indexing multiple files from my tileserver. This is not my intention, and it seems like it's not super easy to configure a robots.txt with this project. I would have to run a second Docker container which runs nginx or some other file-server and configure my reverse proxy to serve it when tileserver.my.tld/robots.txt is requested. This is quite cumbersome.
The simplest fix would be to serve a robots.txt when it's available somewhere (configurable) in the file tree, and return a 404 on the robots.txt route if it's not found. This way, you could easily mount a read-only robots.txt into the container.
The text was updated successfully, but these errors were encountered: