Lucas de los Santos
|
3e268d2acc
|
Fix robots.txt to disallow indexing of all paths (#10150)
Following suit of recent change logged in Consul https://github.com/hashicorp/consul/pull/8958
The path pattern specified in allow / deny lines should start with a forward slash to designate the root.
[1] https://tools.ietf.org/html/draft-koster-rep-00#section-2.2
[2] https://developers.google.com/search/reference/robots_txt
|
2020-10-14 14:55:49 -06:00 |
|