Fix robots.txt to disallow indexing of all paths (#10150)
Following suit of recent change logged in Consul https://github.com/hashicorp/consul/pull/8958 The path pattern specified in allow / deny lines should start with a forward slash to designate the root. [1] https://tools.ietf.org/html/draft-koster-rep-00#section-2.2 [2] https://developers.google.com/search/reference/robots_txt
This commit is contained in:
parent
4d9a2c44ca
commit
3e268d2acc
|
@ -1,3 +1,3 @@
|
||||||
# http://www.robotstxt.org
|
# http://www.robotstxt.org
|
||||||
User-agent: *
|
User-agent: *
|
||||||
Disallow: *
|
Disallow: /
|
||||||
|
|
Loading…
Reference in New Issue