Fix robots.txt to disallow indexing of all paths (#10150)

Following suit of recent change logged in Consul https://github.com/hashicorp/consul/pull/8958

The path pattern specified in allow / deny lines should start with a forward slash to designate the root.

[1] https://tools.ietf.org/html/draft-koster-rep-00#section-2.2
[2] https://developers.google.com/search/reference/robots_txt
This commit is contained in:
Lucas de los Santos 2020-10-14 16:55:49 -04:00 committed by GitHub
parent 4d9a2c44ca
commit 3e268d2acc
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions

View File

@ -1,3 +1,3 @@
# http://www.robotstxt.org # http://www.robotstxt.org
User-agent: * User-agent: *
Disallow: * Disallow: /