From 3e268d2acc61b0ba22a7d846c7f68f65cbb70c01 Mon Sep 17 00:00:00 2001 From: Lucas de los Santos Date: Wed, 14 Oct 2020 16:55:49 -0400 Subject: [PATCH] Fix robots.txt to disallow indexing of all paths (#10150) Following suit of recent change logged in Consul https://github.com/hashicorp/consul/pull/8958 The path pattern specified in allow / deny lines should start with a forward slash to designate the root. [1] https://tools.ietf.org/html/draft-koster-rep-00#section-2.2 [2] https://developers.google.com/search/reference/robots_txt --- ui/public/robots.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ui/public/robots.txt b/ui/public/robots.txt index 207b72685..217b42f0e 100644 --- a/ui/public/robots.txt +++ b/ui/public/robots.txt @@ -1,3 +1,3 @@ # http://www.robotstxt.org User-agent: * -Disallow: * +Disallow: /