Browse code

Added subheads to the FAQ page; applied ln5's patch

Matt Pagan authored on 09/02/2014 03:55:13
Showing 1 changed files
... ...
@@ -1,3 +1,2 @@
1 1
 User-Agent: *
2 2
 Allow: /
3
-Sitemap: https://www.torproject.org/sitemap.xml
Browse code

Added an automatically generated sitemap.

Matt Pagan authored on 07/02/2014 23:06:10
Showing 1 changed files
... ...
@@ -1,2 +1,3 @@
1 1
 User-Agent: *
2 2
 Allow: /
3
+Sitemap: https://www.torproject.org/sitemap.xml
Browse code

sitemap no longer exists. remove from robots.

Andrew Lewman authored on 05/07/2012 21:23:30
Showing 1 changed files
... ...
@@ -1,4 +1,2 @@
1
-Sitemap: http://www.torproject.org/sitemap.xml
2
-Sitemap: https://www.torproject.org/sitemap.xml
3 1
 User-Agent: *
4 2
 Allow: /
Browse code

update robots.txt to make it valid.

Andrew Lewman authored on 12/07/2010 20:35:11
Showing 1 changed files
... ...
@@ -1,3 +1,4 @@
1 1
 Sitemap: http://www.torproject.org/sitemap.xml
2 2
 Sitemap: https://www.torproject.org/sitemap.xml
3 3
 User-Agent: *
4
+Allow: /
Browse code

update robots.txt with sitemap, thanks to BarkerJr

Andrew Lewman authored on 18/07/2009 22:46:15
Showing 1 changed files
... ...
@@ -1 +1,3 @@
1
+Sitemap: http://www.torproject.org/sitemap.xml
2
+Sitemap: https://www.torproject.org/sitemap.xml
1 3
 User-Agent: *
Browse code

Update robots.txt.

Andrew Lewman authored on 16/10/2007 04:24:51
Showing 1 changed files
... ...
@@ -0,0 +1 @@
1
+User-Agent: *
Browse code

add an empty robots.txt file, since we've got one of the website already, just not in cvs.

Roger Dingledine authored on 31/07/2006 23:26:01
Showing 1 changed files
1 1
new file mode 100644