eomyidae

a gopher crawler software
git clone git://r-36.net/eomyidae
Log | Files | Refs | LICENSE

commit 0dac4a637d7e25983b563286bb0539d53ddf8d3e
parent 571cf2bbd98fa39ddf9ea9812ae8ade9be36cf87
Author: Christoph Lohmann <20h@r-36.net>
Date:   Sun,  4 Aug 2019 07:53:04 +0200

There is no intention to implement sitemap.xml for gopherspace.

Diffstat:
eomyidae | 1-
1 file changed, 0 insertions(+), 1 deletion(-)

diff --git a/eomyidae b/eomyidae @@ -136,7 +136,6 @@ def menu2text(menu): # Disallow: /path # Allow: /path # Crawl-delay: seconds -# Sitemap: /some/sitemap.xml def parserobots(data): robots = [] lines = data.split("\n")