User-agent: * Allow: / # Robots.txt file from http://xyxl.net/bbs/index1.asp # All robots will spider the domain User-agent: * Disallow: