This robots.txt file generator automatically produces a robots.txt file based on the inputs you provide. Using robots.txt, you can improve the way search engines index your site by placing it in the root folder. There are parts of your website that you do not want them to crawl to include in user search results, such as the admin page, which search engines such as Google use to crawl. 

You can add these pages to a robots.txt file to be explicitly ignored. The file will be created for you with inputs of pages to be excluded by this website. Allow All Robots: Allow Disallow Crawl Delay (seconds, optional): Select Search Engines for Blogger: Googlebot Googlebot-Image Googlebot-Mobile Bingbot Yahoo Baidu Naver Enter Your Sitemap URL: Generate Robots.txt Generated Robots.txt File: Download Robots.txt