While the robots.txt file was invented to tell search engines what webpages not to crawl, the robots.txt file can be used to place engines like google to the XML sitemap. This is supported by Google, Bing, Yahoo and Ask. A sitemap that’s build appropriately need to consist of only the https://rankingseotoolz.com/robots-txt-generator