website SEO analysis & review tools

robots.txt Test

iwebchk

robots.txt file is used to provide instructions to web crawlers and bots on how to properly crawl the website.