Website Auditing
Just like search engine bots, WebSite Auditor tool digs diligently through your entire site to find and audit every resource, both internal and external: HTML, CSS, JavaScript, Flash, images, videos, and more. The Website Auditor tool also lets you crawl your site as Google, Bing, or Yahoo, following robots.txt instructions for any bot, so you can see and analyze the same pages of your site that search engines see.