ClicBot allows you to crawl and index your website and presents reports to analyze your structural SEO, Internal links, navigation, semantics...
Add your server's logs to get more informations about crawl & visit rates, and matched / non matched crawl from Google.
More informations Try for free !
You can test Clicbot for free, 20 credits are automaticaly offered when you subscribe. this credits allow you to crawl up to 30 000 pages.