Good to Know: Search Engine Robots, and others, 2017

0 comments

There's a comprehensive assessment, from a webmaster perspective, of search engine robots detailed in a thread on WebmasterWorld. It also includes other bots that may be legitimate, or acceptable to some.

 

It includes Bot Names, IPs, UAs and comments about the source and bot behaviour.

https://www.webmasterworld.com/search_engine_spiders/4849056.htm