Robots.txt Répartition des utilisations aux îles Cocos

Distribution pour les sites Web utilisant Robots.txt technologies

Technologie	Sites Web	%
Googlebot Disallow	41 643	13,93
Baidu Bot Disallow	40 793	13,64
Sogou Bot Disallow	28 447	9,51
Some Robots Disallow	24 855	8,31
Yandex Bot Disallow	23 339	7,81
Microsoft BingBot Disallow	22 383	7,49
Yahoo Slurp Disallow	22 360	7,48
Alexa Bot Disallow	21 322	7,13
Soso Bot Disallow	19 510	6,53
Robots Disallow	15 898	5,32
Ahrefs Bot Disallow	5 374	1,8
Majestic Bot Disallow	4 921	1,65
Google Media Partners Disallow	4 625 ^*	1,55
Semrush Bot Disallow	4 389	1,47
360 Bot Disallow	2 675	0,89
Microsoft MSNBot Disallow	2 287	0,76
Google AdsBot Disallow	1 888	0,63
Google Image Bot Disallow	1 270	0,42
Common Crawl Bot Disallow	1 036	0,35
Google AdsBot Mobile Disallow	1 028	0,34
GPTBot Disallow	1 018	0,34
Youdao Bot Disallow	939	0,31
AppleBot Disallow	806	0,27
Anthropic Claude Bot Disallow	748	0,25
ByteDance Bytespider Disallow	714	0,24
Facebook Bot Disallow	606	0,2
Google Bard Disallow	548	0,18
Exalead Bot Disallow	533	0,18
Apple AI Bot Disallow	439	0,15
Cohere AI Disallow	389	0,13
Perplexity	373	0,12
Google AdsBot Mobile Apps Disallow	371	0,12
Webz Omgili Disallow	246	0,08
BuiltWithBot Disallow	198	0,07
DuckDuckGo Bot Disallow	137	0,05
YOU Disallow	126	0,04
Google News Bot Disallow	123	0,04
Google Video Bot Disallow	119	0,04
Google Favicon Disallow	116	0,04
Diffbot Disallow	111	0,04
Google API Bot Disallow	108	0,04
Google DuplexWeb Disallow	107	0,04
Google FeedFetcher Disallow	43	0,01
Pipl Disallow	31	0,01