Robots.txt Répartition des utilisations au Pérou

Distribution pour les sites Web utilisant Robots.txt technologies

Technologie	Sites Web	%
Google AdsBot Disallow	3 695	13,5
Googlebot Disallow	3 166	11,57
Robots Disallow	2 465	9,01
Ahrefs Bot Disallow	2 350	8,59
Google AdsBot Mobile Disallow	1 554	5,68
Google Media Partners Disallow	1 539 ^*	5,62
Baidu Bot Disallow	1 176	4,3
Sogou Bot Disallow	1 054	3,85
Microsoft BingBot Disallow	951	3,48
Yandex Bot Disallow	894	3,27
Yahoo Slurp Disallow	816	2,98
Alexa Bot Disallow	785	17,18
Soso Bot Disallow	742	2,71
Some Robots Disallow	713	2,61
GPTBot Disallow	530	11,6
Common Crawl Bot Disallow	503	11,01
Anthropic Claude Bot Disallow	449	9,83
Google Bard Disallow	448	9,81
AppleBot Disallow	429	1,57
Facebook Bot Disallow	415	9,08
Apple AI Bot Disallow	405	8,86
Perplexity	399	8,73
Cohere AI Disallow	387	8,47
Google AdsBot Mobile Apps Disallow	374	1,37
Semrush Bot Disallow	277	1,01
Majestic Bot Disallow	272	0,99
Microsoft MSNBot Disallow	111	0,41
Google Image Bot Disallow	79	0,29
Exalead Bot Disallow	67	0,24
ByteDance Bytespider Disallow	58	1,27
Webz Omgili Disallow	50	1,09
360 Bot Disallow	39	0,14
BuiltWithBot Disallow	30	0,11
Youdao Bot Disallow	27	0,1
Google News Bot Disallow	23	0,08
Google Video Bot Disallow	23	0,08
Diffbot Disallow	20	0,44
YOU Disallow	17	0,37
DuckDuckGo Bot Disallow	11	0,04
Google Favicon Disallow	7	0,03
Google API Bot Disallow	5	0,02
Google DuplexWeb Disallow	5	0,02
Pipl Disallow	2	0,01
LLMS Text	2	0,01