Robots.txt Répartition de l'utilisation dans Monaco

Distribution pour les sites Web utilisant Robots.txt les technologies

Technologie	Sites Internet	%
Google AdsBot Disallow	470	15,25
Google AdsBot Mobile Disallow	392	12,72
Googlebot Disallow	363	11,78
Robots Disallow	148	4,8
Ahrefs Bot Disallow	115	3,73
Google AdsBot Mobile Apps Disallow	108	3,51
Google Media Partners Disallow	104 ^*	3,38
GPTBot Disallow	98	3,18
Common Crawl Bot Disallow	96	3,12
AppleBot Disallow	90	2,92
Google Bard Disallow	90	2,92
Anthropic Claude Bot Disallow	89	2,89
Facebook Bot Disallow	89	2,89
Apple AI Bot Disallow	89	2,89
Cohere AI Disallow	89	2,89
Perplexity	89	2,89
Baidu Bot Disallow	68	2,21
Yandex Bot Disallow	51	1,66
Yahoo Slurp Disallow	46	1,49
Microsoft BingBot Disallow	44	1,43
Some Robots Disallow	39	1,27
Alexa Bot Disallow	35	1,14
Sogou Bot Disallow	30	0,97
Soso Bot Disallow	30	0,97
Google Image Bot Disallow	24	0,78
Majestic Bot Disallow	22	0,71
Microsoft MSNBot Disallow	22	0,71
Google News Bot Disallow	20	0,65
Google Video Bot Disallow	20	0,65
Google Favicon Disallow	20	0,65
DuckDuckGo Bot Disallow	18	0,58
Google API Bot Disallow	18	0,58
Google DuplexWeb Disallow	18	0,58
Semrush Bot Disallow	15	0,49
Exalead Bot Disallow	11	0,36
360 Bot Disallow	3	0,1
BuiltWithBot Disallow	3	0,1
Youdao Bot Disallow	3	0,1
ByteDance Bytespider Disallow	1	0,03
Webz Omgili Disallow	1	0,03