Robots.txt Nutzungsverteilung in Monaco

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Google AdsBot Disallow	368	14,4
Google AdsBot Mobile Disallow	307	12,02
Googlebot Disallow	285	11,15
Robots Disallow	179	7,01
Ahrefs Bot Disallow	108	4,23
Google AdsBot Mobile Apps Disallow	96	3,76
GPTBot Disallow	89	12,97
Common Crawl Bot Disallow	87	12,68
Anthropic Claude Bot Disallow	79	11,52
AppleBot Disallow	78	3,05
Google Bard Disallow	78	11,37
Cohere AI Disallow	78	11,37
Perplexity	78	11,37
Facebook Bot Disallow	77	11,22
Apple AI Bot Disallow	77	11,22
Baidu Bot Disallow	57	2,23
Yandex Bot Disallow	46	1,8
Google Media Partners Disallow	40 ^*	1,57
Microsoft BingBot Disallow	39	1,53
Yahoo Slurp Disallow	39	1,53
Alexa Bot Disallow	30	4,37
Soso Bot Disallow	27	1,06
Google Image Bot Disallow	21	0,82
Majestic Bot Disallow	20	0,78
Microsoft MSNBot Disallow	19	0,74
Google News Bot Disallow	18	0,7
Google Video Bot Disallow	18	0,7
DuckDuckGo Bot Disallow	18	0,7
Google Favicon Disallow	18	0,7
Google API Bot Disallow	18	0,7
Google DuplexWeb Disallow	18	0,7
Semrush Bot Disallow	17	0,67
Exalead Bot Disallow	11	0,43
360 Bot Disallow	4	0,16
ByteDance Bytespider Disallow	3	0,44
BuiltWithBot Disallow	3	0,12
Youdao Bot Disallow	3	0,12
YOU Disallow	2	0,29
Webz Omgili Disallow	1	0,15
Diffbot Disallow	1	0,15