Robots.txt Nutzungsverteilung In Belize

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Googlebot Disallow	953	11,69
Baidu Bot Disallow	778	9,55
Sogou Bot Disallow	607	7,45
Yandex Bot Disallow	537	6,59
Alexa Bot Disallow	532	6,53
Some Robots Disallow	527	6,47
Robots Disallow	506	6,21
Microsoft BingBot Disallow	502	6,16
Soso Bot Disallow	502	6,16
Yahoo Slurp Disallow	494	6,06
Google Media Partners Disallow	304 ^*	3,73
Google AdsBot Disallow	294	3,61
Google AdsBot Mobile Disallow	247	3,03
Common Crawl Bot Disallow	112	1,37
GPTBot Disallow	111	1,36
Anthropic Claude Bot Disallow	106	1,3
Google Bard Disallow	105	1,29
Ahrefs Bot Disallow	102	1,25
Facebook Bot Disallow	101	1,24
Cohere AI Disallow	95	1,17
AppleBot Disallow	93	1,14
Apple AI Bot Disallow	85	1,04
Google AdsBot Mobile Apps Disallow	79	0,97
Perplexity	77	0,94
Majestic Bot Disallow	75	0,92
Semrush Bot Disallow	63	0,77
Webz Omgili Disallow	28	0,34
Exalead Bot Disallow	26	0,32
360 Bot Disallow	25	0,31
ByteDance Bytespider Disallow	22	0,27
BuiltWithBot Disallow	18	0,22
Microsoft MSNBot Disallow	13	0,16
Google Image Bot Disallow	10	0,12
DuckDuckGo Bot Disallow	5	0,06
Diffbot Disallow	3	0,04
Google News Bot Disallow	2	0,02
Google Video Bot Disallow	2	0,02
Google Favicon Disallow	2	0,02
Youdao Bot Disallow	2	0,02
Google API Bot Disallow	2	0,02
Google DuplexWeb Disallow	2	0,02
YOU Disallow	1	0,01