Robots.txt Nutzungsverteilung in Belgien

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Googlebot Disallow	67.070	15,48
Google AdsBot Disallow	63.936	14,75
Google AdsBot Mobile Disallow	54.105	12,48
Baidu Bot Disallow	21.078	4,86
Robots Disallow	17.115	3,95
Sogou Bot Disallow	16.014	3,69
Yandex Bot Disallow	13.906	3,21
Ahrefs Bot Disallow	12.351	2,85
Microsoft BingBot Disallow	11.756	2,71
Yahoo Slurp Disallow	11.603	2,68
Alexa Bot Disallow	11.315	2,61
GPTBot Disallow	10.465	2,41
Soso Bot Disallow	10.413	2,4
Some Robots Disallow	9.726	2,24
Common Crawl Bot Disallow	9.662	2,23
Anthropic Claude Bot Disallow	9.536	2,2
Google Bard Disallow	9.170	2,12
AppleBot Disallow	8.935	2,06
Facebook Bot Disallow	8.920	2,06
Apple AI Bot Disallow	8.590	1,98
Perplexity	8.547	1,97
Cohere AI Disallow	8.433	1,95
Google AdsBot Mobile Apps Disallow	8.248	1,9
Semrush Bot Disallow	5.195	1,2
Majestic Bot Disallow	4.955	1,14
Google Media Partners Disallow	1.844 ^*	0,43
Microsoft MSNBot Disallow	1.438	0,33
ByteDance Bytespider Disallow	1.365	0,31
Exalead Bot Disallow	1.056	0,24
Google Image Bot Disallow	990	0,23
Webz Omgili Disallow	970	0,22
DuckDuckGo Bot Disallow	647	0,15
Diffbot Disallow	560	0,13
360 Bot Disallow	523	0,12
Google News Bot Disallow	482	0,11
Google Video Bot Disallow	473	0,11
YOU Disallow	452	0,1
Google Favicon Disallow	436	0,1
Google API Bot Disallow	383	0,09
Google DuplexWeb Disallow	382	0,09
BuiltWithBot Disallow	210	0,05
Pipl Disallow	86	0,02
Youdao Bot Disallow	53	0,01
Google Read Aloud Disallow	3	0
Google FeedFetcher Disallow	1	0
Google Web Light Disallow	1	0