Robots.txt Nutzungsverteilung In Bulgarien

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Ahrefs Bot Disallow	4.038	11,11
Googlebot Disallow	3.680	10,13
Google AdsBot Disallow	3.203	8,82
Robots Disallow	2.470	6,8
Majestic Bot Disallow	2.258	6,21
Google AdsBot Mobile Disallow	1.979	5,45
Yandex Bot Disallow	1.898	5,22
Baidu Bot Disallow	1.776	4,89
Microsoft BingBot Disallow	1.628	4,48
Sogou Bot Disallow	1.436	3,95
Alexa Bot Disallow	1.401	3,86
Semrush Bot Disallow	1.355	3,73
Yahoo Slurp Disallow	1.219	3,35
Soso Bot Disallow	1.190	3,28
Common Crawl Bot Disallow	692	1,9
Some Robots Disallow	589	1,62
GPTBot Disallow	558	1,54
Anthropic Claude Bot Disallow	462	1,27
Google Media Partners Disallow	417 ^*	1,15
Google Bard Disallow	372	1,02
AppleBot Disallow	368	1,01
Facebook Bot Disallow	361	0,99
Google AdsBot Mobile Apps Disallow	343	0,94
Exalead Bot Disallow	338	0,93
Apple AI Bot Disallow	319	0,88
Cohere AI Disallow	317	0,87
Perplexity	316	0,87
Microsoft MSNBot Disallow	244	0,67
Google Image Bot Disallow	166	0,46
ByteDance Bytespider Disallow	151	0,42
360 Bot Disallow	149	0,41
BuiltWithBot Disallow	88	0,24
YOU Disallow	85	0,23
Webz Omgili Disallow	66	0,18
Google News Bot Disallow	64	0,18
Google Video Bot Disallow	63	0,17
DuckDuckGo Bot Disallow	60	0,17
Google Favicon Disallow	55	0,15
Google API Bot Disallow	52	0,14
Google DuplexWeb Disallow	52	0,14
Diffbot Disallow	31	0,09
Pipl Disallow	13	0,04
Youdao Bot Disallow	11	0,03
Google FeedFetcher Disallow	1	0