Robots.txt Nutzungsverteilung In Albanien

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Googlebot Disallow	698	7,96
Google AdsBot Disallow	651	7,43
Robots Disallow	536	6,12
Ahrefs Bot Disallow	489	5,58
Google AdsBot Mobile Disallow	453	5,17
Baidu Bot Disallow	419	4,78
Yandex Bot Disallow	396	4,52
Anthropic Claude Bot Disallow	365	4,16
Sogou Bot Disallow	350	3,99
Semrush Bot Disallow	281	3,21
GPTBot Disallow	272	3,1
Common Crawl Bot Disallow	270	3,08
Some Robots Disallow	264	3,01
Alexa Bot Disallow	248	2,83
Yahoo Slurp Disallow	237	2,7
Microsoft BingBot Disallow	236	2,69
Soso Bot Disallow	235	2,68
Google Bard Disallow	228	2,6
AppleBot Disallow	226	2,58
Facebook Bot Disallow	222	2,53
Google AdsBot Mobile Apps Disallow	220	2,51
Google Media Partners Disallow	220 ^*	2,51
Apple AI Bot Disallow	217	2,48
Cohere AI Disallow	213	2,43
Perplexity	213	2,43
Majestic Bot Disallow	155	1,77
Exalead Bot Disallow	74	0,84
Microsoft MSNBot Disallow	53	0,6
Webz Omgili Disallow	45	0,51
360 Bot Disallow	44	0,5
BuiltWithBot Disallow	36	0,41
Google Image Bot Disallow	29	0,33
YOU Disallow	26	0,3
ByteDance Bytespider Disallow	23	0,26
Google News Bot Disallow	18	0,21
Google Video Bot Disallow	18	0,21
DuckDuckGo Bot Disallow	18	0,21
Google Favicon Disallow	18	0,21
Google API Bot Disallow	18	0,21
Google DuplexWeb Disallow	18	0,21
Diffbot Disallow	9	0,1
Pipl Disallow	3	0,03
Youdao Bot Disallow	1	0,01