Robots.txt Nutzungsverteilung In Dominikanische Republik

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Google AdsBot Disallow	1.679	12,41
Googlebot Disallow	1.325	9,8
Google AdsBot Mobile Disallow	934	6,91
AppleBot Disallow	671	4,96
Baidu Bot Disallow	670	4,95
Ahrefs Bot Disallow	667	4,93
Yandex Bot Disallow	596	4,41
Microsoft BingBot Disallow	571	4,22
Yahoo Slurp Disallow	562	4,15
GPTBot Disallow	519	3,84
Robots Disallow	495	3,66
Google Image Bot Disallow	356	2,63
ByteDance Bytespider Disallow	355	2,62
Common Crawl Bot Disallow	330	2,44
Google Bard Disallow	318	2,35
Anthropic Claude Bot Disallow	312	2,31
Facebook Bot Disallow	311	2,3
Apple AI Bot Disallow	308	2,28
Google AdsBot Mobile Apps Disallow	304	2,25
Perplexity	297	2,2
Cohere AI Disallow	293	2,17
Semrush Bot Disallow	293	2,17
Majestic Bot Disallow	246	1,82
Alexa Bot Disallow	235	1,74
Sogou Bot Disallow	233	1,72
Soso Bot Disallow	213	1,57
Some Robots Disallow	160	1,18
Google Media Partners Disallow	104 ^*	0,77
Microsoft MSNBot Disallow	40	0,3
Webz Omgili Disallow	29	0,21
Exalead Bot Disallow	21	0,16
DuckDuckGo Bot Disallow	12	0,09
360 Bot Disallow	11	0,08
Google News Bot Disallow	8	0,06
Google Video Bot Disallow	8	0,06
Google Favicon Disallow	8	0,06
Google API Bot Disallow	8	0,06
Google DuplexWeb Disallow	8	0,06
Pipl Disallow	6	0,04
Diffbot Disallow	4	0,03
YOU Disallow	3	0,02
Youdao Bot Disallow	2	0,01
BuiltWithBot Disallow	1	0,01