Robots.txt Nutzungsverteilung in Uruguay

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Google AdsBot Disallow	2.884	18,65
Googlebot Disallow	2.686	17,37
Google AdsBot Mobile Disallow	2.219	14,35
Robots Disallow	967	6,25
Ahrefs Bot Disallow	705	4,56
Sogou Bot Disallow	429	2,77
Yandex Bot Disallow	423	2,74
Baidu Bot Disallow	415	2,68
Yahoo Slurp Disallow	351	2,27
Soso Bot Disallow	335	2,17
Microsoft BingBot Disallow	317	2,05
Alexa Bot Disallow	306	13,43
Common Crawl Bot Disallow	274	12,03
GPTBot Disallow	246	10,8
Some Robots Disallow	240	1,55
Anthropic Claude Bot Disallow	226	9,92
Google Bard Disallow	222	9,75
AppleBot Disallow	214	1,38
Facebook Bot Disallow	211	9,26
Perplexity	196	8,6
Apple AI Bot Disallow	187	8,21
Cohere AI Disallow	181	7,95
Semrush Bot Disallow	177	1,14
Google AdsBot Mobile Apps Disallow	173	1,12
Majestic Bot Disallow	168	1,09
Google Media Partners Disallow	161 ^*	1,04
Microsoft MSNBot Disallow	81	0,52
Exalead Bot Disallow	76	0,49
ByteDance Bytespider Disallow	66	2,9
360 Bot Disallow	66	0,43
BuiltWithBot Disallow	61	0,39
Google Image Bot Disallow	53	0,34
Webz Omgili Disallow	44	1,93
YOU Disallow	25	1,1
Diffbot Disallow	24	1,05
DuckDuckGo Bot Disallow	9	0,06
Google News Bot Disallow	7	0,05
Google Video Bot Disallow	7	0,05
Youdao Bot Disallow	7	0,05
Google Favicon Disallow	7	0,05
Google API Bot Disallow	7	0,05
Google DuplexWeb Disallow	7	0,05
Pipl Disallow	1	0,01
Google FeedFetcher Disallow	1	0,01