Robots.txt Nutzungsverteilung in Uruguay

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Google AdsBot Disallow	2.125	16,41
Googlebot Disallow	1.971	15,22
Google AdsBot Mobile Disallow	1.600	12,35
Robots Disallow	954	7,37
Ahrefs Bot Disallow	681	5,26
Yandex Bot Disallow	405	3,13
Sogou Bot Disallow	384	2,96
Baidu Bot Disallow	359	2,77
Yahoo Slurp Disallow	331	2,56
Soso Bot Disallow	320	2,47
Microsoft BingBot Disallow	304	2,35
Alexa Bot Disallow	293	13,22
Common Crawl Bot Disallow	262	11,82
GPTBot Disallow	229	10,33
Anthropic Claude Bot Disallow	220	9,93
Google Bard Disallow	209	9,43
Some Robots Disallow	206	1,59
AppleBot Disallow	204	1,58
Facebook Bot Disallow	203	9,16
Perplexity	191	8,62
Apple AI Bot Disallow	189	8,53
Cohere AI Disallow	183	8,26
Majestic Bot Disallow	168	1,3
Semrush Bot Disallow	166	1,28
Google AdsBot Mobile Apps Disallow	164	1,27
Google Media Partners Disallow	133 ^*	1,03
Exalead Bot Disallow	74	0,57
Microsoft MSNBot Disallow	69	0,53
ByteDance Bytespider Disallow	65	2,93
360 Bot Disallow	56	0,43
BuiltWithBot Disallow	51	0,39
Webz Omgili Disallow	44	1,99
Google Image Bot Disallow	42	0,32
Diffbot Disallow	27	1,22
YOU Disallow	27	1,22
Google News Bot Disallow	7	0,05
DuckDuckGo Bot Disallow	7	0,05
Google Video Bot Disallow	6	0,05
Google Favicon Disallow	6	0,05
Google API Bot Disallow	6	0,05
Google DuplexWeb Disallow	6	0,05
Youdao Bot Disallow	3	0,02
Pipl Disallow	1	0,01
Google FeedFetcher Disallow	1	0,01