Robots.txt Nutzungsverteilung in Venezuela

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Googlebot Disallow	695	10,02
Google AdsBot Disallow	687	9,91
Robots Disallow	447	6,45
Google AdsBot Mobile Disallow	417	6,01
Baidu Bot Disallow	352	5,08
Ahrefs Bot Disallow	341	4,92
Sogou Bot Disallow	329	4,75
Yandex Bot Disallow	255	3,68
Some Robots Disallow	232	3,35
Yahoo Slurp Disallow	213	3,07
Alexa Bot Disallow	210	3,03
Microsoft BingBot Disallow	203	2,93
GPTBot Disallow	197	2,84
Common Crawl Bot Disallow	195	2,81
Soso Bot Disallow	188	2,71
Google Media Partners Disallow	186 ^*	2,68
Anthropic Claude Bot Disallow	183	2,64
AppleBot Disallow	179	2,58
Google Bard Disallow	174	2,51
Facebook Bot Disallow	172	2,48
Apple AI Bot Disallow	169	2,44
Cohere AI Disallow	166	2,39
Perplexity	166	2,39
Google AdsBot Mobile Apps Disallow	134	1,93
Majestic Bot Disallow	96	1,38
Semrush Bot Disallow	67	0,97
ByteDance Bytespider Disallow	47	0,68
Webz Omgili Disallow	46	0,66
Microsoft MSNBot Disallow	37	0,53
Diffbot Disallow	36	0,52
Google Image Bot Disallow	25	0,36
Exalead Bot Disallow	22	0,32
YOU Disallow	10	0,14
360 Bot Disallow	9	0,13
BuiltWithBot Disallow	7	0,1
Google News Bot Disallow	7	0,1
Google Video Bot Disallow	6	0,09
DuckDuckGo Bot Disallow	6	0,09
Google Favicon Disallow	6	0,09
Youdao Bot Disallow	5	0,07
Google API Bot Disallow	5	0,07
Google DuplexWeb Disallow	5	0,07
Pipl Disallow	1	0,01