Robots.txt Nutzungsverteilung In Guatemala

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Google AdsBot Disallow	1.880	17,42
Google AdsBot Mobile Disallow	1.377	12,76
Googlebot Disallow	1.155	10,7
Ahrefs Bot Disallow	546	5,06
GPTBot Disallow	452	4,19
Common Crawl Bot Disallow	439	4,07
AppleBot Disallow	438	4,06
Anthropic Claude Bot Disallow	433	4,01
Google Bard Disallow	432	4
Facebook Bot Disallow	431	3,99
Apple AI Bot Disallow	429	3,97
Perplexity	416	3,85
Cohere AI Disallow	414	3,84
Google AdsBot Mobile Apps Disallow	411	3,81
Robots Disallow	357	3,31
Baidu Bot Disallow	170	1,57
Yandex Bot Disallow	150	1,39
Some Robots Disallow	135	1,25
Semrush Bot Disallow	110	1,02
Sogou Bot Disallow	82	0,76
Alexa Bot Disallow	74	0,69
Yahoo Slurp Disallow	74	0,69
Microsoft BingBot Disallow	73	0,68
Soso Bot Disallow	65	0,6
Google Media Partners Disallow	60 ^*	0,56
Majestic Bot Disallow	57	0,53
Microsoft MSNBot Disallow	39	0,36
Webz Omgili Disallow	24	0,22
ByteDance Bytespider Disallow	14	0,13
Google Image Bot Disallow	11	0,1
Exalead Bot Disallow	10	0,09
Diffbot Disallow	7	0,06
360 Bot Disallow	5	0,05
BuiltWithBot Disallow	4	0,04
YOU Disallow	4	0,04
Google News Bot Disallow	2	0,02
Google Video Bot Disallow	2	0,02
DuckDuckGo Bot Disallow	2	0,02
Google Favicon Disallow	2	0,02
Youdao Bot Disallow	2	0,02
Google API Bot Disallow	2	0,02
Google DuplexWeb Disallow	2	0,02
Pipl Disallow	2	0,02