Robots.txt Nutzungsverteilung In Irak

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Googlebot Disallow	305	18,83
Google AdsBot Disallow	295	18,21
Robots Disallow	130	8,02
Yandex Bot Disallow	121	7,47
Majestic Bot Disallow	99	6,11
Some Robots Disallow	65	4,01
Ahrefs Bot Disallow	63	3,89
Alexa Bot Disallow	41	2,53
Microsoft BingBot Disallow	40	2,47
Sogou Bot Disallow	36	2,22
Yahoo Slurp Disallow	33	2,04
Soso Bot Disallow	33	2,04
GPTBot Disallow	32	1,98
AppleBot Disallow	32	1,98
Common Crawl Bot Disallow	31	1,91
Anthropic Claude Bot Disallow	31	1,91
Google Bard Disallow	30	1,85
Facebook Bot Disallow	30	1,85
Apple AI Bot Disallow	30	1,85
Perplexity	29	1,79
Google AdsBot Mobile Apps Disallow	28	1,73
Cohere AI Disallow	27	1,67
Semrush Bot Disallow	20	1,23
Exalead Bot Disallow	7	0,43
Microsoft MSNBot Disallow	7	0,43
Google Image Bot Disallow	7	0,43
Webz Omgili Disallow	3	0,19
360 Bot Disallow	3	0,19
ByteDance Bytespider Disallow	2	0,12
Google News Bot Disallow	2	0,12
Google Video Bot Disallow	2	0,12
BuiltWithBot Disallow	1	0,06
DuckDuckGo Bot Disallow	1	0,06
Google Favicon Disallow	1	0,06
Youdao Bot Disallow	1	0,06
Google API Bot Disallow	1	0,06
Google DuplexWeb Disallow	1	0,06