Robots.txt Nutzungsverteilung In Sint Maarten

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Googlebot Disallow	201	11,85
Baidu Bot Disallow	188	11,08
Microsoft BingBot Disallow	166	9,79
Yandex Bot Disallow	165	9,73
Sogou Bot Disallow	162	9,55
Alexa Bot Disallow	159	9,38
Yahoo Slurp Disallow	158	9,32
Soso Bot Disallow	155	9,14
Robots Disallow	78	4,6
Some Robots Disallow	59	3,48
Google Media Partners Disallow	30 ^*	1,77
Google AdsBot Disallow	24	1,42
Google AdsBot Mobile Disallow	23	1,36
GPTBot Disallow	14	0,83
Anthropic Claude Bot Disallow	10	0,59
Google Bard Disallow	10	0,59
Perplexity	10	0,59
Common Crawl Bot Disallow	8	0,47
AppleBot Disallow	8	0,47
Facebook Bot Disallow	8	0,47
Apple AI Bot Disallow	8	0,47
Cohere AI Disallow	8	0,47
Google AdsBot Mobile Apps Disallow	8	0,47
Semrush Bot Disallow	6	0,35
Ahrefs Bot Disallow	4	0,24
Majestic Bot Disallow	4	0,24
Microsoft MSNBot Disallow	3	0,18
ByteDance Bytespider Disallow	2	0,12
Webz Omgili Disallow	2	0,12
Google Image Bot Disallow	2	0,12
Google News Bot Disallow	2	0,12
Google Video Bot Disallow	2	0,12
DuckDuckGo Bot Disallow	2	0,12
Google Favicon Disallow	2	0,12
Google API Bot Disallow	2	0,12
Google DuplexWeb Disallow	2	0,12
360 Bot Disallow	1	0,06