Robots.txt Répartition des utilisations à Niue

Distribution pour les sites Web utilisant Robots.txt technologies

Technologie	Sites Web	%
Googlebot Disallow	2 214	10,97
Baidu Bot Disallow	1 938	9,6
Sogou Bot Disallow	1 655	8,2
Robots Disallow	1 436	7,11
Yandex Bot Disallow	1 426	7,06
Alexa Bot Disallow	1 361	6,74
Yahoo Slurp Disallow	1 319	6,53
Microsoft BingBot Disallow	1 316	6,52
Soso Bot Disallow	1 254	6,21
Ahrefs Bot Disallow	605	3
Majestic Bot Disallow	591	2,93
Some Robots Disallow	555	2,75
Google AdsBot Disallow	540	2,68
Semrush Bot Disallow	507	2,51
Google AdsBot Mobile Disallow	422	2,09
GPTBot Disallow	253	1,25
Anthropic Claude Bot Disallow	238	1,18
Common Crawl Bot Disallow	234	1,16
AppleBot Disallow	215	1,07
Google Bard Disallow	213	1,06
Facebook Bot Disallow	207	1,03
Google Media Partners Disallow	190 ^*	0,94
Apple AI Bot Disallow	188	0,93
Cohere AI Disallow	177	0,88
Perplexity	175	0,87
ByteDance Bytespider Disallow	143	0,71
Webz Omgili Disallow	125	0,62
Google AdsBot Mobile Apps Disallow	100	0,5
Diffbot Disallow	91	0,45
YOU Disallow	87	0,43
Exalead Bot Disallow	78	0,39
Google Image Bot Disallow	74	0,37
Microsoft MSNBot Disallow	67	0,33
DuckDuckGo Bot Disallow	36	0,18
Google News Bot Disallow	31	0,15
Google Video Bot Disallow	30	0,15
Google Favicon Disallow	26	0,13
Google API Bot Disallow	22	0,11
Google DuplexWeb Disallow	22	0,11
360 Bot Disallow	18	0,09
BuiltWithBot Disallow	4	0,02
Pipl Disallow	2	0,01