Robots.txt Répartition des utilisations en Nouvelle-Zélande

Distribution pour les sites Web utilisant Robots.txt technologies

Technologie	Sites Web	%
Google AdsBot Disallow	64 667	16,2
Google AdsBot Mobile Disallow	46 800	11,73
Googlebot Disallow	29 490	7,39
GPTBot Disallow	18 938	4,75
Common Crawl Bot Disallow	18 754	4,7
Anthropic Claude Bot Disallow	18 594	4,66
Google Bard Disallow	18 401	4,61
AppleBot Disallow	18 387	4,61
Facebook Bot Disallow	18 323	4,59
Apple AI Bot Disallow	18 201	4,56
Perplexity	18 070	4,53
Cohere AI Disallow	18 056	4,52
Google AdsBot Mobile Apps Disallow	17 986	4,51
Ahrefs Bot Disallow	15 281	3,83
Baidu Bot Disallow	10 495	2,63
Microsoft BingBot Disallow	7 952	1,99
Robots Disallow	7 044	1,76
Yandex Bot Disallow	5 648	1,42
Yahoo Slurp Disallow	4 829	1,21
Sogou Bot Disallow	4 509	1,13
Alexa Bot Disallow	3 866	0,97
Soso Bot Disallow	3 590	0,9
Some Robots Disallow	2 651	0,66
Majestic Bot Disallow	2 202	0,55
Semrush Bot Disallow	1 709	0,43
Google Media Partners Disallow	761 ^*	0,19
Microsoft MSNBot Disallow	637	0,16
ByteDance Bytespider Disallow	617	0,15
Google Image Bot Disallow	553	0,14
Exalead Bot Disallow	524	0,13
Webz Omgili Disallow	435	0,11
360 Bot Disallow	167	0,04
Google News Bot Disallow	149	0,04
Google Video Bot Disallow	143	0,04
Diffbot Disallow	139	0,03
BuiltWithBot Disallow	108	0,03
Google Favicon Disallow	106	0,03
DuckDuckGo Bot Disallow	86	0,02
YOU Disallow	69	0,02
Google API Bot Disallow	59	0,01
Google DuplexWeb Disallow	58	0,01
Youdao Bot Disallow	30	0,01
Pipl Disallow	26	0,01
Google FeedFetcher Disallow	2	0
Google Read Aloud Disallow	1	0