Robots.txt Distribuzione dell'utilizzo In Nuova Zelanda

Distribuzione per i siti Web che utilizzano Robots.txt tecnologie

Tecnologia	Siti web	%
Google AdsBot Disallow	67.980	14,26
Google AdsBot Mobile Disallow	49.113	10,3
Googlebot Disallow	30.762	6,45
GPTBot Disallow	24.822	5,21
Common Crawl Bot Disallow	24.671	5,17
Anthropic Claude Bot Disallow	24.238	5,08
Google Bard Disallow	24.216	5,08
AppleBot Disallow	24.200	5,08
Facebook Bot Disallow	24.134	5,06
Apple AI Bot Disallow	23.928	5,02
Google AdsBot Mobile Apps Disallow	23.789	4,99
Perplexity	23.768	4,98
Cohere AI Disallow	23.742	4,98
Ahrefs Bot Disallow	19.529	4,1
Baidu Bot Disallow	12.220	2,56
Robots Disallow	10.581	2,22
Microsoft BingBot Disallow	7.837	1,64
Yandex Bot Disallow	6.490	1,36
Yahoo Slurp Disallow	5.464	1,15
Alexa Bot Disallow	4.167	0,87
Sogou Bot Disallow	3.957	0,83
Soso Bot Disallow	3.814	0,8
Majestic Bot Disallow	2.853	0,6
Some Robots Disallow	2.802	0,59
Semrush Bot Disallow	2.188	0,46
ByteDance Bytespider Disallow	849	0,18
Microsoft MSNBot Disallow	765	0,16
Exalead Bot Disallow	736	0,15
Google Media Partners Disallow	729 ^*	0,15
Google Image Bot Disallow	702	0,15
Webz Omgili Disallow	548	0,11
360 Bot Disallow	242	0,05
Google News Bot Disallow	168	0,04
Google Video Bot Disallow	157	0,03
BuiltWithBot Disallow	154	0,03
Google Favicon Disallow	111	0,02
Diffbot Disallow	109	0,02
DuckDuckGo Bot Disallow	87	0,02
Google API Bot Disallow	52	0,01
Google DuplexWeb Disallow	51	0,01
Youdao Bot Disallow	40	0,01
YOU Disallow	40	0,01
Pipl Disallow	27	0,01
Google FeedFetcher Disallow	2	0