Robots.txt Distribuzione dell'utilizzo In Giappone

Distribuzione per i siti Web che utilizzano Robots.txt tecnologie

Tecnologia	Siti web	%
Google AdsBot Disallow	179.612	20,26
Googlebot Disallow	178.921	20,18
Google AdsBot Mobile Disallow	153.027	17,26
Ahrefs Bot Disallow	51.950	5,86
Robots Disallow	37.363	4,21
Majestic Bot Disallow	28.855	3,26
Baidu Bot Disallow	28.795	3,25
Semrush Bot Disallow	28.010	3,16
Microsoft BingBot Disallow	23.221	2,62
Yandex Bot Disallow	19.077	2,15
Alexa Bot Disallow	17.710	2
Sogou Bot Disallow	16.839	1,9
Yahoo Slurp Disallow	16.179	1,83
Soso Bot Disallow	15.593	1,76
Some Robots Disallow	14.782	1,67
GPTBot Disallow	8.151	0,92
Common Crawl Bot Disallow	6.963	0,79
AppleBot Disallow	6.652	0,75
Anthropic Claude Bot Disallow	6.311	0,71
Google Bard Disallow	5.822	0,66
Google Media Partners Disallow	5.507 ^*	0,62
Facebook Bot Disallow	5.499	0,62
Apple AI Bot Disallow	5.484	0,62
Perplexity	4.146	0,47
Cohere AI Disallow	3.635	0,41
Google AdsBot Mobile Apps Disallow	3.557	0,4
Microsoft MSNBot Disallow	3.494	0,39
ByteDance Bytespider Disallow	2.293	0,26
Webz Omgili Disallow	2.146	0,24
Exalead Bot Disallow	1.995	0,23
Google Image Bot Disallow	1.863	0,21
360 Bot Disallow	1.066	0,12
Youdao Bot Disallow	528	0,06
Google News Bot Disallow	184	0,02
DuckDuckGo Bot Disallow	171	0,02
Google Video Bot Disallow	165	0,02
Pipl Disallow	164	0,02
Diffbot Disallow	153	0,02
Google Favicon Disallow	131	0,01
Google API Bot Disallow	128	0,01
Google DuplexWeb Disallow	120	0,01
BuiltWithBot Disallow	119	0,01
YOU Disallow	82	0,01
Google FeedFetcher Disallow	3	0