Robots.txt Distribuzione dell'utilizzo In Iran

Distribuzione per i siti Web che utilizzano Robots.txt tecnologie

Tecnologia	Siti web	%
Majestic Bot Disallow	5.654	13,64
Ahrefs Bot Disallow	4.965	11,97
Semrush Bot Disallow	4.495	10,84
Microsoft BingBot Disallow	2.526	6,09
Common Crawl Bot Disallow	2.457	5,93
Exalead Bot Disallow	2.361	5,69
Microsoft MSNBot Disallow	2.163	5,22
Robots Disallow	2.003	4,83
Yandex Bot Disallow	1.738	4,19
Googlebot Disallow	1.554	3,75
Sogou Bot Disallow	1.404	3,39
Baidu Bot Disallow	1.321	3,19
Soso Bot Disallow	1.037	2,5
Yahoo Slurp Disallow	1.034	2,49
ByteDance Bytespider Disallow	892	2,15
Some Robots Disallow	705	1,7
360 Bot Disallow	699	1,69
BuiltWithBot Disallow	657	1,58
GPTBot Disallow	645	1,56
Alexa Bot Disallow	558	1,35
AppleBot Disallow	386	0,93
DuckDuckGo Bot Disallow	325	0,78
Google Media Partners Disallow	287 ^*	0,69
Google Image Bot Disallow	218	0,53
Google Bard Disallow	170	0,41
Anthropic Claude Bot Disallow	143	0,34
Google News Bot Disallow	142	0,34
Google Video Bot Disallow	141	0,34
Google AdsBot Disallow	117	0,28
Google AdsBot Mobile Disallow	82	0,2
Perplexity	75	0,18
Apple AI Bot Disallow	73	0,18
Facebook Bot Disallow	70	0,17
Cohere AI Disallow	68	0,16
Google AdsBot Mobile Apps Disallow	65	0,16
Webz Omgili Disallow	54	0,13
Google Favicon Disallow	37	0,09
Google API Bot Disallow	37	0,09
Google DuplexWeb Disallow	37	0,09
Diffbot Disallow	28	0,07
Pipl Disallow	21	0,05
YOU Disallow	17	0,04
Youdao Bot Disallow	4	0,01