Robots.txt Distribución de uso en Israel

Distribución para sitios web que utilizan Robots.txt tecnologías

Tecnología	Sitios web	%
Google AdsBot Disallow	21.125	24,79
Googlebot Disallow	19.256	22,6
Google AdsBot Mobile Disallow	18.645	21,88
Ahrefs Bot Disallow	3.916	4,6
Robots Disallow	2.755	3,23
Majestic Bot Disallow	2.177	2,55
Semrush Bot Disallow	1.548	1,82
Yandex Bot Disallow	1.166	1,37
Common Crawl Bot Disallow	963	1,13
GPTBot Disallow	924	1,08
Baidu Bot Disallow	890	1,04
Anthropic Claude Bot Disallow	871	1,02
Google Bard Disallow	859	1,01
AppleBot Disallow	855	1
Facebook Bot Disallow	837	0,98
Apple AI Bot Disallow	819	0,96
Perplexity	809	0,95
Cohere AI Disallow	794	0,93
Google AdsBot Mobile Apps Disallow	790	0,93
Some Robots Disallow	770	0,9
Microsoft BingBot Disallow	661	0,78
Alexa Bot Disallow	582	0,68
Soso Bot Disallow	557	0,65
Sogou Bot Disallow	531	0,62
Yahoo Slurp Disallow	524	0,61
Exalead Bot Disallow	351	0,41
Google Media Partners Disallow	253 ^*	0,3
Google Image Bot Disallow	191	0,22
Microsoft MSNBot Disallow	187	0,22
Webz Omgili Disallow	154	0,18
YOU Disallow	100	0,12
360 Bot Disallow	90	0,11
ByteDance Bytespider Disallow	62	0,07
BuiltWithBot Disallow	50	0,06
Google News Bot Disallow	24	0,03
DuckDuckGo Bot Disallow	24	0,03
Google Video Bot Disallow	21	0,02
Google Favicon Disallow	21	0,02
Diffbot Disallow	19	0,02
Google API Bot Disallow	12	0,01
Google DuplexWeb Disallow	12	0,01
Pipl Disallow	7	0,01
Youdao Bot Disallow	6	0,01
Google Read Aloud Disallow	1	0