Robots.txt Nutzungsverteilung In Philippinen

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Some Robots Disallow	80.908	48,48
Googlebot Disallow	10.934	6,55
Baidu Bot Disallow	7.919	4,75
Google AdsBot Disallow	6.914	4,14
Sogou Bot Disallow	6.213	3,72
Yandex Bot Disallow	5.938	3,56
Alexa Bot Disallow	5.796	3,47
Microsoft BingBot Disallow	5.744	3,44
Yahoo Slurp Disallow	5.718	3,43
Soso Bot Disallow	5.658	3,39
Robots Disallow	4.712	2,82
Google AdsBot Mobile Disallow	4.200	2,52
Ahrefs Bot Disallow	2.895	1,73
GPTBot Disallow	1.170	0,7
Common Crawl Bot Disallow	1.155	0,69
Anthropic Claude Bot Disallow	1.112	0,67
Google Bard Disallow	1.108	0,66
AppleBot Disallow	1.097	0,66
Facebook Bot Disallow	1.081	0,65
Apple AI Bot Disallow	1.064	0,64
Perplexity	1.021	0,61
Cohere AI Disallow	1.004	0,6
Google AdsBot Mobile Apps Disallow	997	0,6
Google Media Partners Disallow	655 ^*	0,39
Semrush Bot Disallow	431	0,26
Majestic Bot Disallow	400	0,24
360 Bot Disallow	358	0,21
Microsoft MSNBot Disallow	120	0,07
Exalead Bot Disallow	112	0,07
Webz Omgili Disallow	104	0,06
Google Image Bot Disallow	97	0,06
ByteDance Bytespider Disallow	96	0,06
BuiltWithBot Disallow	26	0,02
Diffbot Disallow	21	0,01
DuckDuckGo Bot Disallow	20	0,01
Google News Bot Disallow	17	0,01
Google Video Bot Disallow	14	0,01
Google Favicon Disallow	11	0,01
Youdao Bot Disallow	11	0,01
Google API Bot Disallow	11	0,01
Google DuplexWeb Disallow	9	0,01
Pipl Disallow	9	0,01
YOU Disallow	8	0
Google FeedFetcher Disallow	3	0