31% of business websites turn at least one AI crawler away

161,416 reachable business websites in 675 indexes, as scanned by 17 October 2026.

Main points

That is 31% of 161,416 reachable business websites.
By kind of business: highest among hairdressers and beauty salons (34%), lowest among tours, attractions and museums (28%).
Among 33 clinic indexes, Nuremberg is highest at 40%; the median is 30%.

Figure 1

By kind of business

Kind	Share	Of
restaurants	29%	74,726
cafes and bars	33%	29,566
hairdressers and beauty salons	34%	17,764
dentists, clinics and vets	30%	29,450
tours, attractions and museums	28%	9,910

Figure 2

Dentists, clinics and vets, city by city

another clinic indexmedian, 30%

0%25%50%

33 clinic indexes. The median is 30%; the highest is Nuremberg at 40%.

#	Index	Share	Of
1	Nuremberg DE	40%	211
2	Hamburg DE	39%	1,045
3	Leipzig DE	38%	315
4	Warsaw PL	38%	419
5	Zurich CH	37%	207
6	Berlin DE	37%	7,425
7	Lisbon PT	35%	459
8	Tokyo JP	35%	265
9	Frankfurt DE	34%	360
10	Toronto CA	34%	461

All 33 clinic indexes

#	Index	Share	Of
1	Nuremberg DE	40%	211
2	Hamburg DE	39%	1,045
3	Leipzig DE	38%	315
4	Warsaw PL	38%	419
5	Zurich CH	37%	207
6	Berlin DE	37%	7,425
7	Lisbon PT	35%	459
8	Tokyo JP	35%	265
9	Frankfurt DE	34%	360
10	Toronto CA	34%	461
11	Cologne DE	33%	467
12	Munich DE	32%	922
13	Dortmund DE	32%	323
14	Los Angeles US	30%	212
15	Budapest HU	30%	303
15	Dresden DE	30%	357
15	Stuttgart DE	30%	370
18	Seattle US	29%	205
19	Miami US	29%	2,768
20	Calgary CA	29%	190
21	Dallas US	29%	220
22	Bremen DE	28%	254
23	Brisbane AU	28%	217
24	Madrid ES	27%	285
25	Hanover DE	27%	368
26	New York US	26%	585
27	Düsseldorf DE	26%	247
28	Amsterdam NL	23%	249
29	London GB	22%	905
30	Graz AT	21%	240
31	Vienna AT	19%	834
32	Sydney AU	18%	302
33	Melbourne AU	17%	460

Indexes under 200 scanned websites are left out. The "Of" column is the base each figure rests on.

Data, CSV JSON Method Link to this figure

About these numbers
A site turns a crawler away when its robots.txt disallows a known AI crawler or its firewall answers that crawler with an error. Many owners don't know their host or template does it.

Cite this

31% of business websites turn at least one AI crawler away (The Direct Index, 17 October 2026, directindex.org/stats/ai-blocked).

CSV JSON Markdown Method Every number

Data: ODbL. Business data from OpenStreetMap contributors and Overture Maps; website facts from the Direct Index scanner, which reads each site's public pages.