An assistant's crawler fetches the page once, without running JavaScript, and reads the text, the structured data and the links. Below is what it gets from lonelyplanet.com, taken from the copy the scanner kept.
characters of text without JavaScript
structured data for the business
of 3 AI crawlers turned away at the edge (GPTBot)
Title: Travel Advice & Inspiration | Lonely Planet · language en. First 1,800 of 26,532 characters:
Travel Advice & Inspiration | Lonely Planet Skip to Main Content Destinations Trending Europe Asia The Americas Australia & Oceania Africa & The Middle East The Caribbean Our Favorite Places Iceland Italy Japan London Portugal Peru Spain Algarve Mexico City Costa Rica Corfu Dubrovnik Cinque Terre Tokyo Budapest Lisbon Andalucía Morocco New York City Rio de Janeiro Vietnam View all destinations Not sure where to start? Get inspired by destinations chosen in this year's Best in Travel list. Start Discovering Popular Amalfi Coast The Azores Cinque Terre Puglia Sardinia Sicily Tuscany Countries Croatia Greece Iceland Ireland Italy Portugal Spain Cities Barcelona London Madrid Paris Porto Rome Santorini View all destinations in Europe Europe Experience Europe's ancient history, amazing beauty, divine food and incredible culture. EXPLORE Popular Bali Hong Kong Japan Maldives Phuket Southeast Asia Thailand Countries China India Indonesia Philippines South Korea Sri Lanka Vietnam Cities Bangkok Beijing Chiang Mai Ho Chi Minh City Kyoto Seoul Tokyo View all destinations in Asia Asia Uncover Asia's iconic attractions, must-visit places and epically awesome experiences. EXPLORE Popular California Central America Florida Hawaii North America Patagonia South America Countries Argentina Brazil Canada Costa Rica Mexico Peru United States Cities Buenos Aires Chicago Lima Mexico City Miami New Orleans San Francisco View all destinations New York City See the city's iconic sites, sample famous foods and marvel at world-class museums. EXPLORE Popular Bora Bora Fiji Gold Coast Oceania South Pacific Tahiti Tasmania Countries Cook Islands French Polynesia New Zealand Palau Papua New Guinea Samoa Tonga Cities Auckland Brisbane Melbourne Perth Queensland Sydney Victoria View all destinations i
| A business type in JSON-LD | Organization |
| Name | no |
| Telephone | no |
| Address | no |
| Opening hours | no |
| Menu or price list | nothing |
| Cuisine in structured data | no |
| JSON-LD blocks | 1 (Organization) |
| Link | Role | Where it goes |
|---|---|---|
| www.lonelyplanet.com/articles/best-time-to-book-a-flight | booking | the business's own site |
Everything here comes from fetching the homepage once as a crawler. The report turns it into a score and a list of fixes, with code to paste.