An assistant's crawler fetches the page once, without running JavaScript, and reads the text, the structured data and the links. Below is what it gets from sydney.com.au, taken from the copy the scanner kept.
characters of text without JavaScript
structured data for the business
of 3 AI crawlers turned away at the edge (GPTBot)
Title: Sydney the Harbour City : Sydney Harbour Bridge : Sydney Opera House : The Rocks : Blue Mountains : Darling Harbour : Sydney.com.au. First 1,800 of 2,311 characters:
Sydney the Harbour City : Sydney Harbour Bridge : Sydney Opera House : The Rocks : Blue Mountains : Darling Harbour : Sydney.com.au sydney.com.au Est 1994, Sydney's oldest tourism website HOME Start Here Hop on, hop off bus Essentials Essential information Getting around City Circle Underground Railway Light rail service Weather & climate Emergency phone numbers Consulates and embassies Airport links Airport rail link Airport transfers Driving In Australia Distances between Australian cities and driving times Car Hire Maps of Sydney Secure car parking Campervan Hire Caravan park locations Sights Main attractions Sydney Harbour Sydney Harbour Bridge About Sydney Harbour Bridge Climb the Sydney Harbour Bridge and walk across the arch Sydney Opera House About Sydney Opera House New Year's Eve fireworks in Sydney Circular Quay The Rocks Darling Harbour Sydney Beaches Manly Beach Bondi Beach Balmoral Beach Sydney's Victorian-era shopping centres Royal Botanic Garden Taronga Zoo Sydney Tower Blue Mountains Jenolan Caves Popular Destinations Manly Manly Manly Beach The Corso Shelly Beach walking path Watsons Bay Newtown Kings Cross Other Sights Sydney Aquarium Sydney Observatory Museums & art galleries Chinese Garden of Friendship Featherdale Wildlife Park Fort Denison Bradleys Head Sydney wedding locations Historic buildings Islands in Sydney Harbour and Botany Bay Museums Markets Harbour Cruises Sydney Harbour cruises Hop-on hop-off Sydney Harbour cruise Whale watching cruises Jet boat rides Wedding cruises on Sydney Harbour Corporate cruises on Sydney Harbour New Year's Eve on Sydney Harbour Scenic seaplane flights over Sydney Harbour Reminiscing about Sydney Harbour Trips & Tours Blue Mountains Jenolan Caves Hotels Hotels in Sydney Boutique Hotels Campervan hire Pet friend
| A business type in JSON-LD | no |
| Name | no |
| Telephone | no |
| Address | no |
| Opening hours | no |
| Menu or price list | nothing |
| Cuisine in structured data | no |
| JSON-LD blocks | 0 |
| Link | Role | Where it goes |
|---|---|---|
| www.viator.com/en-AU/tours/Sydney/Sydney-Harbour-Tour-by-Helicopter/d3 | booking | viator, a marketplace |
Everything here comes from fetching the homepage once as a crawler. The report turns it into a score and a list of fixes, with code to paste.