An assistant's crawler fetches the page once, without running JavaScript, and reads the text, the structured data and the links. Below is what it gets from homeaffairs.gov.au, taken from the copy the scanner kept.
characters of text without JavaScript
structured data for the business
of 3 AI crawlers turned away at the edge (GPTBot, ClaudeBot, PerplexityBot)
Title: Department of Home Affairs · language en-AU. First 1,800 of 3,454 characters:
Department of Home Affairs Skip to navigation Skip to main content Loading  Select language English You have entered feedback mode. Right click on translated text to report Menu  Menu Home Affairs Portfolio ImmiAccount Visa Entitlement Verification Online (VEVO) Department of Home Affairs Search Popular searches Portfolio functions Border Watch - make a reportIf it doesn’t feel right, flag it anonymously with Border Watch. Report suspicious immigration, customs and border related activity. National Security HotlineIf it doesn't add up, speak up. Call National Security Hotline. News and updates See all news Our other websites National Security nationalsecurity.gov.auAustralian National Security acknowledges the Traditional Custodians of Country throughout Australia. Critical Infrastructure Security Centre cisc.gov.auProtecting Australia's Critical Infrastructure Security. AusCheck auscheck.gov.auWe help to treat trusted-insider risks in Australia’s security-sensitive critical infrastructure sectors. Office of the Migration Agents Registration Authority mara.gov.auFind out about registered Migration Agents and how we protect consumers of migration advice services. Protective Security Policy Framework protectivesecurity.gov.auThe PSPF sets out Australian Government policy across six security domains. Hosting Certification Framework hostingcertification.gov.auSafeguarding Australian Government data. Illicit Tobacco and E-cigarette Commissioner itec.gov.auWe fight against illicit tobacco and e cigarettes. Disaster Assist disasterassist.gov.auFind out how we assist Australians affected by natural disasters in Australia and overseas . Harmony Week harmonyday.gov.auHarmony Week celebrates Australia’s cultural diversity. It's about inc
| A business type in JSON-LD | no |
| Name | no |
| Telephone | no |
| Address | no |
| Opening hours | no |
| Menu or price list | nothing |
| Cuisine in structured data | no |
| JSON-LD blocks | 0 |
| Link | Role | Where it goes |
|---|---|---|
| www.homeaffairs.gov.au/_layouts/15/FIXUPREDIRECT.ASPX?WebId=f6759357-0 | ordering | the business's own site |
Everything here comes from fetching the homepage once as a crawler. The report turns it into a score and a list of fixes, with code to paste.