An assistant's crawler fetches the page once, without running JavaScript, and reads the text, the structured data and the links. Below is what it gets from fwc.gov.au, taken from the copy the scanner kept.
characters of text without JavaScript
structured data for the business
of 3 AI crawlers turned away at the edge (GPTBot, ClaudeBot, PerplexityBot)
Title: Fair Work Commission | Australia's national workplace relations tribunal · language en. First 1,800 of 49,490 characters:
Fair Work Commission | Australia's national workplace relations tribunal Skip to main content MyFWC unavailable this weekend while we implement changes (8pm Friday to 8am Monday AEDT). Close Top Menu About us Contact us Support and legal help MyFWC Online Learning Portal Search results Search Show/hide search box Menu Home × Job loss or dismissal Toggle submenu Unfair dismissal Toggle submenu About unfair dismissal Toggle submenu What is unfair dismissal? Who the law protects from unfair dismissal The process for unfair dismissal claims Check you are ready to apply for unfair dismissal Unfair dismissal – can you apply? Respond to an unfair dismissal claim Toggle submenu What to do when an employee claims unfair dismissal Respond to a claim for unfair dismissal (Form F3) Toggle submenu Help with Form F3 – Employer response to unfair dismissal Object to an unfair dismissal claim Toggle submenu Reasons to object to an unfair dismissal claim Jurisdiction hearings in unfair dismissal cases Conciliation for unfair dismissal Toggle submenu What is conciliation? Tips to prepare for conciliation What happens in a conciliation meeting Options at conciliation for unfair dismissal The role of the independent conciliator Ask to delay a conciliation Withdraw your application for unfair dismissal Possible results of unfair dismissal claims Toggle submenu Compensation for unfair dismissal Toggle submenu The formula to calculate compensation Reinstatement after unfair dismissal Outcomes or remedies at an unfair dismissal hearing Unfair deactivation or termination for regulated workers Toggle submenu Unfair deactivation for employee-like workers Toggle submenu Apply for unfair deactivation (for regulated workers) (Form F89) Response to an unfair deactivation application (Form F89A)
| A business type in JSON-LD | no |
| Name | no |
| Telephone | no |
| Address | no |
| Opening hours | no |
| Menu or price list | a menu or price-list page in HTML www.fwc.gov.au/hearings-decisions/major-cases/annual-wage-re |
| Cuisine in structured data | no |
| JSON-LD blocks | 0 |
| Link | Role | Where it goes |
|---|---|---|
| www.fwc.gov.au/job-loss-or-dismissal/unfair-deactivation-or-terminatio | booking | the business's own site |
| www.fwc.gov.au/workplace-disputes/bullying/what-do-if-youre-bullied-wo | ordering | the business's own site |
Everything here comes from fetching the homepage once as a crawler. The report turns it into a score and a list of fixes, with code to paste.