An assistant's crawler fetches the page once, without running JavaScript, and reads the text, the structured data and the links. Below is what it gets from hudson.com, taken from the copy the scanner kept.
characters of text without JavaScript
structured data for the business
of 3 AI crawlers turned away at the edge
Title: Recruitment Agency Australia | Hudson · language en-US. First 1,800 of 20,301 characters:
Recruitment Agency Australia | Hudson Skip to content TIMESHEETS Find talent | Find Job Applications Saved jobs Job alerts Profile details Work preferences Documents Account settings Logout Job seekers Go back Search jobs in Go back Search jobs inSales, Marketing & Communications Admin, Accounting Support/Business Support/Customer Service, Contact Center Human Resources Supply Chain & Procurement Engineering Finance & Accounting Legal, Risk & Compliance Technology & Project Services Executive & Board Appointments Policy & Program – State & Federal Government Advice & Resources Contractor Hub Search Jobs Find your dream job Employers Go back Our expertise Go back Our expertiseAdmin, Customer Service & Contact Centre Assessment Services Engineering Executive and Board Appointment Finance and Accounting Graduate Recruitment Human Resources Legal, Risk and Compliance Policy & Program Sales, Marketing & Communications Supply Chain and Procurement Talent Solutions Technology and Project Services Unbundled Services Volume & Project Recruitment Managed Service Programs Recruitment Outsourcing Statement of Work (SOW) Timesheet Portal Insights & Resources Find Talent Hire world class talent Company Go back About Us For 40 years, we’ve helped people and businesses grow through great work—connecting the right people with the right opportunities. About Us Privacy Policy Contact +61 2 8233 2222 Contact us Find talent | Find Job Applications Saved jobs Job alerts Profile details Work preferences Documents Account settings Logout 40 years of finding the right fit. We help people realise their potential and help businesses grow with the right talent—because the right fit changes everything. For employers Hire world class talent For job seekers Find your dream job For 40 years, we've hel
| A business type in JSON-LD | no |
| Name | no |
| Telephone | +61 2 8233 2222 (a tel: link, not structured) |
| Address | no |
| Opening hours | no |
| Menu or price list | nothing |
| Cuisine in structured data | no |
| JSON-LD blocks | 1 (WebSite, microdata:CreativeWork, microdata:WPFooter, microdata:WPHeader, microdata:WebPage) |
| Link | Role | Where it goes |
|---|---|---|
| au.hudson.com/executive-and-board-appointment/ | booking | the business's own site |
Everything here comes from fetching the homepage once as a crawler. The report turns it into a score and a list of fixes, with code to paste.