An LLM described a website in detail. The website doesn't exist.

Short summary

An LLM evaluation harness testing six Chinese LLM APIs found GLM hallucinated a detailed Airtable China website (airtable.cn) with fake pricing, terms, and privacy policy — the domain belongs to a private individual and returns a 502. The author demonstrates a simple three-step DNS/whois/curl check to catch fabricated domain citations that rate-based quality metrics miss. Seven of eight international SaaS brands' .cn domains are owned by third parties, yet LLMs confidently cite them as official.

• •GLM hallucinated a detailed fake Airtable China website with specific pricing and legal pages
• •Rate-based quality checks cannot catch stable wrong answers; assertion-based checks can
• •Simple dig/whois/curl verification catches fabricated domain citations in minutes

Generated with AI, which can make mistakes.