AI Hallucinations Could Cause Mortgage Lenders Millions in Potential Financial Harm: A Cautionary Tale of Unperfected Technology gone Awry

The rush to embrace and adopt artificial intelligence technology solutions to make mortgage operations faster, better and leaner has primarily focused on operational cost savings.  Some lenders are entranced by the vision of the promise of reduced labor costs, increased operational efficiency, and modernization. There is also an inherent concern about competitive disadvantage as lenders see their peers adopting new AI tools where they have none.

I have addressed AI regulatory and compliance risks elsewhere however it is worth also addressing AI hallucinations because actual applications of generative AI tools have resulted in serious financial losses where businesses trusted the “black box” more than their own experience and common sense.

As a reminder, AI hallucinations occur when generative AI models (think ChatGPT) produce plausible sounding but factually incorrect, fabricated, or nonsense outputs. The errors have caused embarrassment, lost business, false financial reporting, customer refunds, legal actions, and direct financial losses.   Beyond financial costs, companies have also experienced reputation damage, lost contracts, market value erosion, regulatory fines and penalties, and public embarrassment.

How big is the risk in dollars?  According to Korra, an AI data analytics firm, businesses faced $67.4 billion in losses for 2024.  covering various industries), documented cases often involve smaller direct payouts but significant broader impacts. Meanwhile, Blue Ocean Media estimated that worldwide, AI hallucination losses cost businesses over $100 billion dollars.  That’s not chump change!

Need more evidence?  Let’s examine some of the more public cases where generative AI tools caused financial and reputational heartaches for major businesses.

The Summer Reading List that Wasn’t (2025)

In May 2025, the Chicago Sun-Times readers found a “Summer Reading List for 2025” to include fake books attributed to real authors. Only 5 of the 15 titles were genuine works – the rest were fabricated with convincing descriptions. 

The newspapers’ management explained that the list came from another publisher that acknowledged using AI to generate it. Although the newspaper has removed the list from its online edition, readers of the printed version expressed their disappointment paying for the AI-generated content. 

Some newspapers around the country, including the Chicago Sun-Times and at least one edition of The Philadelphia Inquirer, have published a syndicated summer book list that includes made-up books by famous authors. Only five of the 15 titles on the list are real.   

Error-Laden Financial Report Causes Aussie Government Scandal and Major CPA Firm Embarrassment (2025)

In October 2026, the British newspaper The Guardian reported that Deloitte was obligated to provide a refund to the Australian government over a $440,000 report which contained several errors; after admitting it used generative artificial intelligence to help produce it.  The country’s Department of Employment and Workplace Relations (DEWR) confirmed the CPA firm giant would repay fees received when one Labor Party senator accused the consultancy firm of having a “human intelligence problem”. 

Deloitte had been commissioned by the government to review its technology platform used to automate penalties in the welfare system if mutual obligations weren’t met by jobseekers.  

Sullivan and Cromwell Legal Brief Hallucinations (April 2026)

Most recently, on April 21, 2026, Reuters news agency reported that Sullivan & Cromwell, a premier Wall Street law firm, was forced to apologize to a federal ‌judge in the Manhattan federal bankruptcy court for submitting a court filing with inaccurate citations and other errors generated by artificial intelligence.

According to the law firm, the errors included made up case citations, misquotes of the law, and non-existent legal sources culled from generative AI platform searches.  The mistakes were not caught by the court but rather by opposing counsel.

Hallucinations stem from how machine learning tools predict questions and inquiries based on limited human programming as well as the extraction of data from many sources which include correct and faulty data.  AI is not an organic being; it does not "know" facts and can confidently invent details when uncertain. Risks rise with complex queries, faulty data inputs, undisciplined data scraping, outdated knowledge cutoffs, and lack of grounding in real-time data.  So, then what should businesses do with this technology limitation and attendant risk?

Take it slowly and conduct due diligence.  Examine the “black box” and make sure it is developed to properly address the needs of your business while carefully considering the regulatory and compliance risk that does not permit offloading liability for mistakes to third parties. Evaluate the vendor and the vendor contract; understand who is responsible for what and whether insurance and indemnification exist if harm occurs. 

Also ensure human-in-the-loop verification and monitoring.  Develop, implement, test and maintain clear and appropriate for your industry and company size, policies and training on AI use, plus controls and disclaimers for users. When it comes to consumer facing tools, disclose, obtain consent, and build in off-ramps for opting out.   

As AI integrates deeper into business operations, the financial stakes continue to grow—prompting more focus on robust guardrails rather than blind reliance.  Buyer beware!