The human handoff problem: When AI automation goes too far

It’s hard to go anywhere these days without running into mentions of artificial intelligence and automation. From customer service chatbots to call summaries and scheduling tools, AI adoption is moving fast as companies look for ways to save time, cut costs and help employees work more efficiently.

Like with most business decisions, though, the pros and cons of AI usage are nuanced. AI can be helpful when applied to the right tasks, but it’s not a fool-proof solution to every problem.

In this article, we’ll explore how AI is helpful, where it can create problems and how businesses can strike the right balance between automation and human oversight.

How is AI helpful?

According to McKinsey, 88 percent of organizations report using AI regularly in at least one business function. It’s an unsurprising stat, considering how AI has been touted as a transformative tool.

Take the Centers for Disease Control and Prevention (CDC), for instance. The organization launched an enterprise-wide generative AI chatbot to help staff brainstorm ideas, draft content, write code, analyze data and summarize documents. For every $1 invested in the chatbot, the CDC estimates a return of $6.27 in benefits, with more than 41,000 staff hours redirected toward higher-value work like strategic planning and decision-making.

The case is a good example of where AI tends to be the most reliable: handling repetitive, time-consuming tasks quickly and consistently and freeing people up to do the work that requires nuanced judgment, creativity and human connection.

Use cases can be as simple as AI transcriptions that automatically document and summarize customer calls, or an automated telephone system with an AI receptionist that can answer missed calls, handle basic questions, and flag urgent requests that need immediate attention.

For many organizations, especially those exploring AI for small business applications, these tools can help teams do more without adding complexity or stretching limited resources.

Ultimately, AI automation is most helpful when it reduces manual overhead and gives people more time to focus on the parts of their jobs that matter most.

When automation goes wrong

While AI use has grown rapidly, trust hasn’t necessarily kept pace. A recent Pew Research Center survey revealed that, despite nearly half of U.S. adults now using AI chatbots, 40 percent predict AI will have a negative societal impact, and 63 percent believe it’s advancing too fast.

Much of that skepticism comes from experiencing the disadvantages for artificial intelligence firsthand. More than half of organizations using AI report seeing at least one negative consequence tied to its use, and many cite inaccuracies as the root cause.

In customer service, the consequences have already prompted some backtracking. One study found that 74 percent of organizations had rolled back or shut down at least one AI communication tool after deployment, often because of governance concerns or the challenge of maintaining reliable performance at scale.

In fact, tales of bad customer service from AI chatbots have even attracted the attention of regulators. According to a report by the Consumer Financial Protection Bureau, automated customer service systems used by banks have trapped consumers in conversational “doom loops,” forced them to pay fees by mistake and more.

There was also a high-profile case involving Air Canada, where the airline’s chatbot gave a passenger inaccurate advice about bereavement fares. While Air Canada argued the chatbot was a “separate legal entity that is responsible for its own actions,” a tribunal ruled that the airline was responsible for the misinformation and ordered it to pay damages and fees.

Even outside customer service, overreliance on AI has led to consequences. For instance, a Massachusetts lawyer was sanctioned after submitting court filings that cited cases generated by an AI tool that didn’t actually exist.

These examples highlight an important lesson: when AI makes a mistake, the responsibility still falls on the organization using it. Automation can be powerful, but handing over critical decisions without oversight can create new risks just as quickly as it solves old problems.

Talk to a human: A framework for deciding what to automate

The most successful organizations don’t automate everything. In fact, McKinsey found that companies seeing the greatest returns from AI are more likely to keep a “human in the loop,” with 65 percent of high performers having defined processes for deciding when AI outputs require human validation before action is taken.

In other words, the human vs AI debate isn’t really about choosing one over the other. It’s about knowing where each adds the most value and figuring out how they should work together.

Before automating a task, ask:

• Is it repetitive? Routine, rules-based work is often a good candidate for automation.
• What’s the cost of getting it wrong? The higher the stakes, the more human oversight matters.
• Does it require empathy or judgment? Difficult conversations and nuanced decisions are better handled by people.
• Will customers expect flexibility? If exceptions are common, human involvement is essential.
• Who owns the outcome? Someone should remain accountable, even if AI completes part of the task.

Just as important is planning what happens when AI reaches its limits. A good handoff system should:

• Give customers an easy way to reach a person. Don’t force them into a “doom loop” of useless answers and unhelpful menus.
• Transfer context along with the conversation. Share transcripts, notes and the reason for escalation so customers don’t have to start over.
• Set clear triggers for escalation. Decide in advance which situations should automatically be passed to a human, such as when the AI can’t answer the question after multiple attempts, a customer expresses frustration, the issue is time-sensitive, or the request involves an important account or decision.
• Review handoffs regularly. Use customer feedback and support data to identify where the process is breaking down.

Ultimately, effective automation shouldn’t replace people but ensure they’re available at the moments that matter most. This isn’t theoretical, either. Companies have learned this firsthand.

Take Klarna. In 2024, the fintech went all-in on AI-first customer service, launching an OpenAI-powered assistant that handled 2.3 million conversations in its first month and had resolution times drop from 11 minutes to under two. Two-thirds of customer-service chats were handled by AI that month, which, according to the company, was the equivalent to the work of 700 human agents.

A year later, though, the company was recruiting human agents again. CEO Sebastian Siemiatkowski told Bloomberg, “[I]t’s so critical that you are clear to your customer that there will be always a human if you want.” Focusing too much on the cost factor led to “lower quality,” with the CEO saying that “[r]eally investing in the quality of the human support is the way of the future for us.” Today, Klarna runs a hybrid model where AI handles the routine and seamlessly hands off to a person as soon as they’re needed.

Keeping humans in the loop

AI is well-suited to handling repetitive tasks, processing information at scale and improving operational efficiency. But people still need to be present to review outputs and decide what to do next, especially when interactions are complex or high-stakes.

At the end of the day, the future of AI and business won’t come down to choosing between human and machine. It’ll depend on how well companies design systems where automation understands its limits, real people can step in naturally and the handoff between the two feels seamless.