Inside the surge of AI mental health tools in the workplace

The Rise of AI in Mental Health Support

Since the emergence of tools like ChatGPT and Claude, there has been a growing debate about whether artificial intelligence (AI) should be used to support mental health. Can a chatbot truly replace a therapist? This question has been asked many times, yet it still lacks a straightforward answer.

However, AI tools may have the potential to do more than just respond to distress — they might even be able to anticipate it. A new wave of tools, many of which are aimed at workplaces, could detect early signs of depression, anxiety, or even suicide risk before someone is aware of their struggles. These systems analyze patterns in behavior, language, voice, and daily activity, searching for subtle signals that something may be wrong.

On paper, this idea seems promising. But the reality is far more complex, and the questions go beyond whether the technology works or not.

How Can AI Detect a Mental Health Crisis?

It’s important to note that these tools are not all the same. However, many of them rely on similar principles.

Most AI mental health tools collect data in two ways. The first involves information that users actively provide, such as mood check-ins, sleep logs, journal entries, or conversations with a chatbot. The second method involves passive sensing, where data is gathered in the background. This includes movement, frequency of messages, speech patterns, and typing speed. The data collected depends on what the tools can access, whether it's from wearables, computers, or apps.

The underlying premise is simple: changes in behavior often occur before someone consciously realizes they're struggling. An AI system continuously scanning these signals may be able to detect shifts early, flag an issue, and get help more quickly.

In addition to this data layer, many tools use AI chatbots trained on therapeutic approaches like Cognitive Behavioral Therapy (CBT) to offer immediate support. They might suggest coping strategies, help reframe thoughts, or prompt reflection.

Some elements of this technology are already being used. For example, Meta has long used text and behavioral signals to identify users who may be at risk, while companies like Kintsugi focus on analyzing voice for signs of mental health conditions. Workplace platforms like Unmind have also explored similar approaches.

However, mapping the full picture is challenging. Many of these capabilities are integrated into broader AI systems and aren't always visible to users, so their use may be more extensive than what is publicly known.

Does It Actually Work?

When it comes to whether these tools work, the answer is: it depends.

There is some evidence that AI can detect patterns linked to mental health risks, particularly in areas like symptom monitoring and suicide risk screening. However, the results are mixed, and performance varies widely depending on the population, the data being used, and how the system is deployed.

In practice, most research suggests these tools work best as a supplement to clinicians rather than a replacement for professional judgment. Reliable, real-world prediction remains much harder.

So, the conclusion is that much more research is needed before AI-driven mental health prediction can be considered robust or widely dependable.

"There are so many nuanced issues that this technology brings up," says psychologist and AI risk advisor Genevieve Bartuski of Unicorn Intelligence Tech Partners. "My fear is that it's hitting the market before they are fully addressed."

What Are the Concerns?

"When people know they are being watched, they tend to perform. It is an automatic response and often, people don't even realize they are doing it," explains therapist Amy Sutton from Freedom Counselling.

This is known as the Hawthorne Effect. In the context of AI monitoring mental health, this could mean people masking signs of distress, consciously or not.

On the flip side, if these tools are rolled out as part of workplace wellbeing programs and people don’t know they’re being monitored, that raises serious questions about consent. It also raises a more fundamental question: whose interests are these systems really serving — the individual’s wellbeing, or the organization’s risk management?

"It bothers me that this could be deployed by employers," Bartuski tells me. "This is information that employers do not need to have or to know. They do not need information about a person's mental health, especially when it can be used against the employee."

Even when participation is presented as optional, consent can quickly become murky. "Does it put the employee at risk of being negatively impacted if they do not want to participate? If so, that isn't really consent. It's coercive consent," she says.

Sutton adds that workplace monitoring could actually worsen the problem it's trying to solve. "With mental health stigmas still rife, AI observation would likely lead to greater efforts to hide evidence of struggles. This could create a dangerous spiral, where the greater our efforts to hide low mood or anxiety, the worse it becomes."

There’s also the risk of false positives when it comes to AI — where someone is flagged as being at risk when they’re not — and the consequences of that can be serious, particularly in systems that trigger intervention.

Where Does This Leave Us?

The pressure to develop these tools is real. The WHO estimates that depression and anxiety cost the global economy $1 trillion a year in lost productivity. That’s a number that makes early warning systems look attractive to a lot of employers.

But there’s a risk that prediction tools become a shortcut. An alternative to the slower, more expensive work of building environments where people feel able to say they’re struggling, investing in human support, and creating the conditions where someone notices when a colleague isn’t okay.

"We are being encouraged to give up a basic need of real human connection to be productive, and in turn productivity decreases due to the impact of loneliness and disconnection," Sutton says.

It echoes a broader pattern I've noticed during my AI reporting over the past year. People often turn to AI for support when real-world networks fall short — sometimes with benefits, but often as a substitute rather than a solution.

AI systems that could genuinely flag a mental health crisis early — with meaningful consent and proper safeguards — might have a place. But without that, they risk doing the opposite of what they promise: making problems harder to see, and giving organizations a reason not to look.