Today’s column delves into the critical policy considerations for regulating AI that offers mental health support. This examination is crucial for understanding the breadth and depth of policies that demand thorough attention.
It is important to note that new state-level laws are emerging rapidly, often adopting a hit-or-miss strategy in governing AI’s role in mental health. This approach is problematic because it tends to address only select aspects of the necessary policies, leaving significant regulatory gaps and creating ambiguity about the intentions behind these omissions.
In response, this discussion offers a comprehensive perspective intended for policymakers and other relevant stakeholders. Researchers from fields such as AI, governance, law, ethics, behavioral sciences, and psychology can also utilize this framework to further investigate and analyze these urgent issues.
Let’s delve into this topic further.
This analysis of AI advancements is part of my ongoing coverage in a Forbes column, where I explore and elucidate the various complexities introduced by AI (see the link here for more).
By way of background, I have been extensively examining the multifaceted nature of modern AI developments, particularly those that generate mental health advice and conduct AI-driven therapy. The increasing use of AI in this field is largely driven by the rapid evolution and widespread acceptance of generative AI technologies. For a brief overview of some of my numerous columns on this dynamic topic, see the link here, which summarizes about forty of the over one hundred articles I have penned on the subject.
AI And Mental Health
As a quick background, I’ve been extensively covering and analyzing a myriad of facets regarding the advent of modern-era AI that produces mental health advice and performs AI-driven therapy. This rising use of AI has principally been spurred by the evolving advances and widespread adoption of generative AI. For a quick summary of some of my posted columns on this evolving topic, see the link here, which briefly recaps about forty of the over one hundred column postings that I’ve made on the subject.
There is little doubt that this is a rapidly developing field and that there are tremendous upsides to be had, but at the same time, regrettably, hidden risks and outright gotchas come into these endeavors, too. I frequently speak up about these pressing matters, including in an appearance last year on an episode of CBS’s 60 Minutes, see the link here.
Background On AI For Mental Health
I’d like to set the stage on how generative AI and large language models (LLMs) are typically used in an ad hoc way for mental health guidance. Millions upon millions of people are using generative AI as their ongoing advisor on mental health considerations (note that ChatGPT alone has over 800 million weekly active users, a notable proportion of which dip into mental health aspects). The top-ranked use of contemporary generative AI and LLMs is to consult with the AI on mental health facets; see my coverage at the link here.
This popular usage makes abundant sense. You can access most of the major generative AI systems for nearly free or at a super low cost, doing so anywhere and at any time. Thus, if you have any mental health qualms that you want to chat about, all you need to do is log in to AI and proceed forthwith on a 24/7 basis.
Compared to using a human therapist, AI usage is a breeze and readily undertaken.
AI For Consumers Versus Therapists
There are generic versions of AI, and there are also non-generic or customized versions of LLMs for mental health. Generic AI is used for all kinds of everyday tasks by consumers, and just so happens to also encompass providing a semblance of mental health advice. On the other hand, there are tailored AIs specifically for performing therapy; see my discussion at the link here.
A consumer typically chooses to use generic AI for generalized purposes and then discovers that they can also tap into the LLM for mental health guidance. This is becoming so popular that some consumers seek out the use of generic AI primarily for mental health advice.
Therapists are opting to make use of AI as part of their therapeutic practice. They might encourage clients to use generic AI or establish customized AI that specifically focuses on mental health. Controversy is associated with this approach. Some believe that the therapist-client dyad is sacrosanct and should not be marred by AI.
Others, such as in my view, assert that we are inexorably heading toward a new triad, the therapist-AI-client relationship, which is the future of therapy. See my detailed analysis at the link here.
Weighty Concerns About AI
There are significant worries that AI can readily go off the rails or otherwise dispense unsuitable or even egregiously inappropriate mental health advice.
Banner headlines in August of this year accompanied a lawsuit filed against OpenAI for their lack of AI safeguards when it came to providing cognitive advisement. Despite claims by AI makers that they are gradually instituting AI safeguards, there are still a lot of downside risks of the AI doing untoward acts, such as insidiously helping users in co-creating delusions that can lead to self-harm.
For the details of the OpenAI lawsuit and how AI can foster delusional thinking in humans, see my analysis at the link here. I have been earnestly predicting that eventually all of the major AI makers will be taken to the woodshed for their paucity of robust AI safeguards.
Regulators are starting to catch up with the advent of AI for mental health. I have done a series of in-depth assessments of the mainstay AI mental health laws at the state level, consisting of the recently enacted Illinois law, see my analysis at the link here, the Nevada law, see my analysis at the link here, and the Utah law, see my analysis at the link here.
There isn’t yet an equivalent federal law. An existing concern is that each state will craft its own idiosyncratic laws, creating a confounding and conflicting patchwork of such laws. A federal law could presumably provide an across-the-board standardized approach. Though numerous attempts at forging a federal law encompassing AI for mental health have been undertaken, the matter is still unresolved and appears to be stuck in limbo for now.
Messiness Of These Laws
None of these latest state laws is comprehensive in its breadth of AI mental health considerations. Gaps exist. This means that the public and AI makers are in a lurch regarding what is permitted versus what is restricted.
Another difficulty is that the way in which AI for mental health is defined in these laws is disparate and typically full of loopholes. AI makers can readily use legal acumen to try and wiggle out of the imposed conditions. You might be surprised to know that there isn’t already a standardized, legally tested definition of AI per se. See my comments on this challenging predicament at the link here.
It gets worse.
A noted concern that I have voiced is that states opting to craft such a law are likely to assume that they can simply grab a copy of another state’s AI mental health law and use that as a principled basis for devising their own state law. Unbeknownst to them, they are inadvertently starting with a rocky and faulty base. It’s an unsound foundation.
The odds are that not only will they by default incorporate the pitfalls and quandaries of an existing such law, but they will attempt to add their own particulars and make for a murkier and messier law. Double trouble. Begin with a law that hasn’t been ably worked out. Then add and subtract, adjust, and make things worse.
A triple trouble arises partially since the lawmaking process itself adds a forbidding, torturous, and convoluted layer to these valiant pursuits. As per the famed remark by statesman Otto von Bismarck: “Laws are like sausages. It’s better not to see them being made.”
Bottom line is that policymakers, regulators, lawyers, and even AI experts are frequently ill-informed about how to best devise AI-related laws, especially in the case of AI for mental health. They might be entirely sincere in their endeavors, but they aren’t well-armed for the crucial task at hand.
Framework For Policy And Legal Formulation
Based on my detailed analysis of the existing state laws that entail AI for mental health, I have derived a comprehensive framework for guidance on formulating regulations in this realm. The idea is to provide a one-stop shopping experience. Those aiming to devise regulations should mindfully consider every nook and cranny of this framework.
Doing so will tend to ensure that a comprehensive perspective is being undertaken.
Whether the policymakers and lawmakers choose to cover every aspect in their legislation is not necessarily going to be the case. Some might decide that they wish to leave particular aspects outside the scope of their efforts. There also might be other allied laws already on the books that cover this or that aspect. The essence is that at least they will be cognizant of what they have opted to cover and what they are also expressly not covering.
I have shaped the policies into twelve distinctive categories:
- (1) Scope of Regulated Activities
- (2) Licensing, Supervision, and Professional Accountability
- (3) Safety, Efficacy, and Validation Requirements
- (4) Data Privacy and Confidentiality Protections
- (5) Transparency and Disclosure Requirements
- (6) Crisis Response and Emergency Protocols
- (7) Prohibitions and Restricted Practices
- (8) Consumer Protection and Misrepresentation
- (9) Equity, Bias, and Fair Treatment
- (10) Intellectual Property, Data Rights, and Model Ownership
- (11) Cross-State and Interstate Practice
- (12) Enforcement, Compliance, and Audits
Due to space limitations here, I will provide a brief summary of each category. A complete stipulation of each category and its associated subcategories will be provided in a subsequent posting. Keep on the watch for that posting.
Let’s now take a look at each of the categories.
1. Scope of Regulated Activities
Establishing proper scope is crucial, otherwise the proposed policy or law will wander afield of where it needs to be. A pivotal but often shoddily written element of policies and regulations concerning AI for mental health involves providing an improper definition of terms. I realize that might seem like a trivial aspect, but the truth is that tightly woven definitions make-or-break the matter.
If the definition of AI is overly broad, this opens the door to all types of tech being construed as pertinent to the policy or regulation. Technology of all kinds will suddenly be considered within the scope at hand. A broad-brush definition risks undermining technology adoption at large and creates undue exposures for tech makers that have nothing to do with the circumstance.
Meanwhile, a definition of AI that is overly narrow will potentially allow AI makers to exploit a loophole. For example, many times the AI definition specifically and solely refers to LLMs. That’s a problem because AI for mental health might be implemented via other means, such as expert systems. An AI maker will slyly lean into legal minutiae to avoid accountability.
Another twist is how the AI is being applied in mental health. In the case of therapists, they might use AI for their administrative tasks, rather than for therapy services. If the policy is intended to be about the mental health realm per se, inadvertently encompassing billing chores could be out of place.
You must decide which battle or battles are being fought.
Scope aspects go much further. Does mental health equate to the purview of mental well-being? An AI maker might proclaim that their AI is built exclusively for mental well-being, and not for mental health. This is the crafty slipperiness that gets undertaken. Various other angels exist. Suppose that an AI does triaging for mental health, which might be argued as not performing mental health acts. That is certainly debatable since the triaging is likely to assess alleged mental health conditions.
2. Licensing, Supervision, and Professional Accountability
A policy or regulation in AI for mental health must clearly stipulate the details associated with licensing, supervision, and professional accountability. Omitting any of those factors is going to allow slippage and murkiness.
Who is to be held legally responsible for the AI when the AI produces mental health guidance that is incorrect, harmful, misleading, etc.?
You cannot just imply or stipulate that the AI itself is responsible. That makes no sense in today’s world, namely, we do not currently recognize legal personhood for AI (see my coverage at the link here). Humans must be held accountable. Which humans? In what way are they to be held responsible? And so on
In the case of therapists who opt to use AI for therapy, do they need to disclose to their clients that AI is being utilized? In what manner and how communicated? Is the AI for diagnosis, treatment, or what purposes? For my discussion on the therapist-AI-triad, which is replacing the traditional dyad of therapist-client, see the link her
3. Safety, Efficacy, and Validation Requirements
AI in mental health carries inherent risks. It is not a risk-free or zero-risk setting. A policy or law that stipulates the risk must be completely removed is making a hyper leap that cannot be attained. Essentially, any use of AI for mental health would be an instant violation.
The focus should be on the levels of risk. What level of risk is acceptable? What level of risk is unacceptable? The highest risk elements should naturally receive the most attention.
What is the range and depth of AI safety precautions that are expected to be undertaken? How are those to be validated and detected when they possibly go awry? If the AI is used for training purposes, does that still constitute as within the boundaries, or does the educational use fit into a different bracket?
The latest trend in AI for mental health is toward evidence-based validation, see my discussion at the link here and the link here. Bright-line rules are important to establish in a given policy or regulation.
4. Data Privacy and Confidentiality Protection
AI for mental health will almost assuredly capture deeply personal information. The AI is devised to find out as much as possible about the person during the mental health dialogue. People are willing to open their hearts and minds to the AI, pouring out private details that they wouldn’t even tell a fellow human.
Is the data to be properly stored and protected?
A policy or regulation needs to speak to the data privacy and confidentiality considerations. By and large, most of the major AI makers tend to have online licensing agreements that tend to indicate that users do not have privacy or confidentiality when using the AI. The AI maker can inspect the entered data. The data can even be used for further data training of the AI.
A solid policy or regulation must stipulate where it stands on these aspects. Is there to be explicit, informed consent for data collection, limit secondary uses, and prohibit the sale of mental-health-related data to advertisers or data brokers? Should encryption, secure storage, and minimization principles be mandated to prevent breaches? Does HIPAA apply to this realm of AI usage?
5. Transparency and Disclosure Requirements
Some AI makers have a tiny message at the login page that warns users about the tradeoffs of using the AI for mental health guidance. Does that constitute sufficient heads-up for users? Maybe, maybe not. Or the warning is buried on a webpage that houses their online licensing agreement. Again, is this sufficient notification?
A policy or regulation must stipulate what kinds of disclosure and transparency requirements are expected of AI that performs mental health guidance.
6. Crisis Response and Emergency Protocols
The odds are high that an AI performing mental health guidance is going to encounter users who express self-harm or other endangering thoughts. Some LLMs are devised to do nothing about this. Other LLMs will tell the person they should consider visiting a therapist. A wide variety of responses are somewhat arbitrarily being implemented by AI makers.
A policy or regulation needs to identify how the AI is expected to handle crisis detection and what the AI is to do as a crisis response.
An interesting approach was recently announced by OpenAI. They intend to shape ChatGPT and GPT-5 to seamlessly hand over an online chat to a curated network of therapists when needed, see my coverage at the link here. Should this approach be mandated via policy and regulations for all AI makers, or be it up to each AI maker to decide to undertake?
7. Prohibitions and Restricted Practices
A policy or regulation must establish boundaries for AI that performs mental health guidance, consisting of stated prohibitions that are clearly delineated.
What are the allowed practices, and what are the disallowed practices?
For example, is it permitted for the AI to make clinical diagnoses on its own, or does a human therapist need to be in the loop? Are minors allowed to use the AI, or is it restricted to adult use only? Does parental consent need to be obtained, and if so, how is this to be undertaken?
8. Consumer Protection and Misrepresentation
AI makers are tempted to tout that their AI ably assists in overcoming mental health problems. Marketing and advertising can be over-the-top and make promises that cannot reasonably be kept. This has already brought the attention of the FTC; see my coverage at the link here.
A policy or regulation must identify whether those making the AI or promulgating the AI are to be held responsible for any deceptive or unsupported therapeutic claims.
Can an AI maker imply that their AI is equivalent to licensed mental health professionals? Or make unsupported claims of therapeutic efficacy? Marketing and advertising ought to accurately reflect what the AI can and cannot do. Vulnerable users are especially readily misled by false claims.
9. Equity, Bias, and Fair Treatment
It is larger known that AI often veers into algorithmic biases about factors affecting racial, gender, disability, or socioeconomic groups. I’ve extensively examined this and been covering efforts to retool AI to reduce these proclivities; see the link here.
A policy or regulation overseeing AI for mental health should include parameters associated with the assessment and mitigation of bias across the development life cycle of the AI, including model training, evaluation, and deployment. This includes monitoring for demographic performance gaps in symptom assessment, risk detection, or triage recommendations.
Bias-mitigation mechanisms should be ongoing because model behavior can drift over time.
10. Intellectual Property, Data Rights, and Model Ownership
Imagine that a therapist uses AI as a therapeutic tool, and in so doing opts to essentially “train” the AI on how to be a therapist. Does the AI maker now own that capability, or does the therapist “own” it?
Many AI makers are gradually allowing users to indicate whether the data they enter can or cannot be used for further data training of the AI by the AI maker. But this is an ad hoc rule haphazardly adopted by AI makers. Some allow an opt-out, while others do not.
A policy or regulation for AI in mental health needs to explicitly identify the nature of intellectual property rights, data rights, and model ownership rights.
Allied considerations include whether users are to have the ability to access, correct, or delete their data. And whether they can request human review of AI-influenced decisions and obtain explanations about how AI contributed to the generated results. There should also be an indication of whether users can file complaints with governmental agencies, seek remediation when harmed, and opt out of automated profiling. Redress mechanisms are vital since they tend toward accountability and safeguard users from opaque or harmful AI behaviors.
11. Cross-State and Interstate Practice
State-level and local-level policies and regulations in AI for mental health are replete with complex jurisdictional intricacies.
Suppose an AI maker in one state opts to make their AI available for those in other states. A person in one of those other states uses the AI for mental health guidance. The person receives foul advice. Does a state law in that person’s state regarding AI mental health provide applicable redress concerning an AI maker that resides in a different state?
The same kind of question applies to therapists. A therapist in one state performs mental health guidance to people in other states. The therapist sets up an AI for their therapeutic practice. A client from a different state receives counseling from the therapist and uses the AI. Issues arise with both the therapist and their AI. Will the client be able to pursue the matter in their state or have to do so in the originating state?
These jurisdictional issues partially stem from the lack of a federal overarching policy or regulation concerning AI for mental health, which presumably would establish interstate provisions. In any case, for now, states or local policymakers and regulators must explicitly stipulate what they believe their jurisdictional boundaries consist of in their AI for mental health provisions.
Without sufficiently addressing this, legal gray zones will create easy routes of escape from accountability.
12. Enforcement, Compliance, and Audits
When it comes to the classic carrot or the stick, policies and regulations about AI for mental health are only likely to have any teeth if they include adequate enforcement provisions. The harsh stick approach can be a sizable motivator.
A policy or regulation must specify how claimed harms will be investigated and what penalties will be imposed for validated violations.
Does the policy or regulation authorize named agencies to investigate harm and audit AI systems, require documentation, etc.? Are there fines, mandatory corrective actions, suspension of deployment, or permanent bans for egregious misconduct?
Keep in mind that if the potential penalties are considered inconsequential by an AI maker, they are likely to believe that violating the policy or regulation isn’t a big deal. They will simply ignore it and be willing to absorb the pesky but trivial imposition. The envisioned enforcement must have sharp enough teeth. Plus, it must be perceived as a credible threat and not just a flimsy or unlikely-to-be-enforced provision.
The State Of AI For Mental Health
We are all immersed right now in a grandiose experiment. AI for mental health is being globally made available. Is this good for society? Will AI help at scale to improve our collective mental health? Some see the other side of the coin as more likely. They view that this Wild West and wanton AI usage for mental health guidance is going to worsen mental status massively.
A vacuum exists right now about how AI for mental health is to be suitably devised and promulgated. A few states have chosen to compose and enact relevant laws. Despite the sincerity, those laws have issues. Nearly all the states do not yet even have any such laws on their books.
The famous American statesman J. William Fulbright made this pointed remark: “Law is the essential foundation of stability and order both within societies and in international relations.” My prediction is that we are going to rapidly see states and local entities jump on the AI for mental health policymaking and lawmaking bandwagon. That’s good news when done properly. It could be bad news if done poorly.
I recommend that, as a foundation for stability, my listed twelve categories and corresponding provisions should be mindfully taken into consideration. Our societal mental health depends on doing so.
