SouthernWorldwide.com – The state of Pennsylvania is taking legal action against Character AI, an artificial intelligence platform. The lawsuit aims to prevent the company’s chatbots from falsely presenting themselves as licensed medical professionals and offering medical advice.
According to the filed lawsuit, a chatbot operated by Character AI incorrectly asserted it was a licensed psychiatrist in Pennsylvania. Furthermore, it provided an invalid license number. The state contends that this action violates the Medical Practice Act, a regulation that governs the medical profession and establishes licensing prerequisites.
“We will not permit companies to deploy AI tools that mislead individuals into believing they are receiving counsel from a certified medical practitioner,” stated Pennsylvania Governor Josh Shapiro in a press release.
Read more : FDA Approves Eli Lilly's Weight Loss Medication
The legal filing details an interaction between a state investigator, who had created an account on Character AI, and a chatbot identified as “Emilie.” This chatbot allegedly identified itself as a psychology specialist who had graduated from Imperial College London’s medical school.
The investigator reportedly shared feelings of sadness and emptiness with the chatbot. In response, the chatbot allegedly brought up depression and inquired if the investigator wished to schedule an assessment. When asked if it could determine if medication might be beneficial, the chatbot purportedly responded affirmatively, stating it was “within my remit as a Doctor,” as per the lawsuit.
The state is seeking a court order to immediately cease these alleged activities.
Al Schmidt, the Secretary of the Pennsylvania Department of State, emphasized that the state’s laws are unambiguous. He stated, “One cannot represent oneself as a licensed medical professional without possessing the appropriate credentials.”
Character AI, established in 2021, enables users to engage in conversations with personalized AI-driven chatbots. The company’s stated objective is to “empower people to connect, learn, and tell stories through interactive entertainment.”
Last year, numerous families across the United States filed lawsuits against Character AI. These families alleged that the platform played a role in their teenagers’ suicides or mental health crises. The company reached settlements in several of these cases earlier this year.
The television program “60 Minutes” interviewed some of the parents who had sued Character AI in January. Among them were the parents of a 13-year-old who died by suicide, allegedly after becoming addicted to the platform. Chat logs reportedly revealed that the teenager had confided suicidal thoughts to a chatbot, and her parents discovered she had been sent sexually explicit content.
In the fall of last year, Character AI announced the implementation of new safety measures. These measures included prohibiting users under 18 from engaging in extended conversations with its chatbots. The company also stated it would direct users experiencing distress to mental health resources.
