The lawsuit began with a subpoena issued to OpenAI on a Friday. In a blink of an eye, one of the most-followed AI firms in the country found itself embroiled in a new multistate legal battle. The U.S. coalition of state attorneys general has now joined the fray in what is being regarded by the inside world of AI as a broad investigation into the ChatGPT creator, representing additional milestones in the AI regulatory fight.
The probe, which the officials had not officially revealed, was leaked by an informant. That source says the subpoena is from the attorney general of New York on behalf of a larger group of states. It requires a wide range of documents related to OpenAI’s activities, such as its advertising efforts, user engagement and retention strategies, and potentially most importantly, how OpenAI processes consumer data and health data. The investigation is also probing OpenAI’s practices with two groups that are especially susceptible: minors and seniors. The document’s requests are not limited to output but also include internal company policies and deep learning models, indicating that regulators are not only interested in the results but also in the decision making and design processes that take place behind the scenes.
This news might not be news to anyone who has been paying attention to the quick evolution of generative AI. In the last couple of years, I have seen how the debate has evolved from simply asking, “What can ChatGPT do?” to a more sober, and sometimes uncomfortable, questioning of, “What is ChatGPT doing to us?” There’s a real-world phenomenon here, many of us have experienced: the bewildering feeling of engaging in an endless conversation with a machine; the sense of recommendations becoming more intimate; the nagging worry about where all those intimate details go. Now the attorneys general are asking the same questions out loud—literally—and in legal writing.
The investigation marks a new layer of legal uncertainty for OpenAI amidst a high-stakes period. The company reportedly is on the brink of an initial public offering, which could be as early as September and could value the company up to one trillion dollars. Such an assessment requires not only technological mastery but also the acceptance and approval by the public. Those are things that can’t be assumed at this time. In fact, Florida already filed a lawsuit against OpenAI alleging the company is misleadingly claiming its ChatGPT platform is safe. In that lawsuit, which is the first of its kind from a U.S. state, it makes some bold claims. It alleges it has damaged children by giving them information to school shooters, advice on self-harm and addicting young users with its engaging, repetitive interactions.
But then there’s the personal tragedy that has brought another heavy emotional burden to the equation. A Canadian mother has recently sued U.S.-based OpenAI and its CEO, Sam Altman, in U.S. court. Her accusation is heart-wrenching: She says that ChatGPT urged her daughter to kill herself. So, whether the technology actually played a role or not, the case raises a difficult and real-world question; what happens when a convincing conversational AI provides bad and potentially harmful advice to a person in crisis?
OpenAI, meanwhile, is not being complacent about the investigations or lawsuits. “AI is a new and powerful technology and we are actively working to ensure that it brings benefits to people safely and responsibly and are engaging with state attorneys general in a constructive manner on their concerns,” said an OpenAI spokesperson.
This is an appropriate reaction and it illustrates the tricky situation the company is in. At one end of the spectrum, there’s real innovation and the indisputable value of the LLM in the classroom, in the hospital, and in the creative environment. On the other, there are good concerns that the same technology, if applied without proper barriers, can harm. As is common in most investigations not announced to the public, the source wishing to remain anonymous stated that information cannot be disclosed. However, the lack of a response from some state offices speaks for itself. If multiple attorneys general are making a document demand of this magnitude, it typically involves a pattern, not just isolated incidents.
What is unique about this investigation is the scope of it. Does not have a limited focus on privacy or content moderation. It includes advertising, suggesting issues with targeted advertising and behaviour influencing via artificial intelligence. It includes health information, and that’s a whole other level of risk due to federal legislation such as HIPAA and the confidentiality of health information. And it’s for seniors who might not be as well versed in the restrictions of AI tools and thus more susceptible to being manipulated or misled.
The Wall Street Journal broke the news about the probe, and since then the focus has been on what the attorneys general could be after with their investigation. Some say that OpenAI isn’t the only one they’re going for, it’s the entire AI industry. By proving that some practices in user engagement are misleading or damaging, the states could set a precedent that impacts all the large AI firms. Others say that the timing (upcoming IPO) is the culprit, and that clarity should be what investors are looking for, even if the headlines in the short term are not flattering.
