Anthropic has updated its policy to prevent users from engaging in "sustained and needless abusive or cruel behavior" towards its AI models. This change comes as the company's leadership continues to explore the concept of machine consciousness.
The Verge was the first to report this policy shift. A spokesperson for Anthropic, the creators of the AI chatbot Claude, did not immediately respond to questions regarding what qualifies as "abusive or cruel."
In its updated user policy, the San Francisco-based company clarified that the ban would not cover typical user frustrations, model testing, or themes with a "dark creative" aspect. Previously, Anthropic's policy included restrictions on abusive conduct.
Anthropic's large language models can terminate conversations if a user is persistently harmful. This feature, introduced last August, was described as a protective measure for the AI's welfare.
"On the issue of AI consciousness, we are highly uncertain about the potential moral status of Claude and other large language models, now or in the future. Nonetheless, we take this matter seriously and are working to identify and implement low-cost interventions to mitigate risks to model welfare, in case such welfare is possible. Allowing models to end or exit potentially distressing interactions is one such intervention," states its website.
The idea of AI systems possessing consciousness remains a contentious topic both within and outside the tech industry. Anthropic CEO Dario Amodei has stated that he cannot dismiss the possibility. In contrast, Sam Altman, CEO of OpenAI, has shown reluctance to consider the idea.
Read next
OpenAI Anticipated to Miss Revenue Target by $20bn
OpenAI has disclosed a $20 billion shortfall in projected revenue compared to earlier estimates provided to investors, prompting scrutiny over the anticipated rapid growth in AI demand.
The company behind ChatGPT informed investors that it expects this year’s revenue to reach $50 billion, based on sales up to September.
"AI Surveillance: Co-op Joins Firms Monitoring Employees in 'Dystopian' Move"
The Co-op has introduced AI surveillance to monitor its workers, utilizing automated listening technology to evaluate phone calls between employees and customers. This system, criticized as "oppressive and dystopian" by a whistleblower, is employed by Co-op Legal Services to assess interactions concerning probate, wills, and estates.
US officers worry Meta smart glasses could covertly record them
In January, the NYPD’s counterterrorism unit warned officers about a new security threat: Ray-Ban Meta glasses. A memo directed officers to “conduct thorough inspections of all eyewear permitted within inmate cells,” cautioning that recently arrested individuals could use the devices’ small cameras and microphones to record inside police