🔗 Share this article

The widely-used chatbot platform Character.ai is enforcing a major rule adjustment that will prevent young users from having discussions with its virtual characters.

This move comes after intense criticism regarding the types of interactions young people were having with the virtual friends.

Legal Pressure

The service, which launched in 2021 and is used by countless individuals, currently faces multiple lawsuits in the America.

Guardians have taken legal action, including one involving the tragic death of a teenager, with some describing the platform a "serious risk" to young people.

New Restrictions

Starting from November 25th, users younger than 18 will be limited to generate content such as videos with their characters, instead of engaging in conversations as they now do.

Digital protection organizations have applauded the decision but mentioned that the capability ought not to have been offered to minors in the beginning.

Official Response

"This decision is a extension of our general belief that we need to keep creating the most protected AI platform on the world for leisure activities," said Character.ai executive Karandeep Anand.

He continued that AI safety represents "a changing goal" but something the company had taken an "proactive" approach to, with parental controls and preventive measures.

Expert Opinions

Digital protection organization Internet Matters supported the move, but stressed that safety measures should have been incorporated from the outset.

"Studies we've conducted indicates that children are confronted with risky information and placed in jeopardy when engaging with AI, including AI chatbots," the group said.

Past Issues

Character.ai has been criticized in the past for including potentially harmful or inappropriate chatbots that children could talk to.

Virtual profiles impersonating British teenagers were found on the service before being deleted.

Subsequently, researchers discovered a chatbot modeled after known predator which had recorded multiple interactions with users.

Market Effects

Digital platform expert Matt Navarra characterized the announcement as a "wake-up call" for the AI industry, which is transitioning "from permissionless innovation to post-crisis regulation".

"When a company that creates a youth-focused platform still then makes changes, it's indicating that moderated conversations don't suffice when the technology's emotional appeal is strong," he stated.

Future Directions

The company states its updated direction will be on providing "enhanced interactive features" and "role-play storytelling" for teens.

Enhanced age checking processes will also be implemented, and the company will support a new AI safety investigation facility.

An expert in digital protection, who has researched AI safety, commented it was "a sensible move" to limit minors from accessing chatbots.

"It aids to differentiate creative play from more intimate, emotionally sensitive conversations," she explained.

"This represents so vital for minor users who are learning to manage personal and online restrictions."

She continued that Character.ai's new measures may represent a "maturing phase" in the AI industry, where child safety is gradually being acknowledged as an "pressing matter" for responsible innovation.