The story so far: California’s governor Gavin Newsom promised to take action on deepfake AI-generated content in late July after X owner and billionaire Elon Musk re-posted a deepfake video of U.S. Vice President Kamala Harris, in which she refers to herself as a “diversity hire”. In response to the governor’s criticism, Musk made an obscene remark, claiming parody was legal in the country.

On September 18, Newsom re-upped his X post criticising Musk and said he had made such content illegal in California by signing a bill. He signed three pieces of legislation to take action against deepfakes or digitally altered content in election campaigns that could be “deceptive”.

“You can no longer knowingly distribute an ad or other election communications that contain materially deceptive content -- including deepfakes,” Newsom said.

What are the new anti-deepfake bills about?

According to Newsom, users can no longer distribute ads or election communications with “materially deceptive content,” including deepfakes.

Newsom signed three bills in total: AB 2655 that will make large online platforms remove or label election-related deepfakes during “specific periods”; AB 2839 to expand the interval during which posters cannot distribute election ads or material with “deceptive” AI-generated or manipulated content; and AB 2355 that requires electoral ads with AI or altered content to disclose the same.

The three bills also give regulators and law enforcement bodies the power to handle such media through legal means or have them taken down.

“Safeguarding the integrity of elections is essential to democracy, and it’s critical that we ensure AI is not deployed to undermine the public’s trust through disinformation – especially in today’s fraught political climate. These measures will help to combat the harmful use of deepfakes in political ads and other content, one of several areas in which the state is being proactive to foster transparent and trustworthy AI,” said Newsom in an official press note on September 17.

How watertight are the bills?

The bills largely apply to California and expand already existing laws regarding election-related conduct that prevent highly altered media from being circulated for a stipulated number of days before or after elections in the state.

While Musk claimed that the governor’s actions are in violation of the Constitution of the United States, a number of compliant AI companies have already established filters and content restrictions that stop users from making deepfakes of election candidates, officials, heads of state, celebrities, etc.

AB-2655 stated: “This bill, to be known as the Defending Democracy from Deepfake Deception Act of 2024, would require a large online platform, as defined, to block the posting of materially deceptive content related to elections in California, during specified periods before and after an election. The bill would require a large online platform to label certain additional content inauthentic, fake, or false during specified periods before and after an election in California.”

These are largely standard requirements across most jurisdictions, and major search engine platforms have also committed to identifying or watermarking AI-generated images in order to prevent user confusion and the dissemination of false news.

Musk has used parody as an excuse to defend the deepfake campaign ad showing Harris in a negative light. In fact, AB 2655 stated the bill “would also exempt content that is satire or parody.”

While the original deepfake video’s poster stated that it was a parody, one version that Musk shared on X on July 27 did not have any Community Notes or platform-based disclosures that made it clear the media was fake.

A screenshot of Elon Musk sharing a deepfake video featuring Harris | Photo Credit: Elon Musk on X

What will happen if someone violates the regulations?

Though the newly signed bills and the existing laws have laid out measures for violations, it is not yet clear how this will play out in reality if someone violates the new regulations. The bills were signed almost fifty days before the U.S. election is set to take place, so it will be difficult to assess how effective or rigorous they are in such a short time span.

Adding to this, the bills may not severely penalise non-compliant individuals or entities.

While AB 2355 states that the following disclosure - “Ad generated or substantially altered using artificial intelligence” - should be added to applicable media content, the bill noted that violating the disclosure requirements “does not constitute a misdemeanor.”

Though powerful stakeholders such as Musk are not likely to be curtailed by the new bills, the legislation may be used to shine a light on the rising volume of AI-generated misinformation spreading across X. A significant portion of this is enabled by the Grok AI chatbot, which has fewer content restrictions than its counterparts from more compliant AI companies.

Ultimately, repeated violations of California’s laws could land X in legal hot water over time, and affect its already shaky standing with more risk-averse advertisers.

In response to Newsom’s news of the bills being signed, Musk again amplified the parody campaign ad featuring Harris that prompted criticism from the governor in the first place.

Musk shared the deepfake video with his more than 180 million followers, saying it “would be a shame if it went viral.”

However, the version he shared this time clearly identified the video as a parody.

A screenshot of Elon Musk criticising Newsom’s new legislation | Photo Credit: Elon Musk on X