San Francisco City Attorney David Chiu Demands Apple and Google Remove AI Nudification Apps to Halt Facilitation of Nonconsensual Sexual Imagery

In a significant escalation of the legal and regulatory pressure facing the world’s largest technology firms, San Francisco City Attorney David Chiu has issued formal cease-and-desist letters to Apple and Google. The directives mandate the immediate removal of 13 specific applications from the App Store and Google Play Store that enable users to create AI-generated nonconsensual intimate imagery, a process colloquially referred to as "nudification" or "undressing." These legal notices serve as a stern warning to Silicon Valley giants, asserting that by hosting and profiting from such software, the companies are effectively aiding and abetting the distribution of sexually explicit deepfakes.

The move represents a coordinated effort by municipal authorities to address the growing crisis of AI-driven harassment. Chiu’s office, which has previously targeted the infrastructure of the deepfake industry, argues that the current moderation systems employed by Apple and Google are fundamentally inadequate. The demand is not merely for the removal of the 13 flagged apps, but for a comprehensive overhaul of the vetting processes that allow such technologies to bypass platform security protocols in the first place.

A Chronology of Escalating Concerns

The proliferation of "nudify" technology has been a persistent issue over the past five years, characterized by a cat-and-mouse game between watchdog organizations and tech platforms. While the technology has become increasingly accessible, the public outcry reached a breaking point following several high-profile reports.

In January, research revealed that the widespread availability of these apps on mobile storefronts had enabled the creation of millions of sexualized images, often targeting women and children without their consent. The Tech Transparency Project (TTP) conducted an extensive investigation in both January and April, identifying approximately 100 applications across both platforms that facilitated this behavior. Despite the companies’ promises to strengthen safety measures, the TTP found that the problem had not abated, with these apps collectively amassing an estimated 480 million downloads and generating upwards of $120 million in revenue.

The City Attorney’s latest action is the culmination of these recurring findings. By targeting specific apps that function as "face-swapping" tools but contain hidden or easily accessible features for generating nonconsensual nudity, the city is shifting the focus from general content moderation to the economic facilitation of sexual abuse.

The Mechanism of Harm and "Dual-Use" Technology

The danger of these applications lies in their deceptive marketing and "dual-use" nature. According to a preprint research paper published in May by academics from Cornell and Georgetown Universities, 420 apps offering face-swapping capabilities were analyzed, with researchers finding that 70 percent of those tested could be weaponized to create deepfake nudes without any internal safeguards.

These apps rarely market themselves as "nudification" tools, opting instead for benign labels like "AI portrait editing" or "creative face-swapping." This strategic ambiguity allows them to evade the initial submission review process of the App Store and Google Play. Once installed, however, the software often unlocks features that permit users to manipulate photos with just a few clicks. The results, as noted by security researchers, have become alarmingly realistic, with some apps providing these services for free while others utilize subscription models that generate profit for both the developer and the platform operator, which typically takes a commission of 15 to 30 percent on in-app purchases.

The human cost of this technology is immense. Reports have surfaced of deepfake sexual abuse images being created in at least 90 different school districts, with victims suffering severe psychological trauma, reputational damage, and, in some tragic instances, suicidal ideation.

Official Responses and Corporate Accountability

Both Apple and Google maintain that they strictly prohibit pornography, harassment, and nonconsensual imagery on their platforms. In response to the cease-and-desist letters, both firms have taken varying degrees of action.

Google spokesperson Dan Jackson confirmed that the company has deleted hundreds of apps identified as violating their policies, including the five specifically flagged by the San Francisco City Attorney. "Google Play does not allow apps that contain sexual content, and we continually take proactive steps to detect and remove apps with harmful content," Jackson stated. He added that the company has also restricted the use of certain search terms, such as "nudify," to prevent users from discovering such software.

Apple, meanwhile, has taken a more targeted approach, confirming the removal of three of the eight apps flagged by the city. Apple spokesperson Adam Dema stated that the company is currently in the process of terminating the developer accounts associated with those apps. Regarding the remaining four apps, Apple indicated that it is giving the developers a chance to address policy violations, though it reserves the right to remove them if they fail to comply. "The App Store was designed to be a safe and trusted place for users," Dema noted, emphasizing that the company relies on a combination of automated detection and user reporting to enforce its guidelines.

The Broader Implications for Tech Governance

The tension between the City Attorney’s office and the tech giants highlights a fundamental debate regarding the responsibility of "gatekeepers" in the digital age. Under California law, there are provisions that prohibit the provision of services that facilitate the creation of deepfake pornography. Chiu argues that because these platforms take a cut of the revenue generated by these apps, they are not merely passive hosts but active participants in the economic engine of sexual harassment.

Legal experts suggest that if these platforms do not tighten their vetting processes, they may face a growing wave of litigation. The argument is that the "dual-use" nature of these apps is a known vulnerability, and failing to account for it constitutes a breach of the duty of care that these companies owe to their users.

Furthermore, the scale of the issue—hundreds of millions of downloads and nine-figure revenues—suggests that current manual review processes are insufficient. The demand for "proactive" rather than "reactive" moderation implies that Apple and Google may need to invest in more sophisticated AI-based detection systems designed specifically to identify the underlying architecture of nudification software before it is ever published to the public.

Future Outlook and Regulatory Pressure

As of mid-2026, the legislative landscape is also shifting. New laws and international bans are being enacted to combat the scourge of explicit deepfakes, but the pace of technological development continues to outstrip regulatory response. The San Francisco City Attorney’s office has signaled that it will continue to monitor the situation closely, with Chiu explicitly warning that if the companies do not move to permanently secure their platforms against these tools, the city will consider all available legal avenues to force compliance.

The outcome of this confrontation will likely serve as a precedent for how municipalities and state governments interact with global tech conglomerates regarding the harms of generative AI. Whether this results in a permanent sanitization of app stores or a prolonged legal battle over the limits of platform liability remains to be seen. What is clear, however, is that the era of treating deepfake "nudification" as an inevitable by-product of AI innovation is coming to an end, with stakeholders increasingly demanding that those who host the technology take full responsibility for its devastating real-world consequences.