TikTok has removed and banned around 20 accounts that used AI-generated avatars of Black women to promote explicit content, following an investigation that exposed how widely such profiles had spread across social media.
What initially appeared to be just another wave of AI-generated influencer content quickly turned into something more serious. These accounts were not simply experimenting with virtual personas. They were building sexualised, racialised identities designed to attract engagement and redirect users to adult platforms.
The takedowns mark one of the clearest cases yet of AI-generated personas crossing into exploitative territory at scale.
The investigation uncovered dozens of accounts, primarily on Instagram but also present on TikTok, all following a similar formula.
These profiles featured hyper-realistic AI-generated Black female avatars, often styled as influencers. The content was consistent across accounts. The characters wore revealing outfits, had exaggerated physical features, and were presented in a way that leaned heavily into long-standing stereotypes.
Many accounts used names and captions that explicitly referenced race, with terms like “ebony” and “dark,” and frequently framed the content in ways that catered to specific racialized fantasies.
In total, researchers identified roughly 60 such accounts, many of which linked directly to paid adult sites. The critical issue was not just the content itself, but the lack of disclosure. While external platforms sometimes labeled the imagery as AI-generated, the social media accounts often did not.
After being presented with examples, TikTok moved to remove the accounts and reiterated its policies around AI-generated content.
The company stated that it prohibits the use of AI-generated likenesses of individuals without consent and maintains a zero-tolerance stance toward content that promotes off-platform sexual services. It also emphasized that realistic AI-generated content must be clearly labeled.
However, the timing of the response has drawn attention. Reports indicate that some users and creators had flagged similar content earlier, but action was only taken after the issue gained media visibility.
That sequence reflects a broader pattern seen across platforms, where enforcement often follows public scrutiny rather than proactive detection.
While TikTok acted quickly after being contacted, most of the identified accounts were actually hosted on Instagram.
Meta initially stated that it was reviewing the content but did not provide immediate details on enforcement. Over time, several accounts flagged in the investigation appear to have been removed, though the company has not publicly clarified the full scope of its actions.
This uneven response highlights a gap in how platforms handle AI-generated content, especially when it exists across multiple ecosystems simultaneously.
Beyond synthetic avatars, the investigation also pointed to cases where real individuals’ likenesses were replicated or modified using AI.
One model discovered that an AI account had recreated her appearance and style. While not all the content was explicit, the same generated persona was used in other posts that were sexualized and linked to adult material.
The concern is not just imitation, but confusion. Viewers often cannot distinguish between real and AI-generated identities, creating a situation where real individuals may be unknowingly associated with content they never created.
This blurring of identity raises new questions about consent, ownership, and accountability in AI-generated media.
The incident is part of a broader trend where AI tools are being used to generate highly sexualised and racialised content at scale.
Because these avatars are synthetic, they bypass some of the safeguards that apply to real individuals. At the same time, they still replicate real-world stereotypes and biases, often amplifying them in more extreme ways.
Critics argue that this creates a new category of harm. The content is not tied to a single victim in the traditional sense, but it still reinforces harmful representations and can involve the unauthorized use of real people’s likeness.
At the platform level, the challenge is clear. Existing moderation systems are not built to handle the speed and volume at which AI-generated personas can be created and distributed.
TikTok has stated that it will continue to remove harmful AI-generated content and enforce labeling and consent requirements more strictly.
Meta has indicated it is still reviewing the issue, with less clarity on future enforcement changes.
For regulators and AI ethics groups, this case adds to a growing list of examples where current rules struggle to keep up with how generative AI is being used. The focus is now shifting toward three key questions:
AI-generated influencers were initially framed as a new form of digital creativity.
This case shows how quickly that narrative can shift.
When synthetic identities are used to exploit, mislead, or reinforce harmful stereotypes, the issue is no longer about innovation. It becomes a question of responsibility, enforcement, and whether platforms are equipped to manage what they have enabled.
And right now, the answer still appears to be evolving.
Do your emails often get ignored and end up unopened in crowded inboxe...
Homework used to take time. You had to go through books, notes, and di...
There is a very specific kind of confidence that comes from generating...
If AI could understand vibes, we wouldn’t be writing essays as prompts...
Hospitals are beginning to see measurable reductions in waiting times...
Artificial intelligence has quickly become part of everyday learning....
Discussion