- TikTok will test improvements in detecting accounts that post AI-generated spam about politics, health, and finance.
- Scams using fake videos of people with Down syndrome to sell overpriced products have been uncovered.
- The European Union will implement mandatory labeling of AI-generated content from December 2026.
- Platforms are opting for transparency through labeling instead of total restriction.
The proliferation of AI-generated content on TikTok has led the platform to take stronger measures. In the coming weeks, the social network will test improvements to its detection systems to identify accounts dedicated to posting AI-generated spam, especially on sensitive topics such as politics, health, current events, and financial advice. This decision responds to the need to protect user trust and curb so-called "digital garbage" or "AI slop . "
The problem isn't limited to low-quality content. Recent investigations have uncovered a network of scams using AI-generated videos featuring people who appear to have Down syndrome to emotionally manipulate users and redirect them to online stores selling overpriced products. Faced with this situation, TikTok is not only strengthening its detection capabilities but also aligning itself with new regulatory requirements in Europe, where mandatory labeling of synthetic content will become a reality by the end of the year.
TikTok strengthens its fight against AI-generated spam
The company has announced it will test improvements to its detection systems to identify accounts posting AI-generated spam on topics that could affect public trust or user well-being. According to the official statement, these accounts produce "low-quality and highly standardized" content and exhibit a concentration of high-risk topics. Furthermore, TikTok has already removed more than 86 million fake accounts in the first three months of the year, although the evolution of technology necessitates constant updates to its tools.
The phenomenon known as "digital junk" or "AI slop" has become a growing concern for platforms. Not only TikTok, but also Instagram, Facebook, and YouTube are facing the proliferation of AI-generated content that seeks to build an audience at a low cost. In the case of TikTok, the platform has already labeled more than 3.000 billion videos as AI-generated content, using a combination of content credentials, labeling tools for creators, and invisible watermarking technology.
The case of the fake videos featuring people with Down syndrome
One of the most worrying manifestations of this spam is the creation of videos showing supposed young people with Down syndrome crying or being insulted, in order to then sell supposedly handcrafted products. Investigations by Maldita.es and AFP have revealed that this content is generated by AI and follows the same pattern: it appeals to the viewer's compassion to get them to buy bags, flowerpots, or other items that can actually be found on platforms like Amazon or AliExpress at lower prices.
The accounts that post these videos often include links in their bios to online stores that lack mandatory legal sections, such as privacy or return policies. The National Cybersecurity Institute (INCIBE) warns that these websites don't always allow users to identify the owner, which is a red flag. Furthermore, the videos accumulate thousands of views and comments, allowing them to be monetized and go viral.
The regulatory context in Europe
While TikTok strengthens its systems, the European Union is preparing to implement the Artificial Intelligence Regulation (AI Act) starting December 2, 2026. This legislation will require the clear and visible labeling of any audiovisual content generated or substantially manipulated by AI, with penalties that can reach 3% of a company's global revenue or up to €35 million. Meanwhile, Spain has submitted a reform of the right to honor to Congress to curb the impersonation of individuals using deepfakes, excluding satirical or explicitly fictional uses.
Major platforms are opting for transparency through labeling instead of outright restrictions on AI-generated content. Instagram, TikTok, and YouTube plan to implement identification systems, but without offering users complete filtering options. Their strategy focuses on personalization: those who wish to avoid this type of content will be able to configure their preferences, while those who appreciate it will have unrestricted access.
The combination of technical measures, such as improved spam account detection, and regulatory advancements in Europe paints a picture where authenticity and trust become the primary assets for creators and platforms. TikTok has already taken concrete steps by joining the C2PA Steering Committee and investing over $4 million in AI literacy programs. However, the challenge remains significant: ensuring that users can distinguish between real and synthetic content in an ecosystem where AI produces increasingly compelling and inexpensive material.
