Meta's AI-Powered Stickers Stir Controversy Early On
Meta's latest foray into the realm of AI-driven features seems to have hit a stumbling block, raising concerns about the company's ability to ensure responsible AI usage. In September, Meta announced its intention to introduce more AI features on its platforms, but the newly unveiled sticker generation tool, powered by an algorithm called Emu (short for "expressive media universe"), has encountered issues in monitoring user-generated content.
The concept behind Emu is simple: users on Meta platforms like Instagram, Facebook Stories, WhatsApp, and Messenger can input a phrase or word, and the algorithm generates a selection of stickers to be used in conversations.
However, the implementation of content safeguards appears to be inconsistent, resembling more of a sieve than a protective barrier. Some controversial phrases are blocked, while their synonyms are inexplicably allowed. Although this AI stickers feature is only accessible to a select group of users on a limited basis, it enables individuals to create AI-generated sticker images from text-based descriptions in Facebook Messenger and Instagram Messenger. These stickers are then shared in chats, akin to emojis.
Meta employs its Emu image synthesis model to create these stickers and has integrated filters to identify and block potentially offensive content. Nevertheless, numerous unconventional combinations seem to evade these safeguards.
While the phenomenon of users attempting to bypass content filters in AI-generated imagery is not entirely new, with open-source image models enabling such experiments for over a year, Meta's decision to publicly release a model that can create such content within flagship apps like Instagram and Messenger raises eyebrows.
Similar challenges have emerged with OpenAI's DALL-E 3, as users have tested its limits by creating images featuring real people or containing violent elements. The difficulty lies in trying to anticipate and prevent all potentially harmful or culturally offensive content when an image generator can produce virtually any combination of objects, scenarios, or individuals. This predicament underscores the ongoing challenges faced by moderation teams in the evolving landscape of AI-powered apps and online spaces.
