Speakers
Synopsis
The widespread use of mobile services has significantly impacted many aspects of life, including entertainment, communication, finance, and shopping, with children increasingly using smart devices and apps. Surveys from 2013 and 2019 show that over 75% of children under 8 and 69% of teens owned smartphones, raising concerns about the appropriateness of the content they access. Content rating labels, regulated by bodies such as the Entertainment Software Rating Board (ESRB) in North America, Pan-European Game Information (PEGI) in Europe, and the Australian Classification Board (ACB), are critical for guiding age-appropriate usage. However, in mobile app ecosystems like the Google Play Store, where millions of apps are available and constantly updated, the process of assigning these ratings remains largely self-regulated by developers. Efforts by consumer protection agencies, privacy advocates, and regulators to identify content compliance violations are often hampered by the sheer scale of the app market. Many violations are only uncovered after customer complaints or lengthy investigations, by which time millions of users may have already been affected.
Deploying an automated method to determine content rating labels based on mobile app metadata—such as app descriptions, developer information, app icons, and screenshots—represents a significant advancement in cybersecurity to identify potential compliance issues and enhance the safety of apps, particularly those designed for children. Existing research in automatic maturity labelling typically relies on unimodal data, focusing on either textual or visual metadata. Compliance checks are often limited to examining differences across app stores or geographic regions. However, recent advancements in artificial intelligence, particularly with multimodal transformer networks like OpenAI’s CLIP, make automated tools for content rating more promising and effective.
However, adapting multimodal frameworks to mobile app data presents several unique challenges. Unlike traditional media, such as Instagram feeds where text and images are typically aligned in the context and the meaning, mobile app descriptions, icons, and screenshots often lack this correlation. Additionally, determining the correct content rating requires not only identifying the content but also analysing the style of app visuals, as we have observed that cartoonish themes are often exploited to attract children to mature content.
In this talk, we will present our findings and discuss the challenges of using multimodal networks to predict content rating labels based on textual and visual metadata from app information pages. Our research highlights two concerning trends: developers undermining the content rating to misrepresent the actual maturity level of the app, and deceptive practices aimed at misleading users into downloading apps unsuitable for their age group. Additionally, our quantitative findings reveal that approximately 35% of the apps flagged by our classifier for violating content rating guidelines were subsequently removed from the Play Store. Furthermore, we emphasise the importance of such automated methods in supporting e-safety regulators and policymakers to identify these malpractices at scale.
