DeepMind SL2T Natively Translates Sign Language on Pixel 11

Google DeepMind has integrated real-time sign language recognition into consumer hardware with the release of SL2T, a multilingual sign-language-to-text model. Debuting on the Pixel 11 smartphone series, the system initially supports American Sign Language (ASL) to English translation. The model operates as a core accessibility feature across Android, enabling dictation in Gboard and face-to-face translation in Live Transcribe.

By embedding the model directly into Gboard, users can sign into the front-facing camera to draft emails, search the web, or trigger OS-level Gemini actions.

Direct-to-Text Architecture

SL2T utilizes a direct-to-text architecture that omits intermediate glosses. Glosses are fixed text labels applied to individual signs, which systematically fail to capture the spatial grammar and non-manual markers inherent to sign languages. The direct-to-text approach maps continuous motion directly to natural language, allowing the model to interpret facial expressions and body posture as grammatical modifiers.

DeepMind trained the model on over 100,000 hours of video spanning 50 different sign languages, with ASL comprising 25 percent of the dataset. On the FLEURS-ASL benchmark, SL2T achieved a 70 BLEURT score without task-specific tuning.

Pipeline Stage	Processing Location	Data Handled
Optical Tracking	On-Device	Raw Video Feed
Pose Estimation	On-Device	130 Geometric Key Points
Semantic Translation	Google Servers	Coordinate Wireframes

On-Device Processing

Translation relies on a hybrid pipeline optimized for the Tensor G6 chip found in the Pixel 11, 11 Pro, 11 Pro XL, and 11 Pro Fold. To protect user privacy, the system does not process or transmit raw video to the cloud.

Instead, local AI execution handles the initial computer vision phase. An on-device MediaPipe Holistic model tracks 130 key points across the user’s hands, face, and body. The smartphone discards the raw video feed instantly, sending only the resulting geometric coordinate data to Google’s servers for the final text translation. The vision model is tuned for real-world constraints, maintaining tracking for one-handed signing while holding the device and fully supporting left-handed signers.

Current Technical Boundaries

A joint impact report published alongside the launch details several operational limits. The optical tracking degrades in harsh backlighting or low-light environments. The pose estimation follows only the largest subject in the frame, preventing it from transcribing simultaneous multi-signer conversations.

The model also operates with strict context constraints. Inputs are capped at 60 seconds of continuous signing, and the system lacks conversational memory across previous turns. Furthermore, SL2T was neither trained nor evaluated on signers under the age of 18, meaning accuracy for pediatric users is undocumented.

Early testers and the AI Sign Language Advisory Committee noted the speed advantage of signing over typing, though organizations like Deaf Connect expressed concerns regarding the preservation of deep cultural nuances and the risk of replacing human interpreters in critical settings.

If you build computer vision applications, SL2T demonstrates that highly sensitive inference tasks can safely scale to consumers by decoupling local spatial extraction from cloud-based semantic translation.