Could Google’s New Sign-Language AI Change Everyday Life for Deaf People?

Most hearing people probably do not realise how much effort Deaf people can put into an ordinary conversation.

At a family dinner, in a shop, at a hotel or in a taxi, we may have to type messages, pass a phone backwards and forwards, try to lip-read or ask somebody else to explain what was said.

Google’s new sign-language technology on the Pixel 11 made me think about how this could change.

A Deaf person can sign in American Sign Language (ASL) towards the phone’s camera. The AI then translates the message into written English.

It is not perfect, and it currently supports ASL rather than BSL, but I believe it could be an important step.

SLR and SL2T: what is the difference?

You may have seen two different terms.

SLR means Sign Language Recognition. This usually means recognising signs, handshapes or movements. Some systems may recognise one sign at a time or produce a list of recognised signs.

SL2T means Sign Language-to-Text. This goes further. It attempts to understand continuous signing and translate the full meaning into natural written language.

Google’s feature is SL2T.

This difference matters because sign languages are not spoken languages presented on the hands.

ASL has its own grammar, sentence structure, use of space, facial grammar and body movements. BSL does too, but it is a different language from ASL.

Google is trying to translate the complete ASL message into natural English rather than simply matching individual signs with English words.

How could a conversation work?

Google has connected SL2T to Gboard and Live Transcribe.

Gboard is the keyboard used across many Android apps. Instead of typing, an ASL user can sign towards the phone. The translated English can then be entered into a message, email, document, online search or another app.

Live Transcribe works in the other direction. A hearing person speaks, and the phone displays their words as text for the Deaf person to read.

A conversation could therefore work like this:

Deaf person signs in ASL → English text → audio for the hearing person

Hearing person speaks → Live Transcribe → text for the Deaf person

One phone could potentially support both sides of an everyday conversation.

The English text produced from ASL could also be read aloud through text-to-speech. This would mean that the hearing person would not always need to look at the screen.

Imagine this at a family dinner

This is the example that interests me most.

Imagine a Deaf ASL user sitting at a dinner table with hearing family members who do not know ASL.

The Deaf person signs naturally towards the phone. Their message appears in English and is read aloud. A family member replies, and Live Transcribe turns the spoken response into text.

The Deaf person would not need to type every message in English. They may also be able to follow more of the conversation without depending on one relative to keep explaining what everyone else is saying.

In the future, the translated conversation might be cast onto a television or another large screen so that everyone around the table can follow it.

That large-screen idea is not a confirmed Google feature. It is my own example of what might become possible if SL2T is eventually connected to more devices.

For some Deaf people, it could create an emotional feeling of finally being included in an ordinary family conversation.

Where else could it help?

I can imagine it being useful for simple situations such as:

• Ordering food

• Asking a quick question in a shop

• Checking into a hotel

• Communicating with a taxi driver

• Talking to a neighbour

• Having an informal conversation at work

• Writing an email or document through signing

• Communicating with relatives who do not know sign language

These are everyday situations with relatively low risk.

Some current sign-language AI systems are designed mainly for organisations. They may involve a commercial cost and often provide information in one direction, such as turning written information into an AI-generated sign-language video.

Google is approaching the problem from another direction. The Deaf person signs naturally, and the phone translates their message into English.

That gives the Deaf person more control.

Google says there is no additional charge for the feature, although a Pixel 11 is currently needed. More devices are expected later.

Could it help people learn sign language?

It also made me think about education.

In the future, Deaf and hearing people might use AI to practise sign language at home. They could sign towards a phone and see whether the system understands what they mean.

This might help people practise between lessons, learn at their own pace and build confidence.

Someone may not need to attend a class every week if they can do some of their practice independently. They could then meet a qualified Deaf tutor to check their progress and prepare for an assessment.

However, SL2T is currently a translation tool, not a complete sign-language teacher.

If the AI understands your message, that does not necessarily mean your signing is fully accurate, natural or appropriate for the situation.

A Deaf tutor can look at much more than the basic meaning, including:

• Facial grammar

• Signing space

• Regional variation

• Natural conversation

• Deaf culture

• Formal and informal language

• Interaction with other signers

Anybody wanting a recognised qualification would still need to meet the awarding organisation’s requirements and complete the correct assessment.

AI could be useful for additional practice, but it should work alongside Deaf teachers rather than replace them.

What could happen if Google releases an API?

At the moment, SL2T is available through Gboard and Live Transcribe on the Pixel 11.

Google has not publicly announced an SL2T API.

An API would allow approved developers to connect the technology to other apps, platforms and devices.

If this happens in the future, SL2T might become available through:

• Computers and tablets

• Video-meeting platforms

• Customer-service systems

• Workplace apps

• Smart televisions

• Smart-home devices

• Education platforms

• Public-information services

A Deaf person might be able to sign directly into a computer to write an email, create a document or communicate during a meeting.

Hearing people have been able to speak to phones, computers and smart-home devices for years. Sign-language input could begin to give Deaf people similar access.

But opening an API would also create new risks.

• Who would have access to the signing data?

• What would be stored?

• How would consent work?

• Could an employer or public service rely on the technology when it should provide an interpreter?

Google would need strong safeguards and continued involvement from Deaf communities.

Opening the technology to developers could create many opportunities, but it would need to be done carefully.

Would we still need interpreters?

Yes, absolutely.

Qualified sign-language interpreters remain essential when a conversation is complex, sensitive or high risk.

That includes:

• Healthcare

• Legal matters

• Education

• Employment meetings

• Workplace assessments

• Financial decisions

• Safeguarding

• Emergencies

• Important family discussions

  
  

An interpreter does much more than change one language into another. They understand context, culture, tone and the people involved.

AI cannot currently offer the same professional judgement or responsibility.

However, interpreters are a limited resource. If a Deaf person chooses to use technology for a simple conversation in a shop or at a family dinner,

interpreters could focus on situations where their professional skills are most needed.

The Deaf person should be able to choose what works best: SL2T, written English, an interpreter, a video relay service or another form of support.

AI should give us another option. It should not take existing options away.

A contradiction in the sign-language AI sector

There is something else that concerns me.

Some companies and organisations speak publicly about the future of sign-language AI, but their own podcasts, interviews, webinars and announcements still have no sign-language interpreter or signed version.

I find that disappointing.

Captions are important, but captions are not the same as sign language.

BSL and English have different grammar, syntax and ways of explaining information. ASL and English are also different languages.

Long English captions may not provide the same experience as receiving information directly in somebody’s first or preferred sign language.

If an organisation promotes sign-language accessibility, it should demonstrate that accessibility in its own communication.

It should not only discuss what AI may provide in the future while its current content remains inaccessible to sign-language users.

For an international organisation, this may involve International Sign, national sign languages or different accessible formats depending on its audience.

What are the limitations?

Google has been clear that SL2T can still make mistakes.

Problems may include:

• Rare signs

• Fast fingerspelling

• Names

• Specialist vocabulary

• Classifiers

• Tense and context

• Regional signs

• Complex signing

Facial expressions are also part of sign-language grammar. They are not simply emotions added to hand movements.

Google says the phone first converts the signer’s movements into pose landmarks. These are points representing parts of the face, hands and body.

The pose information is sent to Google’s server for translation, while the original camera video is discarded.

This may reduce some privacy concerns, but it does not answer every question.

People will still want to know what data is stored, how long it is kept, who can access it and whether it might be used for other purposes.

The technology should also be tested independently by a wide range of Deaf native signers, not only by Google or its selected partners.

It is not an emergency-alert system

It is important to be clear about this.

SL2T translates a Deaf person’s signing into text. It is not an emergency-alert system.

Live Transcribe may help a Deaf person understand nearby speech or a tannoy announcement, but the result will depend on the sound quality.

It may struggle with:

• Background noise

• Distance

• Echo

• Several people speaking

• Fast announcements

• Place names

• Numbers and times

• Poor-quality tannoy systems

An error involving a train platform, medication or emergency instruction could have serious consequences.

A railway, employer, hospital or public service must not expect a Deaf person to rely on their own phone.

Organisations must continue to provide visual alerts, accessible information, interpreters, reasonable adjustments and appropriate human support.

When will BSL be available?

This is the big question for me.

I am a Deaf native BSL user in the UK. I know some ASL, but BSL is my main language.

ASL and BSL are completely different languages, so the current feature would not understand my natural BSL.

Google says the underlying model was trained using more than 100,000 hours of data across more than 50 sign languages. It has also said that more languages and devices will follow.

However, there is currently no confirmed date for BSL, French Sign Language (LSF) or any African sign languages.

It is also important not to speak about “African Sign Language” as if there were only one. Africa has many different national, regional and local sign languages.

Each language will need suitable data, testing and direct involvement from its own Deaf community.

I hope future development also creates paid opportunities for Deaf signers, teachers, translators, researchers, linguists and accessibility specialists in different countries.

My final thoughts

I believe Google’s SL2T could become one of the most important developments in everyday Deaf communication.

For the first time, many Deaf ASL users may be able to sign naturally to an everyday phone and use that signing to write, search and communicate with people who do not know ASL.

It could help at home, in public places, in education and eventually across many different apps and devices.

But we should not treat it as perfect.

It is not a replacement for interpreters.

It is not a replacement for Deaf tutors.

It is not an emergency-alert system.

It is not an excuse for organisations to remove reasonable adjustments.

It is not a replacement for Deaf people’s rights.

It is another communication choice, and that choice should remain with the Deaf person.