Google is bringing a new wave of upgrades to its translation service with machine learning (ML)-driven additions. This makes it easier for users to translate text that occurs in the real world, such as storefront signs, menus, documents, business cards, and other items. Even better, the new feature doesn’t simply overlay the translation on top of the original text but instead uses an AI-generated background to reconstruct pixels for a more natural-looking experience.
Cathy Edwards, vice president and general manager of Google Search, noted:
"In many cases, contextual combinations of words—such as background images—really give greater meaning to what the user sees. At this point you definitely don't want the important background that might appear in the image to be obscured by the translated text."
To make things even smoother, Google is also using a machine learning technique known as “generative adversarial networks” (aka GAN models).
Powered by this technology, the “Magic Eraser” feature intelligently erases unwanted objects from photos taken with a Pixel smartphone.
But now, Google is applying it to the translation feature as well. By incorporating translations into complex background images, it provides users with a more natural and seamless experience.
In the screen rendered by the software, you can view the translation as the original text, rather than adding another layer of occlusion to the current image. Speaking of which, some people may have thought of the company’s intentional new AR glasses.
"Google says more than 1 billion people use the company's translation service every month, involving text and images in more than 100 languages. Starting this year, Google also began evaluating its AR prototype devices in public spaces with a handful of employees and trusted testers."
Sadly, the company’s first foray into consumer smartglasses with Google Glass products turned out to be short-lived.
Lastly, Google didn’t share long-term plans for its translation feature, saying only that the feature will be available later this year.
