Although Android 10 is out, a couple of features aren't ready yet, and it looks like Google is going to make both of these exclusive to the Pixel 4 family when that launches later this fall. At least at first. These might eventually roll out to older Pixels, but don't hold your breath because both seem to require a lot of on-device processing power, which older models may simply not have.
The Google Assistant made its debut on the original Pixel phones in 2016, and since then it's grown into the best virtual assistant there is. On its third anniversary, a massive update is coming, but we have to warn you, Google said this is launching on the Pixel 4 phones first. So like the original rollout of the Assistant, it might be limited to the upcoming Pixels for a while, until the company is ready for a rollout across all Android devices.
The big news is that the Assistant will be fully contained on your phone, so it will no longer send your queries to the Google servers to interpret your speech. The company has managed to develop completely new speech recognition and language understanding models that don't need the cloud to function anymore because they take up as little as half a gigabyte of space. Hence, these will live on your handset, replacing the 100GB models that the company has so far been using in the cloud.
This means the new Assistant will process requests with near-zero latency since there's no waiting around for the round-trip the data makes to the cloud and back. So its answers should feel much more like they are part of a natural conversation - Google says it will reply up to ten times faster than before. And of course, since it's all on-device, it will understand your query even when there's no internet connection - although the lack of connectivity will still prevent it from showing you results on the web.
Live Caption was announced on stage at Google I/O, and its name is very descriptive. Thanks to the new on-device machine learning capability and audio playback capture API, you'll get support for live captions of any media right on your phone, with no internet connection required.
It will automatically caption media playing on your device - videos, podcasts, audio messages, you name it - across any app. It even works with stuff you record yourself. As soon as speech is detected, captions will just automagically appear. The captions and audio never leave your phone. This is going to be helpful for the deaf and hard of hearing people, 466 million across the world, according to Google's numbers. But it will also help those who watch video without sound out of privacy reasons.