<![CDATA[Microsoft Files Patent to Rewrite What Speakers Say in Live Video, Lips Included]]>The claims describe a system that watches a live video feed and its audio track while people are speaking. An AI agent (software that makes decisions on its own) listens to the words and picks out the ones to change, based on a set of rules and on a profile of the user.

Then a generative AI system (AI that creates new text, audio or images) does two jobs in 'near-real-time,' which the filing defines as within a few seconds, roughly 1 to 10:

  • It replaces the chosen words with new ones, in both the audio and the video.

  • It reworks the audio and video just before and after the swap, so the join is smooth.

The filing lists what could get swapped: filler words like 'um,' slang, mumbled or mispronounced words, offensive words, and background noise. Replacement words can come from an internet search, and the filing also floats instant fact-checking.

A second independent claim covers translation. Words in a language missing from your profile are swapped for translated words, and the video is redrawn, lips included, to match. It could run on AR glasses, a phone, a tablet or a video camera, using the chips on the device. One dependent claim keeps the original recording as 'ground truth.'

Link Preview Image
Microsoft Files Patent to Rewrite What Speakers Say in Live Video, Lips Included

Microsoft's new patent would rewrite a speaker's words in live video, lips and all, cutting 'ums' or translating a foreign language for you.

favicon

Patentlyze (patentlyze.com)
]]>https://citiverse.it/topic/992e9a38-ff8b-4653-bc11-f64d78e15c82/microsoft-files-patent-to-rewrite-what-speakers-say-in-live-video-lips-includedRSS for NodeWed, 11 Nov 2026 04:00:23 GMTMon, 09 Nov 2026 10:15:54 GMT60<![CDATA[Reply to Microsoft Files Patent to Rewrite What Speakers Say in Live Video, Lips Included on Mon, 09 Nov 2026 11:32:07 GMT]]>And, of course, at what cost?

]]>https://citiverse.it/post/https://lemmy.ml/comment/28892263https://citiverse.it/post/https://lemmy.ml/comment/28892263Mon, 09 Nov 2026 11:32:07 GMT<![CDATA[Reply to Microsoft Files Patent to Rewrite What Speakers Say in Live Video, Lips Included on Mon, 09 Nov 2026 11:01:16 GMT]]>Humanity just can't help but fuck itself, can it?

]]>https://citiverse.it/post/https://lemmy.today/comment/27778781https://citiverse.it/post/https://lemmy.today/comment/27778781Mon, 09 Nov 2026 11:01:16 GMT<![CDATA[Reply to Microsoft Files Patent to Rewrite What Speakers Say in Live Video, Lips Included on Mon, 09 Nov 2026 10:37:22 GMT]]>To be employed by the Ministry of Truth.

]]>https://citiverse.it/post/https://sopuli.xyz/comment/26726082https://citiverse.it/post/https://sopuli.xyz/comment/26726082Mon, 09 Nov 2026 10:37:22 GMT<![CDATA[Reply to Microsoft Files Patent to Rewrite What Speakers Say in Live Video, Lips Included on Mon, 09 Nov 2026 10:16:03 GMT]]>Why would the CCP do this?

]]>https://citiverse.it/post/https://lemmy.ml/comment/28890809https://citiverse.it/post/https://lemmy.ml/comment/28890809Mon, 09 Nov 2026 10:16:03 GMT