Microsoft taps AI to make you look your best in video meetings

If you’ve ever been scared to look away for a second at your notes while in a Teams meeting, worry no longer. Microsoft is making AI-driven eye contact and a number of other productivity features part of the Windows platform, the company said in a press briefing accompanying its Future of Hybrid Work presentation on Tuesday.

The new features represent some nifty uses of artificial intelligence, on a PC platform that has struggled somewhat to integrate them. On smartphones, AI is generally used to enhance photo-taking and minimize background noise. Microsoft is adapting these features to Windows 11 PCs, bringing improved background blur alongside a feature called voice clarity. Two other new features use machine learning more directly: an AI-driven Eye Contact feature, and an automatic framing of speakers as they move around.

To be clear, Microsoft has not said exactly when these features will roll out. The company also did not say whether these new features will be specific to Teams or system-wide, and didn’t divulge whether they’ll be a paid feature of Microsoft 365 or not.

Microsoft has signaled before that such features were coming. In July 2020, Microsoft rolled out the Eye Contact technology as one of the selling points of the Surface Pro X. Moreover, Microsoft made it part and parcel of Windows; the company claimed that the technology would work with any video chat app. (Several customers vociferously disagreed, claiming the technology didn’t really work even after the device was updated.) Microsoft also offers noise-suppression options within Teams, where users can tell the app to eliminate all noise besides speech, or just let Teams decides what sounds should be cut out and which should be left in place.

Teams, like other videoconferencing apps, now allows virtual backgrounds, which uses AI to sense the speaker and “replace” what the camera sees with a mix of animated or static backgrounds. Zoom became the app of the pandemic with its easy videoconferencing and funny virtual backgrounds, but Microsoft also implemented virtual backgrounds in 2019. Now, Microsoft is implementing an improved “portrait background mode,” where it will look like your background was naturally blurred, rather than artificially.

Eye Contact works somewhat in the same manner. Portrait mode “looks” for your silhouette. Eye Contact looks at your eyes. The difference here is that Eye Contact will actually “replace” your eyes. As you move your gaze around the screen, Eye Contact is supposed to replace your eyes so that they appear to be staring directly at the screen. Given the trouble users have had with it before, we don’t know whether the technology will actually work, or whether it will work so well as to be creepy.

Automatic framing applies the same techniques slightly differently. Dell’s $200 WB7022 4K UltraSharp Webcam performs automatic framing to center your face, and now Windows will use the same approach. “[This] basically means that you will [be] right in the center of focus, even as you move around or you stand up,” said Wangui McKelvey, general manager of Microsoft 365.

Users have already complained that Microsoft Teams performs automatic framing, but only for the host, and that it can be distracting. Again, we’ll have to see if this ability is configurable in some way. Still, CPU suppliers like AMD, Intel, and Qualcomm have talked about implementing AI on the PC for years now. It’s nice to see technologies starting to take advantage.