Vizrt AI Keyer enhances the way viewers engage with broadcast content in most types of environment – simplifying real-time talent separation, background switching and AR insertion.
Vizrt AI Keyer helps broadcasters enhance the way viewers engage with their broadcast productions in most types of spaces – indoors or outdoors – and simplifies such techniques as real-time talent separation, background switching and AR insertion. Presenters can interact naturally with AR elements, for instance, and guests can be teleported direct to a virtual environment.
As an AI tool, the Keyer has been trained to recognise human shapes and motion in different scenarios from regular activities like walking and sitting to sports, presentations, news reports and so on. It then isolates the people in shots as foreground elements – they do not have to be shot separately against green screen. Later, these elements can be preserved, the background replaced and AR elements inserted into the foreground, mid-ground or background.
Automated Keying
The backgrounds can be replaced with a virtual scene or with photography from other shots. The user doesn’t need to spend time keying out the background – the Keyer performs that step automatically. This deliberate kind of scene construction can make messages clearer, more memorable and, where AR elements are incorporated, more interactive.
The proprietary Vizrt AI Keyer is trained on footage captured in very diverse environments and lighting conditions to improve the engine’s ability to recognise human shapes and movements. This means that, if offices, venues, lobbies, stadium sidelines or temporary locations need to be used as the virtual stage, the AI Keyer will still be able to separate the talent from the background. Note that user’s data remains protected and will not be used in new AI training.
Specialist cameras or crews are also unnecessary. Vizrt AI Keyer works with PTZ cameras, which can be set up so teams are able to deliver the required shots with less manual input.
Augmented Reality
Augmented reality elements can be placed into the indoor or outdoor scene to make data and imagery more relevant. When combined with Vizrt’s Reality Connect feature, which also runs on Viz AI, talent can move in front of, behind and around AR elements, as well as interact with 3D AR elements in a natural way.
Reality Connect uses AI algorithms to apply immersive techniques like casting shadows and reflections within virtual studios and XR sets.
“Reality Connect solves the problem of people appearing to float in a virtual studio by enabling realistic shadow casting and reflections,” said Gerhard Lang, CTO at Vizrt. “Reality Connect anchors the talent within the virtual studio through the use of virtual 3D models of the talent, which are continuously updated using Vizrt’s AI pose estimation. Pose estimation functions operate on the same GPU and the same live video feed used for keying.”
Applications
On-location news pieces are an example application for the AI Keyer, where a presenter brings studio-style visuals to outside broadcasts such as elections coverage. AR elements and backgrounds may be switched from wherever the story breaks to add depth to a story.
When interviewing sports personalities at events, users can quickly create shots with sponsor backdrops directly from the venue, without setting up physical boards. These are usually placed behind players being interviewed, and expand sponsorship value through virtual ad placement options.
Internal corporate communications also look more interesting when presenters can stand inside branded environments, either alongside key messages or interacting with AR visuals. Also, due to the deeper AR integration, existing studio infrastructure can be modernised or extended. www.vizrt.com
