LED virtual shooting technology integrates real-time digital image, real-time light and shadow rendering, real-time camera motion tracking and real-time computing and so on. Through XR + LED, stay at home, can also shoot a large film, can significantly reduce the time and economic costs of film and television production.

01

what is XR XR (Extended Reality) Extended Reality is a term used to describe the fusion or interaction of virtual and real elements. It is commonly referred to as AR (augmented Reality)/VR (virtual Reality)/(mixed Reality) , by means of the technology of camera tracking and real-time image rendering, the display medium (LED/projection bowl screen) and the virtual scene outside the display medium can track the camera's visual angle in real time, and the real-time picture is synthesized in front of the camera lens, to create an infinite sense of space technology.

02

XR fundamentals understanding perspective is an important prerequisite for understanding XR technology. The bridge between virtual reality and reality is based on the perspective principle. In daily life, the person P in front of the tourist attractions of synthetic photos, some can be false, some can be seen at a glance fake. One of the things that makes a composite look realistic is the spatial relationship between the person and the background. That is, perspective. All objects in a composite must have a common perspective in space. If the background in the picture is taken from an overhead angle, then the person who needs to be composited should never be viewed from above. Otherwise, it looks like the picture will feel incongruous, there is an unreal feeling. All the objects in the picture must have a unified perspective point in space, otherwise the picture will produce unreal feeling this is in the two-dimensional plane, confusion between virtual and real practice. So how do you realistically mix virtual and real in a 3D?

Hong Kong 2021 Andy Lau performs with CCTV Spring Festival gala performers thousands of miles away using XR technology if you want to synthesize a video, the actor is indoors, but behind the camera is a forest, is there any alternative to using the traditional Green Screen + matting method? If the camera is fixed and in the right position, you can even hang a huge canvas with a forest pattern directly behind the actors, as long as the actors and the background planes show the right perspective under the camera, to meet some of the most basic needs. But once the camera moves, the fact that the background forest is a flat canvas can not be concealed. In fact, the realistic feel of the scene background is precisely shaped and enhanced as the camera moves. In each frame of the shot, such as the panorama when looking down, the mid-shot when following-the actor's back should present a background scene in this position, this angle, to meet the real perspective relationship. It would be nice if the background behind the actor could be synchronized with the camera as the camera rotates or changes angle. And that's what XR technology is all about. The reality of the video scene background is created and enhanced as the camera moves

03

XR technology and equipment conditions to understand the concept of XR and the principle of the foundation, in the actual application, how to achieve it? At present, the technology and equipment needed to implement the mainstream XR technology mainly include LED display, LED control system, virtual rendering system, camera and tracking system, synchronization system and other auxiliary systems. With the above technical equipment. Through real-time workflow, performers can be integrated into the virtual world without green screen and post-production, and real-time output of the final screen. The overall technical framework is as follows: XR scene technical framework consists of LED display composed of the shooting site, camera, object location tracking and positioning technology, real-time graphics rendering engine (AR) , high-speed low-latency network and other elements

Among them:

● LED control system

 mainly undertake to receive the virtual rendering screen, and output to the LED display, an excellent control system must possess excellent video processing ability, can carry on the accurate color management and brightness adjustment to the LED screen body, and display the LED screen body to the greatest extent, and in the hardware side of the XR special application needs (such as multi-station, high frame rate, etc.) to provide technical support.

● the virtual rendering system

mainly includes media server and real-time rendering engine. The master media server mainly controls the rendering engine, processes the camera data, and outputs the LED video source images. Real-time rendering engine provides the integration and production of virtual digital assets, building a complete virtual expansion space.

● The camera system

takes real images and sends them back to the virtual rendering system, requiring the use of professional-grade video cameras in order to achieve clear and non-destructive real image acquisition. A camera with a frame rate of 120 Hz or more will be used. : : The Tracking and positioning system is mainly used to locate and realize real camera displacement and camera rotation data. The key indicators of the positioning system are accuracy, degree of freedom and low delay, more commonly used are mechanical tracking (head, ARM, etc.) , optical positioning, as well as other infrared tracking, visual camera tracking.

● The synchronization system

mainly used for the reference time and frame of the overall workflow of XR. An independent and accurate synchronization system is necessary for the XR project, through the synchronization system can effectively reduce the camera in the LED screen when shooting the scanning line, black field and other problems, can also control the synchronization time with the LED control system to achieve multi-camera shooting effect. XR scene technology actual operation flow XR technology is an emerging technology which develops rapidly in recent years. Compared with the traditional green screen + matting virtual shooting method, XR + LED display has significant advantages. In the future, driven by the rapid iteration of technology and demand innovation, the emerging XR technology will welcome its own vast world.