Samsung Science & Technology Center

First, we will discuss general challenging issues of the meta-vision systems such as ultra-high resolution (higher than 8K x 8K) micro-displays with variable frame rate driving schemes up to 480 Hz, ultra-high-speed wireless and wired communication, and compact optical system with various sensors. We developed the 1.2-in diagonal size, 4K X 4K resolution, and 120 frame rate OLEDoS micro-display [1, 2]. However, the meta-vision micro-display devices require less than a 1.6-inch diagonal backplane size, higher than 8K X 8K resolution, and a frame rate of up to 480 Hz. Therefore, there is no solution to achieve the specification of a meta-vision micro-display. In this presentation, we want to discuss the current practical limitations and the pathway to the challenging requirements of OLEDoS micro-display systems. We will also discuss video data reduction schemes' foveated rendering technology [3].

The prerequisite for opening the metaverse era is convenient and easy accessibility of the VR/AR micro-display devices in commercial market. LCoS and OLEDoS microdisplays are available, but not popular yet. The major reasons may be due to the uncomfortable image qualities as well as high street price. The major tech companies are building up the technologic connections between the real world and the metaverse. LEDoS is also one of the candidates for the meta-vision system application. Its wide color gamut, high reliability, high efficacy and flexibility in form factor are favorable to immersive display over other display devices. In this talk, the state-of-theart technology with technical challenges of a LEDoS micro-display will be presented and its roadmap will be discussed for applying it to the meta-vision system. Attentions are put on the monolithic approach [1] and wafer-scaled integration for high resolution [2]. Also, cost effective transfer processing will be introduced [3].

The holographic meta-vision display creates three-dimensional visual images by generating an optical wave field distribution in free space [1, 2]. Since the human eye is highly dynamic, with vigorous and active binocular movement in real-time, the holographic image light field distribution must be adaptively generated to match the constantly changing visual perception system. Therefore, a highly advanced concept of "complete dynamic control or synthesis of image light field" has been introduced, with holographic three-dimensional displays being actively researched as a candidate. The key technology for this kind of display, from a wave optic point of view, is complex spatial light modulation, which involves independent modulation of amplitude and phase of the light field in free space [3]. Essential factors for evaluating the performance of full color holographic threedimensional displays [4] include field of view, dynamic range of amplitude and phase modulation, full color modulation, resolution, noise reduction, and high framerate. In particular, the field of view is limited by the pixel size, with wavelength-scale pixels being required to provide binocular holographic vision. Unfortunately, the physical limitations make it extremely difficult to fabricate spatial light modulators at the wavelength scale, and even more challenging to achieve complete complex light modulation on such small scale display pixels. In this presentation, current ideas about complex spatial light modulation in the context of full-color holographic meta-vision displays are being examined, and future directions for research will be discussed.

Holographic displays have been considered as an ultimate type of display for neareye display (NED), particularly in augmented/virtual (AR/VR) reality, by utilizing electromagnetic wave properties of light. However, most of the holographic displays rely on spatial light modulators (SLMs) to achieve holographic video display, which requires a large computation power and has a large form factor to achieve highresolution realistic display performance. Recent advances in nanofabrication have made it possible to develop metasurface-based flat optics for high-resolution and large field-of-view hologram display in a compact form factor. However, there are trade-off of shortage of information capacity and tunability. To this end, orbital angular momentum of light, which has a high degree of freedom, is used as a virtual information carrier in two image planes by manipulating both amplitude and phase of light [1], which has the potential to be extended to real-3D display without vergenceaccommodation-conflict issues. Another, challenge that must be addressed for the metasurface to be used is tunability. Here, polarization-multiplexed metasurfaces are integrated with specifically designed liquid crystals to modulate the polarization of light in response to various stimuli to achieve virtual image formation over multiple depths [2, 3]. The reconstruction of depth information could be achieved using an active stereo system that utilizes dot-projecting metasurfaces [4]. At the end of the talk, discussion on low-loss materials within the ultraviolet to visible to near-infrared wavelength range [5, 6, 7, 8], as well as the development of large-area and highthroughput fabrication methods [9, 10] for metasurfaces will be provided.

The human eye is a remarkable product of evolution, possessing high resolution, a wide field of view, and the ability to focus [1]. However, implementing a metavision system that provides a natural scene to our eyes is a challenging task. The human visual field is approximately 120 degrees, and while commercial VR devices offer a viewing angle of up to 100 degrees, most of these devices fail to provide the necessary optical resolution. To implement an optical system with both a wide viewing angle and long working distance, multiple lenses are typically required to correct for chromatic and geometric aberrations. For instance, smartphones, microscopes, and short-throw projectors usually employ 5 to 12 lenses. However, the eyepieces of most VR optical systems are composed of only 2 to 3 elements, mainly due to the large aperture size and limited track length, as well as the imprecision of the human eye. In metavision, we unconsciously compare virtual images to actual vision and perceive any optical aberrations as significant. Although it is often stated that the resolution of the eye is approximately 1/60 degree, this number is based on the high photoreceptor density in the fovea, which is relatively small with a diameter of only around 2 mm [2, 3]. The photoreceptor density decreases rapidly towards the periphery of the visual field. Display manufacturers typically aim to achieve the maximum possible visual cell density in their displays, as it is difficult to match the position of the display to the rotations of the eye. One of the most intriguing challenges in metavision is the ability to control focus, which requires a way to visualize 3D information on a 2D retina. Traditionally, two approaches have been used to express 3D information: stacking information in a depth direction (volumetric display technology) or providing different information based on direction (multi-view technology). However, the most ideal method is to establish a direct optical connection between the foveated rendering information and the photoreceptors in the eye.

Optical see-through is a main element of mixed-reality metaverse AR/MR vision technology. applications. In particular, although various optical methods have been developed to enable optical AR/MR with wide field of view and large eyebox, limited monocular focus cues and lack of mutual occlusion are still challenging issues of AR/MR. Limited monocular focus cue hinders the depth perception mechanism of the human visual system, prohibiting long-time use and optical matching of virtual images to real objects. Lack of mutual occlusion significantly degrades the reality of the displayed real-virtual mixed world by only presenting translucent virtual images without proper occlusion of the real world. It also forces the AR/MR meta-vision system to compete with bright outdoor light, increasing the required luminance of the display devices well above the practically reasonable level. In this talk, I will discuss these two challenging issues. First, I will introduce possible approaches to present optical principles of monocular focus cues to users. Maxwellian [1], multi-focal [2], light field [3], and holographic techniques will be discussed with their principle, recent advancement, and remaining challenges. Next, the mutual occlusion will be covered. Techniques that enable the occlusion of real-world view by the displayed virtual images will be introduced [4]. Recent advancement that extends the conventional 2D occlusion to 3D will also be discussed along with their current limitations. Lastly, I will conclude my talk with an outlook on the realization of an ultimate metaverse vision technology that can be adopted in our everyday life.

The realization of immersive metaverse requires an optical system with a large degrees of freedom to deceive the human vision system. In contrast to conventional 2-D display systems, optical systems for metaverse visual experience necessitates 3-D graphic rendering in various illumination conditions. In this regard, the conventional imaging and display paradigm, relying on the 1-to-1 correspondence between an target domain and a measurement (or modulation) domain, often fails to address large degrees of freedom or results in bulky optical systems that are inadequate for wearable form-factor.

One of the fundamental reasons of these difficulty could be found in the traditional optical components used for conventional imaging and display systems - lens, grating, and prism - where the design paradigm has been dictated by a set of basic principles of refraction and diffraction. This talk will explore a novel way of using designed complex nanostructures to unlock an optical space that is inaccessible using conventional optics. In particular, I will explain how the optical complexity can be designed and implemented using a metasurface platform and how wavefront shaping through the engineered complexity enables control over an unprecedentedly large number of spatial input-output channels (>10¹³; 5 orders of magnitude larger than the previous record) [1]. I will also introduce the characteristic and the usability of a stacked disordered metasurface platform in spatio-spectral domain. Beyond that, I will also discuss the development at the junction of engineered disorder and computation that can be potentially used for solving high computational cost issue in 3-D rendering [2]. Lastly, I will conclude with the prospect of brain-computer interface for futuristic metaverse implementation

The human finger is the most sensitive tactile sensor, and the engineering realization of human tactile perception mechanism is a key technology of the future virtual reality world in which the tactile sense is synchronized. Human skin can detect physical stimuli including pressure and vibration sensitively and selectively through different types of cutaneous mechanoreceptors [1]. In biological sensory systems, tactile stimuli are converted to action potentials through somatosensory transduction and the resulting signals are transmitted via afferent nerves. While slow adaptive (SA) mechanoreceptors generate voltage spikes in different rates and patterns depending on the strength of the sustained stimulus such as static pressure, fast adaptive (FA) mechanoreceptors generate most voltage spikes at initial contact and final release of physical stimuli due to their vibration-sensitive characteristics. These signals are delivered separately, and then integrated for multi-level information on tactile perception in the brain. There have been attempts to mimic the complex human tactile perception systems and implement them engineeringly [2, 3], however, constructing such systems with human-like capabilities remains challenging. In this talk, I will introduce a method for designing a tactile sensor that mimics the function of human tactile mechanoreceptors and discuss a core idea for mechanically or electrically stimulating the skin to deliver realistic tactile information to the central nervous system. Finally, I will suggest that authoring tactile signals for each object is essential for implementing tactile interfaces in electronic communication devices.

In recent times, there has been active development of metaverse technologies owing to their potential to revolutionize the way we experience and interact with our surroundings. The key technologies required for building a metaverse include high-speed communication, big data processing, high-quality graphics, artificial intelligence, haptics, and sensory feedback. Among these, haptics and sensory feedback technologies play a crucial role in creating a more immersive experience. These technologies provide realistic tactile, olfactory, and gustatory sensations to the users, thus enhancing their immersion in the metaverse. To achieve advanced haptics and sensory feedback, accurate and reliable sensor technologies are required to detect physical and chemical parameters of the target objects. Micro/nano-technologies based on functional nanomaterials and micro/nano-fabricated structures have enabled significant advancements in these sensor technologies. In this talk, I will discuss recent advances in micro/nano-enabled sensor technologies and their potential applications in the metaverse. Firstly, I will provide an overview of state-of-the-art micro/nano-enabled sensor technologies for metaverse applications. Secondly, some of our research group’s achievements in the micro/nano-enabled sensor technologies will be explained, including (1) sensitive and reliable strain sensors based on piezo-transmittance mechanisms of mechanical metamaterials and optical sensing for accurate measurement of human motion and object deformation towards advanced motion sensing in advanced healthcare/entertainment IoT and metaverse systems [1, 2]; (2) ultra-sensitive pressure sensors based on micro/nano-hierarchical structures for large-area, multi-point pressure monitoring towards advanced tactile sensing in metaverse systems [3-5]; (3) high-performance, multiplexed electronic nose (e-nose) systems featuring nanomaterial-based, ultra-low power gas sensor array, neuromorphic sensing mechanism, and deep learning algorithms towards advanced environmental/healthcare IoT and realistic olfactory sensation in metaverse systems [6-8]. Lastly, I will summarize my talk with an outlook and my personal perspective on the advanced and multimodal sensor technologies for the realization of ultra-high performance IoT and truly immersive metaverse experience in the future.

6G wireless systems are expected to be deployed by 2030. Among popular applications for 6G, virtual reality (VR) and augmented reality (AR) are two technologies that promise to transform many aspects of our lives, from entertainment and education to work and health. However, to deliver immersive and interactive experiences, VR and AR applications require high bandwidth, low latency and reliable wireless connectivity. To meet demanding requirements, several techniques have been developed including massive multi-input multi-output (MIMO) transmission and mobile edge computing, etc. Also sematic communication has been introduced which works based on the semantic-meaning passing concept. The core of semantic communication is to extract the meanings of sent information at a transmitter with the help of a knowledge base between a transmitter and a receiver and thus it enables task-oriented communication instead of traditional message oriented communication. In this talk, we will explore how such new wireless transmission technologies can realize various applications of next generation communication systems by acheiving perfect connectivity, high performance distributed computing and ultra low latency. Discussions on some of the examples and trends will be presented in this emerging field of extended reality (XR). We will conclude the talk by providing open problems and research directions for realizing future high speed communication systems.

The rapid advancement of high-performance computing and networking systems has led to an increased demand for high-bandwidth connectivity between electronic components. To meet this demand, current technology trends suggest that more than 100Gb/s per-channel bandwidth will be necessary for data-center networking and high-performance computing applications. This presentation aims to provide insights into the development of high-speed wireline connectivity by introducing research projections and key enabler techniques for achieving 100+Gb/s data rates. The presentation will first focus on the introduction of high-order pulse-amplitude modulation (PAM) and its associated signal-processing techniques. The utilization of PAM signaling allows for higher channel capacities compared to traditional binary signaling, as it accommodates more bits per symbol [1]. However, high-order PAMs are susceptible to a high level of inter-symbol interference (ISI), and thus additional signal-processing techniques, such as Transition-Limited PAM (TL-PAM) [2] and Supply-Noise-Reduced PAM (SNRPAM), are required to overcome the BER penalty, which will be presented in this talk with their advantages and challenges. After that, the critical role of circuit design innovations in achieving ultra-high bandwidth data transmission will be covered. Topics include transmitter circuit-level techniques such as protective cascade and high-speed serialization in advanced CMOS technology [3], as well as recent advancements in high-speed converters [4] and analog front-ends for multi-GHz operation on the receive side.

The wireless industry is continuously striving to meet the growing demand for ultrahigh-speed, global coverage, and extremely-low latency data transmission, catering to a wide range of applications, such as virtual reality, metaverse, and holoportation. To fulfill these demands, cutting-edge wireless technologies beyond the Shannon theory, which was developed under unrealistic assumptions, are necessary. For example, the channel capacity with finite block length, a recently developed post-Shannon theory, envisions the maximum data rate for short packet transmission. Another example is goal-oriented communication. This approach changes the traditional view of communication defined by Shannon into a message containing minimum information to perform a required task reliably. The Post-Shannon theory can be found in multiple-input multiple-output (MIMO) communications. Thomas Marzetta, the inventor of massive MIMO, conjectured that channel state information (CSI) feedback is necessary to achieve substantial massive MIMO gain in frequency division duplexing (FDD). Recently, we found that FDD massive MIMO gain is attainable even with no CSI feedback by harnessing geometrical channel reciprocity in uplink and downlink. In this talk, I will introduce a set of new wireless transmission technologies for the post-Shannon theory, which includes full-duplexing [1], low-latency channel codes [2], zero-feedback multi-antenna communications [3], terahertz multi-antenna communications [4], information-theoretical approach to user positioning [5], goaloriented source compression, communication-efficient distributed learning [6], and satellite-integrated cellular networks [7]. I will elaborate on their potential to meet the requirements for ultra-high-speed, global coverage, and extremely-low latency data transmission and discuss potential research directions for each of these technologies, providing insights into the future of wireless communication.

Metaverse is expected to evolve in three phases: a virtual world that allows P1) interaction of 3D-rendered objects in pseudo real-time, P2) interaction of photorealistic objects in pseudo real-time, and P3) interaction of photo-realistic (and hyperrealistic) objects in real-time. The current implementation of metaverses such as Roblox, Zepeto, and Second Life can be classified as a crude version of P1 metaverse. Realizing P3 metaverse that can be accessed from anywhere anytime is a challenging research topic in communications networking, especially under cellular networks. The fundamental question behind this challenge is how to guarantee the delivery of photo- or hyper-realistic object information, which is sizable and dynamic in size, over the mobile network within the interaction time requirement. Unfortunately, this question is unanswerable with the current architecture of cellular networks including 5G, which we regard as cellular network 1.0. In cellular network 1.0, increasing the channel bandwidth and the number of antennas would improve the probability of intime delivery, but guaranteeing the delivery is not achievable due to its infeasibility of compensating the uncertainties in the object size and the channel quality. In this talk, to give a new perspective to the challenge of realizing the P3 metaverse, we introduce the concept of cellular network 2.0 [1] and discuss its essential component technologies such as latency-guaranteed congestion control [2] and performanceguaranteed computing [3][4].