The ray-space method represents the "visual information" of 3D spaces as information on rays of light propagating through space, and is represented by four parameters: position (x, y) and direction (θ, ϕ) (Figure 1). Before this concept was proposed, 3D images were discussed based on multi-view images taken by many cameras, and those with different camera parameters, CG, holography, etc. were discussed independently. In 1994, the awardee et al. proposed the ray-space method[1], and various types of 3D images can now be discussed in a unified manner within the framework of ray-space[2]. The theory states that images from various viewpoints are merely subspaces of the ray space, and that the generation of images from any viewpoint can be realized by extracting these subspaces. This theory has been extended to camera images taken from 360 degrees surrounding an object and used as a unified format for various 3D images (Figure 1)[5]. The concept of light field presented at SIGGRAPH in August 1996 was shown to be equivalent to the concept of the ray-space method, and this concept is also called light field. Various studies have been conducted around this light field, some of which are listed below.

(1) Research on 3D video communication, information compression and coding

Figure 2 shows a conceptual diagram of 3D visual communication based on the ray-space method[3]. 3D visual communication is realized through a series of processes such as acquisition, compression/encoding, transmission, novel view synthesis[4], and display centered on the ray-space data format. The most important issue is the huge amount of data, and research on the compression and encoding of light ray information has been recognized as an important issue since the early days. In recent years, the awardee has studied ray-space compression using an auto-encoder with deep learning[6] and has achieved compression of several hundred multi-view images down to a data volume of a few images. The awardee has also actively made proposals to MPEG (Moving Picture Experts Group), an international standardization organization, and has contributed to the establishment of an international standard for MVC (multiview video coding), which has had a significant industrial effect.

(2) Research on the acquisition and display of ray space

The problem of acquiring and displaying ray space has been addressed since the early 2000's. For acquisition, the awardee has constructed a multi-camera system and a time-division ray-space imaging system and has succeeded in acquiring moving images of 100 views at 30[fps]. In addition, the awardee has studied a coded aperture camera that reconstructs the original ray space by computational processing based on 2D signals acquired by controlling the optical aperture[7]. For display, the awardee developed a prototype of MIT-type layered 3D display (tensor display), and by speeding up the computation of layer patterns, the awardee has succeeded in performing the full-chain process from acquisition to display.

(3) Research on 3D imaging systems

The awardee has been studying a series of systems from acquisition to the display of ray information. For free viewpoint image acquisition, the awardee has developed a multi-view image acquisition system consisting of 100 cameras and has also developed a free-viewpoint video system which is based on the acquisition, transmission, distribution, and display of sports video from multiple viewpoints. The awardee has also developed a system that acquires 3D depth information of objects using active illumination and displays it in real time on a 3D display using integral photography (The Institute of Image Electronics Engineers of Japan, 2017 Image Electronics Technology Award).

The achievements of the awardee are highly evaluated as pioneering works that began with the proposal of the concept of ray space, continued from the dawn of the field to the accumulation of high-level research achievements, had an effect on various fields of 3D imaging, and laid the foundation for the cutting-edge research still being conducted at various institutions around the world today.

As described above, the awardee's achievements are extremely remarkable and worthy of the Achievement Award.

References

1. Fujii, “A Basic Study of Integrated 3D Image Coding,” Ph.D Thesis, The University of Tokyo (1994.12).
2. Fujii, Kaneko, Harashima, “Representation of 3D Spatial Information by Rays and Its Application,” The Journal of the Institute of Television Engineers of Japan, Vol. 50, No. 9 pp. 1312-1318 (1996.9).
3. T. Fujii, M. Tanimoto, “Free-Viewpoint TV System based on Ray-Space Representation,” SPIE Three-Dimensional TV, Video, and Display Vol. 4864, pp. 175-189 (2002.11).
4. Y. Mori, N. Fukushima, T. Yendo, T. Fujii, M. Tanimoto, “View Generation with 3D Warping Using Depth Information for FTV,” Signal Processing: Image Communication. Vol. 24, Issues 1-2, pp. 65-72 (2009.1)
5. M. Tanimoto, M. P. Tehrani, T. Fujii, T. Yendo, “Free-Viewpoint TV,” IEEE Signal Processing Magazine, Vol. 28, Issue 1 (2011.1).
6. T. Fujii, “3D Image Processing Using Deep Neural Network,” SPIE 3D Imaging, Visualization, and Display. Fumio Okano Best 3D Paper Prize (2020).
7. R. Mizuno, K. Takahashi, M. Yoshida, C. Tsutake, T. Fujii, H. Nagahara, “Acquiring a Dynamic Light Field through a Single-Shot Coded Image,” CVPR 2022 (2022.6).