RESEARCH

__________________

Immersive Content Analysis for 3D/VR/Metaverse

Stereoscopic 3D Depth Editing (AAAI'21)

360º Video Analysis for VRSA (AAAI'21)

Panoramic Image Analysis (IEEE TCSVT'20)

Immersive Reality including 3D/AR/VR seems like distinct experience today, over time it will converge and allow us to combine real and virtual worlds freely, and interact between them, i.e., Metaverse. In the context of the immersive reality with AI technology for intelligent reality, CAU IRIS Lab Research is focused on developing all the technologies needed to enable breakthrough 3D/AR/VR/Metaverse contents analysis and creation including AI/ML, digital twin, robotics & computer vision, 360º image and video analysis, 3D image editing and generation, human visual perception, etc.

Selected Publication Lists

  • H. G. Kim, M. Park, S. Lee, S. Kim and Y. M. Ro, Visual Comfort Aware-Reinforcement Learning for Depth Adjustment of Stereoscopic 3D Images, AAAI 2021

  • H. G. Kim, S. Lee, S. Kim, H. Lim and Y. M. Ro, Towards a Better Understanding of VR Sickness: Physical Symptom Prediction for VR Contents, AAAI 2021

  • H. G. Kim*, H. Lim* and Y. M. Ro, Deep Virtual Reality Image Quality Assessment with Human Perception Guider for Omnidirectional Image, IEEE TCSVT 2020

  • H. G. Kim, H. Lim, S. Lee and Y. M. Ro, VRSA Net: VR Sickness Assessment Considering Exceptional Motion for 360° VR Video, IEEE TIP 2019

  • H. G. Kim*, H. Jeong*, H. Lim and Y. M. Ro, Binocular Fusion Net: Deep Learning Visual Comfort Assessment for Stereoscopic 3D, IEEE TCSVT 2018

  • H. G. Kim and Y. M. Ro, Multiview Stereoscopic Video Hole Filling Considering Spatiotemporal Consistency and Binocular Symmetry for Synthesized 3D Video, IEEE TCSVT 2017

  • H. G. Kim, W. J. Baddar, H. Lim, H. Jeong and Y. M. Ro, Measurement of Exceptional Motion in VR Video Contents for VR Sickness Assessment Using Deep Convolutional Autoencoder, ACM VRST 2017

  • H. G. Kim and Y. M. Ro, Ultrafast Layer-based Computer-Generated Hologram Calculation with Sparse Template Holographic Fringe Pattern for 3-D Object, Optics Express 2017

Attention-Aware Image/Video Understanding

Video Prediction (CVPR'21)

Video Frame Interpolation (IEEE TCSVT'21)

Abnormal Event Detection (IEEE TIP'20)

Attention mechanisms originated from the investigations of human vision in cognitive science. Inspired by human visual attention mechanism, the performance of the models can be improved in various computer vision and natural language processing fields. Based on the knowledge of human vision, CAU IRIS Lab Research is focused on developing intelligent decision making systems in both of industry and research fields including video prediction, abnormal event detection, traffic situation prediction, autonomous driving, weather forecasting, frame rate interpolation, etc.

Selected Publication Lists

  • S. Lee, H. G. Kim, D. H. Choi, H. Kim and Y. M. Ro, Video Prediction Recalling Long-term Motion Context via Memory Alignment Learning, CVPR 2021

  • M. Park, H. G. Kim and Y. M. Ro, Robust Video Frame Interpolation with Exceptional Motion Map, IEEE TCSVT 2021

  • S. Lee, H. G. Kim and Y. M. Ro, BMAN: Bidirectional Multi-scale Aggregation Networks for Abnormal Event Detection, IEEE TIP 2020

  • J. U. Kim, J. Kwon, H. G. Kim and Y. M. Ro, BBC Net: Bounding-Box Critic Network for Occlusion-Robust Object Detection, IEEE TCSVT 2020

Domain Knowledge (Multi-modal) Learning

Medical Image Segmentation (CVPR'20)

Video-Physiological Signal Fusion (ECCV'20)

Satellite Image Prediction (IEEE TGRS'20)

Big data analysis is a field that lies at the intersection of computer science, mathematics/statistics, and domain-specific expertise. The domain knowledge in software engineering is the knowledge about the environment in which the data is processed to reveal secrets of the data. CAU IRIS Lab is focused on helping to go beyond the performance limits of AI/ML itself by integrating AI/ML with domain knowledge of human expert or domain data (i.e., multi-modal) for domain specific tasks including visio-linguistic learning, visual-speech recognition, multi-spectral imaging, etc.

Selected Publication Lists

  • S. Lee, S. Kim, H. G. Kim and Y. M. Ro, Assessing Individual VR Sickness through Deep Feature Fusion of VR Video and Physiological Response, IEEE TCSVT 2021

  • H. J. Lee, J. U. Kim, S. Lee, H. G. Kim and Y. M. Ro, Structure Boundary Preserving Segmentation for Medical Image with Ambiguous Boundary, CVPR 2020

  • S. Lee, S. Kim, J. U. Kim, H. G. Kim, S. Kim and Y. M. Ro, SACA Net: Cybersickness Assessment of Individual Viewers for VR Content via Graph-based Symptom Relation Embedding, ECCV 2020

  • J.-H. Lee, S. S. Lee, H. G. Kim, S. K. Song, S. Kim and Y. M. Ro, MCSIP Net: Multichannel Satellite Image Prediction via Deep Neural Network, IEEE TGRS 2020