Publications

2025

1. ICCV

   

   Few-Shot Pattern Detection via Template Matching and Regression

   Eunchan Jo, Dahyun Kang, Sanghyun Kim , Yunseon Choi, and 1 more author

   In Proceedings of the IEEE/CVF Conference on International Conference on Computer Vision (ICCV), 2025

   Website
2. CVPR

   

   Locality-Aware Zero-Shot Human-Object Interaction Detection

   Sanghyun Kim , Deunsol Jung, and Minsu Cho

   In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

   Invited Poster Presentation, 3rd RHOBIN Workshop @ CVPR2025

   Abs arXiv Code

   Recent methods for zero-shot Human-Object Interaction (HOI) detection typically leverage the generalization ability of large Vision-Language Model (VLM), i.e., CLIP, on unseen categories, showing impressive results on various zero-shot settings. However, existing methods struggle to adapt CLIP representations for human-object pairs, as CLIP tends to overlook fine-grained information necessary for distinguishing interactions. To address this issue, we devise, LAIN, a novel zero-shot HOI detection framework designed to enhance the locality and interaction awareness of CLIP representations. The locality awareness, which involves capturing fine-grained details and the spatial structure of individual objects, is achieved by aggregating the information and spatial priors of adjacent neighborhood patches. The interaction awareness, which involves identifying whether and how a human is interacting with an object, is achieved by capturing the interaction pattern between the human and the object. By infusing locality and interaction awareness into CLIP representation, LAIN captures detailed information about the human-object pairs. Our extensive experiments on existing benchmarks show that LAIN outperforms previous methods in various zero-shot settings, demonstrating the importance of locality and interaction awareness for effective zero-shot HOI detection.

2024

1. CVPRW

   

   Burst Image Super-Resolution with Base Frame Selection

   Sanghyun Kim , Minjung Lee, Woohyeok Kim, Deunsol Jung, and 3 more authors

   In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshop on New Trends in Image Restroation and Enhancement (NTIRE), 2024

   Abs arXiv Website

   Burst image super-resolution has been a topic of active research in recent years due to its ability to obtain a high-resolution image by using complementary information between multiple frames in the burst. In this work, we explore using burst shots with non-uniform exposures to confront real-world practical scenarios by introducing a new benchmark dataset, dubbed Non-uniformly Exposed Burst Image (NEBI), that includes the burst frames at varying exposure times to obtain a broader range of irradiance and motion characteristics within a scene. As burst shots with non-uniform exposures exhibit varying levels of degradation, fusing information of the burst shots into the first frame as a base frame may not result in optimal image quality. To address this limitation, we propose a Frame Selection Network (FSN) for non-uniform scenarios. This network seamlessly integrates into existing super-resolution methods in a plug-and-play manner with low computational costs. The comparative analysis reveals the effectiveness of the nonuniform setting for the practical scenario and our FSN on synthetic-/real- NEBI datasets.

2. WACV

   

   Optical Flow Domain Adaptation via Target Style Transfer

   Jeongbeen Yoon, Sanghyun Kim , Suha Kwak, and Minsu Cho

   In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2024

   Abs Website

   Optical flows play an integral role for a variety of motion-related tasks such as action recognition, object segmentation, and tracking in videos. While state-of-the-art optical flow methods heavily rely on learning, the learned optical flow methods significantly degrade when applied to different domains, and the training datasets are very limited due to the extreme cost of flow-level annotation. To tackle the issue, we introduce a domain adaptation technique for optical flow estimation. Our method extracts diverse style statistics of the target domain and use them in training to generate synthetic features from the source features, which contain the contents of the source but the style of the target. We also impose motion consistency between the synthetic target and the source and deploy adversarial learning at the flow prediction to encourage domain-invariant features. Experimental results show that the proposed method achieves substantial and consistent improvements in different domain adaptation scenarios on VKITTI 2, Sintel, and KITTI 2015 benchmarks.

2023

1. CVPR

   

   Relational Context Learning for Human-Object Interaction Detection

   Sanghyun Kim , Deunsol Jung, and Minsu Cho

   In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023

   Abs arXiv Code Website

   Recent state-of-the-art methods for HOI detection typically build on transformer architectures with two decoder branches, one for human-object pair detection and the other for interaction classification. Such disentangled transformers, however, may suffer from insufficient context exchange between the branches and lead to a lack of context information for relational reasoning, which is critical in discovering HOI instances. In this work, we propose the multiplex relation network (MUREN) that performs rich context exchange between three decoder branches using unary, pairwise, and ternary relations of human, object, and interaction tokens. The proposed method learns comprehensive relational contexts for discovering HOI instances, achieving state-of-the-art performance on two standard benchmarks for HOI detection, HICO-DET and V-COCO.

2. CVPR

   

   Devil’s on the Edges: Selective Quad Attention for Scene Graph Generation

   Deunsol Jung, Sanghyun Kim , Won Hwa Kim, and Minsu Cho

   In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023

   Abs arXiv Code Website

   Scene graph generation aims to construct a semantic graph structure from an image such that its nodes and edges respectively represent objects and their relationships. One of the major challenges for the task lies in the presence of distracting objects and relationships in images; contextual reasoning is strongly distracted by irrelevant objects or backgrounds and, more importantly, a vast number of irrelevant candidate relations. To tackle the issue, we propose the Selective Quad Attention Network (SQUAT) that learns to select relevant object pairs and disambiguate them via diverse contextual interactions. SQUAT consists of two main components: edge selection and quad attention. The edge selection module selects relevant object pairs, i.e., edges in the scene graph, which helps contextual reasoning, and the quad attention module then updates the edge features using both edge-to-node and edge-to-edge cross-attentions to capture contextual information between objects and object pairs. Experiments demonstrate the strong performance and robustness of SQUAT, achieving the state of the art on the Visual Genome and Open Images v6 benchmarks