Presentation 2002/3/1
Relating Audio-Visual Events Caused by Multiple Movements : In the Case of Entire Object Movement
Jinji CHEN, Toshiharu MUKAI, Yoshinori TAKEUCHI, Testuya MATSUMOTO, Hiroaki KUDO, Tsuyoshi YAMAMURA, Noboru OHNISHI,
PDF Download Page PDF download Page Link
Abstract(in Japanese) (See Japanese page)
Abstract(in English) Relating audio-visual events is important for constructing an artificial intelligent system, which can acquire the audio-visual knowledge of movement through active observation without teaching. This paper proposes a method for relating multiple audio-visual events observed by a camera and a microphone according to general laws without object-specific knowledge (including the case of entire object movement). As corresponding cues, we use Gestalt's grouping law; simultaneity of the occurrence of the sound and the change in movement or the same motion starting, similarity of repetition between sound and movement. Based on their correlation coefficient, the component of frequency at sound onset is related to the short-term space-time invariants (STSTI) of movement. We conducted experiments in the real environment and obtained satisfactory results showing the effectiveness of the proposed method.
Keyword(in Japanese) (See Japanese page)
Keyword(in English) sensor fusion / event correspondence / short-term space-time invariants / motion invariants / repetition similarity / occurrence simultaneity / correlation coefficient
Paper # HIP2001-97
Date of Issue

Conference Information
Committee HIP
Conference Date 2002/3/1(1days)
Place (in Japanese) (See Japanese page)
Place (in English)
Topics (in Japanese) (See Japanese page)
Topics (in English)
Vice Chair

Paper Information
Registration To Human Information Processing (HIP)
Language JPN
Title (in Japanese) (See Japanese page)
Sub Title (in Japanese) (See Japanese page)
Title (in English) Relating Audio-Visual Events Caused by Multiple Movements : In the Case of Entire Object Movement
Sub Title (in English)
Keyword(1) sensor fusion
Keyword(2) event correspondence
Keyword(3) short-term space-time invariants
Keyword(4) motion invariants
Keyword(5) repetition similarity
Keyword(6) occurrence simultaneity
Keyword(7) correlation coefficient
1st Author's Name Jinji CHEN
1st Author's Affiliation Nagoya University()
2nd Author's Name Toshiharu MUKAI
2nd Author's Affiliation Bio-Mimetic Control Research Center, RIKEN
3rd Author's Name Yoshinori TAKEUCHI
3rd Author's Affiliation Nagoya University
4th Author's Name Testuya MATSUMOTO
4th Author's Affiliation Nagoya University
5th Author's Name Hiroaki KUDO
5th Author's Affiliation Nagoya University
6th Author's Name Tsuyoshi YAMAMURA
6th Author's Affiliation Aichi Prefectural University
7th Author's Name Noboru OHNISHI
7th Author's Affiliation Nagoya University
Date 2002/3/1
Paper # HIP2001-97
Volume (vol) vol.101
Number (no) 699
Page pp.pp.-
#Pages 7
Date of Issue