Presentation 1995/5/12
Automatic Extraction of Uninterrupted and Interrupted Collocations from Very Large Japanese Corpora using N-gram Statistics
Satoru Ikehara, Satoshi Shirai, Tsukasa Kawaoka,
PDF Download Page PDF download Page Link
Abstract(in Japanese) (See Japanese page)
Abstract(in English) In order to extract rigid expressions with a high frequency of use, new algorithms that can efficiently extract both uninterrupted and interrupted collocations from very large Japanese corpora have been proposed. More recently, the technique of applying n-gram statistics for uninterrupted collocation has been proposed. This enables the extraction of collocations in the order of string length and frequency of use. But this method posed problems in that large volumes of fractional and unnecessary expressions are included. To solve this problem, this paper proposes a new algorithm that restrains the extraction of unnecessary expressions. This is followed by the proposal of a method that extracts interrupted collocations combining the uninterrupted collocations thus obtained. These new methods are applied to newspaper articles containing 8.92 million characters. In the case of uninterrupted collocations with string length of 2 or more characters and whose frequency of appearance is 2 or more times, there were 4.4 million expressions (total frequency of 31.2 million times) extracted by the conventional method. In contrast, the new method reduced this to 0.97 million types (total frequency of 2.6 million times) revealing a substantial reduction in fractional and unnecessary expressions. In the case of interrupted collocational substring extractions, combining the substring with frequency of 10 times or more extracted by the first method, yielded 6.5 thousand types of pairs of substrings with the total frequency of 21.8 thousand times.
Keyword(in Japanese) (See Japanese page)
Keyword(in English) Collocation / Corpora / Automatic Extraction / N-gram / Sentence Pattern
Paper #
Date of Issue

Conference Information
Committee NLC
Conference Date 1995/5/12(1days)
Place (in Japanese) (See Japanese page)
Place (in English)
Topics (in Japanese) (See Japanese page)
Topics (in English)
Chair
Vice Chair
Secretary
Assistant

Paper Information
Registration To Natural Language Understanding and Models of Communication (NLC)
Language JPN
Title (in Japanese) (See Japanese page)
Sub Title (in Japanese) (See Japanese page)
Title (in English) Automatic Extraction of Uninterrupted and Interrupted Collocations from Very Large Japanese Corpora using N-gram Statistics
Sub Title (in English)
Keyword(1) Collocation
Keyword(2) Corpora
Keyword(3) Automatic Extraction
Keyword(4) N-gram
Keyword(5) Sentence Pattern
1st Author's Name Satoru Ikehara
1st Author's Affiliation NTT Communication Science Laboratories()
2nd Author's Name Satoshi Shirai
2nd Author's Affiliation NTT Communication Science Laboratories
3rd Author's Name Tsukasa Kawaoka
3rd Author's Affiliation Faculty of Engineering, Dousisha University
Date 1995/5/12
Paper #
Volume (vol) vol.95
Number (no) 29
Page pp.pp.-
#Pages 8
Date of Issue