TSD05.pdf

Automatic Acquisition of a Slovak Lexicon

from a Raw Corpus

Beno ıt Sagot

INRIA-Rocquencourt, Projet Atoll,

Domaine de Voluceau, Rocquencourt B.P. 105

78 153 Le Chesnay Cedex, France

Abstract. This paper presents an automatic methodology we used in

an experiment to acquire a morphological lexicon for the Slovak lan-

guage, and the lexicon we obtained. This methodology extends and re-

nes approaches which have proven e cient, e.g., for the acquisition of

French verbs or Croatian and Russian nouns, adjectives and verbs. It

only relies on a raw corpus and on a morphological description of the

language. The underlying idea is to build all possible lemmas that can

explain all words found in the corpus, according to the morphological

description, and to rank these hypothetical lemmas according to their

likelihood given the corpus. Of course, hand-validation and iteration of

the whole process is needed to achieve a high-quality lexicon, but the

human involvement required is orders of magnitude lower than the cost

of the fully manual development of such a resource. Moreover, this tech-

nique can be easily applied to other languages with a rich morphology

that lack large-coverage lexical resources.

1 Introduction

Among the dierent resources that are needed for Natural Language Process-

ing tasks, the lexicon plays a central role. It is for example a prerequisite to

any wide-coverage parser. However, the development or enrichment of a large

and precise lexicon, even restricted to morphological information, is a di cult

task, in particular because of the huge amount of data that has to be collected.

Therefore, most large-coverage morphological lexicons for NLP concern only a

few languages, such as English. Moreover, these lexicons are usually the result

of the careful work of human lexicographers who develop them manually over

years, and for this reason they are often not freely available.

The aim of this paper is to show that this is not the only possible way to

develop or enrich a morphological lexicon, and that this process can be automa-

tized in such a way that the needed human labor is drastically reduced, at least

for categories that have a rich morphology. 1 The only requirements are a raw

We do not consider here closed classes like prepositions, pronouns or numerals, be-

cause they can be easily described manually and/or because they don’t have a rich

morphology, if any.

corpus and a morphological description of the language. This makes it possi-

ble to build morphological lexicons in relatively little time for languages that

received less attention until now, for example because they are spoken by less

people and/or because they are not supported by a large NLP community. We

applied our methodology to Slovak language.

The idea of learning lexical information (with or without manual validation)

is not new. It has been successfully applied, among other tasks, to terminology

[1], collocations [2] or sub-categorization properties [3]. All assume the avail-

ability of a morphological lexicon. But to our knowledge, very few work has

been published on automatic acquisition of morphological lexicons from raw

corpus. Experiments have been conduced by Oliver and co-workers on Russian

and Croatian [4, 5] to acquire or enlarge lexicons of verbs, nouns and adjectives.

Independently, Clement, Sagot and Lang [6] have published the methodology

they used to acquire a lexicon of French verbs. This paper is an extension of

these methods for at least three reasons. First, we do not take into account only

inectional morphology, but also derivational morphology, which allows a better

precision and recall as well as the acquisition of derivational relations in the lex-

icon. Second, we use a morphological description that is more powerful than the

purely concatenative morphology used in previous works. Third, our algorithm

relies on a very simple but rigorous probabilistic model. 2

The main idea is that the acquisition of the lexicon of a corpus in a given

language can be achieved by the iteration of a three-step loop:

1. Given the morphological description of the language, build all possible lem-

mas that can possibly explain the inected forms found in the corpus,

2. Rank these possible lemmas according to their likelihood given the corpus,

3. Validate manually best ranked lemmas.

In the remainder of this paper, we will describe these steps, our morphological

description of Slovak and our current results, with an emphasis on step 2.

2 Slovak morphology

Like most other Slavic languages, and contrary to English or French, Slovak is

an inected language. This means that nouns and adjectives (among others) are

inected according to their gender and number, but also to their grammatical

function or to the preposition that governs them (case). This inection is mostly

realized by changing the ending of the word according to its inectional class (or

paradigm), but the stem itself can be aected. The latter occurs in particular

for some feminine and neuter nouns in their genitive plural form. For example,

zena (“woman”), in which the stem is zen-, has the genitive plural form zien.

2.1 Slovak language

The Slovak language is a Slavic (and therefore Indo-European) language that

is the o cial language of the Slovak Republic. Its closest relative is the Czech

This is already the case in [6], but the model presented here seems more convincing.

language. Both languages coexisted during a long period within former Czecho-

slovakia. For this reason, and because of the proximity of these languages, most

Slovak understand Czech, and people wishing to learn “the” language spoken in

Czechoslovakia learned Czech. Consequences of this are for example that Slovak

language is under-represented among language manuals, 3 and that it received

less attention than other Slavic languages such as Czech or Russian. The only

big project concerning Slovak in computational linguistics is the Slovak National

Corpus [7], which is a highly valuable resource. Because we think that having

only one resource for a given language is not necessarily satisfying, we decided

not to use this resource and the information it contains. However, we intend to

compare our lexicon with this corpus in the near future.

2.2 Description of Slovak morphology

As already mentioned in the introduction, automatic lexical acquisition of a

morphological lexicon from a raw corpus strongly relies on morphological knowl-

edge. Moreover, this knowledge has to be represented and used in a symmetrical

way, in the sense that we want to be able to inect lemmas (associated with an

inectional class) but also to ambiguously un-inect forms found in the corpus

into all possible lemmas that might explain them. Moreover, the morphological

description of the language must be written by hand, and therefore in a reason-

able format. It must also be exploited in a very e cient way, since we want to

deal with big corpora, and therefore with a big amount of hypothetical lemmas.

Our description of the Slovak morphology, 4 inspired among others from [8]

and validated by a native speaker of the language, is represented by an XML le

that contains three main kinds of objects, which will be described successively:

1. letters, and the classes they belong to,

2. fusion rules that model the interaction of the nal letters of a stem and the

initial letters of an ending,

3. inectional classes.

The list of letters deserves no special comment. We associate with each of

those letters a list of the phonetic classes of the phoneme they denote. We use

six classes: consonants, soft consonants, non-soft consonants, vowels, long vowels

(including diphtongs), short vowels.

The second kind of information we built about Slovak morphology is the

interaction between the nal letters of a stem and the initial letters of an end-

ing. This is done through a set of fusion patterns which allow to model with a

For example, and to our knowledge, Slovak is today the only o cial language of a

European country for which no manual in French language is available.

It is important to state here that this morphological description is not the main

point of the paper. Any other morphological description, possibly better or more

justied from a linguistic point of view, could be used. The only requirement is that

this description must be able to give all inected forms of a given lemma, as well as

all possible lemmas having a given form in their inectional paradigm.

reasonable amount of inectional classes phenomena that can be explained by

standard classes, provided fusion patterns are used. Let us take an example. If a

stem ending in t’ like kost’ (“bone”) gets an ending beginning with i , as in our

example the ending i of the locative singular, then the result is not -t’i- (here

*kost’i) but -ti- (here kosti). Therefore, we can describe a pattern t’i ! ti .

Another example is the plural genitive zien of zena mentioned above: we decide

that the ending is in this case - , and we add the following fusion pattern, where

\ c means “any letter of the class of consonants”: e \ c- ! ie \ c . We also dened

the special operator $ that means “end of word”, and the special class\ * that

matches any letter. An example that uses these operators is the pair of patterns

bec \ * ! bc \ * and bc \ $ ! bec , which allows to model the alternance between

the -bec form of some stems when they get an empty ending and the -bc form

of the same stems when the ending is non-empty (e.g., vrabec, vrabca,. . . ). Both

patterns are needed since we need our morphological description to be usable in

both directions: from a lemma to its forms, using the rst rule, and from a form

to its possible lemmas, using the second rule.

The third set of information we designed is the set of inectional classes. For

each class, we list its name, the ending that has to be removed from the lemma

to get the stem, and (when needed) a regular expression that stems using this

inectional class have to match. To exemplify the latter point, we say that verbs

in -at’ / -iam / -aju (like merat’) have stems that must end with a soft consonant.

Each inectional class contains a set of inected forms dened by their ending

and a morphological tag, supplemented, if needed, by another regular expression

that the stem must match. This allows to merge into one inectional class two

paradigms that dier only on a few forms in a way that can be predicted from

the stem. 5 Classes also contains derived lemmas, dened by their ending and

their inection class. For example, the inectional class of regular -at’ verbs

have (for the moment) two possible derivations, namely the associated noun in

-anie and the associated adjective in -any .

3 Automatic acquisition of the lexicon

As mentioned in the introduction, we iterate a three-step loop as many times

as wanted. The three steps are the generation and the inection of all possible

lemmas, the ranking of these possible lemmas, and a partial manual validation.

Each step takes into account the information given by the manual validator

during previous steps. We shall now describe these steps one after the other.

The probabilistic model we developed that underlies step 2 is described in the

corresponding paragraph.

For example, we have only one inectional class for regular -at’ verbs. The form-

level regular expression checks if the last vowel of the stem is long or short, thus

allowing to decide between the endings -am , -as ,. . . , and the endings -am , -as ,. . . .

Indeed, innitive, participle and 3rd person plural endings are identical, as well as

derived lemmas (see below).

3.1 Generation and inection of all possible lemmas

For our experiments, we use a relatively small corpus of 150,000 words represent-

ing 20,000 dierent words. This corpus includes texts produced by the European

Union (including the project of Constitutional Treaty) and free-of-use articles

found on Internet (both scientic and journalistic style are represented). The rst

step is to remove from the corpus all words that are present in a hand-crafted

list of words belonging to closed classes (pronouns, some adverbs, prepositions,

and so on).

After the extraction of the words of our corpus and their number of occur-

rences, we need to build all hypothetical lemmas that match the morphological

description of Slovak language and have among their inected form at least one

word which is attested in the corpus. We then need to inect these hypothetical

lemmas to build all their inected forms (we call “lemma” a canonical form with

the name of its inection class 6 ). To achieve these goals, we developed a script

that reads our morphological description and turns it into two programs. The

rst one can be seen as a non-deterministic morphological parser (or ambiguous

lemmatizer), and the second one as an inecter. In a few dozens of seconds, the

rst program generates 73,000 hypothetical lemmas out of the 20,000 dierent

words of the corpus. These lemmas are then inected by the second program

in a few other dozens of seconds, thus generating more than 1,500,000 inected

forms, each one being associated with its lemma and a morphological tag.

3.2 Ranking possible lemmas

At this point, our goal is to rank the hypothetical lemmas we generated in such

a way that the best ranked lemmas are (ideally) all correct, and the least ranked

lemmas are all erroneous. Therefore, we need a way to model some kind of

plausibility measure for each lemma. We have chosen to compute the likelihood

of each lemma given the corpus. Since we do not have the required information

to do so directly, we use a x-point algorithm according to the following model.

We consider the following experiment: we choose at random in the corpus

a token (i.e., one occurence of an inected form, hereafter simply ”form”). The

probability to have chosen a given form f is P (f ) = occ(f )/n tot , where occ(f )

is the number of occurrences of f in the corpus and n tot the total number of

words, both known. Let us callL f the set of all hypothetical lemmas that have

f as one of their inected forms. We denote by P (l) the probability that the

token we chose is an inected form of l, and by P (f|l) the probability to have

chosen an occurrence of f given the fact that we have chosen an inected form

of the lemma l. We then have the following equality (for the rst iteration of the

x-point algorithm, P (l) is initialized to an arbitrary value P 0 , typically 0.1):

P (f|l) =

P (f )− P

l 0 2L f ,l 0 6=l P (l 0 )P (f|l 0 )

P (l)

Hence, a same canonical form can come from several dierent lemmas, provided they

do not belong to the same inectional classes.

Plik z chomika:

Inne pliki z tego folderu:

Inne foldery tego chomika: