    \documentclass[english]{jnlp_1.4}

\usepackage{jnlpbbl_1.1}
\usepackage[dvips]{graphicx}
\usepackage{hangcaption_jnlp}
\usepackage{array}
\usepackage{udline}
\setulminsep{1.2ex}{0.2ex}
\let\underline

\Volume{15}
\Number{2}
\Month{Apr.}
\Year{2008}
\received{2007}{7}{3}
\revised{2007}{10}{23}
\accepted{2007}{11}{15}

\setcounter{page}{137}

\etitle{A Web Corpus and Word Sketches for Japanese}
\eauthor{Irena Srdanovi\'{c} Erjavec\affiref{Author_1} \and Toma\v{z} Erjavec\affiref{Author_2}\and Adam Kilgarriff\affiref{Author_3}} 
\eabstract{
Of all the major world languages, Japanese is lagging behind in terms of publicly accessible and searchable corpora. In this paper we describe the development of JpWaC (Japanese Web as Corpus), a large corpus of 400 million words of Japanese web text, and its encoding for the Sketch Engine. The Sketch Engine is a web-based corpus query tool that supports fast concordancing, grammatical processing, `word sketching' (one-page summaries of a word's grammatical and collocational behaviour), a distributional thesaurus, and robot use. We describe the steps taken to gather and process the corpus and to establish its validity, in terms of the kinds of language it contains. We then describe the development of a shallow grammar for Japanese to enable word sketching. We believe that the Japanese web corpus as loaded into the Sketch Engine will be a useful resource for a wide number of Japanese researchers, learners, and NLP developers.
}
\ekeywords{Japanese web corpus, Corpus query tool, Sketch Engine, Word sketches}

\headauthor{Srdanovi\'{c}, Erjavec and Kilgarriff}
\headtitle{A web corpus and word sketches for Japanese}

\affilabel{Author_1}{}{Tokyo Institute of Technology}
\affilabel{Author_2}{}{Jo\v{z}ef Stefan Institute}
\affilabel{Author_3}{}{Lexical Computing Ltd. and Universities of Leeds and Sussex}

\newcommand{\cf}{}
\newcommand{\eg}{}
\newcommand{\ie}{}
    \newcommand{\url}[1]{}
\newcommand{\ex}[1]{}
\newcommand{\reftab}[1]{}
\newcommand{\reffig}[1]{}
\newcommand{\refsec}[1]{}

\begin{document}

\maketitle


\section{The Sketch Engine}

Of all the major world languages, Japanese is lagging behind in terms
of publicly accessible and searchable corpora. This paper reports on
the development of JpWaC (Japanese Web as Corpus), a large corpus of
Japanese web text, which has been loaded into the Sketch Engine, where
it takes its place alongside Chinese, English, French, German, and a
range of other languages.  

The Sketch Engine\footnote{
	\url{http://www.sketchengine.co.uk}
}\shortcite{Kilgarriff_2004} is a corpus tool with several distinctive
features. It is fast, giving immediate responses for most regular
queries for corpora of up to two billion words. It is designed for use
over the web. It works with all standard browsers, so users need no
technical knowledge, and do not need to install any software on their
machine. It has been used for dictionary compilation at, amongst
others, Oxford University
Press\footnote{
	\url{http://www.askoxford.com/oec}
}, Macmillan
\shortcite{Kilgarriff_2002}, Chambers Harrap and Collins, and also
for language teaching \shortcite<e.g.>{Chen_2007} and language technology
development \shortcite<e.g.>{Gatt_2006,Chantree_2005}.

As well as offering standard corpus query functions such as
concordancing, sorting, filtering etc., the Sketch Engine is unique in
integrating grammatical analysis, which makes it possible to produce
\emph{word sketches}, one-page summaries of a word's grammatical and
collocational behaviour as illustrated in Figure 1.



\begin{figure}[b]
\begin{center}
\includegraphics{15-2ia6f1.eps}
\hangcaption{Word sketch for お湯 (\textit{oyu} `hot water').
The first number is the number of instances for the collocation. 
The second is the association score (based on the Dice coefficient) for the collocation, which is used for deciding which collocations to show and sorting.}
\label{fig1}
\end{center}
\end{figure}


Figure 1 gives a word sketch for the noun お湯
(\textit{oyu}). Different grammatical relations, such as Ai and Ana
adjectives modifying a noun, noun-particle-verb collocates
(をverb, でverb, がverb), noun-pronominal relations
(with の) etc., are displayed in order of their significance,
revealing the most frequent and most salient sets of collocations (the
first and second column with numbers respectively). The list of
collocates nicely reveals the different senses of the noun: (1)
`hot/warm water (water that is heated)'
（お湯を沸かす，熱いお湯，ポットのお湯，鍋のお湯，お湯で茹でる，お湯で温める），(2) `bath (water for taking a
bath)' （お湯に入る，お湯から上がる，湯舟のお湯，浴槽のお湯，風呂のお湯，易しいお湯，いいお湯）\footnote{
	Checking word sketches for the
variant 湯 (\textit{yu}), reveals an additional sense, `hot
spring', that is not present in the お湯 (\textit{oyu})
variant.
}The ``coord'' relation reveals the coordinate noun of お湯，that is 水(\textit{mizu} `water').

Based on the grammatical analysis, we also produce a
\textit{distributional thesaurus} for the language, in which words
occurring in similar settings, sharing the same collocates, are put
together \shortcite{Sparck_1986,Grefenstette_1994,Lin_1998,Weeds_2005}, 
and ``sketch diffs'', which compare related words (see
section 3 below). The Sketch Engine is accessible either by a person
using a web browser, or by program, with the output from the Sketch
Engine (in plain text, XML or JSON) being an input to some other NLP
process.




\section{The JpWaC corpus}

In this section we review the steps taken to compile the JpWaC corpus,
give a quantitative account of its contents and discuss the validity
of this kind of corpus for general language research. The corpus has
been gathered using methods as described by
\shortcite{Sharoff_2006a,Sharoff_2006b,Baroni_2006c}, and for Japanese
\shortcite{Ueyama_2005}. The corpus consists of texts obtained from around
50,000 web pages and contains just over 400 million tokens.


\subsection{Compiling the corpus}

For easy web corpus compilation, a great step forward has been made by
the publicly available
BootCat\footnote{
	\url{http://sslmit.unibo.it/{\textasciitilde baroni}/bootcat.html}
}\shortcite{Baroni_2004,Baroni_2006a},
WAC\footnote{
	\url{http://www.drni.de/wac-tk/}
}(Web as Corpus
Toolkit) and related work of the Wacky
Project\footnote{
	\url{http://wacky.sslmit.unibo.it/}
}\shortcite{Baroni_2006b}. 
These open source tools provide the functionality
for compiling a basic web corpus, and are straightforward to install
and use on a Linux platform.

The main steps for compiling JpWaC were the following, explained in more detail below:

\begin{enumerate}
\item obtain list of Japanese language URLs 
\item download the Web pages
\item normalize character encodings
\item extract metadata
\item remove document boilerplate 
\item annotate with linguistic information
\end{enumerate}


1. The list of URLs to build the corpus is critical for any web
corpus, as it determines the overall composition, in terms of
language(s), registers, lexis, etc. In order to obtain a good corpus
of general language, the URL list for JpWaC list was obtained via the
methodology described in \shortcite{Sharoff_2006a}. First, the top 500
non-function words of the British National Corpus (see
\url{http://natcorp.ox.ac.uk}; hereafter BNC) were translated into
Japanese\footnote{
	The words were translated by Moto Ueyama.
}. Random
4-tuples were generated from this list and the Google API was used to
query the internet with them. The set of retrieved pages gave the
final URL list.

2. We downloaded the HTML pages with the WAC toolkit program {\tt paraget}, which implements parallel downloading of pages, while taking care not to overload particular servers; it also separates the large number of retrieved files into separate tree-structured directories.

3. For normalizing the character sets of the downloaded pages we used
the BootCat tool {\tt encoding-sort.pl}, which implements a heuristic
to guess the encoding of each HTML page, and then the standard UNIX
utility {\tt iconv} to convert the actual file to UTF-8.

4. The extraction of meta-data implemented in BootCat/WAC, in particular title and author, did not work well on Japanese pages, so the only metadata we retained is the URL of each document.

5. For boilerplate removal we used BootCat {\tt clean\_pages\_from\_file\_list.pl}, which removes HTML tags, JavaScript code and text-poor portions of the pages, such as navigational frames, leaving only the pure text of each page. 

6. Finally, we annotated the corpus using
ChaSen\footnote{
	\url{http://chasen.naist.jp/}
}, which segments the
text into sentences, tokenises it, lemmatises the words and assigns
them morphosyntactic descriptions (MSDs). The MSDs that ChaSen returns
are in Japanese: to make it easier to understand and use the
annotations by non-Japanese speakers (and those without Japanese
language keyboards), we translated the Japanese MSDs into English.


\subsection{Copyright}

There has been much discussion amongst corpus-builders about copyright. Many have taken the view that, for any text to be included, the copyright holder must be contacted and the document can only be included if their consent is granted.

This perspective dates back to a pre-web era. A corpus tool such as Sketch Engine, when loaded with a web corpus, is performing a task that is equivalent to a search engine such as Google or Yahoo. In each case, the owners of the service have automatically gathered large numbers of web pages, processed them, and indexed them. Then, users are shown small extracts. 

This basic function of search engines has not been legally challenged. Google and Yahoo do not ask copyright owners for permission to index their pages. They, like us, observe the ``no robots'' convention for not trawling pages where the owner has specified in a ``robots.txt'' file that they should not. 

Various related search engine functions have been challenged, such as where Google preserves, and shows users, cached copies of pages which are no longer freely available because the owner of the material charges for access to their archive. There have also been challenges where copyrighted videos have been placed on YouTube: YouTube offers the defence that it did not know that the material was there, which is an acceptable defence (in US law) provided that YouTube promptly removes the offending material when asked to do so.  The current issue in this case (YouTube vs.\ Viacom, a large case which is ongoing as at October 2007) is how quickly and effectively YouTube responds, when the copyright owner brings it to YouTube's attention that it is hosting copyrighted material. Taking note of such cases, Lexical Computing Ltd has mechanisms for promptly removing copyrighted material from its corpora should a copyright owner objects.

There is no legal question over the legitimacy of the core activity of commercial search engines, or, thereby, of the legitimacy of presenting web corpora in the Sketch Engine.



\subsection{Corpus statistics}

In this section we give some statistics over the corpus in terms of overall size, the number of web sites and documents, and the statistics over linguistic categories. 

The total corpus size is 7.3\,GB, or 2\,GB if compressed (.zip). An
overview of the corpus size in terms of web documents is given in
the left side of \reftab{tab1}. 
The documents, just under 50 thousand, correspond to Web
pages, each one identified by its URL,
e.g.\ \url{http://www.arsvi.com/0e/\linebreak[2]ps01.\linebreak[2]htm}. The number of hosts
(16,000) corresponds to distinct Web sites, such as
\url{http://www.\linebreak[2]arsvi.com}. Dividing the former by the latter gives us
the average number of pages per host, 3.1. The last two lines in the
table give the numbers of pages from each of the two domains present
in the corpus, with three quarters of the pages being from .jp and one
quarter from .com.

It is also interesting to see what kinds of websites the documents
come from. The center of \reftab{tab1} gives the keywords, defined as alphabetic
strings appearing in URLs, which cover more than a thousand
documents. The keywords to a certain extent reflect the Web sites with
the greatest number of documents, in particular blog.livedoor.jp
(1,646 documents), www.amazon.co.jp (1,048 documents), d.hatena.ne.jp
(759 documents), and blog.goo.ne.jp (690 documents).

Linguistic processing with ChaSen gave us 12,759,201 sentences and
409,384,411 tokens; statistics over tokens per document is given on
the right side of \reftab{tab1}.

\begin{table}[t]
\begin{center}
\caption{Corpus statistics}
\input{06table1.txt}
\label{tab1}
\end{center}
\end{table}



\subsection{JpWaC validity, and comparison with a newspaper corpus}

At this point, the reader will naturally want to know what sort of language there is in the web corpus. People are often concerned that web corpora will give a partial and distorted view of a language. These are difficult concerns to address because there is no simple way to describe a collection, which is, by design, heterogeneous, and where there has been no process of assigning labels like ``journalism'' or ``novel'', to individual pages. This is a central problem in corpus linguistics \shortcite{Kilgarriff_2001}.

One straightforward strategy is to make comparisons with other
corpora. As newspaper corpora have been widely used in Japanese
language research and we had at our disposal the data from Mainichi
Shimbun newspaper for the year 2002 (hereafter News), we compared
JpWaC with that. The raw News corpus was processed with ChaSen, in the
same way as JpWaC, and has around 30 million tokens. We analyzed the
differences between the two corpora with frequency profiling, as
described in \shortcite{Rayson_2000}. The method can be used to discover
items of interest (say, key words or grammatical categories) in the
corpora, which differentiate one corpus from another. In our case, we
used it to discover lexical items (word form + ChaSen tag) or just
tags in isolation. The method is straightforward: we first produce
frequency lists and then calculate the log-likelihood statistic for
each item. LL takes into account the two frequencies of the item and
the sizes of the respective corpora and the larger it is, the more the
item is salient for one of the two corpora. Starting with the items
that have the highest LL scores, it is then possible to investigate
the major differences between JpWaC and News.

The highest LL scores resulted from differences in coding practices. Both corpora use Unicode, which allows for coding the alphanumeric characters and basic punctuation in the standard ASCII range or, for Japanese, in the ``Halfwidth Katakana'' block (U+FF00 -- U+FFEF). Similarly, the space character exists in ASCII, but also as the Unicode character ``Ideographic space'' U+3000. This space (analysed as a token with the tag Sym.w by ChaSen) is the item with the highest LL score. It appears over 24 times per 1000 words in News, while hardly ever in JpWaC. JpWaC uses, for the most part, ASCII characters for Western spaces, letters and numbers, while News uses their Eastern Unicode equivalents. Furthermore, ChaSen tokenises words written in ASCII into individual characters (and tags them with Sym.a). Being unaware of such differences can have practical consequences in cases where a researcher is interested in patterns including these characters.

\begin{table}[b]
\begin{center}
\hangcaption{Differences in ChaSen tags between the JpWaC and News (first 20 tags). The first column gives the ChaSen tag, the second the log-likelihood score, and the third and fourth the frequencies per thousand words}
\input{06table2.txt}
\label{tab2}
\end{center}
\end{table}

\reftab{tab2} shows the twenty most salient log-likelihood differences
in ChaSen tags between the two corpora. As explained, the first one
marks Ideographic space, the second ASCII letters, and the third
(Sym.g) ASCII punctuation. The fourth shows that unknown words
(i.e. words that are not in the ChaSen dictionary) are much more
frequent on the web than in the newspaper corpus. These three high LL
scores thus stem for a combination of different coding practices,
coupled with differences in ChaSen processing. The fifth row (N.Num)
shows that numerals are more frequent in News. (This is a true
difference in content, as ChaSen correctly tags both ASCII numbers and
their Eastern Unicode equivalents with N.Num.) More frequent in News
are also measure suffixes and proper nouns. Bounding nouns, pronouns,
auxiliaries and adverbs are much more frequent in JpWaC.

In JpWaC the following forms / grammatical categories are more salient than in News:
\begin{itemize}
\item auxiliary verbs forms ます，です，まし，ませ showing that sentence endings in the web data, in contrast to the newspaper data, frequently uses the masu/desu form
\item forms expressing modality か，でしょ，よう，思う，ので，わけ
\item forms expressing politeness お，ござい
\item forms expressing informal language って，よ，ね，ん
\item person and place deixis: the first personal pronoun 私， and place diexis この，その，それ
\item noun bound forms こと and もの
\end{itemize}

In the newspaper corpus the following are more salient than in JpWaC:
\begin{itemize}
\item past tense form た showing that the newspaper data is mainly written in the past tense, in contrast to the web data.
\item various suffixes and prefixes expressing time, place, numerals, measures, people 日，市，県，役，円，メートル，際，人，さん
\item adverbial nouns expressing time 昨年，後，元
\item proper nouns 東京，大阪，鈴木
\item general nouns that are specific for newspapers content and politically oriented 容疑，首相，米国，テロ，自民堂 (No nouns that are specific to web data occurred amongst the 100 highest-LL entries. We believe this is because Web data is more heterogeneous, so, while there are very many nouns which have higher relative frequency in JpWaC than News, none had high enough frequency and bias to reach the `top 100'.)
\end{itemize}

\reftab{tab3} gives the twenty words (together with their ChaSen tag)
that have the highest LL scores. Based on the top hundred words, the
analysis reveals a number of interesting differences between the
corpora.

The comparison of the two corpora shows that newspaper data is more
specific both in terms of form (being written mainly in past tense and
not using masu/desu), as well as content (high proportion of news
specific nouns). On the other hand, JpWaC contains more informal and
interactional material, and more diverse content. The distinction
between formal, distanced language and informal, interactional
language is the most salient dimension of language variation, as has
been explored and established in a number of studies by Biber and
colleagues \shortcite{Biber_1988,Biber_1995}, see also \shortcite{Heylighen_2002}.
The effect for JpWaC matches the differences that \shortcite{Sharoff_2006a} 
found when comparing web corpora with
newspaper corpora for English and German. For English, he also
considers a third point of comparison: the BNC, a corpus carefully
designed to give a balanced picture of contemporary British English
and often cited as a model for other corpus-building projects. Sharoff
finds that his web corpus (prepared using the same method we have
used) is more similar to the BNC, than either of them are to the
newspaper corpus. This suggests that JpWaC gives a fuller picture of
current Japanese than do newspaper corpora such as Mainichi Shimbun.

\begin{table}[t]
\begin{center}
\hangcaption{Differences in entries between JpWaC (left table) and News (right table): the most distinctive twenty words, according to the log-likelihood statistic (LL), in each case. Columns JpWaC and News give frequencies per thousand for the two corpora.}
\input{06table3.txt}
\label{tab3}
\end{center}
\end{table}

The findings are in accordance with several other studies that explore
the nature of web corpora. \shortcite{Kilgarriff_2003} review the field and
show that approved or correct forms are typically orders of magnitude
more common on the web than incorrect or disparaged forms. In \shortcite{Keller_2003} 
they undertake psycholinguistic studies which show that
associations between pairs of words as found on the web, using search
engine hit-counts, correspond closely to associations in the mental
lexicon (and for rarer items, the web associations do a better job of
approximating mental-lexicon associations than do extrapolations from
the BNC). 


The issue of corpus composition is crucial for many potential
users of the corpus. Consider for example dictionary publishers, who
are perhaps the users with the greatest need for large, balanced
corpora. (The impetus for the BNC came from dictionary publishing at
Oxford University Press and Longman.)  For them, newspaper corpora are
inadequate because they include only journalism, and journalism is a
relatively formal text type. Dictionaries need to cover the vast
number of informal words, phrases and constructions that are common in
the spoken language but which rarely make it into newspapers. They
would like the corpora they use to be not only very large, but also to
include a substantial proportion of informal language. In the BNC,
this was addressed by gathering and transcribing (at great expense)
five million words of conversational spoken English.

Size is also an issue. At 100 million words, the BNC was, in the early
1990s, a vast resource, which lexicographers and other language
researchers revelled in. But fifteen years on, it no longer looks
large when compared to the web, and lexicographers who have been using
it extensively at Oxford University Press and Longman, are vividly
aware that for some phenomena, it does not provide enough data. The
leading UK dictionary-makers are now all working with larger (and more
modern) corpora, of between 200 million and 2 billion
words\footnote{
	See e.g.\ \url{http://www.askoxford.com/oec}
}. At 400
million words of heterogenous Japanese, JpWaC is a good size for
contemporary dictionary making.

Our initial investigations suggest that JpWaC will be a good corpus,
in terms of size, composition, and availability, for Japanese
lexicography.  For the same reasons it will be a useful resource for
exercises such as the preparation of word lists for Japanese language
teaching and other research requiring good coverage of current
Japanese vocabulary across the full range of genres.


\subsection{Other Japanese web corpora}

We are aware of two Japanese web corpora that are comparable to
ours. \shortciteA{Ueyama_2005} used very similar methods. The
differences are simply that their corpus is smaller (at 62 million
words) and that they also classify the corpus in terms of topic
domains and genre types.

\shortciteA{Kawahara_2006} crawled the Japanese web very extensively and have
prepared a corpus around five times larger than JpWaC in form of
extracted sentences. The authors kindly allowed us access to the data,
and we have examined samples and believe it to be a high quality
resource, comparable in many ways to JpWaC.



\section{The Sketch Engine for Japanese}

In this section we describe how JpWaC was prepared for the Sketch
Engine and the development of the shallow grammatical relation
definitions to support word sketching.

Loading a corpus into the Sketch Engine enables the use of standard
functions, such as concordances, which include searching for phrases,
collocates, grammatical patterns, sorting concordances according to
various criteria, identifying ``subcorpora'' etc. In addition, after
defining the grammatical relations and loading them into Sketch
Engine, the tool finds all the grammatical relation instances and
offers access to the more advanced functions: word sketches,
thesaurus, and sketch diffs.


\subsection{Loading the corpus}

The Sketch Engine supports loading of any corpora of any language,
either on the command line for local installations of the software, or
using the `CorpusBuilder' web interface. In order to create good word
sketch results, the corpus should be lemmatized and PoS-tagged. (It is
also possible to apply the tool to word forms only, which can still
give useful output.)

The input format of the corpus is based on the ``word per line''
standard developed for the Stuttgart Corpus Tools \shortcite{Christ_1994}. Each
word is on a new line, and for each word, there can be a number of
fields specifying further information about the word, separated by
tabs. The fields of interest here are word form, PoS-tag and
lemma. Other constituents such as paragraphs, sentences and documents,
can be added with associated attribute-value pairs in XML-like
format\footnote{
	Full documentation is available at the Sketch Engine
website (\url{http://www.skechengine.co.uk}).
}. An example is given in
Figure 2.

As described above, JpWaC had been lemmatized and part-of-speech
tagged with the ChaSen tool. It was then converted into word-per-line
format.


\subsection{Preparing grammatical relations}

For producing word sketches, thesaurus and sketch diffs grammatical
relations need to be defined. This mini-grammar of syntactic patterns
enables the system to automatically identify possible relations of
words to a given keyword. It is language and tagset-specific. The
formalism uses regular expressions over PoS-tags. As an example we
give below a simple definition for an adjective modifier relation:

\texttt{=modifies\\
\indent\indent 
2:[tag= "Ai"] 1:[tag="N"]
}

The grammatical relation states that, if we find an adjective (PoS tag
Ai) immediately followed by a noun (PoS tag starts with N), then a
\textit{modifies} relation holds between the noun (labelled 1) and the
adjective (labelled 2).

A more complex definition, as used in JpWaC, is given below. Here we define a pair of relations, \textit{modifier\_Ai }and \textit{modifies\_N}, which are duals of each other: if \textit{w1} is in the relation \textit{modifier\_Ai} to \textit{w2}, then \textit{w2} is in the relation \textit{modifies\_N }to \textit{w1. }We start by declaring that we have two `dual' relations, and give them their names. Then comes the pattern itself, with the two arguments for the two relations labelled 2 and 1. The pattern excludes the \textit{nai} adjectival form (using the operator ``!'') and possibly includes a prefix before the noun (using the operator ``?''). Since the tag ``N.*'' includes also suffixes and bound nouns, we exclude these from the results.

\texttt{
*DUAL\\
\indent\indent
=modifier\_Ai/modifies\_N\\
\indent\indent 
2:[tag="Ai.*" \& word!="ない|無い"] [tag="Pref.*"]?
\\\indent\indent 1:[tag="N.*" \& tag!="N.Suff.*" \& tag!="N.bnd.*"]}

\begin{figure}[t]
\begin{center}
\begin{minipage}{240pt}
\begin{verbatim}
<doc id="http://www.0start-hp.com/voice/index.php">
<s>
月々   月々    N.Adv
２   ２    N.Num
６   ６    N.Num
３   ３    N.Num
円   円    N.Suff.msr
で   だ    Aux
、   、    Sym.c
あなた   あなた    N.Pron.g
も   も    P.bind
ブログデビュー  ブログデビュー  Unknown
し   する    V.free
て   て    P.Conj
み   みる    V.bnd
ませ   ます    Aux
ん   ん    Aux
か   か    P.advcoordfin
？   ？    Sym.g
</s>
\end{verbatim}
\end{minipage}
\caption{An example stretch of the corpus}
\label{fig2}
\end{center}
\end{figure}

The Sketch Engine finds all matches for these patterns in the corpus and stores them in a database, complete with the corpus position where the match was located. The database is used for preparing (at run time) word sketches and sketch differences and (at compile time) a thesaurus. When the user calls up a word sketch for a noun, the software counts how often each adjective occurs in the \textit{modifier\_Ai} relation with the noun, and computes the salience of the adjective for the noun using a salience formula based on the Dice co-efficient. If the adjective is above the salience threshold,\footnote{
	The adjective must also occur with the noun above the frequency threshold, and must be one of the N (default = 25) highest-scoring adjectives meeting these criteria.
} it appears in the word sketch.

As mentioned, the Japanese grammatical relations are prepared using
the ChaSen PoS tags and tokens. The ChaSen tags are quite detailed (88 tags), and the tokenisation is ``narrow'': it splits inflectional morphemes from their stems. This has advantages and disadvantages in the creation of grammatical relations. The advantage is that by using already precisely defined tags and small tokens, it is easier to define desired patterns and the need to specify additional constraints is lower. The main disadvantage is that sometimes a targeted string is divided into several tokens by the analyzer, making it more difficult to define patterns and impossible to retrieve some types of results (for example, 女の人is divided into 3 tokens, \textit{onn}\textit{a}\textit{-no}\textit{-}\textit{hito, }and is therefore not considered as a unit by the system). However, it is possible to search for this kind of strings in the Concordance window, using the CQL functionality.

We defined 22 grammatical relations for Japanese, mostly using the ``dual'' type. There is also one symmetric relation (where a match for relation \textit{R} between \textit{w1} and \textit{w2} is also a match for \textit{R} between \textit{w2} and \textit{w1}, and one unary relation (involving only one word). Although the grammatical relations for other languages in the Sketch Engine name relations by their functions, such as subjects, objects, we found it easier to remain on the level of particles (が,は,を etc.) and avoid the complexity of topic vs.\ subject functions - differences between は and が particles. A similar approach is seen in \shortcite{Kawahara_2006}.

Since the grammatical relations formalism is sequence-based, it is
better suited to languages with fixed word order, such as
English. There are already some reports on addressing the problem of
free word order in creation of Czech and Slovene word sketches
\shortcite{Kilgarriff_2004,Krek_2006}, where the simple mechanism of gaps in
the patterns is employed as one of the solutions. We also use it for
the Japanese word sketches. In the example below, up to 5 tokens are
allowed, using the notation []\{0,5\}, between the case particle で
and the corresponding verb.

\texttt{
*DUAL\\
\indent\indent
=nounで/でverb\\
\indent\indent 
    2:[tag="N.*"][tag="P.c.g" \& word="で"][ ]\{0,5\} \\\indent\indent  1:[tag="V.*"]|[tag="N.Vs"][tag="V.*"]}

This kind of pattern covers an example such as: 

授業　で　，　少し　だけ　講演　さ　せ　て　頂き　まし　た．\\
\indent N.*  P.c.g\& で [ ]\{0,5\}  N.Vs V.*\ldots 

While for some languages trinary relations (between three dependent items) were useful for identifying prepositional phrases, we defined prepositional phrases employing dual relations. In this way, we were able to specify constraints relevant for a specific particle, which gave higher precision output. On the other hand, to define grammatical relations for the phrasal verbs in Japanese (for example, to query most relevant objects and subjects of idioms, such as S が O を気に入る) the existing relations proved to be too weak, as they are limited to 3-token relations. While these relations are not displayed in the word sketch, they can be found using the `sort collocations' functionality in the concordancer.

The grammatical relations would ideally be more sophisticated, but we
have found that very simple definitions, while linguistically
unambitious, produce good results. Linguistically complex instances
are missed when using simple definitions, but it is generally the case
that a small number of simple patterns cover a high proportion of
instances, so the majority of high salience collocates are readily
found, given a large enough corpus \shortcite{Kilgarriff_2004}. For
Japanese, as for other languages, PoS-tagging errors cause more
anomalous output than do weaknesses in the grammar. However these
errors are rare and the quality of the word sketch output is good. We
evaluated the output of six items
（元気，閉める，女の子，多分，温かい，温泉）
that had all together approx. 1000 collocational instances
(grammatical relations' instances) in the word sketches output. The
result showed that only seven output mistakes were due to PoS-tagging
errors (for example,
温泉へ\underline{行う}（おこなう）
instead of \underline{行く}（いく）) and four of them were
due to a shortcoming of the gramrel specification. Fourteen cases
offered valuable collocational information but were incomplete due to
ChaSen's narrow tokenization
（\underline{っぽい}の女の子）. Nonetheless,
in future versions, we plan to add more advanced relations into the
system and cover also the instances that are now missed and not able
to search for (for example, mutli-word units and `suru' verbs).


\subsection{Word sketches}

The word sketch for a word presents a list of all of its salient collocates, organised by the grammatical relations holding between word and collocate. The grammatical relations are as named and defined in the previous section, and the collocates are as found in the corpus. For each collocate listed, the word sketch provides:

\begin{itemize}
\item the statistical salience and frequency with which keyword and collocate occur together; 
\item links to concordances, so we can explore the pattern by looking at the corpus examples.
\end{itemize}

The word sketch also provides links to grammatical relations, where we can see how the pattern is defined inside the system.

We presented word sketches for the noun お湯 in the opening section; here we give another example, this time for the verb 閉める (\textit{shimeru}) (Figure 3). Here the grammatical relations reveal different noun-particle-verb relations, such as ドアを閉める，カーテンは閉める，カーテン・扉・店が閉められる，後ろ手・鍵で閉める．We can also easily find the most relevant bound verbs that appear with the verb: 閉めきる，閉め直す，閉めてくれる・いただく・もらう etc. Adverbs that usually modify the verb, しっかり，きちんと，必ずimply that the action is done/should be done firmly, tightly, definitely. The suffixes that appear with the verb （られる，させる，っぱなし） suggest the frequent passive and causative usage of the verb. After checking the concordance, we can also select a number of useful usage examples:ドアがきちんと閉められています，雨戸を閉めさせる，カーテンを閉めっぱなしにする).

\begin{figure}[t]
\begin{center}
\includegraphics{15-2ia6f3.eps}
\caption{Word sketch for the verb 閉める
(\textit{shimeru}, `to close')}
\label{fig3}
\end{center}
\end{figure}



\subsection{Thesaurus}

The semantic similarity in the Sketch Engine is based on ``shared
triples'' (for example, 雑誌 and 本 share the same triple
$\langle$?を読む$\rangle$). When we find a pair of
grammatical relation instances, such as
$\langle$雑誌を読む$\rangle$ and
$\langle$本を読む$\rangle$ with high salience for both
words, 本 and 雑誌, we use it as a piece of evidence for
assuming the words belong to the same thesaurus category. The
thesaurus is built by computing ``nearest neighbours'' for each word,
and based on the tradition of automatic thesaurus building
\shortcite{Sparck_1986,Grefenstette_1994,Lin_1998}. We present a thesaurus
entry for the word お湯 in Figure 4.

\begin{figure}[b]
\begin{center}
\includegraphics{15-2ia6f4.eps}
\caption{Thesaurus for the noun お湯 (\textit{oyu})}
\label{fig4}
\end{center}
\end{figure}

As can be seen from the list of the words in the thesaurus, they can also suggest different senses of a word. In the case of お湯, these are `bath' （風呂） and お湯 as a liquid `hot/warm water' （液体，熱湯） (see also Section 1). The thesaurus also reveals that the word is semantically most similar to the word 湯(\textit{yu}), which is actually a variation of the お湯(\textit{oyu}), and adds on an additional sense `hot spring' （温泉）．It also offers sets of related words belonging to the same semantic domains, indicating that お湯is semantically related to food/drink and its preparation （スープ，牛乳，お茶，紅茶，ご飯，鍋など），to water flow/liquids （液体，液，海水，水道），to bathing （シャワー，汗，湯船） etc. It also relates to 水，熱湯 and its antonym 冷水．



\subsection{Sketch Differences}

The difference between two near-synonyms can be identified as the triples that have high salience for one word, but no occurrences (or low salience) for the other. Based on this type of data on various grammatical relations and their salience, a one-page summary for sketch differences between two semantically similar words can be presented. The system is also useful for showing differences in language usage for words that are considered semantically similar but different orthographically, for example 良い (\textit{yoi/ii}) and いい(\textit{ii}), as well as for showing differences of transitive/intransitive semantic pairs, such as 閉める・閉まる(\textit{shimeru/shimaru}).

The sketch difference summary offers the list of collocates that are common for the comparing pair showing their salience and frequency variance, as well as the list of collocates that appear only with one word of the comparing pair. 

Figure 5 shows partial results of the sketch difference for 女の子 (\textit{onna no ko}, `girl') and 男の子 (\textit{otoko no ko}, `boy'). Ai adjective-noun relations that are common to both (common patterns), and that apply only to one of the words (``女の子'' only patterns and ``男の子'' only patterns) are displayed. From the results we can see that 可愛い，いい，although common to both, is more present as the collocation to 女の子．``Only patterns'' reveal that かわいらしい，美しい，強い collocate only with 女の子 and thatカッコイイ，かっこいい，弱い appears only with 男の子．

Before we presented word sketches for the word お湯．Comparing it with its coordinate noun 水in the sketch difference shows clearly that only 水collocates with 冷たい，おいしい and only お湯 with 熱い．

\begin{figure}[b]
\begin{center}
\includegraphics{15-2ia6f5.eps}
\caption{Sketch difference for 女の子 and 男の子 (partial)}
\label{fig5}
\end{center}
\end{figure}



\section{Evaluation}

We evaluated the word sketches results by comparing them to ten randomly
selected entries in the Japanese collocational dictionary
『日本語表現活用辞典』
\shortcite{Himeno_2004}. Here we briefly summarize the results, for a
detailed description please refer to \shortcite{Srdanovic_2008}.

The first difference to be noticed is in the number of grammatical relations holding between the words of a collocation, or collocation types. The Sketch Engine offers a richer set. As well as Ana adjectives and verbs, there are nouns, Ai adjectives, adverbs and others. Also for one word class there is a richer variety of collocations. For example, for verbs, there are not just collocations with が，を，と，にas in the dictionary, but also withで，まで，から，へ and はparticles. 

The comparison also suggests that the word sketch is a useful tool for selecting the most relevant collocations---for example, the very frequent かすかな記憶was missed in the dictionary. There are also some collocations in the dictionary, which were not present (or not regarded as significant) in the JpWaC word sketches. For verbs 閉める and 閉まる we find collocations such asドア，まど，カーテン, but we do not find 襖 and 障子 which are present in the dictionary. This suggests differences in the corpora used (the dictionary uses newspaper corpus, modern literature etc.) and indicates that language change has occurred and these two collocations are no longer current. 

In addition, the dictionary usually offers examples for most significant collocations. The Sketch Engine is useful for finding good examples because the user can click on the word in the word sketch to see a concordance of the sentences that the collocation occurs in. 

Lastly, Thesaurus and Sketch Diff functions can also be used to easily obtain relevant data on similar words. This type of data can be used for cross-references and to show the differences between similar words, which are rarely captured in the dictionary. Although it seems that 閉める and 閉まる share almost the same collocations, some differences can be rapidly found in the Sketch Diff---for example, レストラン，商店，図書館 are used only with 閉まる．

In this evaluation, we concentrated on collocational dictionaries and confirmed that word sketches are a useful tool in compiling such dictionaries. As has already been shown for other languages, the application of the tool is in fact broader---in lexicography, for compilation of various dictionary types, as well as for NLP, language research, and language teaching.



\section{Conclusion and further work}

In this paper we presented how JpWaC, a 400-million-word Japanese web corpus, and a set of Japanese grammatical relations were created and employed inside the Sketch Engine. The Sketch Engine uses grammatical relations (defined with regular expressions over part-of-speech tags) and lexical statistics, applied to a large corpus, to find useful linguistic information: the most salient collocation and grammatical patterns for a word. The tool has already proved to be useful for English and other languages, and we believe that the Japanese version of the tool is a step forward in corpus-based lexicography, language learning, and linguistic research for Japanese. Its possible application in the various fields is investigated and exemplified in \shortcite{Srdanovic_2008}. 

As future work, we plan to make the system user-friendly for both native-speakers and learners of Japanese, by providing a Japanese interface and by offering option to choose between English and Japanese tag sets and grammatical relation names, and by providing the corpus also in furigana and romaji transcriptions. We shall also enrich the grammatical relation set.

We also aim to add some other Japanese corpora into the system, which
among other things would be interesting from the point of view of
comparing various corpora. Currently a long-term corpus development
project is in progress at the National Institute of the Japanese
Language \shortcite{Maekawa_2006}. Loading these corpora into the Sketch
Engine tool is being considered \shortcite{Tono_2007}. We will also consider
possible benefits of implementing some other morphological and
structural analysers for the Japanese language. Finally, we shall
explore a direct application of the system to the creation of
learner's dictionaries \shortcite{Erjavec_2006} and CALL systems \shortcite{Nishina_2007}.

\acknowledgment

The authors would like to thank Serge Sharoff for providing the URL
list, which served as the basis for constructing the JpWaC corpus, all
the collaborators of the WAC project for making their software
available, and the anonymous reviewers for their useful comments.


\bibliographystyle{jnlpbbl_1.3}
\begin{thebibliography}{}

\bibitem[\protect\BCAY{Baroni \BBA\ Bernardini}{Baroni \BBA\
  Bernardini}{2004}]{Baroni_2004}
Baroni, M.\BBACOMMA\ \BBA\ Bernardini, S. \BBOP 2004\BBCP.
\newblock \BBOQ BootCat: Bootstrapping corpora and terms from the web\BBCQ\
\newblock In {\Bem Proceedings of the Fourth Language Resources and Evaluation
  Conference, LREC2004}\ Lisbon.

\bibitem[\protect\BCAY{Baroni \BBA\ Bernardini}{Baroni \BBA\
  Bernardini}{2006}]{Baroni_2006b}
Baroni, M.\BBACOMMA\ \BBA\ Bernardini, S.\BEDS\ \BBOP 2006\BBCP.
\newblock {\Bem Wacky! Working papers on the Web as Corpus}.
\newblock GEDIT, Bologna.

\bibitem[\protect\BCAY{Baroni \BBA\ Kilgarriff}{Baroni \BBA\
  Kilgarriff}{2006}]{Baroni_2006c}
Baroni, M.\BBACOMMA\ \BBA\ Kilgarriff, A. \BBOP 2006\BBCP.
\newblock \BBOQ Large linguistically-processed Web corpora for multiple
  languages\BBCQ\
\newblock In {\Bem Proceedings EACL}\ Trento, Italy.

\bibitem[\protect\BCAY{Baroni, Kilgarriff, Pomik\'{a}lek, \BBA\
  Rychl\'{y}}{Baroni et~al.}{2006}]{Baroni_2006a}
Baroni, M., Kilgarriff, A., Pomik\'{a}lek, J., \BBA\ Rychl\'{y}, P. \BBOP
  2006\BBCP.
\newblock \BBOQ WebBootCaT: instant domain-specific corpora to support human
  translators\BBCQ\
\newblock In {\Bem Proceedings of EAMT 2006}, \mbox{\BPGS\ 47--252}\ Oslo.

\bibitem[\protect\BCAY{Biber}{Biber}{1988}]{Biber_1988}
Biber, D. \BBOP 1988\BBCP.
\newblock {\Bem Variation across speech and writing}.
\newblock Cambridge University Press, Cambridge.

\bibitem[\protect\BCAY{Biber}{Biber}{1995}]{Biber_1995}
Biber, D. \BBOP 1995\BBCP.
\newblock {\Bem Dimensions of Register Variation. A Cross-Linguistic
  Comparison}.
\newblock Cambridge University Press, Cambridge.

\bibitem[\protect\BCAY{Chantree, de~Roeck, Kilgarriff, \BBA\ Willis}{Chantree
  et~al.}{2005}]{Chantree_2005}
Chantree, F., de~Roeck, A., Kilgarriff, A., \BBA\ Willis, A. \BBOP 2005\BBCP.
\newblock \BBOQ Disambiguating Coordinations Using Word Distribution
  Information\BBCQ\
\newblock In {\Bem Proceedings RANLP}\ Bulgaria.

\bibitem[\protect\BCAY{Chen, Rychl\'{y}, Huang, Kilgarriff, \BBA\ Smith}{Chen
  et~al.}{2007}]{Chen_2007}
Chen, A., Rychl\'{y}, P., Huang, C.-R., Kilgarriff, A., \BBA\ Smith, S. \BBOP
  2007\BBCP.
\newblock \BBOQ A corpus query tool for SLA: learning Mandarin with the help of
  Sketch Engine\BBCQ\
\newblock In {\Bem Practical Applications of Language Corpora (PALC)}\ Lodz,
  Poland.

\bibitem[\protect\BCAY{Christ}{Christ}{1994}]{Christ_1994}
Christ, O. \BBOP 1994\BBCP.
\newblock \BBOQ A modular and flexible architecture for an integrated corpus
  query system\BBCQ\
\newblock In {\Bem COMPLEX' 94}\ Budapest.

\bibitem[\protect\BCAY{Erjavec, Hmeljak, \BBA\ Srdanovi\'{c}}{Erjavec
  et~al.}{2006}]{Erjavec_2006}
Erjavec, T., Hmeljak, K.~S., \BBA\ Srdanovi\'{c}, I.~E. \BBOP 2006\BBCP.
\newblock \BBOQ jaSlo, A Japanese-Slovene Learners' Dictionary: Methods for
  Dictionary Enhancement\BBCQ\
\newblock In {\Bem Proceedings of the 12th EURALEX International Congress}\
  Turin, Italy.

\bibitem[\protect\BCAY{Gatt \BBA\ van Deemter}{Gatt \BBA\ van
  Deemter}{2006}]{Gatt_2006}
Gatt, A.\BBACOMMA\ \BBA\ van Deemter, K. \BBOP 2006\BBCP.
\newblock \BBOQ Conceptual coherence in the generation of referring
  expressions\BBCQ\
\newblock In {\Bem Proceedings of the COLING-ACL 2006 Main Conference Poster
  Session}.

\bibitem[\protect\BCAY{Grefenstette}{Grefenstette}{1994}]{Grefenstette_1994}
Grefenstette, G. \BBOP 1994\BBCP.
\newblock {\Bem Explorations in Automatic Thesaurus Discovery}.
\newblock Kluwer.

\bibitem[\protect\BCAY{Heylighen}{Heylighen}{2002}]{Heylighen_2002}
Heylighen, F. \BBOP 2002\BBCP.
\newblock \BBOQ Variation in the Contextuality of Language: An Empirical
  Measure\BBCQ\
\newblock {\Bem Foundations of Science}, {\Bbf 7}  (3), \mbox{\BPGS\ 293--340}.

\bibitem[\protect\BCAY{Himeno}{Himeno}{2004}]{Himeno_2004}
Himeno, M. \BBOP 2004\BBCP.
\newblock {\Bem Nihongo hyougen katsuyou jiten}.
\newblock Kenkyusha.

\bibitem[\protect\BCAY{Kawahara \BBA\ Kurohashi}{Kawahara \BBA\
  Kurohashi}{2006}]{Kawahara_2006}
Kawahara, D.\BBACOMMA\ \BBA\ Kurohashi, S. \BBOP 2006\BBCP.
\newblock \BBOQ Case Frame Compilation from the Web using High-Performance
  Computing\BBCQ\
\newblock In {\Bem Proceedings LREC}\ Genoa, Italy.

\bibitem[\protect\BCAY{Keller \BBA\ Lapata}{Keller \BBA\
  Lapata}{2003}]{Keller_2003}
Keller, F.\BBACOMMA\ \BBA\ Lapata, M. \BBOP 2003\BBCP.
\newblock \BBOQ Using the Web to Obtain Frequencies for Unseen Bigrams\BBCQ\
\newblock {\Bem Computational Linguistics}, {\Bbf 29}  (3), \mbox{\BPGS\
  459--484}.

\bibitem[\protect\BCAY{Kilgarriff}{Kilgarriff}{2001}]{Kilgarriff_2001}
Kilgarriff, A. \BBOP 2001\BBCP.
\newblock \BBOQ Comparing Corpora\BBCQ\
\newblock {\Bem International Journal of Corpus Linguistics}, {\Bbf 6}  (1),
  \mbox{\BPGS\ 1--37}.

\bibitem[\protect\BCAY{Kilgarriff \BBA\ Grefenstette}{Kilgarriff \BBA\
  Grefenstette}{2003}]{Kilgarriff_2003}
Kilgarriff, A.\BBACOMMA\ \BBA\ Grefenstette, G. \BBOP 2003\BBCP.
\newblock \BBOQ Introduction to the Special Issue on Web as Corpus\BBCQ\
\newblock {\Bem Computational Linguistics}, {\Bbf 29}  (3).

\bibitem[\protect\BCAY{Kilgarriff \BBA\ Rundell}{Kilgarriff \BBA\
  Rundell}{2002}]{Kilgarriff_2002}
Kilgarriff, A.\BBACOMMA\ \BBA\ Rundell, M. \BBOP 2002\BBCP.
\newblock \BBOQ Lexical profiling software and its lexicographic
  applications---a case study\BBCQ\
\newblock In {\Bem Proceedings EURALEX}, \mbox{\BPGS\ 807--818}\ Copenhagen.

\bibitem[\protect\BCAY{Kilgarriff, Rychly, Smr\v{z}, \BBA\ Tugwell}{Kilgarriff
  et~al.}{2004}]{Kilgarriff_2004}
Kilgarriff, A., Rychly, P., Smr\v{z}, P., \BBA\ Tugwell, D. \BBOP 2004\BBCP.
\newblock \BBOQ The Sketch Engine\BBCQ\
\newblock In {\Bem Proceedings EURALEX}, \mbox{\BPGS\ 105--116}\ Lorient,
  France.

\bibitem[\protect\BCAY{Krek \BBA\ Kilgarriff}{Krek \BBA\
  Kilgarriff}{2006}]{Krek_2006}
Krek, S.\BBACOMMA\ \BBA\ Kilgarriff, A. \BBOP 2006\BBCP.
\newblock \BBOQ Slovene Word Sketches\BBCQ\
\newblock In {\Bem Proceedings 5th Slovenian/First International Languages
  Technology Conference}\ Ljubljana, Slovenia.

\bibitem[\protect\BCAY{Lin}{Lin}{1998}]{Lin_1998}
Lin, D. \BBOP 1998\BBCP.
\newblock \BBOQ Automatic retrieval; and clustering of similar words\BBCQ\
\newblock In {\Bem Proceedings COLING-ACL}, \mbox{\BPGS\ 768--774}\ Montreal.

\bibitem[\protect\BCAY{Maekawa}{Maekawa}{2006}]{Maekawa_2006}
Maekawa, K. \BBOP 2006\BBCP.
\newblock \BBOQ Kotonoha. The Corpus Development Project of the National
  Institute for Japanese Language\BBCQ\
\newblock In {\Bem Proceedings of the 13th NIJL International Symposium:
  Language Corpora: Their Compilation and Application}, \mbox{\BPGS\ 55--62}\
  Tokyo.

\bibitem[\protect\BCAY{Nishina \BBA\ Yoshihashi}{Nishina \BBA\
  Yoshihashi}{2007}]{Nishina_2007}
Nishina, K.\BBACOMMA\ \BBA\ Yoshihashi, K. \BBOP 2007\BBCP.
\newblock \BBOQ Japanese Composition Support System Displaying Occurrences and
  Example Sentences\BBCQ\
\newblock In {\Bem Symposium on Large-scale Knowledge Resources (LKR2007)},
  \mbox{\BPGS\ 119--122}.

\bibitem[\protect\BCAY{Rayson \BBA\ Garside}{Rayson \BBA\
  Garside}{2000}]{Rayson_2000}
Rayson, P.\BBACOMMA\ \BBA\ Garside, R. \BBOP 2000\BBCP.
\newblock \BBOQ Comparing corpora using frequency profiling\BBCQ\
\newblock In {\Bem Proceedings of the ACL Workshop on Comparing Corpora},
  \mbox{\BPGS\ 1--6}\ Hong Kong.

\bibitem[\protect\BCAY{Sharoff}{Sharoff}{2006a}]{Sharoff_2006a}
Sharoff, S. \BBOP 2006a\BBCP.
\newblock \BBOQ Creating general-purpose corpora using automated search engine
  queries\BBCQ\
\newblock In {\Bem WaCky! Working papers on the Web as Corpus}. GEDIT, Bologna.

\bibitem[\protect\BCAY{Sharoff}{Sharoff}{2006b}]{Sharoff_2006b}
Sharoff, S. \BBOP 2006b\BBCP.
\newblock \BBOQ Open-source corpora: using the net to fish for linguistic
  data\BBCQ\
\newblock {\Bem International Journal of Corpus Linguistics}, {\Bbf 11}  (4),
  \mbox{\BPGS\ 435--462}.

\bibitem[\protect\BCAY{Sparck}{Sparck}{1986}]{Sparck_1986}
Sparck, K.~J. \BBOP 1986\BBCP.
\newblock {\Bem Synonymy and Semantic Classification}.
\newblock Edinburgh University Press.

\bibitem[\protect\BCAY{Srdanovi\'c \BBA\ Nishina}{Srdanovi\'c \BBA\
  Nishina}{2008}]{Srdanovic_2008}
Srdanovi\'c, I.~E.\BBACOMMA\ \BBA\ Nishina, K. \BBOP 2008\BBCP.
\newblock \BBOQ Ko-pasu kensaku tsu-ru Sketch Engine no nihongoban to sono
  riyou houhou (The Sketch Engine corpus query tool for Japanese and its
  possible applications)\BBCQ\
\newblock {\Bem Nihongo kagaku (Japanese Linguistics)}, {\Bbf 24}, \mbox{\BPGS\
  59--80}.

\bibitem[\protect\BCAY{Tono}{Tono}{2007}]{Tono_2007}
Tono, Y. \BBOP 2007\BBCP.
\newblock \BBOQ Nihongo ko-pasu de no Sketch Engine jissou no kokoromi (Using
  the Sketch Engine for Japanese Corpora)\BBCQ\
\newblock In {\Bem Tokutei ryouiki kenkyuu ``Nihongo ko-pasu'' Heisei 18 nendo
  koukai wa-kushoppu (Kenkyuu seika houkokukai) yokoushuu). Monbukagakusho
  kagaku kenkyuuhi tokutei ryouiki kenkyuu ``Nihongo ko-pasu''}, \mbox{\BPGS\
  109--112}\ Soukatsuhan.

\bibitem[\protect\BCAY{Ueyama \BBA\ Baroni}{Ueyama \BBA\
  Baroni}{2005}]{Ueyama_2005}
Ueyama, M.\BBACOMMA\ \BBA\ Baroni, M. \BBOP 2005\BBCP.
\newblock \BBOQ Automated construction and evaluation of a Japanese web-based
  reference corpus\BBCQ\
\newblock In {\Bem Proceedings of Corpus Linguistics 2005}\ Birmingham.

\bibitem[\protect\BCAY{Weeds \BBA\ Weir}{Weeds \BBA\ Weir}{2005}]{Weeds_2005}
Weeds, J.\BBACOMMA\ \BBA\ Weir, D. \BBOP 2005\BBCP.
\newblock \BBOQ Co-occurrence Retrieval: A Flexible Framework for Lexical
  Distributional Similarity\BBCQ.

\end{thebibliography}


\begin{biography}

\bioauthor[:]{Irena Srdanovi\'{c} Erjavec}{
received the Bachelor degree in Japanese
Language from University of Belgrade in 1997, and Master degree in
Linguistics from Univesity of Ljubljana in 2005. Since April 2007 she
has been a PhD student at Human System Science Department at Tokyo
Institute of Technology. From 1997 to 2001, she worked as Japanese
language advisor and technical writer at Hermes SoftLab in Ljubljana,
from 2001 to 2002, as a translator and international trainee
coordinator at Pioneer Corporation in Tokyo, and from 2005 to 2006 as
a teacher assistant for Japanese Language at University of
Ljubljana. Her research interests lie in the fields of corpus
linguistics, lexicography and Japanese language education, particulary
concentrating on application of human language technologies.
}

\bioauthor[:]{Toma\v{z} Erjavec}{
received his BSc (1984), MSc (1990), and PhD (1997)
degrees in Computer Science from the University of Ljubljana; he also
received an M.Sc. in Cognitive Science (1992) from the University of
Edinburgh. He works as a scientific associate at the Dept.\ of
Knowledge Technologies at the research Institute Jo\v{z}ef Stefan,
with teaching positions at several universities. He has been a
visiting researcher at the University of Edinburgh, University of Tokyo
and the Joint Research Center of the European Commission. His research
interests lie in the fields of computational linguistics and language
technologies, with a large part of the work devoted to developing
Slovene and multilingual language resources. He has
served two terms on the Board of the EACL, has been a member of the
Text Encoding Initiative Council and was the founding president of the
Slovenian Language Technologies Society. 
See also http://nl.ijs.si/et/
}

\bioauthor[:]{Adam Kilgarriff}{
Director of Lexical Computing Ltd. which has
developed the Sketch Engine http://www.sketchengine.co.uk, a leading
tool for corpus research.  His scientific interests lie at the
intersection of computational linguistics, corpus linguistics, and
dictionary-making.  Following a PhD on ``Polysemy'' from Sussex
University, he has worked at Longman Dictionaries, Oxford University
Press, and the University of Brighton, and is now Director of the
Lexicography MasterClass (http://www.lexmasterclass.com) as well as
Lexical Computing Ltd. He is Visiting Research Fellow at the
Universities of Leeds and Sussex. He started the SENSEVAL initiative
on automatic word sense disambiguation and is now active in moves to
make the web available as a linguists' corpus. He is the founding
chair of ACL-SIGWAC (Association for Computational Linguistics Special
Interest Group on Web as Corpus) and has been chair of ACL-SIG on the
lexicon and Board member of EURALEX (European Association for
Lexicography). See also http://www.kilgarriff.co.uk/ 
}
\end{biography}

\biodate


\end{document}
