\documentclass[english,tombow]{jnlp_2.3}

\usepackage{jnlpbbl_1.3}
\usepackage[dvipdfmx]{graphicx}
\usepackage{amsmath,amssymb,bm}
\usepackage{hangcaption_jnlp}
\usepackage{array}


\newcommand\parentheses[1]{\left(#1\right)}
\newcommand\norm[1]{\left\lVert#1\right\rVert}
    \newcommand\citet[1]{\citeA{#1}}
    \newcommand\citep[1]{\cite{#1}}
\DeclareMathOperator\similarity{sim}
\DeclareMathOperator\rarity{rar}
\DeclareMathOperator\figurativeness{fig}
\DeclareMathOperator\overallscore{os}
\DeclareMathOperator\pmi{PMI}
\usepackage{gb4e,cgloss4e}
\renewcommand{\thexnumi}{}
\renewcommand\eachwordone{\normalsize\upshape}
\renewcommand\eachwordtwo{\normalsize\itshape}
\renewcommand\eachwordthree{\normalsize\upshape}
\setlength{\glossglue}{5pt plus 2pt minus 1pt}
\noautomath
    \newcommand{\formulalabel}[1]{}\newcommand{\formularef}[1]{}
    \newcommand{\exprlabel}[1]{}\newcommand{\exprref}[1]{}
    \newcommand{\figlabel}[1]{}\newcommand{\figref}[1]{}
    \newcommand{\tablabel}[1]{}\newcommand{\tabref}[1]{}
    \newcommand{\seclabel}[1]{}\newcommand{\secref}[1]{}

\Volume{26}
\Number{2}
\Month{June}
\Year{2019}

\received{2018}{10}{31}
\revised{2019}{2}{18}
\accepted{2019}{3}{2}

\setcounter{page}{277}

\esubject{Paper}
\etitle{Automatically Computable Metrics to Generate Metaphorical Verb Expressions}
\eauthor{Akira Miyazawa\affiref{Author_1} \and Yusuke Miyao\affiref{Author_2}}
\eabstract{
The automatic generation of metaphorical expressions helps us write imaginative texts such as poems or novels. This paper proposes a new metaphor generation task, evaluation metrics, and a method to solve the task. Our task is formalized as a problem of finding metaphorical paraphrases for a literal Japanese phrase consisting of a subject, an object, and a verb. We use four evaluation metrics: \emph{synonymousness}, \emph{metaphoricity}, \emph{novelty}, and \emph{comprehensibility}. Our proposed method generates metaphorical expressions by using three automatically computable scores---\emph{similarity}, \emph{figurativeness}, and \emph{rarity}---corresponding to one of the evaluation metrics. By crowdsourcing, we show how these scores are related to those given by humans in terms of the evaluation metrics and how they are useful in finding human's preferred expressions in pairwise comparisons.
}
\ekeywords{Metaphor, Text Generation}

\headauthor{Miyazawa and Miyao}
\headtitle{Metrics to Generate Metaphorical Verb Expressions}

\affilabel{Author_1}{}{The Graduate University for Advanced Studies/National Institute of Informatics}
\affilabel{Author_2}{}{The University of Tokyo}


\begin{document}

\maketitle

\section{Introduction}

Metaphor is an essential tool in writing imaginative works such as poems or novels.
In a practical sense, for example, metaphors can make political speeches more persuasive or attractive \citep{black2011}.
It can be also useful in educating learners and helping them understand through analogy between new ideas and familiar things \citep{williams1986}.
The automatic generation of metaphors would help us use effective ones that are appropriate to the situation.

Automatic metaphor generation poses an advantage in creating novel metaphors.
Once metaphorical expressions become part of general usage, they lose their rhetorical effect and are, therefore, called \emph{dead metaphors} \citep{richards1965,deignan2005}.
Such expressions are easy to conjure or find in ordinary dictionaries or thesauri.
On the other hand, novel or ``living'' metaphors are not, by their nature.
This causes a demand for metaphor generation systems.
Thus we focus on producing such ``living'' metaphors.

Verbs are a part of speech that is often used metaphorically \citep{steen2010}.
For example, we describe emotions using such verbs as ``hatred burns'' or ``being starved for love.''
In the context of computers, we use ``spawn'' and ``kill'' when talking about processes.
We can generate various metaphorical expressions even if we only care about verbal expressions.
On the other hand, previous works in the field have focused on ``\emph{A} like \emph{B}'' similes where \emph{A} and \emph{B} are nouns \citep{abe2006,terai2010}.
The differences in the types of target cause some difficulties.
First, we need to be mindful whether generated results are actually metaphorical, as they do not necessarily have explicit metaphor signals (e.g., ``like'' or ``as'').\footnote{
Strictly speaking, generation of similes also needs to verify if results are metaphorical because signals of simile are also used for exemplification, subcategorization, and many other usages. Refer to \citet{low2010} for how to identify metaphorical similes from simile-like expressions.}
We also need to pay more attention to the \emph{comprehensibility} of generated expressions.
The absence of explicit signals for metaphors discourages us from optimistically expecting readers to understand the meaning of generated expressions in a nonliteral sense.
For example, we have difficulty in understanding ``He is a kangaroo,'' but we can easily understand ``He can jump high as if he were a kangaroo.''

This paper proposes a method to generate metaphorical expressions using verbs.
More precisely, generation targets are phrases of a subject, object, and verb, where verbs are used metaphorically.
To our knowledge, this work is the first regarding the automatic generation of metaphors other than similes.
First, we define a new task for automatic metaphor generation as a paraphrasing task and propose two evaluation metrics---\emph{synonymousness} and \emph{preference}---in addition to \emph{metaphoricity}, \emph{novelty}, and \emph{comprehensibility}, which were proposed by \citet{miyazawa2017}.
Then we introduce a model to solve the task that finds paraphrasing candidates using automatically computable scores corresponding to the metrics: \emph{similarity}, \emph{figurativeness}, \emph{rarity}, and \emph{overall score}.
We conduct three experiments.
The first measures the performance of the metaphor classifier for computing figurativeness score.
In the second and third, we use crowdsourcing.
The second experiment examines how our system's scores correlate with those provided by humans in terms of the evaluation metrics.
In the last experiment, we show the usefulness of our system's scores in finding good metaphorical expressions, that is, expressions preferred by humans in pairwise comparisons.


\section{Related Works}

The term ``metaphor'' is used for several different but closely related meanings across domains.
\emph{Conceptual mapping theory} (CMT) \citep{lakoff1987,lakoff1993} defines metaphors, or \emph{conceptual metaphors}, as mappings from one conceptual domain to another.
Expressions such as ``inflammatory remarks'' or ``he was breathing fire'' are regarded as linguistic realizations of a conceptual metaphor, from \textsc{fire} to \textsc{anger} in this case, and called \emph{linguistic metaphors}.
It is linguistic metaphors that we are going to generate, and we basically use the term ``metaphor'' in this sense hereafter.
However, we do not apply CMT explicitly in the generation because fixing conceptual domains would restrict the variety or novelty of generated expressions.

Identifying metaphors in texts is an essential process in analyzing the real uses of metaphors.
We also require it when collecting literal expressions for input and picking up only metaphorical expressions from output.
\citet{steen2010} suggested a guideline called \emph{MIPVU} (Metaphor Identification Procedure VU University Amsterdam),
which was used to construct the \emph{VU Amsterdam Metaphor Corpus}.\footnote{http://www.vismet.org/metcor/search/showPage.php?page=start}
The main procedure of MIPVU is as follows.
First, annotators determine the contextual meaning of the target word.
Then they look up the word in a dictionary and search for a more \emph{basic meaning} than the contextual meaning.
Basic meanings tend to be concrete and easy to imagine, see, hear, feel, smell, or taste, or are related to bodily action.
If a more basic meaning exists, the contextual meaning differs from the basic meaning, and if it can be understood in comparison, then it is judged as a \emph{metaphor-related word}.

\emph{Automatic metaphor identification} is a field in NLP (Natural Language Processing) that tries to automatically classify expressions into metaphorical or literal, aiming to contribute to tasks such as word sense disambiguation as well as linguistic analysis.
\citet{birke2006} developed a classifier as a by-product of a dataset called the \emph{TroFi Example Base},
which contains literal and nonliteral usages of 50 English verbs, and was first built through human annotation and later augmented by active learning.
\citet{tsvetkov2014} propose a method to identify adjective-noun and subject-object-verb metaphors across languages.
They trained a classifier on the TroFi dataset using four features, the \emph{abstractness}, \emph{imageability}, \emph{supersenses}, and \emph{vector space word representation} of English words.
These features are based on concepts rather than individual words or languages,
and proved to be effective by experiments in Russian, Farsi, and Spanish as well as English.
We employ their method to score how metaphorical generated Japanese expressions are.

For \emph{automatic metaphor generation}, \citet{abe2006} proposed a model that outputs ``\emph{A} like \emph{B}'' expressions from phrases with a noun and adjectives modifying it.
For example, given ``a young, innocent, and fine character,'' the model suggests phrases such as ``a character like a grandchild'' or ``the character like a baby.''
In evaluating the generated results, \citet{abe2006} used \emph{adequacy}, \emph{ease of visualization}, \emph{amusingness}, and \emph{novelty},
showing that some phrases, through their system, get higher scores than those by humans.

To evaluate metaphors unrestricted to similes, \citet{miyazawa2017} suggested using three metrics---\emph{metaphoricity}, \emph{comprehensibility}, and \emph{novelty}---and showed the possibility to obtain reliable scores in terms of these metrics through crowdsourcing.
They composed targets for evaluation by combining a noun mainly taken from the \textsc{emotion} domain and verbs from the \textsc{water} domain.
For example, \emph{zouo}憎悪(hatred) and \emph{X-ga afureru}〇〇があふれる(something spills out) constitute the expression \emph{zouo-ga afureru}憎悪があふれる(hatred spills out).
They also introduce \emph{overall evaluation}, defined as the arithmetic mean of the three metrics, and show that it can indicate ``good metaphors'' in that expressions with high overall evaluation are preferred by a judge in pairwise comparison, and that they are metaphorical in the sense of MIPVU\@.
In this paper, we also use metaphoricity, comprehensibility, novelty, and a modified version of overall evaluation.


\section{Task}
\seclabel{task}

\subsection{Task Definition}

We introduce a new metaphor generation task:
Given a literal expression, a system suggests alternative verbs that make the expression synonymous and metaphorical.
We assume a practical use where writers have expressions in their mind and want to make them rhetorically appealing.
In this paper, the expressions are in Japanese, each of them consisting of a subject-object-verb (SOV) triple.
For example,
\begin{exe}
\exi{(i)}\glll
{彼が} {気持ちを} {\underline{考慮する}。}\\
{Kare-ga} {kimochi-wo} {\underline{kouryo-suru}.}\\
{He-\textsc{nom}} {feelings-\textsc{acc}} {consideration-do}\\
\glt {``He considers someone's feelings.''}
\end{exe}
Given this expression, the system would output some alternative verbs such as \emph{kumitoru}汲み取る(to draw water from a well).
Then it would make an expression \emph{kare-ga kimochi-wo kumitoru},
which can be interpreted as having the same meaning as the input.
Here the verb \emph{kumitoru} is metaphorical, as it is used to mean ``to consider'' while it basically means the physical action of somebody or something drawing water from a well.

The task is formalized as follows.
Let $\varepsilon$ be an expression of the form ``$s$-\emph{ga} $o$-\emph{wo} $v$,'' in Japanese ``$s$が$o$を$v$,'' where $s$, $o$, and $v$ are the subject, object, and verb, respectively.
The suffix \emph{-ga}が and \emph{-wo}を are the nominative case and accusative case markers, respectively.
We denote it as $\varepsilon = (s, o, v)$.
Then, the task is to find an alternative verb $v'$ and make another expression $\varepsilon' = (s, o, v')$ for a given expression $\varepsilon$ so that $\varepsilon'$ achieves high scores in the five evaluation metrics introduced in \secref{evalmet}.

We focus on verb expressions because verbs tend to be metaphorical compared to other parts of speech \citep{steen2010} and expect that this would make it easier for us to construct balanced datasets of metaphorical and literal expressions and generate many and various metaphorical expressions.
Also, we can practically use the cross-lingual metaphor classifier by \citet{tsvetkov2014} with small modifications, which is necessary to compute how metaphorical expressions are.


\subsection{Evaluation Metrics}
\seclabel{evalmet}

In the evaluation of results, we use the same metrics by \citet{miyazawa2017}---\emph{metaphoricity}, \emph{novelty}, and \emph{comprehensibility}---and introduce two additional metrics, \emph{synonymousness} and \emph{preference}.
\tabref{ex} illustrates some alternative verbs for the input expression \emph{kare-ga kimochi-wo kouryo-suru}彼が気持ちを考慮する(he considers someone's feelings) and their expected classes of scores regarding the metrics except preference, which is used in comparison with another result.

\begin{table}[b]
\hangcaption{Examples of alternative verbs and their expected classes of scores for each metric, \texttt{High} or \texttt{Low}, for \emph{kouryo-suru}考慮する in \emph{kare-ga kimochi-wo kouryo-suru}彼が気持ちを考慮する(he considers someone's feelings).}
\tablabel{ex}
\input{01table01.tex}
\end{table}

\emph{Synonymousness} measures how synonymous input and output expressions are.
It is required because we define the task as a paraphrasing problem.
For example, for the source expression (i), \emph{sukuitoru}掬い取る(to scoop up, to consider) and
\emph{suisoku-suru}推測する(to guess) are more appropriate than \emph{tsutaeru}伝える(to tell).

\emph{Metaphoricity}, the metaphorical degree of output expressions, is essential in the task.
Given (i), \emph{sukuitoru} makes more metaphorical expressions than \emph{kouryo-suru}考慮する(to consider), because it is also used to mean a physical action, ``to scoop up'' as well as ``to consider,'' in contemporary Japanese.

\emph{Novelty} measures how novel output expressions are.
As with the other metrics, it is measured as an expression rather than as a verb.
Consequently, a common verb, \emph{tateru}建てる(to build), is evaluated as novel if it is suggested for the source expression (i) because \emph{kare-ga kimochi-wo tateru}彼が気持ちを建てる(he builds someone's feelings) sounds unnatural and is hardly used.

\emph{Comprehensibility} measures the ease with which an expression is understood.
For example, \emph{kare-ga kimochi-wo tsutaeru}彼が気持ちを伝える(he tells someone his feelings) is more comprehensible than \emph{kare-ga kimochi-wo tateru}.
In each metric's evaluation, the other ones are not considered.
However, comprehensibility affects synonymousness and metaphoricity in that low comprehensibility makes it difficult to evaluate them, as they can be assessed after evaluators understand the target expression's meaning.
This implies that we need to divide evaluation into two stages: comprehensibility in the first stage and the others in the second stage, or vice versa.

\emph{Preference} measures how good an expression is compared to others.
Therefore, it is different from the other metrics in that it cannot be measured by one expression.
In a practical sense, users of metaphor generation systems are likely clueless as to which metrics they should give more weight to; in such cases, preference is helpful.
We quantify preference by collecting pairwise comparisons from multiple judges and aggregating them into a single score, the construction of which is described in \secref{ade}.
\citet{miyazawa2017} also used pairwise comparisons in evaluation,
but they collected these comparisons from only one judge.
Thus, our \emph{preference} is more objective.


\subsection{Crowdsourcing}
\seclabel{ade}

We use crowdsourcing to evaluate expressions according to the above-mentioned five metrics.
Target expressions are sampled from bins that are made according to the corresponding \emph{automatically computable scores} (discussed in the next section).
For example, for synonymousness, we have the corresponding score called \emph{similarity}.
Then, to evaluate synonymousness, we divide the range of similarity into 10 bins and sample one from each bin.
This process makes it easier to analyze the relation between these metrics and the system's scores.
We ask evaluators to rate the expressions on a scale of 1 to 5 as illustrated in \figref{que}.
We provide short descriptions of the choices for 1 and 5 and tell evaluators that higher scores indicate higher degrees of synonymousness, metaphoricity, novelty, and comprehensibility.
For each question, we collect answers $a_1, \ldots, a_N$ from $N$ evaluators and aggregate scores by scaling the answers to $[0,1]$ and averaging them:
\begin{align}
\frac{1}{N} \sum_{i = 1}^{N}\frac{a_i - 1}{4}. \formulalabel{aggregation}
\end{align}

\begin{figure}[t]
\begin{center}
\includegraphics{26-2ia1f1.eps}
\end{center}
\caption{Examples of questions for human evaluation.}
\figlabel{que}
\end{figure}

We collect evaluators for the experiments through \emph{Yahoo!\ Crowdsourcing}.\footnote{https://crowdsourcing.yahoo.co.jp/}
Crowdsourcing participants are not restricted by age, sex, or district of residence but are required to have enough ability to give a correct answer in a Japanese grammar quiz.

As stated in \secref{evalmet}, we expect the evaluation process to be in two stages---comprehensi\-bility and the others---for a strict analysis.
In crowdsourcing, however, we conduct the evaluation at the same time.
Evaluating comprehensibility first and removing expressions with low comprehensibility might result in imbalanced data and analysis issues.
Meanwhile, the opposite process---other metrics first and comprehensibility later---has the problem of cost.
Some evaluation results will be discarded if the expressions have low comprehensibility scores.
Thus, we evaluate all the metrics at once.


\subsection{Preference}
\seclabel{sec:preference}

We evaluate expression preference by asking evaluators to choose the superior of two expressions and aggregating the answers.
An example of the style of questioning is shown in \figref{preference}.
To create practical systems, it would be better to ask how good expressions are in some contexts or how good they are for some specific purposes.
However, we phrase the questions as in \figref{preference} to reduce evaluator load and simplify the task.
The target expressions are sampled the same way as the other metrics.

\begin{figure}[t]
\begin{center}
\includegraphics{26-2ia1f2.eps}
\end{center}
\caption{Example of questions for preference.}\figlabel{preference}
\end{figure}

Answers provided by evaluators are aggregated into real-valued scores using the Bradley--Terry model \citep{bradley1952}.
Suppose the input expression is $(s, o, v)$ and the system suggests a set of verbs $\mathcal{V}$;
then we have a set of expressions $\mathcal{E} = \left\{(s, o, v') \mid v' \in \mathcal{V} \right\}$.
We denote that $\varepsilon \in \mathcal{E}$ is preferred to $\varepsilon' \in \mathcal{E}$ by $\varepsilon \succ \varepsilon'$.
Then we model the probability that $\varepsilon \succ \varepsilon'$ is observed as
\begin{align}
P(\varepsilon \succ \varepsilon')= \frac{\pi_{\varepsilon}}{\pi_{\varepsilon} + \pi_{\varepsilon'}},
\end{align}
where $\pi_\varepsilon$ is a positive parameter for each $\varepsilon \in \mathcal{E}$,
which is supposed to represent a kind of quality of expression.
Suppose we observe the $n_{\varepsilon\varepsilon'}$ occurrences of $\varepsilon \succ \varepsilon'$ for each $\varepsilon, \varepsilon' \in \mathcal{E}$.
The log likelihood of this event is given by
\begin{align}
L(\bm{\pi}) = \log\prod_{\substack{(\varepsilon,\varepsilon') \in \mathcal{E} \times \mathcal{E}\\ \varepsilon \ne \varepsilon'}} \parentheses{\frac{\pi_{\varepsilon}}{\pi_{\varepsilon} + \pi_{\varepsilon'}}}^{n_{\varepsilon\varepsilon'}},
\end{align}
where $\bm{\pi} = \parentheses{\pi_\varepsilon}_{\varepsilon \in E}$.
We can find a unique optimal parameter vector $\bm{\pi}^*$ that maximize $L$ according to \citet{ford1957}.
Consequently, we use $\pi_\varepsilon$ as the preference score of expression $\varepsilon$.


\section{Method}
\seclabel{model}

To find verbs that make highly evaluated expressions in terms of the above evaluation metrics, our model uses four automatically computable scores: \emph{similarity}, \emph{figurativeness}, \emph{rarity}, and \emph{overall score}.
The correspondence is summarized in \tabref{correspondence}.
Here we put \emph{commonness} as the counterpart of comprehensibility, but this is a tentative solution.
We do not suggest a specific score for comprehensibility, as it requires much complex word and world knowledge, and is, therefore, difficult to formulate.
When an automatic score for comprehensibility is required, we compute a tentative score, commonness, as a negative monotonic transformation of rarity.
\begin{table}[t]
\hangcaption{Correspondence between evaluation metrics and automatically computable scores used in finding paraphrases.}\tablabel{correspondence}
\input{01table02.tex}
\end{table}

This idea is based on the observation by \citet{miyazawa2017} that novelty is negatively correlated with comprehensibility.

In the model, we assume that evaluation metrics are independent and modeled by a single automatic score.
We check the validity of this hypothesis in \secref{exp2}.


\subsection{Similarity}
\seclabel{sim}

By the task definition, we need to compute the semantic similarity between input and output phrases,
but this is too difficult.
Instead, we consider verbs only and calculate the cosine similarity between their vector representations.
Let $\mathcal{W}$ be the set of all the words in the corpus.
Using the \emph{continuous bag-of-word} (CBOW) algorithm \citep{mikolov2013}, we obtain a map $\gamma\colon \mathcal{W} \to \mathbb{R}^n$, where $n$ is a predefined integer.
Given a source expression with a verb $v$, the similarity between $v$ and its alternative $v'$ is calculated by the affine transformation of cosine similarity:
\begin{align}
\similarity(v, v') =
\frac{1}{2}\parentheses{
1 + \frac{
\gamma\parentheses{v} \cdot \gamma\parentheses{v'}
}{
\norm{\gamma \parentheses{v}}\norm{\gamma\parentheses{v'}}
}
}.
\end{align}
Affine transformation is applied so that the score ranges from 0 to 1 and becomes compatible with the other scores.


\subsection{Rarity}
\seclabel{subsec:methodrarity}

Let $p(*)$ and $p(*,*)$ represent the probabilities of a word and OV pair, respectively, estimated by a corpus.
Let $g(o)$ be the label representing the cluster which $o$ belongs to.
We compute the rarity of an expression whose object is $o$ and verb is $v$ by
\pagebreak
\begin{align}
\rarity\parentheses{o, v}
=
\begin{cases}
A_{o, \mathcal{V}}
{\displaystyle
\frac{\log p(g(o), v)}{\log p(v)}
} + B_{o, \mathcal{V}}
& \text{if $\log p(g(o), v) > 0$ and $\log p(v) > 0$,} \\
1
& \text{otherwise,}
\end{cases}
\formulalabel{rar}
\end{align}
where {$A_{o,\mathcal{V}}$} and {$B_{o, \mathcal{V}}$} are constants
dependent on $o$ and a set of alternative verbs $\mathcal{V}$ to make the score take value in {$[0, 1]$}.
Accurately,
\begin{gather}
A_{o, \mathcal{V}} = - \frac{1}{M_{o, \mathcal{V}} - m_{o, \mathcal{V}}}, \,
B_{o, \mathcal{V}} = \frac{M}{M_{o, \mathcal{V}} - m_{o, \mathcal{V}}}, \\
M_{o, \mathcal{V}} = \max_{v \in \mathcal{V}}\frac{\log p(g(o), v)}{\log p(v)}, \,
m_{o, \mathcal{V}} = \min_{v \in \mathcal{V}}\frac{\log p(g(o), v)}{\log p(v)}.
\end{gather}

We estimate probabilities as follows.
When the corpus contains $L$ words, and a word $w$ occurs $c_w$ times, then
\begin{align}
p(w) = \frac{c_w}{L}.
\end{align}
To obtain probabilities of OV pairs, we first extract all the pairs of a verb and its direct object.
Suppose the corpus has $L'$ pairs in total.
When a pair {$(o, v)$} appears {$c_{o, v}$} times in the corpus, we have
\begin{align}
p(o, v) = \frac{c_{o, v}}{L'}.
\end{align}

\formularef{rar} has a problem that many possible OV pairs are not observed in the corpus, and the probabilities become $0$.
Consequently, we have difficulty in comparing rarity.
To overcome the issue, we reduce OV pair variety by clustering nouns.
First we construct clusters {$\{\mathcal{C}_i\}_{i = 1}^K$} according to vector representations of words.
We obtain vector representations in the same way as \secref{sim}.
After that, we make $K$ clusters by \emph{spherical K-means} \citep{banerjee2005}, a variant of $K$-means clustering algorithm where the nearness between a point and each cluster representative is computed by cosine similarity.
After that, we replace nouns in the text with the corresponding labels {$\mathtt{cluster}_1, \ldots, \mathtt{cluster}_K$}.
This is $g$ in \formularef{rar}.
Probabilities tend to take large values for frequent words and almost zero for most words as illustrated in \figref{probs}.
Thus, we take the logarithm of probabilities and obtain less skewed values representing degree of novelty.

\begin{figure}[t]
\begin{center}
\includegraphics{26-2ia1f3.eps}
\end{center}
\hangcaption{Probabilities of verbs and OV pairs sorted in descending order,where $\mathcal{C} = \{\text{``\emph{kangae}考え(thought)'', ``\emph{hassou}発想(idea)'', …, ``\emph{ishiki}意識(consciousness)''}\}$.}
\figlabel{probs}
\end{figure}


\subsection{Figurativeness}

To automatically score metaphoricity, we build a classifier that categorizes an expression into the metaphorical class (\texttt{Met}) or the literal class (\texttt{Lit}).
The score can be binary but can be continuous when we adopt a discriminative classifier that models the posterior probability of the class \texttt{Met} when the feature of the expression is given. Let $\phi$ be a function that assigns a feature vector to SOV triples.
Then we define the \emph{figurativeness} score of an expression consisting of a subject $s$, object $o$, and verb $v$ by
\begin{equation}
\figurativeness\parentheses{{s, o, v}}
= p\parentheses{\text{\texttt{Met}} \mid \phi\parentheses{s, o, v}},
\end{equation}
where $p$ is the probability estimated by a discriminative classifier.

We construct feature vectors the same way as in \citet{tsvetkov2014}.
We calculate \emph{abstractness and imageability}, \emph{supersenses}, and \emph{vector space word representations} by translating Japanese words into English.
In addition, as in \citet{tsvetkov2013}, we use \emph{named entities} as an additional feature, which is directly calculated from Japanese.
The mapping $\phi$ concatenates each feature of $s$, $o$, and $v$
and then concatenates the three vectors again.

\emph{Abstractness and imageability} are real-valued scores, which human evaluators give to each word according to how abstract and imageable it is.
These scores are good indicators of metaphors.
For example, a verb \emph{afureru}あふれる(to overflow) usually becomes metaphorical when its subject is an abstract word such as \emph{kanjou}感情(emotion) but becomes literal when its subject is a concrete word such as \emph{mizu}水(water).
While abstractness and imageability are similar concepts, they are not redundant.
Some abstract words, such as
``torture,'' call up visual images \citep{tsvetkov2014}.
\emph{Supersenses} are coarse semantic categories in \emph{WordNet} \citep{miller1995},
which consists of 47 categories, including 27 for nouns (\texttt{noun.act}, \texttt{noun.quantity}, etc.) and 15 for verbs (\texttt{verb.motion}, \texttt{verb.cognition}, etc.).
\emph{Vector space representations} are the same as those used in computing similarity and rarity.
\emph{Named entities} are used to specify subject and object types: human, organization, or location.
Such distinction is effective in finding some metaphorical usages of verbs.
For example, \emph{Seto-naikai-wo idaku}瀬戸内海を抱く(to embrace Seto Inland Sea) is metaphorical because \emph{idaku}抱く is interpreted as ``to be located surrounding something'' rather than ``embrace'' or ``hold someone tight.''
\secref{datafig} discusses which language resources are used to retrieve features and how they are preprocessed.


\subsection{Overall Score}

We define \emph{overall score} in two ways: through the arithmetic mean (AM) of similarity, rarity, and figurativeness, and through the harmonic mean (HM) of the same three scores.
Precisely, when the source expression is $\varepsilon = (s, o, v)$, and an alternative verb is $v'$,
the overall scores are calculated as follows.
\begin{gather}
\overallscore_\text{AM}\parentheses{s,o,v,v'}
= \frac{1}{3}\parentheses{
\similarity\parentheses{v, v'}
+ \rarity\parentheses{o, v'}
+ \figurativeness\parentheses{s, o, v'}
}
, \\
\overallscore_\text{HM}\parentheses{s,o,v,v'}
= 3 \parentheses{\displaystyle \frac{1}{\similarity\parentheses{v, v'}} + \frac{1}{\rarity\parentheses{o, v'}} + \frac{1}{\figurativeness\parentheses{s, o, v'}}}^{-1}\formulalabel{os}.
\end{gather}
We do not use them together, but we compare their performances in experiments.
\citet{miyazawa2017} defined \emph{overall evaluation} as the arithmetic mean of the scores for comprehensibility, metaphoricity, and novelty.
However, we exclude the score for comprehensibility, that is, commonness, from \formularef{os}, as we have no automatic score for comprehensibility, and using commonness leads to the mere cancellation of rarity.
On the other hand, we include similarity in \formularef{os}, which was not used by \citet{miyazawa2017} because their task was not paraphrasing.
We use harmonic mean along with the arithmetic mean in expecting that the score becomes robust to outliers.


\section{Dataset}
\seclabel{dataset}

In this section, we describe the resources used in the experiments
and how we preprocessed them.

\subsection{Input and Metaphor Classifier Datasets}
\seclabel{dataclassifier}

We needed two datasets of Japanese SOV triples: one was inputted to the system as input, and the other was used to train a metaphor classifier for computing figurativeness scores.
For these purposes, each verb required a label of metaphoricity, \texttt{Met} or \texttt{Lit}.
We used a dataset by \citet{miyazawa2016}, which was constructed on part of \emph{Kyoto University Text Corpus Version 4.0} (KU Corpus hereafter) \citep{kawahara2002}.
The KU Corpus has lexical and syntactic annotations on text taken from Japanese news articles.
In addition, some sections of the corpus have semantic annotations such as referent and implicit subjects.
This helped us construct SOV triples from Japanese sentences, as they often lack explicit subjects.
We used this dataset rather than TroFi or other datasets for two reasons: First, metaphor criteria were clearer, which made it easy to analyze classification results.
Second, the dataset contained more diverse expressions;
TroFi focuses on literal and nonliteral uses of a limited number of verbs and only contains expressions related to them.
As we explain below, \citet{miyazawa2016} annotated based on MIPVU and referred to dictionary explanations.
As a result, words were often labeled \texttt{Met} even if they were so conventional that people do not take them as metaphorical.
Such usage of words usually make for dull expressions, and they are not our main targets to generate.
However, we had other automatic metrics, such as rarity, to filter out dull or inappropriate expressions from other perspectives, so we did not expect classifiers to filter out many expressions in advance.
In MIPVU and \citet{miyazawa2016}, metaphoricity assessments did not explicitly depend on the conventionality of each usage, which met the policy.

\citet{miyazawa2016} aimed to create a guideline for a Japanese metaphor corpus based on MIPVU\@.
To check its validity, they annotated 764 OV pairs from the first part of fully-annotated sections of the KU Corpus.
They assigned labels to both objects and verbs.
In this paper, we just needed verbs annotation but also briefly review results including objects.
After extracting 764 OV pairs, two annotators made judgments on metaphoricity according to a guideline based on a slightly modified version of MIPVU\@.
One difference between MIPVU and \citet{miyazawa2016} is that the latter used binary labels
while the former allows annotators to assign a special label in ambiguous cases.
Another difference was the dictionary:
to refer to meanings of words, they used an electronic version of \emph{Shin Meikai Kokugo Jiten} \citep{yamada2011}.
In the dictionary, metaphorical meanings are often marked as \emph{hiyu-teki-ni}比喩的に(in a metaphorical sense).
According to their guideline, the annotators labeled the word as \texttt{Met} when the dictionary marked the contextual meaning as metaphorical.

To check the quality of the guideline, they calculated inter-annotator agreement (Fleiss' $\kappa$),
which is shown in \tabref{metcount}.
They had $\kappa$ of 0.477 and 0.587 for objects and verbs respectively.
These were not high compared to the results of \citet{steen2010},\footnote{
For example, they report $\kappa$ was 0.77 in the News register in the first trial.} but they were high enough to recognize moderate agreement and thus we used this result in the later analysis.
When the two annotators differed in judgment, we adopted the labels given by Annotator 1.

\begin{table}[b]
\hangcaption{Results of the \texttt{Met}/\texttt{Lit} annotation on a section of the KU Corpus done by Annotator 1 and Annotator 2.}
\tablabel{metcount}
\input{01table03.tex}
\end{table}

For our experiments, we removed some expressions from the dataset of \citet{miyazawa2016} and obtained 202 metaphorical expressions (whose verbs are labeled \texttt{Met}) and 326 literal expressions (whose verbs are labeled \texttt{Lit}).
Expressions that have labels on \emph{koto}こと(fact) and \emph{mono}もの(thing) were removed.
This is because we cannot assign proper vector representations them.
Our method assumes that subjects and objects are single nouns, but \emph{koto} and \emph{mono} are often preceded by a verb phrase and create such noun phrases as \emph{eiga-wo miru koto}映画を見ること(to watch a movie),
for which it is difficult to acquire good representations.
We also excluded objects containing numerals (e.g., ``20\%'') because sometimes it was challenging to link them to the quantifying nouns and vector representations.
Finally, we removed expressions with labels on formal verbs: \emph{suru}する and \emph{yaru}やる(to do), \emph{motsu}もつ(to have), and \emph{tsukuru}つくる(to make).
However, \emph{suru} was not excluded when linked to \emph{verbal nouns},
which are nouns that function as verbs by taking \emph{suru} at the end, such as \emph{kouryo-suru}考慮する(to consider).

The dataset was divided into a paraphrase dataset and training dataset.
The former consisted of nine expressions listed in \tabref{extracted}, and the latter consisted of the rest.
In \secref{exp1}, we compared the performance of two metaphor classifiers.
The performance of classifier \texttt{JA} was measured by 10-fold cross validation on this training dataset while that of classifier \texttt{EN} was measured on the \emph{TroFi Example Base} \citep{birke2006,birke2007}, which was used by \citet{tsvetkov2014} as training data.
The classifier for computing figurativeness scores for human evaluation (Section \ref{exp2}, Section \ref{exp3})
was trained on the entire training dataset.

\begin{table}[t]
\caption{Source expressions in the paraphrase dataset.}
\tablabel{extracted}
\input{01table04.tex}
\end{table}


\subsection{Candidates for Alternative Verbs in the Generation}

Alternative verbs in the generation were chosen from a verb set collected from the \emph{Balanced Corpus of Contemporary Written Japanese} (BCCWJ).\footnote{http://pj.ninjal.ac.jp/corpus\_center/bccwj/en/}
It is a large Japanese corpus with about 100 million words taken from a wide variety of registers including books, magazines, and online Q\&A forums, for example.
We also used it to obtain word vectors for similarity and rarity.
By using the lexical annotation of the corpus, we collected all the verbs in the corpus, including combinations of verbal nouns and \emph{suru}.

Since our targets are SOV triples, verbs needed to be transitive.
We extracted them from the verbs in BCCWJ using a case frame dictionary, \emph{Kyoto University Case Frames} (KU Case Frames hereafter) \cite{kawahara2006a,kawahara2006b},
which is a collection of predicate-argument examples taken from 1.6 billion sentences of online text.
The top entries of the dictionary are indexed by verb, with each having examples of cases and complement nouns as subentries.
We assumed that transitive verbs have at least one example of the accusative case, which is indicated by \emph{-wo}.
Collecting such examples, we obtain 10,743 transitive verbs including verbal nouns.


\subsection{Similarity}

We obtained the word vectors used in the calculation of similarity 
\pagebreak
scores by training the CBOW model\footnote{We used an implementation provided in the gensim library (https://radimrehurek.com/gensim/models/\linebreak[2]word2vec.html).} on the sentences in BCCWJ\@.
The size of vectors was set to 100, and the window size was set to 8.


\subsection{Rarity}

To calculate rarity, first, we clustered the BCCWJ words by applying spherical $K$-means\footnote{
We used the \texttt{spherecluster} library (https://github.com/clara-labs/spherecluster).} to their vector representations constructed for similarity.
The number of clusters $K$ was 1,000.
Before computing the score, we removed particles such as case markers from the text to put verbs adjacent to their respective objects.
After that, we replaced each noun with the corresponding cluster tag, and finally we had $p(v)$ and $p(t, v)$ for each verb $v$ and tag $t$.


\subsection{Figurativeness}
\seclabel{datafig}

The classifier \texttt{EN} and \texttt{JA} share some language resources but with some differences.
They are summarized in \tabref{resources}.

We retrieved abstractness and imageability scores from the \emph{MRC Psycholinguistic Database}\footnote{http://ota.oucs.ox.ac.uk/headers/1054.xml} \citep{wilson1988}.
The classifier \texttt{JA} also used the dataset, but we translated words into Japanese using \emph{Japanese WordNet} \citep{isahara2008}.
Japanese WordNet has links between words' synsets (set of synonyms) and corresponding English words, so we can get English translations of Japanese words via these links.

We retrieved and encoded supersenses from WordNet the same way as in \citet{tsvetkov2014} for both \texttt{EN} and \texttt{JA}.

To retrieve information on named entities, we did not use translation because the annotation of named entities was available in the KU Corpus.
For the English dataset, we used the parsing result of \texttt{spaCy}\footnote{https://spacy.io/} with the \texttt{en\_core\_web\_sm} model.

\begin{table}[b]
\caption{Language resources used by two classifiers \texttt{EN} and \texttt{JA}.}
\tablabel{resources}
\input{01table05.tex}
\end{table}

As vector representations for \texttt{EN}, 
    we used \fullciteA{huang2012} as 
\citet{tsvetkov2013} did.
Classifier \texttt{JA} used the vector representation constructed for computing similarity.

Both classifiers used gradient boosting decision tree as the classification algorithm.
We used an implementation provided by the \texttt{LightGBM} library\footnote{https://github.com/Microsoft/LightGBM} with default hyperparameters.


\section{Experiment}
\seclabel{exp}

We conducted three experiments.
Experiment 1 measured the performance of the metaphor classifier by applying it to the Japanese test dataset.
Experiments 2 and 3 used crowdsourcing.
In Experiment 2, we examined whether the system's scores actually correlated with scores provided by human evaluators.
Experiment 3 was conducted to see how the system's scores reflect human preferences.


\subsection{Experiment 1}
\seclabel{exp1}

Experiment 1 measured the performance of the two metaphor classifiers, \texttt{EN} and \texttt{JA}.
Performance was measured by 10-fold cross-validation.
The results are shown in \tabref{performance}.
Though the scores were not so high, \texttt{JA} achieved an accuracy greater than 0.5, implying that they work as metaphor classifiers.
Thus, we used \texttt{JA} to compute figurativeness scores in the later experiments.

\begin{table}[b]
\hangcaption{Performance of metaphor classifiers applied to the different datasetsmeasured by 10-fold cross-validation.}
\tablabel{performance}
\input{01table06.tex}
\vspace{4pt}\small
The columns \mbox{\# \texttt{Lit}} and \mbox{\# \texttt{Met}} show the numbers of instances of \texttt{Lit} and \texttt{Met} contained in each dataset, and A, P, R, and F stand for accuracy, precision, recall, and F-measure respectively.
\end{table}

It is not directly comparative, but the performance was worse than that in \citet{tsvetkov2014}.
They reported that their classifier achieved F-measures of 0.85 and 0.86 in English and Russian, respectively.
These results are attributed to the difference in annotating schemes in the English and Japanese datasets.


\subsection{Experiment 2}
\seclabel{exp2}

Experiment 2 was conducted to analyze the relation between human scores 
\pagebreak
in the evaluation metrics and system scores computed in the generation.
We fed nine literal expressions to our system, and generating 10,743 expressions for each.
We sampled the target expressions as described in \secref{ade}.
Thus, we had 10 expressions for each source expression and each score.
We also evaluated the source expressions.
Consequently, we had $9 \times 11 \times 4 = 396$ questions.
For each question, we collected 10 evaluators.
This means $N$ in \formularef{aggregation} was set to 10.

\begin{figure}[b]
\begin{center}
\includegraphics{26-2ia1f4.eps}
\end{center}
\caption{Relationship between scores given by human and our system.}
\figlabel{rel}
\end{figure}

The results are shown in \figref{rel} and \tabref{correlation}.
The Underlines in \tabref{correlation} indicate that the automatic score of the row was introduced to model the evaluation metric in the column.
Regarding synonymousness, metaphoricity, and novelty,
the corresponding automatic scores, similarity, figurativeness, and rarity, have positive correlations and are the highest among the scores.
This means that they work as models of the corresponding evaluation metrics.
The results for comprehensibility were different from what we expected.
The automatic score with the highest correlation coefficient was similarity (0.44)
rather than negative rarity (0.37).
However, this is natural because the original phrases are highly comprehensible, and
similarity was measured in the sense that they often appear in similar contexts.
For example, \emph{X-san-ga ronbun-wo kakiorosu}〇〇さんが論文を書き下ろす(someone newly writes an article) was rated 1.0 in the comprehensibility evaluation, while similarity and commonness were 0.75 and 0.91, respectively.
This result suggests that we need to consider dependencies and a way to mix the scores to model each evaluation metric.

\begin{table}[t]
\caption{Correlation coefficients between scores given by humans and our system.}
\tablabel{correlation}
\input{01table07.tex}
\end{table}

We observed that human judges tended to give high metaphoricity scores when expressions were incomprehensible.\footnote{
Here we regard expressions ``incomprehensible'' based on our intuition rather than in terms of the comprehensibility scores because the target expressions are different in each metrics and expressions rated in terms metaphoricity do not have comprehensibility scores.}
For example, \emph{kaisha-ga seihin-wo aogu}会社が製品を仰ぐ(the company looks up at its product) was rated 0.66 in metaphoricity but 0.10 in figurativeness.
Opposite cases were also observed. \emph{Kazoku-ga kyuuka-wo furikaeru}家族が休暇を振り返る(the family looks back on holidays) was rated 0.19 by human evaluators, but its figurativeness score was 0.89.
The verb in the expression is metaphorical according to MIPVU, but it is so conventional that people usually do not consider it a metaphor.

As for novelty, we hypothesized that there were two reasons for the weak correlation.
First, alternative verb candidates in the generation were sometimes incompatible with the object.
For example, \emph{deru}出る(to get out), which appeared in \emph{X-san-ga giin-wo deru} (the group gets out of a member of the Diet)〇〇さんが議員を出る(novelty: 0.78, rarity: 0.21),
sounds novel or unnatural because it can be transitive only when its object stands for a place.
The second reason is that the set of alternative verb candidates wrongly contains some verbs that are always intransitive, such as  \emph{mieru}見える(to come into sight) in *\emph{X-san-ga bunka-wo mieru} *〇〇さんが文化を見える.\footnote{
The expression makes sense if we use a transitive verb \emph{miru}見る(to see) insted of \emph{mieru}見える(to come into sight) as \emph{X-san-ga bunka-wo miru}〇〇さんが文化を見る(someone sees the culture).}
We regarded a verb as transitive when it had at least one example of \emph{-wo}.
However, we needed to adopt a more elaborate method because the KU Case Frames sometimes contains unnatural examples; it was constructed automatically, and its examples were collected from the Web.
We expect to have more reliable results if we can remove such ungrammatical cases.


\subsection{Experiment 3}
\seclabel{exp3}

Experiment 3 compared human preference and the overall score.
We used the same input expressions as those in the previous experiment and recruited evaluators the same way.
We collected binary comparison answers and constructed aggregate preference orders as described in \secref{sec:preference}.
The evaluation was done using Kendall's $\tau$.
It takes a value in $[-1, 1]$, and a higher value means two orderings are more similar.

\begin{table}[b]
\caption{Comparison of preference and our system' scores using Kendall's $\tau$.}
\tablabel{tau}
\input{01table08.tex}
\end{table}

\tabref{tau} shows $\tau$ calculated between orderings made by preference and the automatically computable scores including the overall scores, AM and HM.
We cannot conclude that overall scores are consistent with human evaluation,
but it helps us find preferable expressions in some cases.
For example, \emph{X-san-ga chousen-wo umeru}〇〇さんが挑戦を埋める(someone buries a challenge) was ranked at the top by the system and human judges (similarity: 0.47, figurativeness: 0.66, rarity: 0.60).

When we looked at mismatching examples,
\emph{kai-ga giini-wo haki-suru}会が議員を破棄する(the group destroys a member of the Diet) was the most preferred by human evaluators, but it was ranked at the bottom by $\overallscore_\text{AM}$ among the results from the same input (similarity: 0.57, figurativeness: 0.00, rarity: 0.73).
The expression ranked at the top by $\overallscore_\text{AM}$ was
\emph{X-san-ga chousen-wo kaku}〇〇さんが挑戦を書く(someone writes a challenge) (similarity: 0.67, figurativeness: 0.34, rarity: 0.73).
This inconsistency was due to significant differences in figurativeness.
We expected that the harmonic mean would weaken the effect, but it did not work well.
This observation reveals that the need to revise the definition of figurativeness so that they become more robust.


\section{Conclusion and Future Work}
\seclabel{concl}

We defined a new task for automatic metaphor generation as a paraphrasing task.
To evaluate the results, we proposed two metrics---synonymousness and preference along with metaphoricity, novelty, and comprehensibility, which were proposed by \citet{miyazawa2017}.
We introduced a model that finds alternative verbs for input expressions by using automatically computable scores corresponding to the metrics.
By applying the method to a Japanese dataset, we showed that automatically computable scores were weakly correlated with human scores, and in some cases the overall score was effective in finding preferable expressions.
On the other hand, it revealed that the definitions of the automatically computable scores need to be revised.
Specifically, we need to solve two problems.
First, we assumed that we can model each evaluation metric using a single automatic score, but this was proven to be invalid.
For example, similarity correlated highly with both synonymousness and comprehensibility.
Second, overall scores were not consistent with human scores.
We simply used arithmetic and harmonic means, but we need to try many patterns to weight the scores.
Making figurativeness more robust may help us improve overall scores.

We tested the task and method in a Japanese dataset, but it is also applicable to other languages with slight modifications, as metaphors are conceptual rather than lexical.
For the same reason, our method would work in syntactic structures other than SOV such as a noun and its modifier.
We will examine whether these are true by conducting experiments in other datasets.
To make our system practical, we are going to evaluate how good generated results are with contexts or in specific purposes such as speeches or poems.

\bibliographystyle{jnlpbbl_1.5}
\begin{thebibliography}{}

\bibitem[\protect\BCAY{Abe, Sakamoto, \BBA\ Nakagawa}{Abe
  et~al.}{2006}]{abe2006}
Abe, K., Sakamoto, K., \BBA\ Nakagawa, M. \BBOP 2006\BBCP.
\newblock \BBOQ A Computational Model of Metaphor Generation Process.\BBCQ\
\newblock In {\Bem Proceedings of the 28th Annual Meeting of the Cognitive
  Science Society}, \mbox{\BPGS\ 937--942}.

\bibitem[\protect\BCAY{Banerjee, Dhillon, Ghosh, \BBA\ Sra}{Banerjee
  et~al.}{2005}]{banerjee2005}
Banerjee, A., Dhillon, I.~S., Ghosh, J., \BBA\ Sra, S. \BBOP 2005\BBCP.
\newblock \BBOQ Clustering on the Unit Hypersphere using von Mises-Fisher
  Distributions.\BBCQ\
\newblock {\Bem Journal of Machine Learning Research}, {\Bbf 6}  (Sep),
  \mbox{\BPGS\ 1345--1382}.

\bibitem[\protect\BCAY{Birke \BBA\ Sarkar}{Birke \BBA\
  Sarkar}{2006}]{birke2006}
Birke, J.\BBACOMMA\ \BBA\ Sarkar, A. \BBOP 2006\BBCP.
\newblock \BBOQ A Clustering Approach for Nearly Unsupervised Recognition of
  Nonliteral Language.\BBCQ\
\newblock In {\Bem 11th Conference of the European Chapter of the Association
  for Computational Linguistics}.

\bibitem[\protect\BCAY{Birke \BBA\ Sarkar}{Birke \BBA\
  Sarkar}{2007}]{birke2007}
Birke, J.\BBACOMMA\ \BBA\ Sarkar, A. \BBOP 2007\BBCP.
\newblock \BBOQ Active Learning for the Identification of Nonliteral
  Language.\BBCQ\
\newblock In {\Bem Proceedings of the Workshop on Computational Approaches to
  Figurative Language}, \mbox{\BPGS\ 21--28}. Association for Computational
  Linguistics.

\bibitem[\protect\BCAY{Bradley \BBA\ Terry}{Bradley \BBA\
  Terry}{1952}]{bradley1952}
Bradley, R.~A.\BBACOMMA\ \BBA\ Terry, M.~E. \BBOP 1952\BBCP.
\newblock \BBOQ Rank Analysis of Incomplete Block Designs: I.\ The Method of
  Paired Comparisons.\BBCQ\
\newblock {\Bem Biometrika}, {\Bbf 39}  (3/4), \mbox{\BPGS\ 324--345}.

\bibitem[\protect\BCAY{Charteris-Black}{Charteris-Black}{2011}]{black2011}
Charteris-Black, J. \BBOP 2011\BBCP.
\newblock {\Bem Politicians and Rhetoric: The Persuasive Power of Metaphor}.
\newblock Springer.

\bibitem[\protect\BCAY{Deignan}{Deignan}{2005}]{deignan2005}
Deignan, A. \BBOP 2005\BBCP.
\newblock {\Bem Metaphor and Corpus Linguistics}.
\newblock John Benjamins Publishing.

\bibitem[\protect\BCAY{Ford}{Ford}{1957}]{ford1957}
Ford, L.~R. \BBOP 1957\BBCP.
\newblock \BBOQ Solution of a Ranking Problem from Binary Comparisons.\BBCQ\
\newblock {\Bem The American Mathematical Monthly}, {\Bbf 64}  (8),
  \mbox{\BPGS\ 28--33}.

\bibitem[\protect\BCAY{Huang, Socher, Manning, \BBA\ Ng}{Huang
  et~al.}{2012}]{huang2012}
Huang, E., Socher, R., Manning, C., \BBA\ Ng, A. \BBOP 2012\BBCP.
\newblock \BBOQ Improving Word Representations via Global Context and Multiple
  Word Prototypes.\BBCQ\
\newblock In {\Bem Proceedings of the 50th Annual Meeting of the Association
  for Computational Linguistics (Volume 1: Long Papers)}, \mbox{\BPGS\
  873--882}. Association for Computational Linguistics.

\bibitem[\protect\BCAY{Isahara, Bond, Uchimoto, Utiyama, \BBA\ Kanzaki}{Isahara
  et~al.}{2008}]{isahara2008}
Isahara, H., Bond, F., Uchimoto, K., Utiyama, M., \BBA\ Kanzaki, K. \BBOP
  2008\BBCP.
\newblock \BBOQ Development of the Japanese WordNet.\BBCQ\
\newblock In {\Bem Proceedings of the 6th International Conference on Language
  Resources and Evaluation}.

\bibitem[\protect\BCAY{Kawahara \BBA\ Kurohashi}{Kawahara \BBA\
  Kurohashi}{2006a}]{kawahara2006a}
Kawahara, D.\BBACOMMA\ \BBA\ Kurohashi, S. \BBOP 2006a\BBCP.
\newblock \BBOQ Case Frame Compilation from the Web using High-Performance
  Computing.\BBCQ\
\newblock In {\Bem Proceedings of the 5th International Conference on Language
  Resources and Evaluation}. European Language Resources Association (ELRA).

\bibitem[\protect\BCAY{Kawahara \BBA\ Kurohashi}{Kawahara \BBA\
  Kurohashi}{2006b}]{kawahara2006b}
Kawahara, D.\BBACOMMA\ \BBA\ Kurohashi, S. \BBOP 2006b\BBCP.
\newblock \BBOQ A Fully-Lexicalized Probabilistic Model for Japanese Syntactic
  and Case Structure Analysis.\BBCQ\
\newblock In {\Bem Proceedings of the Human Language Technology Conference of
  the NAACL, Main Conference}, \mbox{\BPGS\ 176--183}. Association for
  Computational Linguistics.

\bibitem[\protect\BCAY{Kawahara, Kurohashi, \BBA\ Hasida}{Kawahara
  et~al.}{2002}]{kawahara2002}
Kawahara, D., Kurohashi, S., \BBA\ Hasida, K. \BBOP 2002\BBCP.
\newblock \BBOQ Construction of a Japanese Relevance-tagged Corpus.\BBCQ\
\newblock In {\Bem Proceedings of the 3rd International Conference on Language
  Resources and Evaluation}. European Language Resources Association (ELRA).

\bibitem[\protect\BCAY{Lakoff}{Lakoff}{1987}]{lakoff1987}
Lakoff, G. \BBOP 1987\BBCP.
\newblock {\Bem Women, Fire, and Dangerous Things}.
\newblock University of Chicago Press.

\bibitem[\protect\BCAY{Lakoff}{Lakoff}{1993}]{lakoff1993}
Lakoff, G. \BBOP 1993\BBCP.
\newblock \BBOQ The Contemporary Theory of Metaphor.\BBCQ\
\newblock In Ortony, A.\BED, {\Bem Metaphor and Thought\/} (second \BEd).,
  \BCH~11, \mbox{\BPGS\ 202--251}. Cambridge University Press.

\bibitem[\protect\BCAY{Low}{Low}{2010}]{low2010}
Low, G. \BBOP 2010\BBCP.
\newblock \BBOQ Wot No Similes? The Curious Absence of Simile in University
  Lectures.\BBCQ\
\newblock In Low, G., Todd, Z., Deignan, A., \BBA\ Cameron, L.\BEDS, {\Bem
  Researching and Applying Metaphor in the Real World}, \mbox{\BPGS\ 291--308}.
  John Benjamins Publishing.

\bibitem[\protect\BCAY{Mikolov, Chen, Corrado, \BBA\ Dean}{Mikolov
  et~al.}{2013}]{mikolov2013}
Mikolov, T., Chen, K., Corrado, G., \BBA\ Dean, J. \BBOP 2013\BBCP.
\newblock \BBOQ Efficient Estimation of Word Representations in Vector
  Space.\BBCQ\
\newblock {\Bem arXiv preprint arXiv:1301.3781}.

\bibitem[\protect\BCAY{Miller}{Miller}{1995}]{miller1995}
Miller, G.~A. \BBOP 1995\BBCP.
\newblock \BBOQ WordNet: A Lexical Database for English.\BBCQ\
\newblock {\Bem Communications of the ACM}, {\Bbf 38}  (11), \mbox{\BPGS\
  39--41}.

\bibitem[\protect\BCAY{Miyazawa \BBA\ Miyao}{Miyazawa \BBA\
  Miyao}{2017}]{miyazawa2017}
Miyazawa, A.\BBACOMMA\ \BBA\ Miyao, Y. \BBOP 2017\BBCP.
\newblock \BBOQ Evaluation Metrics for Automatically Generated Metaphorical
  Expressions.\BBCQ\
\newblock In {\Bem Proceedings of the 12th International Conference on
  Computational Semantics (Short papers)}.

\bibitem[\protect\BCAY{Miyazawa, Yoshida, \BBA\ Miyao}{Miyazawa
  et~al.}{2016}]{miyazawa2016}
Miyazawa, A., Yoshida, N., \BBA\ Miyao, Y. \BBOP 2016\BBCP.
\newblock \BBOQ Nihongo Metafaa Koopasu Sakusei No Tame No Gaidorain [Guideline
  for Creating Japanese Metaphor Corpus].\BBCQ\
\newblock In {\Bem Proceedings of the 22nd Annual Meeting of the Association
  for Natural Language Processing}. Association for Natural Language
  Processing.

\bibitem[\protect\BCAY{Richards}{Richards}{1965}]{richards1965}
Richards, I.~A. \BBOP 1965\BBCP.
\newblock {\Bem The Philosophy of Rhetoric}.
\newblock Oxford University Press.

\bibitem[\protect\BCAY{Steen, Dorst, Herrmann, Kaal, Krennmayr, \BBA\
  Pasma}{Steen et~al.}{2010}]{steen2010}
Steen, G.~J., Dorst, A.~G., Herrmann, J.~B., Kaal, A., Krennmayr, T., \BBA\
  Pasma, T. \BBOP 2010\BBCP.
\newblock {\Bem A Method for Linguistic Metaphor Identification: From MIP to
  MIPVU}.
\newblock John Benjamins Publishing.

\bibitem[\protect\BCAY{Terai \BBA\ Nakagawa}{Terai \BBA\
  Nakagawa}{2010}]{terai2010}
Terai, A.\BBACOMMA\ \BBA\ Nakagawa, M. \BBOP 2010\BBCP.
\newblock \BBOQ A Computational System of Metaphor Generation with Evaluation
  Mechanism.\BBCQ\
\newblock In {\Bem International Conference on Artificial Neural Networks},
  \mbox{\BPGS\ 142--147}. Springer.

\bibitem[\protect\BCAY{Tsvetkov, Boytsov, Gershman, Nyberg, \BBA\
  Dyer}{Tsvetkov et~al.}{2014}]{tsvetkov2014}
Tsvetkov, Y., Boytsov, L., Gershman, A., Nyberg, E., \BBA\ Dyer, C. \BBOP
  2014\BBCP.
\newblock \BBOQ Metaphor Detection with Cross-Lingual Model Transfer.\BBCQ\
\newblock In {\Bem Proceedings of the 52nd Annual Meeting of the Association
  for Computational Linguistics (Volume 1: Long Papers)}, \mbox{\BPGS\
  248--258}. Association for Computational Linguistics.

\bibitem[\protect\BCAY{Tsvetkov, Mukomel, \BBA\ Gershman}{Tsvetkov
  et~al.}{2013}]{tsvetkov2013}
Tsvetkov, Y., Mukomel, E., \BBA\ Gershman, A. \BBOP 2013\BBCP.
\newblock \BBOQ Cross-Lingual Metaphor Detection Using Common Semantic
  Features.\BBCQ\
\newblock In {\Bem Proceedings of the 1st Workshop on Metaphor in NLP},
  \mbox{\BPGS\ 45--51}. Association for Computational Linguistics.

\bibitem[\protect\BCAY{Williams}{Williams}{1986}]{williams1986}
Williams, L. \BBOP 1986\BBCP.
\newblock {\Bem Teaching for the Two-sided Mind}.
\newblock Simon and Schuster.

\bibitem[\protect\BCAY{Wilson}{Wilson}{1988}]{wilson1988}
Wilson, M. \BBOP 1988\BBCP.
\newblock \BBOQ MRC Psycholinguistic Database: Machine-usable Dictionary,
  version 2.00.\BBCQ\
\newblock {\Bem Behavior Research Methods, Instruments, {\&} Computers}, {\Bbf
  20}  (1), \mbox{\BPGS\ 6--10}.

\bibitem[\protect\BCAY{Yamada, Shibata, Sakai, Kuramochi, Yamada, Uwano, Ijima,
  \BBA\ Sasahara}{Yamada et~al.}{2011}]{yamada2011}
Yamada, T., Shibata, T., Sakai, K., Kuramochi, Y., Yamada, A., Uwano, Z.,
  Ijima, M., \BBA\ Sasahara, H.\BEDS\ \BBOP 2011\BBCP.
\newblock {\Bem Shin Meikai Kokugo Jiten [Shin Meikai Japanese Dictionary]\/}
  (seventh \BEd).
\newblock Sanseido.

\end{thebibliography}


\begin{biography}

\bioauthor[:]{Akira Miyazawa}{He received his B.A.\ degree from Waseda University in 2012. He is currently a Ph.D.\ candidate at the Graduate University for Advanced Studies. His research interests include natural language processing, in particular, computational approaches to metaphor.}

\bioauthor[:]{Yusuke Miyao}{He received his Ph.D.\ from the University of Tokyo in 2006. He has been Assistant Professor at the University of Tokyo from 2001 to 2010, Associate Professor at National Institute of Informatics since 2010, and Professor at the University of Tokyo from 2018. He has been engaged in research on natural language processing, in particular on syntactic parsing and its applications.}
\end{biography}


\biodate




\clearpage












\clearpage
























\end{document}
