    \documentclass[english]{jnlp_2.1}

\usepackage{jnlpbbl_1.3}
\usepackage[dvipdfmx]{graphicx}
\usepackage{amsmath}
\usepackage[multi]{otf}
\usepackage{hangcaption_jnlp}
\usepackage{udline}
\setnami
\newfont{\cyr}{wncyr10}

\usepackage{scalerel} 
\usepackage{multirow} 
\usepackage{colortbl} 
    \newcommand\G[2]{\multicolumn{1}{>{\columncolor[gray]{0.8}}#1}{#2}}
\providecommand{\asis}{}


\Volume{25}
\Number{5}
\Month{December}
\Year{2018}

\received{2018}{5}{1}
\revised{2018}{8}{16}
\accepted{2018}{9}{18}

\setcounter{page}{599}

\esubject{Paper}
\etitle{Syntactic Matching Methods in Pivot Translation}
\eauthor{Akiva Miura\affiref{Author_1} \and Graham Neubig\affiref{Author_1} \affiref{Author_2} \and Katsuhito Sudoh\affiref{Author_1} \and Satoshi Nakamura\affiref{Author_1}}
\eabstract{
The pivot translation is useful method for translating between languages that contain little or no parallel data by utilizing equivalents in an intermediate language such as English. Commonly, phrase-based or tree-based pivot translation methods merge source--pivot and pivot--target translation models into a source--target model. This tactic is known as triangulation. However, the combination is based on the surface forms of constituent words, and it often produces incorrect source--target phrase pairs because of interlingual differences and semantic ambiguities in the pivot language. The translation accuracy is thus degraded. This paper proposes a triangulation approach that utilizes syntactic subtrees in the pivot language to avoid incorrect phrase combina\-tions by distinguishing pivot language words by their syntactic roles. The results of the experiments conducted on the United Nations Parallel Corpus demonstrate that the proposed method is superior to other pivot translation approaches in all tested combinations of languages.
}
\ekeywords{Pivot Translation, Machine Translation, Parallel Corpus, Low-Resourced \linebreak Language Pairs, Syntactic Analysis}

\headauthor{Miura, Neubig, Sudoh and Nakamura}
\headtitle{Syntactic Matching Methods in Pivot Translation}

\affilabel{Author_1}{}{Graduate School of Information Science, Nara Institute of Science and Technology}
\affilabel{Author_2}{}{Language Technologies Institute, Carnegie Mellon University}


\begin{document}

\maketitle



\section{Introduction}
\label{sec:intro}

Translation effected using models trained on larger parallel corpora can achieve greater accuracy \cite{dyer08} 
in statistical machine translation (SMT) \cite{brown93smt}.
Unfortunately, most language pairs are restricted in terms of readily available parallel corpora: some have fewer than 100~k sentence pairs; others do not contain any.
This paucity is especially true of language pairs that do not include English, and the problem is difficult to overcome because it would cost millions of dollars to manually produce a high-quality parallel corpus.

One effective solution to surmount the scarceness of bilingual data is to introduce a pivot language that contains existing parallel data with respect to both the source and target languages \cite{gispert06}.
The triangulation is both popular and effective among the various methods that employ pivot languages \cite{utiyama07pivot,cohn07}.
This process first combines source--pivot and pivot--target translation models (TMs) into a source--target model and then translates data using this merged model.
The procedure of triangulating two TMs into one has been examined for different frameworks of SMT and its effectiveness has been confirmed both for Phrase-Based SMT (PBMT) \cite{koehn03pbmt,utiyama07pivot} and for Hierarchical Phrase-Based SMT (Hiero) \cite{chiang07hiero,miura15acl}.
However, word interlingual differences in word usage and word sense ambiguity cause difficulties in accurately learning the correspondences between the source and target phrases.
Thus, the accuracy of triangulated models is lesser than the precision attained by models trained on direct parallel corpora.

\begin{figure}[b]
\begin{center}
\includegraphics{25-5ia6f1.eps}
\end{center}
\hangcaption{Example of disambiguation by parse subtree matching (Fr-En-Zh), [X1] and [X2] are non-terminals for sub-phrases.}
\label{fig:pivot-tree}
\end{figure}

In the triangulation method, source--pivot and pivot--target phrase pairs are connected as source--target pairs if a common pivot-side phrase is available.
Fig.~\ref{fig:pivot-tree}(a) illustrates a sample standard triangulation on the Hiero TM, which combines the hierarchical rules of phrase pairs by matching  pivot phrases with equivalent surface forms.
This example also demonstrates the problems of ambiguity: the English word ``record'' can correspond to several different parts-of-speech according to the context.
More broadly, phrases that include this word can also potentially take different grammatical structures, 
\pagebreak
but it is impossible to uniquely identify these constructions unless information is provided with regard to the surrounding context.

This varying syntactic structure will influences translation.
For example, the French verb ``enregistrer'' corresponds to the English verb ``record''.
At the same time, the French noun ``dossier'' matches the noun form of the English word ``record''.
In a more extreme instance, Chinese does not incorporate inflections depending on the part-of-speech of the word.
Thus, although the word order changes, the Chinese term ``\UTFC{8BB0}\UTFC{5F55}'' is used even in contexts where record is employed as a different grammatical category.
These specifics might result in the incorrect connection of ``[X1] enregistrer [X2]'' and ``[X2] [X1] \UTFC{8BB0}\UTFC{5F55}'' even though the proper correspondence of ``[X1] enregistrer [X2]'' and ``[X1] dossier [X2]'' would be ``[X1] \UTFC{8BB0}\UTFC{5F55} [X2]'' and ``[X2] [X1] \UTFC{8BB0}\UTFC{5F55}'' respectively.
Hence, a superficial phrase matching method based solely on the surface form of the pivot often combines incorrect phrase pairs, causing translation errors if the translation scores of the mached pairs are estimated to be higher than the actual correspondences.

Given this background, it is hypothesized that the disambiguation of these cases would be simpler if necessary syntactic information such as phrase structures is considered during the pivoting process.
To incorporate this intuition into the models introduced in this paper, the authors propose a method that considers the syntactic information of the pivot phrase as shown in Fig.~\ref{fig:pivot-tree}(b).
In this manner, the model distinguishes the translation rules extracted in contexts within which the English symbol string ``[X1] record [X2]'' behaves as a verbal phrase from situations in which the same string acts as a nominal phrase.

Specifically, the method posited in this paper is based on Synchronous Context-Free Grammars (SCFGs) \cite{aho69scfg,chiang07hiero}, which are widely used in tree-based machine translation frameworks.
First, Section \ref{sec:mt-framework} of the paper provides a quick review of SCFGs.
The baseline triangulation method that only uses the surface forms for performing the triangulation is detailed in Section \ref{sec:pivot}, and two methods for triangulation based on syntactic matching are proposed in Section \ref{sec:proposed}.
The first method places a hard restriction on the exact matching of parse trees (Section \ref{sec:exact-match}) included in translation rules, whereas the second places a softer limitation and allows partial matches (Section \ref{sec:partial-match}).
Experiments of pivot translation on the United Nations Parallel Corpus \cite{ziemski16un} were performed by the authors to investigate the proposed method's impact on pivot translation quality.
The results of these investigations are presented in Section \ref{sec:exp}.
These findings demonstrate that the posited process indeed provides significant gains in accuracy (of up to 2.3 BLEU points), in almost all tested combinations of five languages with English used as the pivot language.
In addition, as an auxiliary result, the authors compared pivot translations effected through the use of the proposed method to those made through zero-shot neural machine translation.
These outcomes confirm that the triangulation of symbolic TMs still significantly outperforms neural MT in the zero-resource scenario.
\footnote{A preliminary version of this paper has presented in (三浦，Neubig，中村2016a) and \cite{miura17wmt}.\nocite{miura16nl07}}


\section{Machine Translation Framework}
\label{sec:mt-framework}


\subsection{Synchronous Context-Free Grammars (SCFGs)}
\label{sec:scfg}

In this section, the authors initially deal with SCFGs, particularly hierarchical phrase-based translation (Hiero) \cite{chiang07hiero}, which are widely used in machine translation.
The elementary structures used in translation in SCFGs are synchronous rewrite rules with aligned pairs of source and target symbols on the right side as in
\begin{equation}
X \to \left< \overline{s},~ \overline{t} \right>
\end{equation}
where $X$ is the head symbol of the rewrite rule, and $\overline{s}$ and $\overline{t}$ are both strings of terminals and non-terminals on the source and target sides respectively.
Each string in the right-side pair has the same number of indexed non-terminals, and identically indexed non-terminals correspond to each-other.
A synchronous rule could also, for example, take the form of
\begin{equation}
X \to \left< X_0 \mbox{~ of ~} X_1 ,~ X_1 \mbox{~的~} X_0 \right> .
\end{equation}

Synchronous rules can be extracted based on parallel sentences and automatically obtained word alignments.
Each extracted rule is scored with phrase translation probabilities in both directions $\phi(\overline{s}|\overline{t})$ and $\phi(\overline{t}|\overline{s})$, lexical translation probabilities in both directions $\phi_{lex}(\overline{s}|\overline{t})$ and $\phi_{lex}(\overline{t}|\overline{s})$, a word penalty counting the terminals in $\overline{t}$, and a constant phrase penalty of $1$.

At the time of translation, the decoder searches for the target sentence that maximizes the derivation probability, which is defined as the sum of the scores of the rules used in the derivation, and the log of the language model (LM) probability over the target strings.
When not considering an LM, it is possible to efficiently find the best translation for an input sentence using the CKY+ algorithm \cite{chappelier98}.
When using an LM, the expanded search space is further reduced based on the limit on expanded edges, or total states per span, through a procedure such as cube pruning \cite{chiang07hiero}.


\subsection{Hierarchical Rules}
\label{sec:hiero}

The rules used in Hiero are specifically discussed in this section.
Hierarchical rules are composed of the initial head symbol $S$ and synchronous rules containing terminals and singular type of non-terminal $X$.
\footnote{It is also standard to include a glue rule $S \to \left<X_0,~ X_0 \right>$, $S \to \left<S_0 ~ X_1 ,~ S_0 ~ X_1 \right>$, $S \to \left<S_0 ~ X_1 ,~ X_1 ~ S_0 \right>$ to fall back on when standard rules cannot result in a proper derivation.}
Hierarchical rules are extracted using the same phrase extraction procedure employed in phrase-based translation \cite{koehn03pbmt} based on word alignments, followed by a step that performs a recursive extraction of hierarchical phrases \cite{chiang07hiero}.

For example, hierarchical rules could take the form of
\begin{align}
& X \to \left< \mbox{Officers},~ \mbox{主席団} ~ \mbox{成員} \right> , \\
& X \to \left< \mbox{the Committee},~ \mbox{委員会} \right> , \\
& X \to \left< X_0 ~\mbox{of}~ X_1,~ X_1 ~\mbox{的}~ X_0 \right> .
\end{align}
From these rules, the input sentence can be translated by the derivation:
\begin{align*}
S & \to         \left< X_0 ,~ X_0 \right> \\
& \Rightarrow \left< X_1 ~\mbox{of}~ X_2 ,~ X_2 ~\mbox{的}~ X_1 \right> \\
& \Rightarrow \left< \mbox{Officers of}~ X_2 ,~ X_2 ~ \mbox{的} ~ \mbox{主席団} ~ \mbox{成員} \right> \\
& \Rightarrow \left< \mbox{Officers of the Committee} ,~ \right. \\ 
&             \left. ~~~~~ \mbox{委員会} ~ \mbox{的} ~ \mbox{主席団} ~ \mbox{成員} \right> .
\end{align*}

The advantage of Hiero is that it is able to achieve relatively superior word reordering accuracy (compared to other symbolic SMT alternatives such as standard phrase-based MT) without language-dependent processing.
On the other hand, since it does not use syntactic information and tries to extract all possible combinations of rules, Hiero tends to extract very large translation rule tables, and it is also likely to be less syntactically faithful in its derivations.


\subsection{Explicitly Syntactic Rules}
\label{sec:t2s}

The use of synchronous context-free grammar or synchronous tree-substitution grammar \cite{graehl04} rules forms an alternative to the Hiero rules.
These options explicitly take into account the syntax of the source-side (tree-to-string rules), the target-side (string-to-tree rules), or both (tree-to-tree rules).
The tree-to-string (T2S) rules, for example, utilize parse trees on the source language side, and the head symbols of the synchronous rules are not limited to $S$ or $X$, but instead use non-terminal symbols corresponding to the phrase structure tags of a given parse tree.
Thus, T2S rules could take the form of
\begin{align}
& X_{\mbox{NP}} \to \left< \mbox{(NP (NNS Officers))},~ \mbox{主席団} ~ \mbox{成員} \right> , \\
& X_{\mbox{NP}} \to \left< \mbox{(NP (DT the) (NNP Committee))},~ \mbox{委員会} \right> , \\
& X_{\mbox{PP}} \to \left< \mbox{(PP (IN of) $X_{\mbox{NP,0}}$)},~ X_0~ \mbox{的} \right> , \\
& X_{\mbox{NP}} \to \left< \mbox{(NP $X_{\mbox{NP,0}}$ $X_{\mbox{PP,1}}$)},~ X_1~ X_0 \right> .
\end{align}
Here, parse subtrees of the source language rules are set in the form of S-expressions.
From these rules, the translation can be effected from the parse tree of the input sentence by the derivation:

{ \scriptsize
    \begin{eqnarray*}
    	X_{\mbox{ROOT}} & \to         & \left< \mbox{$X_{\mbox{NP,0}}$},~ X_0 \right> \\
    	                & \Rightarrow & \left< \mbox{(NP $X_{\mbox{NP,1}}$ $X_{\mbox{PP,2}}$)},~ X_2~ X_1 \right> \\
    	                & \Rightarrow & \left< \mbox{(NP (NP (NNS Officers) $X_{\mbox{PP,2}}$))},~ X_2~ \scalebox{1.4}{\mbox{主席団} ~ \mbox{成員}} \right> \\
    	                & \overset{*}{\Rightarrow} &
    		{ \scaleleftright[2ex]
    		  {\langle}
    		    {
   		      {\fontsize{6.5pt}{8pt}\selectfont
    		        \begin{array}{l}
    		          \mbox{(NP} \\
    		          ~~\mbox{(NP (NNS Officers))} \\
    		          ~~\mbox{(PP (IN of)} \\
    		          ~~~~\mbox{(NP (DT the)} \\ 
    		          ~~~~~~\mbox{(NNP Committee))))}
    		        \end{array}
   		      },~ \scalebox{1.4}{\mbox{委員会} ~ \mbox{的} ~ \mbox{主席団} ~ \mbox{成員}}
    		    }
    		  {\rangle}
    		}
    \end{eqnarray*}
}

It is hence possible in T2S translation to obtain a result that conforms to the grammar of the source language.
Also, as an advantage of this method, the number of less-useful synchronous rules extracted by syntax-agnostic methods such as Hiero is reduced.
This decrease makes it possible to learn more compact rule tables and allows for faster translation.


\section{Pivot Translation Methods}
\label{sec:pivot}

Several methods for SMT using pivot languages have been proposed, including \textit{cascade} methods that consecutively translate from source to pivot then pivot to target \cite{gispert06}, \textit{synthetic} methods that machine-translate the training data to generate a pseudo-parallel corpus \cite{gispert06}, and \textit{triangulation} methods that obtain a source--target phrase/rule table by merging source--pivot and pivot--target table entries with identical pivot language phrases \cite{cohn07}.
The triangulation method is particularly notable for producing higher quality translation results in comparison to other pivot methods \cite{utiyama07pivot,miura15acl}, and this approach has thus been employed as the grounding for the work presented in this paper.


\subsection{Triangulation of TMs}
\label{sec:triangulation}

SCFG training procedure stores the extracted and scored phrase pairs from a bilingual corpus into a structured file called the \textit{rule table}.
Fig.~\ref{fig:pivot-triangulation} presents a diagram of the \textit{triangulation} of the source--pivot and pivot--target rule tables into the source--target table.

\begin{figure}[b]
\begin{center}
\includegraphics{25-5ia6f2.eps}
\end{center}
\caption{Triangulation of translation models}
\label{fig:pivot-triangulation}
\end{figure}

Triangulation for SCFGs searches $T_{SP}$ and $T_{PT}$ for source--pivot and pivot--target rules that have common pivot symbols and synthesizes these into source--target rules to create rule table $T_{ST}$:
\begin{equation}
\begin{split}
& X \to \left< \overline{s} ,~ \overline{t} \right> \in T_{ST} \\
& s.t.~ X \to \left< \overline{s}, \overline{p}\right> \in T_{SP} ~\wedge~ X \to \left< \overline{p}, \overline{t} \right> \in T_{PT} .
\end{split}
\label{eqn:rule-triangulation}
\end{equation}

Phrase translation probability $\phi(\cdot)$ and lexical translation probability $\phi_{lex}(\cdot)$ are estimated for all combined source--target phrases according to:
\begin{alignat}{2}
\label{eqn:triangulation-begin}
\phi \left( \overline{t} | \overline{s} \right) 
& = \sum_{ \overline{p} \in T_{SP} \cap T_{PT} } \phi \left( \overline{t} | \overline{p},\overline{s} \right) \phi \left( \overline{p} | \overline{s} \right)
&& \approx \sum_{ \overline{p} \in T_{SP} \cap T_{PT} } \phi \left( \overline{t} | \overline{p} \right) \phi \left( \overline{p} | \overline{s} \right)
, \\
\phi \left( \overline{s} | \overline{t} \right)
& = \sum_{ \overline{p} \in T_{SP} \cap T_{PT} } \phi \left( \overline{s} | \overline{p},\overline{t} \right) \phi \left( \overline{p} | \overline{t} \right)
&& \approx \sum_{ \overline{p} \in T_{SP} \cap T_{PT} } \phi \left( \overline{s} | \overline{p} \right) \phi \left( \overline{p} | \overline{t} \right)
, \\
\phi_{lex} \left( \overline{t} | \overline{s} \right)
& = \sum_{ \overline{p} \in T_{SP} \cap T_{PT} } \phi_{lex} \left( \overline{t} | \overline{p},\overline{s} \right) \phi_{lex} \left( \overline{p} | \overline{s} \right)
&& \approx \sum_{ \overline{p} \in T_{SP} \cap T_{PT} } \phi_{lex} \left( \overline{t} | \overline{p} \right) \phi_{lex} \left( \overline{p} | \overline{s} \right)
, \\
\phi_{lex} \left( \overline{s} | \overline{t} \right)
& = \sum_{ \overline{p} \in T_{SP} \cap T_{PT} } \phi_{lex} \left( \overline{s} | \overline{p},\overline{t} \right) \phi_{lex} \left( \overline{p} | \overline{t} \right)
&& \approx \sum_{ \overline{p} \in T_{SP} \cap T_{PT} } \phi_{lex} \left( \overline{s} | \overline{p} \right) \phi_{lex} \left( \overline{p} | \overline{t} \right)
,
\label{eqn:triangulation-end}
\end{alignat}
where $\overline{s}$, $\overline{p}$, and $\overline{t}$ are the phrases in the source, pivot, and target, respectively, and the construction $\overline{p} \in T_{SP} \cap T_{PT}$ indicates that $\overline{p}$ is contained in both phrase tables $T_{SP}$ and $T_{PT}$.
Word penalty and phrase penalty $X \rightarrow \left<\overline{s},~ \overline{t}\right>$ are set as the same values of $X \rightarrow \left<\overline{p},~ \overline{t}\right>$.


\subsection{Problems of Pivot-Side Ambiguity}
\label{sec:ambiguity}

Although triangulation is known to achieve higher translation accuracy than other simple methods and although it has become a popular and standard form of pivot translation nowadays, the problem of ambiguity still remains.
This subsection describes the causes of the difficulties and provides pertinent examples.

In triangulation, Equations (\ref{eqn:triangulation-begin})--(\ref{eqn:triangulation-end}) are based on the memoryless channel model, which assumes
\begin{eqnarray}
\phi \left( \overline{t} | \overline{p}, \overline{s} \right) & = & \phi \left( \overline{t} | \overline{p} \right) , \label{eqn:memoryless-begin} \\
\phi \left( \overline{s} | \overline{p}, \overline{t} \right) & = & \phi \left( \overline{s} | \overline{p} \right) .
\end{eqnarray}
In Equation (\ref{eqn:memoryless-begin}), for example, it is presumed that, given the pivot and source phrases, the translation probability of the target phrase is not affected by the source phrase.
Nonetheless, it is easy to produce examples where this assumption does not hold.

Fig.~\ref{fig:ambiguity} exemplifies three words in German and Italian.
Each of the terms corresponds to the polysemic English word ``approach''.
In such a case, it is extremely difficult to find associated source--target phrase pairs and to estimate the translation probabilities properly.
As a result, pivot translation is significantly more ambiguous than standard translation.

\begin{figure}[b]
\begin{center}
\includegraphics{25-5ia6f3.eps}
\end{center}
\caption{An example of ambiguity in De-En-It triangulation.}
\label{fig:ambiguity}
\end{figure}

\begin{figure}[b]
\begin{center}
\includegraphics{25-5ia6f4.eps}
\end{center}
\caption{An example of the triangulation method remembering the pivot.}
\label{fig:remember}
\end{figure}

The authors of this paper have previously proposed a pivot translation method that uses the triangulation of Synchronous CFG rule tables to a Multi-Synchronous CFG (MSCFG) \cite{neubig15mscfg} rule table that remembers the pivot as shown in Fig.~\ref{fig:remember}.
This method performs the translation using pivot LMs \cite{miura15acl}, and experiments have established that the process is effective in cases when a strong pivot LM exists.

This previously conceived method is effective in instances where the existing source--pivot and pivot--target parallel corpora are not large (i.e., containing less than hundreds of thousands of sentence pairs) and, conversely, where the available pivot monolingual corpus is sizable.
Since the MSCFG decoder demands an immense quantity of memory and computational time and it is difficult to accomplish distributed processing, it is not realistic to scale up within the same framework.
Further, although the MSCFG decoder helps in selecting the appropriate translation rules in a pivot LM, it cannot essentially solve the problem of ambiguity and of inappropriately connected rules, which remain in the triangulated rule table as noise.
This paper attempts to elucidate the manner in which the noisy rules in triangulated TMs can be reduced to bring them closer to the accuracy attained by directly trained TMs.


\section{Triangulation with Syntactic Matching}
\label{sec:proposed}

The previous section outlined the standard triangulation method and marked that the pivot-side ambiguity causes an incorrect estimation of translation probability and that the translation accuracy might decrease for this reason.
To address this problem, it is desirable to be able to distinguish pivot-side phrases that have different syntactic roles or meanings, even if the symbol strings are equivalent.
The next two subsections describe two methods of discerning pivot phrases that take on syntactically discrete roles: the first technique involves exact matching of parse trees; the second pertains to soft matching.


\subsection{Exact Matching of Parse Subtrees}
\label{sec:exact-match}

In the exact matching method, the pivot--source and pivot--target T2S TMs are first trained by parsing the pivot side of parallel corpora.
Next, these data are stored into rule tables as $T_{PS}$ and $T_{PT}$, respectively.
The synchronous rules of $T_{PS}$ and $T_{PT}$ correspondingly take the form of $X \to \left< \hat{p}, \overline{s} \right>$ and $X \to \left< \hat{p}, \overline{t} \right>$, where $\hat{p}$ is a symbol string that expresses the pivot-side parse subtree (S-expression), and $\overline{s}$ and $\overline{t}$ express the source and target symbol strings in that order.
The procedure of synthesizing source--target synchronous rules essentially follows Equations (\ref{eqn:triangulation-begin})--(\ref{eqn:triangulation-end}), except that it utilizes $T_{PS}$ instead of $T_{SP}$ (the direction of probability features is reversed) and the pivot subtree $\hat{p}$ instead of pivot phrase $\overline{p}$.
In this case, $\overline{s}$ and $\overline{t}$ do not have syntactic information, and thus, the synthesized synchronous rules should be hierarchical rules as explained in Section \ref{sec:hiero}.

The matching conditions of this method are more stringent in their constraints than the correspondence of superficial symbols in standard triangulation and thus potentially lessen incorrect connections of phrase pairs, resulting in a more reliable triangulated TM.
Conversely, the number of connected rules decreases as well in this restricted triangulation, and hence, the coverage of the triangulated model might be reduced.
Therefore, it is important to create TMs that are both reliable and that comprise superior coverage.


\subsection{Partial Matching of Parse Subtrees}
\label{sec:partial-match}

To prevent the problem of the reduction of coverage in the exact matching method, the authors of this paper propose a partial matching method that retains the coverage of standard triangulation by allowing the connection of incompletely equivalent pivot subtrees.
To estimate translation probabilities in partial matching, the \textit{weighted triangulation} generalizing Equations (\ref{eqn:triangulation-begin})--(\ref{eqn:triangulation-end}) of standard triangulation with the weight function $\psi(\cdot)$ must first be defined as in
\begin{align}
\label{eqn:weighted-triangulation-begin}
\phi \left( \overline{t} | \overline{s} \right) & = \sum_{\hat{p_T}} \sum_{\hat{p_S}} \phi \left( \overline{t} | \hat{p_T} \right) \psi \left( \hat{p_T} | \hat{p_S} \right) \phi \left( \hat{p_S} | \overline{s} \right) , \\
\phi \left( \overline{s} | \overline{t} \right) & = \sum_{\hat{p_S}} \sum_{\hat{p_T}} \phi \left( \overline{s} | \hat{p_S} \right) \psi \left( \hat{p_S} | \hat{p_T} \right) \phi \left( \hat{p_T} | \overline{t} \right) , \\
\phi_{lex} \left( \overline{t} | \overline{s} \right) & = \sum_{\hat{p_T}} \sum_{\hat{p_S}} \phi_{lex} \left( \overline{t} | \hat{p_T} \right) \psi \left( \hat{p_T} | \hat{p_S} \right) \phi_{lex} \left( \hat{p_S} | \overline{s} \right) , \\
\phi_{lex} \left( \overline{s} | \overline{t} \right) & = \sum_{\hat{p_S}} \sum_{\hat{p_T}} \phi_{lex} \left( \overline{s} | \hat{p_S} \right) \psi \left( \hat{p_S} | \hat{p_T} \right) \phi_{lex} \left( \hat{p_T} | \overline{t} \right) ,
\label{eqn:weighted-triangulation-end}
\end{align}
where $\hat{p_S} \in T_{SP}$ and $\hat{p_T} \in T_{PT}$ are, respectively, the pivot parse subtrees of source--pivot and pivot--target synchronous rules.
By adjusting $\psi(\cdot)$, the magnitude of the penalty for instances of incompletely matched connections can be controlled.
If it is defined that $\psi(\hat{p_T}|\hat{p_S}) = 1$ when $\hat{p_T}$ is equal to $\hat{p_S}$ and that otherwise $\psi(\hat{p_T}|\hat{p_S}) = 0$, Equations (\ref{eqn:weighted-triangulation-begin})--(\ref{eqn:weighted-triangulation-end}) are equivalent to Equations (\ref{eqn:triangulation-begin})--(\ref{eqn:triangulation-end}).

The better estimation of $\psi(\cdot)$ is not trivial, and the co-occurrence counts of $\hat{p_S}$ and $\hat{p_T}$ are not available.
Therefore, a heuristic estimation method is introduced as
{\allowdisplaybreaks
\begin{align}
\label{eqn:psi-begin}
\psi(\hat{p_T}|\hat{p_S}) & = \frac {w(\hat{p_S},\hat{p_T})} {\sum_{\hat{p} \in T_{PT}} w(\hat{p_S}, \hat{p})} \cdot \max_{\hat{p} \in T_{PT}} {w(\hat{p_S},\hat{p})} , \\
\psi(\hat{p_S}|\hat{p_T}) & = \frac {w(\hat{p_S},\hat{p_T})} {\sum_{\hat{p} \in T_{SP}} w(\hat{p}, \hat{p_T})} \cdot \max_{\hat{p} \in T_{SP}} {w(\hat{p},\hat{p_T})} , \\
w(\hat{p_S},\hat{p_T})    & =
\begin{cases}
0 ~~~~~~~~~~~~~~ \left(flat(\hat{p_S}) \neq flat(\hat{p_T})\right) \\
\exp \left(- d \left(\hat{p_S},\hat{p_T}\right)\right) ~~~~ (otherwise)
\end{cases} , \\
d(\hat{p_S},\hat{p_T})    & = TreeEditDistance(\hat{p_S},\hat{p_T}) ,
\label{eqn:psi-end}
\end{align}
}
where $flat(\hat{p})$ returns only the sequence of leaf elements, or the symbol string of $\hat{p}$ keeping non-terminals,
\footnote{For example, given $\hat{p} = (NP (NP (NNS \mbox{Officers})) (PP (IN \mbox{of}) (NP (DT \mbox{the}) (NNP \mbox{Committee}))))$, then, $flat(\hat{p}) = \mbox{``Officers of the Committee''}$.}
and $TreeEditDistance(\hat{p_S},\hat{p_T})$ is the minimum cost of a sequence of operations (contract, un-contract, and modify the label of an edge) that are required to transform $\hat{p_S}$ into $\hat{p_T}$ \cite{klein98distance}.

According to Equations (\ref{eqn:psi-begin})--(\ref{eqn:psi-end}), it may be assured that the incomplete match of pivot subtrees leads to $d(\cdot) \geq 1$ and penalizes in a manner that $\psi(\cdot) \leq 1 / e^d \leq 1 / e$, and an exact match of subtrees causes a value of $\psi(\cdot)$ that is at least $e \approx 2.718$ times larger than those obtained by the utilization of partially matched subtrees.


\section{Experiments}
\label{sec:exp}

\subsection{Experimental Set-Up}
\label{sec:exp-syntax-setup}

\subsubsection{Evaluation of Pivot SMT Methods}

To investigate the effect of the proposed approach, the authors evaluated translation accuracy through pivot translation experiments conducted on the United Nations Parallel Corpus (UN6Way) \cite{ziemski16un}.
UN6Way is a line-aligned multilingual parallel corpus that includes data in English (En), Arabic (Ar), Spanish (Es), French (Fr), Russian (Ru) and Chinese (Zh), and accounts for different families of languages.
The corpus contains over 11~M sentences for each language pair and is therefore deemed suitable for multilingual translation tasks such as pivot translation.
English was fixed as the pivot language for the experiments reported in this paper because it is the language most frequently employed for this function.
Its utilization as the pivot language thus yields the positive side-effect of the readily available accurate phrase structure parsers, benefits the proposed method.
Pivot translation was performed on all combinations of the other five languages, and the accuracy of each method was compared.
For tokenization, SentencePiece,\footnote{https://github.com/google/sentencepiece} an unsupervised text tokenizer and detokenizer, was adopted.
Although it is designed primarily for use in neural MT, it was confirmed that SentencePiece also helps in reducing training time and that it also improves translation accuracy in the authors' previously posited Hiero model.
A single shared tokenization model was first trained by feeding a total of 10~M sentences from the data of all six languages, setting the maximum shared vocabulary size to be 16~k, and all available text was tokenized with the trained model.
English raw text was used without SentencePiece tokenization for phrase structure analysis and for training Hiero and T2S TMs on the pivot side.
To generate parse trees, the Ckylark PCFG-LA parser \cite{oda15ckylark} was utilized, and lines over 60 tokens in length were filtered out from all parallel data to ensure the accuracy of parsing and alignment.
Once the sorting was accomplished, about 7.6~M lines remained.
Since Hiero requires a large quantity of computational resources for training and decoding, the decision was taken to use only the first 1~M lines to train each TM, instead of the entirety of the available training data.
\footnote{The authors use the same 1~M pivot-side sentences in source-pivot and pivot-target parallel data, even though it is not a realistic situation in pivot translation. This setting is useful to investigate how pivot translation tasks degrade accuracy from the ideal condition by comparing with scores of Direct methods. The previous work of the authors (Miura et al. 2015; 三浦，Neubig，Sakti，戸田，中村 2016b) has not found large differences in translation accuracy between situations of using the same pivot sentences and no common pivot sentences.\nocite{miura15acl,miura16jnlp}}
Travatar \cite{neubig13travatar} was employed as a decoder.
Hiero and T2S TMs were utilized and were trained with Travatar's rule extraction code.
5-gram LMs were trained over the target side of the same parallel data utilized for training TMs using KenLM \cite{heafield11}.
The first 1,000 lines of the 4,000 lines of test and dev sets were used for testing and parameter tuning, respectively.
For the evaluation of translation results, the text was de-tokenized with SentencePiece and re-tokenized with the tokenizer from the Moses toolkit \cite{koehn07moses} for Arabic, Spanish, French and Russian.
The Chinese text was re-tokenized with KyTea tokenizer \cite{neubig11kytea} and was then evaluated with the utilization of case-sensitive BLEU-4 \cite{papineni02bleu}, RIBES \cite{isozaki10ribes} and NIST \cite{doddington02nist}.

Five translation methods were assessed:

\begin{description}
\item[\bf ~~ Cascade Hiero:]\mbox{}\\
Sequential pivot translation with source--pivot and pivot--target Hiero TMs (weak baseline).
\item{\bf Tri. Hiero:}\mbox{}\\
Triangulating source--pivot and pivot--target Hiero TMs into a source--target Hiero TM using the traditional method (baseline, Section \ref{sec:triangulation}).
\item{\bf Tri. TreeExact}\mbox{}\\
Triangulating pivot--source and pivot--target T2S TMs into a source--target Hiero TM using the proposed exact matching of pivot subtrees (proposed 1, Section \ref{sec:exact-match}).
\item{\bf Tri. TreePartial}\mbox{}\\
Triangulating pivot--source and pivot--target T2S TMs into a source--target Hiero TM using the proposed partial matching of pivot subtrees (proposed 2, Section \ref{sec:partial-match}).
\item{\bf Direct Hiero:}\mbox{}\\
Translating with a Hiero TM directly trained on the source-target parallel corpus without using a pivot language (as an oracle).
\end{description}


\subsubsection{Comparison with Neural MT}

Recent investigations \cite{firat16zero,johnson17zeroshot} have found that neural machine translation systems can gain the ability to perform translations with zero parallel resources by training on multiple sets of bilingual data.
However, previous work has not examined the competitiveness of these methods in comparison to pivot-based symbolic SMT frameworks such as PBMT or Hiero.
In this section, a zero-shot NMT model and other pivot NMT methods are compared to the pivot-based Hiero models.
The NMTKit \footnote{https://github.com/odashi/nmtkit} was adopted to train and evaluate NMT models.
The detailed parameters of training the NMT models are shown in Table \ref{tab:nmt-setting}.

\begin{table}[b]
\caption{Main parameters of NMT training}
\label{tab:nmt-setting}
\input{06table01.tex}
\end{table}

For additional translation methods were assessed:

\begin{description}
\item[\bf ~~ Cascade NMT:]\mbox{}\\
Sequential pivot translation with source-pivot and pivot-target NMTs.
\item{\bf Synthetic NMT:}\mbox{}\\
Generating pseudo-parallel corpus synthesized by translating pivot-side of source-pivot parallel corpus with pivot-target NMT \cite{sennrich16monolingual,firat16zero}.
\item{\bf Zero-Shot NMT:}\mbox{}\\
Training single shared model with pvt \asis{$\leftrightarrow$} \{src,target\} parallel data according to \cite{johnson17zeroshot}.
\item{\bf Direct NMT:}\mbox{}\\
Translating with NMT directly trained on the source-target parallel corpus without using pivot language (for comparison).
\end{description}

The training data for Cascade NMT, Synthetic NMT, and Zero-Shot NMT were the same 1~M sentences used for the Pivot Hiero methods (source--pivot and pivot--target corpora).
The training data for Direct NMT was identical to that utilized for Direct Hiero.


\subsection{Results}
\label{sec:exp-syntax-results}

\noindent
\textbf{Comparison of Direct models among different language pairs and frameworks:}
Before pivot translation tasks were compared, the performances of SMT and NMT in Direct translation tasks were ascertained.
Table \ref{tab:direct-smt-vs-nmt} demonstrates the BLEU-4, RIBES, and NIST scores for each Direct translation task and language pair.
The results confirm the tendency that directly trained NMT models achieve high translation accuracy even in the case of translation between languages of different families
On the other hand, these scores are drastically reduced in situations that do not offer source--target parallel corpora for training.

\begin{table}[t]
\caption{Comparison of SMT and NMT in multilingual Direct translation tasks}
\label{tab:direct-smt-vs-nmt}
\input{06table02.tex}
\end{table}

Since the parameters were optimized for the BLEU score and not for RIBES and NIST, higher RIBES/NIST scores do not necessarily imply the superior performance of the MT framework.
They merely demonstrate the side effects obtained by optimizing for BLEU.
However, the comparison of RIBES and NIST scores shows the obvious propensities of SMT and NMT frameworks.
In almost language pairs, Direct Hiero outperformed Direct NMT in NIST scores, even though its BLEU and RIBES scores were lower than NMT.
It is known that NMT faces difficulties with rare words and that it tends to fail in the translation of vocabulary that is not frequently used.
This discovery reflects the fact that NIST is recognized as a measurement tool that values the translation accuracy of content words that generally occur in lower frequencies in comparison to function words.
Conversely, NMT is capable of acquiring higher naturalness and fluency in word sequences that include function words, since BLEU provides weightage to the accuracy of word sequences ($n$-grams) and RIBES is known to give importance word order.

The table also demonstrates that the evaluation scores of language-pairs that do not contain English are much lower than those that include English.
For example, the BLEU scores of Arabic--English, English--French and Arabic--French in Direct Hiero are 40.00, 40.57, and 33.34, respectively.
\pagebreak
It is thought that sentence pairs that do not include English are more noisy than those that contain English since the multilingual corpus was primarily constructed by sourcing from English documents as the pivot.

\vspace{1\Cvs}\noindent
\textbf{Performance of English-related translation tasks:}
Pivot translation tasks should depend strongly on the performance of the source--pivot translation and should rely even more compellingly on the pivot--target translation since the pivot--target translation essentially comprises the upper bound performance of generating target sentences for the given pivot-side input.
It is natural that the translation for pairs of languages belonging to different families exhibits a different trend with regard to accuracy.
Table \ref{tab:direct-smt-vs-nmt} illustrates that the TMs of English--Spanish and English--French achieve higher evaluation scores, perhaps because they exhibit relatively closer language structures than the other evaluated English-relative language pairs.
Although English--Arabic, English--Russian, and English--Chinese translation achieve poorer accuracy, each of these likely result from different language features, such as morphology, word order, and diversity of expression.

\begin{table}[b]
\caption{Components of BLEU score for English-related translation tasks}
\label{tab:english-related}
\input{06table03.tex}
\end{table}

Table \ref{tab:english-related} illustrates the components of the BLEU score evaluation, including the precision of 1-grams through 4-grams and the brevity penalty \cite{papineni02bleu}.
This table demonstrates that English--Chinese translation achieved higher accuracy in translating words, or 1-gram precision than language pairs that comprises Arabic, French, and Russian targets.
However, the precision of 2-grams through 4-grams, or the accuracy of translating word sequences is relatively lower in English--Chinese and this low BLEU score is primarily caused by the low 4-gram precision.
This result reflects the fact that word inflections do not exist in Chinese, and instead, the word order takes on significant syntactic roles.

Conversely, the table also clarifies that English--Arabic and English--Russian translation 
\linebreak
achieved lower precision even relating to 1-grams.
This lack of accuracy could be caused by the fact that Arabic and Russian are known for their morphological richness, and it is thus more difficult for MT to translate the source words into correct forms of target words than in the case of other, more morphologically simple languages.

\begin{table}[b]
\caption{Comparison of each method}
\label{tab:result-pivot}
\input{06table04.tex}
\vspace{4pt}
\small
Bold face indicates the highest BLEU score in pivot translation, and daggers indicate statistically significant gains over Tri. Hiero (${}^{\dagger}p < 0.05, {}^{\ddagger}p < 0.01$).
\end{table}

\vspace{1\Cvs}\noindent
\textbf{Translation accuracy of pivot translation methods:}
The results of the experiments that used all combinations of pivot translation tasks via English for five languages are shown in Table \ref{tab:result-pivot}.
These outcomes exhibit that the proposed partial matching method of pivot subtrees in triangulation outperformed the standard triangulation method for all language pairs and that it achieved higher or almost equal scores than the proposed exact matching method.
The exact matching method also outperformed the standard triangulation method in the majority of the language pairs, but has a lesser improvement than the partial matching method.
As demonstrated by the authors' previous research undertaking, the sequential pivot translation was uniformly weaker than all triangulation methods.

\begin{table}[b]
\caption{Comparison of rule table coverage in proposed triangulation methods}
\label{tab:result-coverage}
\input{06table05.tex}
\end{table}

\vspace{1\Cvs}\noindent
\textbf{Effect on coverage:}
Table \ref{tab:result-coverage} presents the outcomes of the comparison of the coverage achieved by each proposed triangulation method.
This table confirms that Tri.~TreeExact reduced the number of unique phrases by several percentage points and Tri.~TreePartial kept the same coverage as Tri.~Hiero.
Especially, triangulated TMs from Chinese with exact matching contain substantially fewer source phrases and significantly source words, and harmed coverage up to 4.884\% as Chinese contains many characters and values the reordering of short tokens instead of inflections.
This anomaly could constitute one of the reasons for the difference in improvement stability with regard to the partial and exact matching methods.

\begin{table}[b]
\caption{Comparison of noise ratio in triangulated rule table}
\label{tab:result-noise}
\input{06table06.tex}
\end{table}

\vspace{1\Cvs}\noindent
\textbf{Noise reduction:}
The main motivation of using parse trees in the proposed methods is to prevent the inappropriate connection of phrase correspondences and reduce the noise in rule tables.
To investigate the manner in which the syntactic matching methods succeed in removing noisy rules, an analysis of noise ratio was conducted.
Noisy rules must contain source and target phrases having no correspondence in meaning, though this decision cannot be made for all phrase pair candidates in rule tables.
It was therefore assumed that directly trained TMs that could avail of a source--target parallel corpus would demonstrate a fine approximation close to the ideal distribution of translation probability.
To compute the noise ratio \asis{$noise(T_{tri}|T_{dir})$} of triangulated rule table \asis{$T_{tri}$} with directly trained table \asis{$T_{dir}$}
\begin{equation}
noise \left( T_{tri} | T_{dir} \right) = 
\frac
{
\sum_{\left(\overline{s},\overline{t}\right) \in T_{tri} \setminus T_{dir}} \phi \left(\overline{t}|\overline{s}\right)
}
{
\sum_{\left(\overline{s},\overline{t}\right) \in T_{tri}} \phi \left(\overline{t}|\overline{s}\right)
}
\end{equation}
was defined, where \asis{$\phi(\overline{t}|\overline{s})$} represents the forward translation probability that can be considered the most important feature of the rule table.
Table \ref{tab:result-noise} displays the calculated noise ratio of the rule table for each triangulation method and language pair.
This result shows that, although triangulated rule tables contain many noisy rules, the syntactic matching methods are indeed successful in reducing them.
Tri.~TreeExact decreased noisy rules, up to  $-16.26$\%, and Tri.~TreePartial lessened noisy rules up to $-10.94$\%.
The reason why the noise reduction rate of Tri.~TreePartial is lower than that of Tri.~TreeExact is that the former weakens the influence of noisy rules instead of removing them to retain the coverage.

\begin{table}[b]
\caption{Comparison of distribution error rate in triangulated rule table}
\label{tab:result-error}
\input{06table07.tex}
\end{table}

\vspace{1\Cvs}\noindent
\textbf{Improvement of probability estimation:}
Although syntactic matching methods help in reducing noisy rules, there is no guarantee that they can improve the estimation of translation probabilities.
Table \ref{tab:result-error} exhibits the mean absolute error (MAE) and the root-mean-square error (RMSE) for the distribution of forward translation probability scores of triangulated rule tables in comparison to directly trained rule tables.
To calculate MAE and RMSE, the noisy rules that were not contained in directly trained rule tables were ignored to separate the different factors.
The results evince that Tri.~TreePartial reduced MAE and RMSE, making the distribution closer to the ideal in almost all language pairs.
On the other hand, Tri.~TreeExact did not diminish the errors in a stable manner.
This consequence may be induced by the fact that the restricted matching conditions of Tri.~TreeExact exclude many unmatched phrase pair candidates and may remove even those translation rules that are not noisy.
It may therefore be posited that the softening of restrictions pertaining to matching conditions aids in the improvement of the estimation of translation probabilities.

\begin{table}[b]
\caption{Comparison of SMT and NMT in pivot translation tasks}
\label{tab:smt-vs-nmt}
\input{06table08.tex}
\end{table}

\vspace{1\Cvs}\noindent
\textbf{Comparison with NMT:}

Table \ref{tab:smt-vs-nmt} presents the BLEU score of each translation task and language pair.
Perhaps, Synthetic NMT outperformed Cascade NMT for the majority of language pairs because multi-layer NNs are robust for noisy training data and can optimize the trained model with fine-tuning techniques.
On the other hand, the fine-tuning is available for Cascade NMT only separately for the source--pivot and pivot--target TMs and not for the whole pipelined system.

In the setting of the current experiments, although bilingually trained NMT systems were competitive to or outperformed Hiero-based models, the zero-shot translation was uniformly weaker.
This outcome could be the result of using only a single LSTM layer for each encoder and decoder or because there was an insufficient quantity of parallel corpora or language pairs.
It may therefore be posited that, although zero-shot translation demonstrated reasonable results in some settings, successful zero-shot translation systems are difficult to build, and pivot-based symbolic MT systems such as PBMT or Hiero might still be competitive alternatives.

\vspace{1\Cvs} \noindent
\textbf{Qualitative analysis:}
A translated sentence for which pivot-side ambiguity is resolved in the syntactic matching methods is presented as an example:

\begin{description}  
\item{\bf Source Sentence in French:}\mbox{}\\
La Suisse \ul{encourage {\bfseries tous les \'{E}tats}} \textbf{parties} \unc{{\`{a}} soutenir le travail conceptuel que fait} 
\linebreak
\unc{actuellement le Secr{\'{e}}tariat} .
\item{\bf Corresponding Sentence in English:}\\
Switzerland encourages all parties to support the current conceptual work of the secretariat.
\item{\bf Reference in Spanish:}\mbox{}\\
Suiza \ul{alienta a {\bfseries todos los Estados}} \textbf{partes} \unc{a que apoyen la actual labor {\itshape conceptual} de} \unc{la Secretar{\'{i}a}} .
\item{\bf Direct Hiero:}\mbox{}\\
Suiza alienta a todos los Estados partes a que apoyen el trabajo conceptual que se examinan en la Secretar\'{\i}a . (BLEU+1: 55.99, RIBES: 91.47, NIST: 9.687)
\item{\bf Tri. Hiero:}\mbox{}\\
Suiza \unc{{\itshape conceptuales} para apoyar la labor que en estos momentos la Secretar{\'{\i}}a} \ul{alienta a todos los} Estados Partes . (BLEU+1: 29.74, RIBES: 53.92, NIST: 6.204)
\item{\bf Tri. TreeExact:}\mbox{}\\
Suiza alienta a \textbf{todos los Estados Partes} a apoyar la labor conceptual que actualmente la Secretar\'{\i}a . (BLEU+1: 43.08, RIBES: 88.36, NIST: 8.625)
\item{\bf Tri. TreePartial:}\mbox{}\\
Suiza alienta a \textbf{todos los Estados Partes} a apoyar la labor conceptual que actualmente la Secretar\'{\i}a . (same with Tri. TreeExact)
\item{\bf Direct NMT:}\mbox{}\\
Suiza alienta a todos los Estados partes a que apoyen la labor conceptual que est\'{a} realizando la Secretar\'{\i}a . (BLEU+1: 66.94, RIBES: 95.79, NIST: 11.22)
\item{\bf Synthetic NMT:}\mbox{}\\
Suiza alent\'{o} a todos los Estados Partes a que apoyen el trabajo conceptual que actualmente est\'{a} a la Secretar\'{\i}a . (BLEU+1: 28.94, RIBES: 88.01, NIST: 7.267)
\end{description}

The results of Tri.~TreeExact and Tri.~TreePartial are identical in this example.
The digest of the derivation process in Tri. Hiero is
\begin{align*}
S & \Rightarrow \left< X_0 ,~ X_0 \right> \\
& \Rightarrow \left< La ~ Suisse ~ X_1 ,~ Suiza ~ X_1 \right> \\
& \Rightarrow \left< La ~ Suisse ~ X_2 ~ . ,~ Suiza ~ X_2 ~ . \right> \\
& \Rightarrow \left< La ~ Suisse ~ \text{\ul{\mbox{$X_3$}}} ~ partie ~ \text{\unc{\mbox{$X_4$}}} ~ . ,~ Suiza ~ \text{\unc{\mbox{$X_4$}}} ~ \text{\ul{\mbox{$X_3$}}} ~ Partes ~ . \right> \\
& \Rightarrow ...
\end{align*}

On the other hand, the digest of the derivation process in Tri.~TreeExact/Tri.~TreePartial is
{\small
\begin{align*}
S & \Rightarrow \left< X_0 ,~ X_0 \right> \\
& \Rightarrow \left< La ~ Suisse ~ X_1 ,~ Suiza ~ X_1 \right> \\
& \Rightarrow \left< La ~ Suisse ~ encourage ~ X_2 ~ X_3 ,~ Suiza ~ alienta ~ a ~ X_2 ~ X_3 \right> \\
& \Rightarrow \left< La ~ Suisse ~ encourage ~ tous ~ les ~ X_4 ~ partie ~ X_3,~ Suiza ~ alienta ~ a ~ todos ~ X_4 ~ Partes ~ X_3 \right> \\
& \Rightarrow \left< La ~ Suisse ~ encourage ~ tous ~ les ~ Etats ~ partie ~ X_3,~ Suiza ~ alienta ~ a ~ todos ~ los ~ Estados ~ Partes ~ X_3 \right> \\
& \Rightarrow ... 
\end{align*}
}

Here, it is observed that the derivation in Tri.~Hiero uses rule $X \to \langle X_0 ~\mbox{parties}~ X_1 , \allowbreak X_1 ~ X_0$ $\mbox{Partes} \rangle$\footnote{The words emphasized with underline and wavy-underline in the example correspond to $X_0$ and $X_1$ respectively.} which causes the incorrect reordering of phrases, followed by steps of incorrect word selection.
\footnote{For example, the word ``conceptuales'' with italic face in Tri.~Hiero takes the wrong form and position.}
On the other hand, the derivation in Tri.~TreeExact and Tri.~TreePartial uses rule $X \to \left<\mbox{tous} ~ \mbox{les}~ X_0 ~\mbox{parties} \right.$, $\left. \mbox{todos}~ X_0 ~\mbox{Partes}\right>$\footnote{The words emphasized in bold face in the example correspond to the rule.} as synthesized from the T2S rules with the common pivot subtree \linebreak $(\mbox{NP} ~ (\mbox{DT all}) ~ (\mbox{NP'} ~ X_{\mbox{NNP}} \allowbreak (\mbox{NNS} ~ \mbox{parties}))$.
It can thus be proved that the derivation improves the word-selection and word-reordering using this rule.

The following example presents a translated sentence for which the exact matching method loses the necessary translation rule and instead degrades accuracy.

\begin{description}  
\item{\bf Source Sentence in Chinese}\mbox{}\\
秘\UTFC{4E66}\UTFC{957F} \UTFC{5173}于 `` 人力 \UTFC{8D44}源 管理 改革 概\UTFC{89C8} : \ul{\mbox{流\UTFC{52A8}性}} '' 的 \UTFC{62A5}告
\item{\bf Corresponding Sentence in English:}\\
Report of the Secretary-General on the overview of human resources management reform : \ul{mobility}
\item{\bf Reference in Russian}\mbox{}\\
Доклад Генерального секретаря об обзоре хода реформы системы управления людскими ресурсами : \ul{мобильность}
\item{\bf Direct Hiero:}\mbox{}\\
Доклад Генерального секретаря об реформе управления людскими ресурсами : обзор \ul{мобильность} {\cyr {\symbol{062}}} (BLEU+1: 45.47, RIBES: 92.29, NIST: 9.143)
\item{\bf Tri. Hiero:}\mbox{}\\
Доклад Генерального секретаря о реформе управления людскими ресурсами обзор : \ul{Мобильность} {\cyr {\symbol{062}}} (BLEU+1: 26.25, RIBES: 86.67, NIST: 5.962)
\item{\bf Tri. TreeExact:}\mbox{}\\
Доклад Генерального секретаря \ul{мобильности} обзор реформе управления людскими ресурсами {\cyr {\symbol{062}}} : '' . (BLEU+1: 26.19, RIBES: 85.66, NIST: 5.652)
\item{\bf Tri. TreePartial:}\mbox{}\\
Доклад Генерального секретаря о Обзор реформе системы управления людскими ресурсами : \ul{мобильность} '' . (BLEU+1: 48.76, RIBES: 89.54, NIST: 8.107)
\item{\bf Direct NMT}\mbox{}\\
Доклад Генерального секретаря о `` Обзоре реформы управления людскими ресурсами : мобильность '' (BLEU: 45.37, RIBES: 91.22, NIST: 8.652)
\item{\bf Synthetic NMT}\mbox{}\\
Доклад Генерального секретаря о реформе управления людскими ресурсами : доклад Генерального секретаря о реформе управления людскими ресурсами : доклад Генерального секретаря (BLEU+1: 21.16, RIBES: 61.48, NIST: 3.608)
\end{description}

In this example, the corresponding Russian word form of the Chinese word ``流\UTFC{52A8}性'' (mobility) is ``мобильность.''
However, Tri.~TreeExact places this word in the incorrect case form ``мобильности'' and also positions it far from the correct placing since the translation rule connecting ``流\UTFC{52A8}性'' with the correct form ``мобильность'' is lost in the process of exact matching, and this rule is maintained in Tri.~Hiero and Tri.~TreePartial.
The selection of the incorrect case form often causes misplacing because of LMs, and it is affirmed that the results obtained by the use of Tri.~TreeExact contain more incorrect word forms and positions.


\subsection{Related Work}
\label{sec:pivot-related}

Up to this point, representative pivot translation methods in SMT have been explained.
Other related research studies in pivot translation are primarily based on the triangulation for PBMT and focuse on discussions to further improve accuracy \cite{zhu14,levinboim15,dabre15}.
The process of correctly estimating the translation probability is a problem in triangulation.

\shortciteA{zhu14} have proposed an estimation method of source--target translation probability by estimating source-target co-occurrence counts first instead of the direct estimation from source--pivot and pivot--target translation probabilities (Equations \ref{eqn:triangulation-begin}--\ref{eqn:triangulation-end}).
They have reported that stable translation accuracy can be obtained even in the triangulation of two phrase tables with unbalanced table size.

\shortciteA{levinboim15} have asserted that it is especially difficult to estimate word-level translation probability for phrase correspondence in the triangulation stage.
Subsequently, they have proposed a method for improving the quality of the triangulation by estimating the translation probability even for the correspondence of words which cannot be directly observed, using a distributed expression of words \cite{mikolov13}.

This paper focuses on pivot translation using English as the pivot language, though it is also known that translation accuracy varies on the manner in which a pivot language is selected.
The influence of the choice of the pivot language on pivot translation has been discussed in detail by \citeA{paul09acl}.
In reality, there are few situations in which the pivot language can be selected from multiple viable candidates, though in the ideal scenario where bilingual corpora of the same scale can be obtained via several languages, a pivot language having a similar language structure as the source or target language should be chosen.

Additionally, it is not necessary to limit the number of pivot languages to one, and methods that consider the simultaneous use of multiple pivot languages have also been proposed.
Representatives of such purposing include methods such as aggregating multiple source--target phrase/rule tables obtained by triangulation with respective pivot languages into one table with linear interpolation and those that accomplish searching via the simultaneouse use of multiple TMs \cite{dabre15}.

Alternatively, training methods of multilingual NMT, which improve translation accuracy by causing translation tasks of multiple language pairs to be trained as a common encoder, have been also proposed \cite{dong15,zoph16,johnson17zeroshot}.


\section{Conclusion}
\label{sec:conclustion}

In this paper, the authors have proposed a new method of pivot translation to resolve the pivot-side syntactic ambiguity.
This proposed method introduces an explicitly syntax-aware matching condition to find the correct correspondences between source-pivot and pivot-target translation rules.
It can thus produce more reliable models.
The results on the multilingual translation experiments revealed significant improvements in MT evaluation scores for the proposed method for all tested language pairs that possessed larger indirectly parallel corpora than \cite{miura15acl}, of 1~M sentence pairs and that had access to the additional resource of English syntactic parsing.
A syntactic matching method that allowed partial matching successfully reduced the number of noisy translation rules.
This noise decrease enhanced the estimation of translation probabilities and better translation accuracy.
This method is effective instances where that accurate syntactic parsers for the pivot language are available, and it is practical to use for pivot translations that allow access to larger quantities of parallel corpora.

To estimate translation probabilities, a heuristic that had no guarantee of being optimal was introduced.
The smoothing technique of TMs could present an effective solution as one of the directions to improve the estimation of probability scores.
It is common to apply smoothing methods such as back-off and interpolation when the coverage of a single model is poor.
In their experimental setting, the authors have incorporated reliable and high-coverage models, and it should be easy to combine them by linear interpolation with fixed coefficients.
In in the future, therefore, the authors are planning to explore more refined estimation methods that utilize machine learning.

Incidentally, the previously proposed method of pivot translation \cite{miura15acl} uses MSCFG models that possess the potential to access information from the source, target, and pivot languages.
For example, it should be possible to combine the proposed methods in Section \ref{sec:proposed} and \shortcite{miura15acl} and to let the MSCFG model remember the pivot tree structures.

As a more advanced method for future research, it should be possible to devise compounded MSCFG models that can store both pivot-side and source-side syntactic information, thereby realizing translations with higher reproducibility of source information.
The authors have mentioned that pivot translation presents the problem of the loss of source language information, which is affected by the expressiveness of the pivot language.
In fact, this problem is frequent not just in MT but also for human translators.
For example, since modern English is known for its simple morphology that does not include complicated grammatical conjugations such as personal suffixes, linguistic modalities such as number, case, and gender are lost when translating into English.
This deficiency yields a translation that is different from the original meaning from English into another language.
With the method posited in this paper, the authors aim to achieve machine translation outcomes that preserve the linguistic information of the original content by combining it with pivot-side syntactic structures.





\acknowledgment

Part of this work was supported by JSPS KAKENHI Grant Numbers JP16H05873 and JP17H06101.
The authors also thank the anonymous reviewers for their insightful comments.


\bibliographystyle{jnlpbbl_1.5}
\newcommand{\MiuraB}[1]{}\newcommand{\MiuraD}[1]{}
\begin{thebibliography}{}

\bibitem[\protect\BCAY{Aho \BBA\ Ullman}{Aho \BBA\ Ullman}{1969}]{aho69scfg}
Aho, A.~V.\BBACOMMA\ \BBA\ Ullman, J.~D. \BBOP 1969\BBCP.
\newblock \BBOQ Syntax Directed Translations and the Pushdown Assembler.\BBCQ\
\newblock {\Bem Journal of Computer and System Sciences}, {\Bbf 3}  (1),
  \mbox{\BPGS\ 37--56}.

\bibitem[\protect\BCAY{Brown, Pietra, Pietra, \BBA\ Mercer}{Brown
  et~al.}{1993}]{brown93smt}
Brown, P.~F., Pietra, V. J.~D., Pietra, S. A.~D., \BBA\ Mercer, R.~L. \BBOP
  1993\BBCP.
\newblock \BBOQ The Mathematics of Statistical Machine Translation: Parameter
  Estimation.\BBCQ\
\newblock {\Bem Computational Linguistics}, {\Bbf 19}, \mbox{\BPGS\ 263--312}.

\bibitem[\protect\BCAY{Chappelier \BBA\ Rajman}{Chappelier \BBA\
  Rajman}{1998}]{chappelier98}
Chappelier, J.-C.\BBACOMMA\ \BBA\ Rajman, M. \BBOP 1998\BBCP.
\newblock \BBOQ A Generalized CYK Algorithm for Parsing Stochastic CFG.\BBCQ\
\newblock In {\Bem Proceedings of TAPD}, \lowercase{\BVOL}~98, \mbox{\BPGS\
  133--137}. Citeseer.

\bibitem[\protect\BCAY{Chiang}{Chiang}{2007}]{chiang07hiero}
Chiang, D. \BBOP 2007\BBCP.
\newblock \BBOQ Hierarchical Phrase-Based Translation.\BBCQ\
\newblock {\Bem Computational Linguistics}, {\Bbf 33}  (2), \mbox{\BPGS\
  201--228}.

\bibitem[\protect\BCAY{Cohn \BBA\ Lapata}{Cohn \BBA\ Lapata}{2007}]{cohn07}
Cohn, T.\BBACOMMA\ \BBA\ Lapata, M. \BBOP 2007\BBCP.
\newblock \BBOQ Machine Translation by Triangulation: Making Effective Use of
  Multi-Parallel Corpora.\BBCQ\
\newblock In {\Bem Proceedings of ACL}, \mbox{\BPGS\ 728--735}.

\bibitem[\protect\BCAY{Dabre, Cromieres, Kurohashi, \BBA\ Bhattacharyya}{Dabre
  et~al.}{2015}]{dabre15}
Dabre, R., Cromieres, F., Kurohashi, S., \BBA\ Bhattacharyya, P. \BBOP
  2015\BBCP.
\newblock \BBOQ Leveraging Small Multilingual Corpora for SMT Using Many Pivot
  Languages.\BBCQ\
\newblock In {\Bem Proceedings of NAACL}, \mbox{\BPGS\ 1192--1202}.

\bibitem[\protect\BCAY{de~Gispert \BBA\ Mari{\~{n}}o}{de~Gispert \BBA\
  Mari{\~{n}}o}{2006}]{gispert06}
de~Gispert, A.\BBACOMMA\ \BBA\ Mari{\~{n}}o, J.~B. \BBOP 2006\BBCP.
\newblock \BBOQ Catalan-English Statistical Machine Translation without
  Parallel Corpus: Bridging through Spanish.\BBCQ\
\newblock In {\Bem Proceedings of LREC 5th Workshop on Strategies for
  Developing Machine Translation for Minority Languages}, \mbox{\BPGS\ 65--68}.

\bibitem[\protect\BCAY{Doddington}{Doddington}{2002}]{doddington02nist}
Doddington, G. \BBOP 2002\BBCP.
\newblock \BBOQ Automatic Evaluation of Machine Translation Quality Using
  N-gram Co-Occurrence Statistics.\BBCQ\
\newblock In {\Bem Proceedings of ARPA Workshop on Human Language Technology},
  \mbox{\BPGS\ 138--145}.

\bibitem[\protect\BCAY{Dong, Wu, He, Yu, \BBA\ Wang}{Dong
  et~al.}{2015}]{dong15}
Dong, D., Wu, H., He, W., Yu, D., \BBA\ Wang, H. \BBOP 2015\BBCP.
\newblock \BBOQ Multi-Task Learning for Multiple Language Translation.\BBCQ\
\newblock In {\Bem Proceedings of ACL}, \mbox{\BPGS\ 1723--1732}.

\bibitem[\protect\BCAY{Dyer, Cordova, Mont, \BBA\ Lin}{Dyer
  et~al.}{2008}]{dyer08}
Dyer, C., Cordova, A., Mont, A., \BBA\ Lin, J. \BBOP 2008\BBCP.
\newblock \BBOQ Fast, Easy, and Cheap: Construction of Statistical Machine
  Translation Models with MapReduce.\BBCQ\
\newblock In {\Bem Proceedings of WMT}, \mbox{\BPGS\ 199--207}.

\bibitem[\protect\BCAY{Firat, Sankaran, Al-Onaizan, Yarman~Vural, \BBA\
  Cho}{Firat et~al.}{2016}]{firat16zero}
Firat, O., Sankaran, B., Al-Onaizan, Y., Yarman~Vural, F.~T., \BBA\ Cho, K.
  \BBOP 2016\BBCP.
\newblock \BBOQ Zero-Resource Translation with Multi-Lingual Neural Machine
  Translation.\BBCQ\
\newblock In {\Bem Proceedings of EMNLP}, \mbox{\BPGS\ 268--277}.

\bibitem[\protect\BCAY{Graehl \BBA\ Knight}{Graehl \BBA\
  Knight}{2004}]{graehl04}
Graehl, J.\BBACOMMA\ \BBA\ Knight, K. \BBOP 2004\BBCP.
\newblock \BBOQ Training Tree Transducers.\BBCQ\
\newblock In {\Bem Proceedings of NAACL}, \mbox{\BPGS\ 105--112}.

\bibitem[\protect\BCAY{Heafield}{Heafield}{2011}]{heafield11}
Heafield, K. \BBOP 2011\BBCP.
\newblock \BBOQ KenLM: Faster and Smaller Language Model Queries.\BBCQ\
\newblock In {\Bem Proceedings of WMT}, \mbox{\BPGS\ 187--197}.

\bibitem[\protect\BCAY{Isozaki, Hirao, Duh, Sudoh, \BBA\ Tsukada}{Isozaki
  et~al.}{2010}]{isozaki10ribes}
Isozaki, H., Hirao, T., Duh, K., Sudoh, K., \BBA\ Tsukada, H. \BBOP 2010\BBCP.
\newblock \BBOQ Automatic Evaluation of Translation Quality for Distant
  Language Pairs.\BBCQ\
\newblock In {\Bem Proceedings of EMNLP}, \mbox{\BPGS\ 944--952}.

\bibitem[\protect\BCAY{Johnson, Schuster, Le, Krikun, Wu, Chen, Thorat,
  Vi{\'{e}}gas, Wattenberg, Corrado, Hughes, \BBA\ Dean}{Johnson
  et~al.}{2017}]{johnson17zeroshot}
Johnson, M., Schuster, M., Le, Q., Krikun, M., Wu, Y., Chen, Z., Thorat, N.,
  Vi{\'{e}}gas, F., Wattenberg, M., Corrado, G., Hughes, M., \BBA\ Dean, J.
  \BBOP 2017\BBCP.
\newblock \BBOQ Google's Multilingual Neural Machine Translation System:
  Enabling Zero-Shot Translation.\BBCQ\
\newblock {\Bem TACL}, {\Bbf 5}, \mbox{\BPGS\ 339--351}.

\bibitem[\protect\BCAY{Klein}{Klein}{1998}]{klein98distance}
Klein, P.~N. \BBOP 1998\BBCP.
\newblock \BBOQ Computing the Edit-Distance Between Unrooted Ordered
  Trees.\BBCQ\
\newblock In {\Bem Proceedings of of European Symposium on Algorithms},
  \mbox{\BPGS\ 91--102}.

\bibitem[\protect\BCAY{Koehn, Hoang, Birch, Callison-Burch, Federico, Bertoldi,
  Cowan, Shen, Moran, Zens, Dyer, Bojar, Constantin, \BBA\ Herbst}{Koehn
  et~al.}{2007}]{koehn07moses}
Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi,
  N., Cowan, B., Shen, W., Moran, C., Zens, R., Dyer, C., Bojar, O.,
  Constantin, A., \BBA\ Herbst, E. \BBOP 2007\BBCP.
\newblock \BBOQ Moses: Open Source Toolkit for Statistical Machine
  Translation.\BBCQ\
\newblock In {\Bem Proceedings of ACL}, \mbox{\BPGS\ 177--180}.

\bibitem[\protect\BCAY{Koehn, Och, \BBA\ Marcu}{Koehn
  et~al.}{2003}]{koehn03pbmt}
Koehn, P., Och, F.~J., \BBA\ Marcu, D. \BBOP 2003\BBCP.
\newblock \BBOQ Statistical Phrase-Based Translation.\BBCQ\
\newblock In {\Bem Proceedings of NAACL}, \mbox{\BPGS\ 48--54}.

\bibitem[\protect\BCAY{Levinboim \BBA\ Chiang}{Levinboim \BBA\
  Chiang}{2015}]{levinboim15}
Levinboim, T.\BBACOMMA\ \BBA\ Chiang, D. \BBOP 2015\BBCP.
\newblock \BBOQ Supervised Phrase Table Triangulation with Neural Word
  Embeddings for Low-Resource Languages.\BBCQ\
\newblock In {\Bem Proceedings of EMNLP}, \mbox{\BPGS\ 1079--1083}.

\bibitem[\protect\BCAY{Mikolov, Yih, \BBA\ Zweig}{Mikolov
  et~al.}{2013}]{mikolov13}
Mikolov, T., Yih, W.-t., \BBA\ Zweig, G. \BBOP 2013\BBCP.
\newblock \BBOQ Linguistic Regularities in Continuous Space Word
  Representations.\BBCQ\
\newblock In {\Bem Proceedings of NAACL}, \mbox{\BPGS\ 746--751}.

\bibitem[\protect\BCAY{三浦\JBA {Graham~Neubig}\JBA 中村}{三浦 \Jetal
  }{2016a}]{miura16nl07}
三浦明波\JBA {Graham~Neubig}\JBA 中村哲 \BBOP 2016a\BBCP.
\newblock 木構造を中間表現とするピボット翻訳手法.\
\newblock \Jem{情報処理学会 第227回自然言語処理研究会 (SIG-NL)}, 9\JNUM,
  \mbox{\BPGS\ 1--6}.

\bibitem[\protect\BCAY{\MiuraB{Miura}, Neubig, Sakti, Toda, \BBA\
  Nakamura}{\MiuraB{Miura} et~al.}{2015}]{miura15acl}
\MiuraB{Miura}, A., Neubig, G., Sakti, S., Toda, T., \BBA\ Nakamura, S. \BBOP
  2015\BBCP.
\newblock \BBOQ Improving Pivot Translation by Remembering the Pivot.\BBCQ\
\newblock In {\Bem Proceedings of ACL}, \mbox{\BPGS\ 573--577}.

\bibitem[\protect\BCAY{三浦\JBA {Graham~Neubig}\JBA {Sakriani~Sakti}\JBA
  戸田\JBA 中村}{三浦 \Jetal }{2016b}]{miura16jnlp}
三浦明波\JBA {Graham~Neubig}\JBA {Sakriani~Sakti}\JBA 戸田智基\JBA 中村哲 \BBOP
  2016b\BBCP.
\newblock 中間言語情報を記憶するピボット翻訳手法.\
\newblock \Jem{自然言語処理}, {\Bbf 23}  (5), \mbox{\BPGS\ 499--528}.

\bibitem[\protect\BCAY{\MiuraD{Miura}, Neubig, Sudoh, \BBA\
  Nakamura}{\MiuraD{Miura} et~al.}{2017}]{miura17wmt}
\MiuraD{Miura}, A., Neubig, G., Sudoh, K., \BBA\ Nakamura, S. \BBOP 2017\BBCP.
\newblock \BBOQ Tree as a Pivot: Syntactic Matching Methods in Pivot
  Translation.\BBCQ\
\newblock In {\Bem Proceedings of WMT}, \mbox{\BPGS\ 90--98}.

\bibitem[\protect\BCAY{Neubig}{Neubig}{2013}]{neubig13travatar}
Neubig, G. \BBOP 2013\BBCP.
\newblock \BBOQ Travatar: A Forest-to-String Machine Translation Engine based
  on Tree Transducers.\BBCQ\
\newblock In {\Bem Proceedings of ACL Demo Track}, \mbox{\BPGS\ 91--96}.

\bibitem[\protect\BCAY{Neubig, Arthur, \BBA\ Duh}{Neubig
  et~al.}{2015}]{neubig15mscfg}
Neubig, G., Arthur, P., \BBA\ Duh, K. \BBOP 2015\BBCP.
\newblock \BBOQ Multi-Target Machine Translation with Multi-Synchronous
  Context-free Grammars.\BBCQ\
\newblock In {\Bem Proceedings of NAACL}, \mbox{\BPGS\ 484--491}.

\bibitem[\protect\BCAY{Neubig, Nakata, \BBA\ Mori}{Neubig
  et~al.}{2011}]{neubig11kytea}
Neubig, G., Nakata, Y., \BBA\ Mori, S. \BBOP 2011\BBCP.
\newblock \BBOQ Pointwise Prediction for Robust, Adaptable Japanese
  Morphological Analysis.\BBCQ\
\newblock In {\Bem Proceedings of ACL}, \mbox{\BPGS\ 529--533}.

\bibitem[\protect\BCAY{Oda, Neubig, Sakti, Toda, \BBA\ Nakamura}{Oda
  et~al.}{2015}]{oda15ckylark}
Oda, Y., Neubig, G., Sakti, S., Toda, T., \BBA\ Nakamura, S. \BBOP 2015\BBCP.
\newblock \BBOQ Ckylark: A More Robust PCFG-LA Parser.\BBCQ\
\newblock In {\Bem Proceedings of NAACL}, \mbox{\BPGS\ 41--45}.

\bibitem[\protect\BCAY{Papineni, Roukos, Ward, \BBA\ Zhu}{Papineni
  et~al.}{2002}]{papineni02bleu}
Papineni, K., Roukos, S., Ward, T., \BBA\ Zhu, W.-J. \BBOP 2002\BBCP.
\newblock \BBOQ BLEU: A Method for Automatic Evaluation of Machine
  Translation.\BBCQ\
\newblock In {\Bem Proceedings of ACL}, \mbox{\BPGS\ 311--318}.

\bibitem[\protect\BCAY{Paul, Yamamoto, Sumita, \BBA\ Nakamura}{Paul
  et~al.}{2009}]{paul09acl}
Paul, M., Yamamoto, H., Sumita, E., \BBA\ Nakamura, S. \BBOP 2009\BBCP.
\newblock \BBOQ On the Importance of Pivot Language Selection for Statistical
  Machine Translation.\BBCQ\
\newblock In {\Bem Proceedings of NAACL}, \mbox{\BPGS\ 221--224}.

\bibitem[\protect\BCAY{Sennrich, Haddow, \BBA\ Birch}{Sennrich
  et~al.}{2016}]{sennrich16monolingual}
Sennrich, R., Haddow, B., \BBA\ Birch, A. \BBOP 2016\BBCP.
\newblock \BBOQ Improving Neural Machine Translation Models with Monolingual
  Data.\BBCQ\
\newblock In {\Bem Proceedings of ACL}, \mbox{\BPGS\ 86--96}.

\bibitem[\protect\BCAY{Utiyama \BBA\ Isahara}{Utiyama \BBA\
  Isahara}{2007}]{utiyama07pivot}
Utiyama, M.\BBACOMMA\ \BBA\ Isahara, H. \BBOP 2007\BBCP.
\newblock \BBOQ A Comparison of Pivot Methods for Phrase-Based Statistical
  Machine Translation.\BBCQ\
\newblock In {\Bem Proceedings of NAACL}, \mbox{\BPGS\ 484--491}.

\bibitem[\protect\BCAY{Zhu, He, Wu, Zhu, Wang, \BBA\ Zhao}{Zhu
  et~al.}{2014}]{zhu14}
Zhu, X., He, Z., Wu, H., Zhu, C., Wang, H., \BBA\ Zhao, T. \BBOP 2014\BBCP.
\newblock \BBOQ Improving Pivot-Based Statistical Machine Translation by
  Pivoting the Co-occurrence Count of Phrase Pairs.\BBCQ\
\newblock In {\Bem Proceedings of EMNLP}, \mbox{\BPGS\ 1665--1675}.

\bibitem[\protect\BCAY{Ziemski, Junczys-Dowmunt, \BBA\ Pouliquen}{Ziemski
  et~al.}{2016}]{ziemski16un}
Ziemski, M., Junczys-Dowmunt, M., \BBA\ Pouliquen, B. \BBOP 2016\BBCP.
\newblock \BBOQ The United Nations Parallel Corpus v1.0.\BBCQ\
\newblock In {\Bem Proceedings of LREC}, \mbox{\BPGS\ 3530--3534}.

\bibitem[\protect\BCAY{Zoph \BBA\ Knight}{Zoph \BBA\ Knight}{2016}]{zoph16}
Zoph, B.\BBACOMMA\ \BBA\ Knight, K. \BBOP 2016\BBCP.
\newblock \BBOQ Multi-Source Neural Translation.\BBCQ\
\newblock In {\Bem Proceedings of NAACL}, \mbox{\BPGS\ 30--34}.

\end{thebibliography}



\begin{biography}

\bioauthor[:]{Akiva Miura}{
received his B.Sc. from Technion - Israel Institute of Technology in 2013, and his M.Eng. and Ph.D. in information science from Nara Institute of Science and Technology in 2016 and 2018 respectively.
From 2018, he has been a researcher at Fujitsu Laboratories Ltd., where he is pursuing research in machine translation and knowledge technology.
}
\bioauthor[:]{Graham Neubig}{
received his B.E. from University of Illinois, Urbana-Champaign in 2005, and his M.S. and Ph.D. in informatics from Kyoto University in 2010 and 2012 respectively.
He is currently an assistant professor at Carnegie Mellon University and an affiliate associate professor at Nara Institute of Science and Technology.
His research interests include speech and natural language processing, with a focus on machine learning approaches for applications such as machine translation, speech recognition, and spoken dialog.
}
\bioauthor[:]{Katsuhito Sudoh}{
is an associate professor of Nara Institute of Science and Technology.
He received a bachelor's degree in engineering in 2000, and a master's and Ph.D. degree in informatics in 2002 and 2015, respectively, from Kyoto University.
He was in NTT Communication Science Laboratories from 2002 to 2017.
He currently works on machine translation and natural language processing.
He is a member of the Association for Computational Linguistics (ACL), the Association of Natural Language Processing (ANLP), the Information Processing Society of Japan (IPSJ) and the Acoustical Society of Japan (ASJ)
}
\bioauthor[:]{Satoshi Nakamura}{
is Professor of Graduate School of Information Science, Nara Institute of Science and Technology, Honorarprofessor of Karlsruhe Institute of Technology, Germany, and ATR Fellow.
He received his B.S. from Kyoto Institute of Technology in 1981 and Ph.D. from Kyoto University in 1992.
He was Director of ATR Spoken Language Communication Research Laboratories in 2000--2008 and Vice president of ATR in 2007--2008.
He was Director General of Keihanna Research Laboratories and the Executive Director of Knowledge Creating Communication Research Center, National Institute of Information and Communications Technology in 2009--2010.
He is currently Director of Augmented Human Communication laboratory and a full professor of Graduate School of Information Science at Nara Institute of Science and Technology.
He has been Elected Board Member of International Speech Communication Association, ISCA, since June 2011 and IEEE Signal Processing Magazine Editorial Board member since April 2012, IEEE SPS Speech and Language Technical Committee Member since 2013, and IEEE Fellow since 2016.
}

\end{biography}

\biodate



\end{document}
