WMT19

Conference on Machine Translation


Location

  • Florence, Italy

Links

Shared Tasks

Schedule

Day 1

   
8:45 Opening Remarks
9:00 Session 1: Shared Tasks Overview Presentations I
Barry Haddow
9:00 Findings of the 2019 Conference on Machine Translation (WMT19)
Loïc Barrault, Ondřej Bojar, Marta R. Costa-jussà, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Philipp Koehn, Shervin Malmasi, Christof Monz, Mathias Müller, Santanu Pal, Matt Post, Marcos Zampieri
9:35 Test Suites
9:50 Results of the WMT19 Metrics Shared Task: Segment-Level and Strong MT Systems Pose Big Challenges
Qingsong Ma, Johnny Wei, Ondřej Bojar, Yvette Graham
10:10 Findings of the First Shared Task on Machine Translation Robustness
Xian Li, Paul Michel, Antonios Anastasopoulos, Yonatan Belinkov, Nadir Durrani, Orhan Firat, Philipp Koehn, Graham Neubig, Juan Pino, Hassan Sajjad
10:30 ☕️
11:00 Session 2: Shared Task Poster Session I
11:00 Shared Task: News Translation
The University of Edinburgh’s Submissions to the WMT19 News Translation Task
Rachel Bawden, Nikolay Bogoychev, Ulrich Germann, Roman Grundkiewicz, Faheem Kirefu, Antonio Valerio Miceli Barone, Alexandra Birch
GTCOM Neural Machine Translation Systems for WMT19
Chao Bei, Hao Zong, Conghu Yuan, Qingming Liu, Baoyong Fan
Machine Translation with parfda, Moses, kenlm, nplm, and PRO
Ergun Biçici
LIUM’s Contributions to the WMT2019 News Translation Task: Data and Systems for German-French Language Pairs
Fethi Bougares, Jane Wottawa, Anne Baillot, Loïc Barrault, Adrien Bardet
The University of Maryland’s Kazakh-English Neural Machine Translation System at WMT19
Eleftheria Briakou, Marine Carpuat
DBMS-KU Interpolation for WMT19 News Translation Task
Sari Dewi Budiwati, Al Hafiz Akbar Maulana Siagian, Tirana Noor Fatyanosa, Masayoshi Aritsugi
Lingua Custodia at WMT’19: Attempts to Control Terminology
Franck Burlot
The TALP-UPC Machine Translation Systems for WMT19 News Translation Task: Pivoting Techniques for Low Resource MT
Noe Casas, José A. R. Fonollosa, Carlos Escolano, Christine Basta, Marta R. Costa-jussà
Kyoto University Participation to the WMT 2019 News Shared Task
Fabien Cromieres, Sadao Kurohashi
NICT’s Supervised Neural Machine Translation Systems for the WMT19 News Translation Task
Raj Dabre, Kehai Chen, Benjamin Marie, Rui Wang, Atsushi Fujita, Masao Utiyama, Eiichiro Sumita
The University of Sydney’s Machine Translation System for WMT19
Liang Ding, Dacheng Tao
UdS-DFKI Participation at WMT 2019: Low-Resource (en-gu) and Coreference-Aware (en-de) Systems
Cristina España-Bonet, Dana Ruiter
The IIIT-H Gujarati-English Machine Translation System for WMT19
Vikrant Goyal, Dipti Misra Sharma
Kingsoft’s Neural Machine Translation System for WMT19
Xinze Guo, Chang Liu, Xiaolong Li, Yiran Wang, Guoliang Li, Feng Wang, Zhitao Xu, Liuyi Yang, Li Ma, Changliang Li
The AFRL WMT19 Systems: Old Favorites and New Tricks
Jeremy Gwinnup, Grant Erdmann, Tim Anderson
Evaluating the Supervised and Zero-shot Performance of Multi-lingual Translation Models
Chris Hokamp, John Glover, Demian Gholipour Ghalandari
The MLLP-UPV Supervised Machine Translation Systems for WMT19 News Translation Task
Javier Iranzo-Sánchez, Gonçal Garcés Díaz-Munío, Jorge Civera, Alfons Juan
Microsoft Translator at WMT 2019: Towards Large-Scale Document-Level Neural Machine Translation
Marcin Junczys-Dowmunt
CUNI Submission for Low-Resource Languages in WMT News 2019
Tom Kocmi, Ondřej Bojar
CUNI Systems for the Unsupervised News Translation Task in WMT 2019
Ivana Kvapilíková, Dominik Macháček, Ondřej Bojar
A Comparison on Fine-grained Pre-trained Embeddings for the WMT19Chinese-English News Translation Task
Zhenhao Li, Lucia Specia
The NiuTrans Machine Translation Systems for WMT19
Bei Li, Yinqiao Li, Chen Xu, Ye Lin, Jiqiang Liu, Hui Liu, Ziyang Wang, Yuhao Zhang, Nuo Xu, Zeyang Wang, Kai Feng, Hexuan Chen, Tengbo Liu, Yanyang Li, Qiang Wang, Tong Xiao, Jingbo Zhu
Multi-Source Transformer for Kazakh-Russian-English Neural Machine Translation
Patrick Littell, Chi-kiu Lo, Samuel Larkin, Darlene Stewart
Incorporating Word and Subword Units in Unsupervised Machine Translation Using Language Model Rescoring
Zihan Liu, Yan Xu, Genta Indra Winata, Pascale Fung
JUMT at WMT2019 News Translation Task: A Hybrid Approach to Machine Translation for Lithuanian to English
Sainik Kumar Mahata, Avishek Garain, Adityar Rayala, Dipankar Das, Sivaji Bandyopadhyay
Johns Hopkins University Submission for WMT News Translation Task
Kelly Marchisio, Yash Kumar Lal, Philipp Koehn
NICT’s Unsupervised Neural and Statistical Machine Translation Systems for the WMT19 News Translation Task
Benjamin Marie, Haipeng Sun, Rui Wang, Kehai Chen, Atsushi Fujita, Masao Utiyama, Eiichiro Sumita
PROMT Systems for WMT 2019 Shared Translation Task
Alexander Molchanov
JU-Saarland Submission to the WMT2019 English–Gujarati Translation Shared Task
Riktim Mondal, Shankha Raj Nayek, Aditya Chowdhury, Santanu Pal, Sudip Kumar Naskar, Josef van Genabith
Facebook FAIR’s WMT19 News Translation Task Submission
Nathan Ng, Kyra Yee, Alexei Baevski, Myle Ott, Michael Auli, Sergey Edunov
eTranslation’s Submissions to the WMT 2019 News Translation Task
Csaba Oravecz, Katina Bontcheva, Adrien Lardilleux, László Tihanyi, Andreas Eisele
Tilde’s Machine Translation Systems for WMT 2019
Marcis Pinnis, Rihards Krišlauks, Matiss Rikters
Apertium-fin-eng–Rule-based Shallow Machine Translation for WMT 2019 Shared Task
Tommi Pirinen
English-Czech Systems in WMT19: Document-Level Transformer
Martin Popel, Dominik Macháček, Michal Auersperger, Ondřej Bojar, Pavel Pecina
The RWTH Aachen University Machine Translation Systems for WMT 2019
Jan Rosendahl, Christian Herold, Yunsu Kim, Miguel Graça, Weiyue Wang, Parnia Bahar, Yingbo Gao, Hermann Ney
The Universitat d’Alacant Submissions to the English-to-Kazakh News Translation Task at WMT 2019
Víctor M. Sánchez-Cartagena, Juan Antonio Pérez-Ortiz, Felipe Sánchez-Martínez
CUED@WMT19:EWC&LMs
Felix Stahlberg, Danielle Saunders, Adrià de Gispert, Bill Byrne
Baidu Neural Machine Translation Systems for WMT19
Meng Sun, Bojian Jiang, Hao Xiong, Zhongjun He, Hua Wu, Haifeng Wang
University of Tartu’s Multilingual Multi-domain WMT19 News Translation Shared Task Submission
Andre Tättar, Elizaveta Korotkova, Mark Fishel
Neural Machine Translation for English–Kazakh with Morphological Segmentation and Synthetic Data
Antonio Toral, Lukas Edman, Galiya Yeshmagambetova, Jennifer Spenader
The LMU Munich Unsupervised Machine Translation System for WMT19
Dario Stojanovski, Viktor Hangya, Matthias Huck, Alexander Fraser
Combining Local and Document-Level Context: The LMU Munich Neural Machine Translation System at WMT19
Dario Stojanovski, Alexander Fraser
IITP-MT System for Gujarati-English News Translation Task at WMT 2019
Sukanta Sen, Kamal Kumar Gupta, Asif Ekbal, Pushpak Bhattacharyya
The University of Helsinki Submissions to the WMT19 News Translation Task
Aarne Talman, Umut Sulubacak, Raúl Vázquez, Yves Scherrer, Sami Virpioja, Alessandro Raganato, Arvi Hurskainen, Jörg Tiedemann
Microsoft Research Asia’s Systems for WMT19
Yingce Xia, Xu Tan, Fei Tian, Fei Gao, Di He, Weicong Chen, Yang Fan, Linyuan Gong, Yichong Leng, Renqian Luo, Yiren Wang, Lijun Wu, Jinhua Zhu, Tao Qin, Tie-Yan Liu
The En-Ru Two-way Integrated Machine Translation System Based on Transformer
Doron Yu
DFKI-NMT Submission to the WMT19 News Translation Task
Jingyi Zhang, Josef van Genabith
11:00 Shared Task: Test Suites
Linguistic Evaluation of German-English Machine Translation Using a Test Suite
Eleftherios Avramidis, Vivien Macketanz, Ursula Strohriegel, Hans Uszkoreit
A Test Suite and Manual Evaluation of Document-Level NMT at WMT19
Kateřina Rysová, Magdaléna Rysová, Tomáš Musil, Lucie Poláková, Ondřej Bojar
Evaluating Conjunction Disambiguation on English-to-German and French-to-German WMT 2019 Translation Hypotheses
Maja Popović
The MuCoW Test Suite at WMT 2019: Automatically Harvested Multilingual Contrastive Word Sense Disambiguation Test Sets for Machine Translation
Alessandro Raganato, Yves Scherrer, Jörg Tiedemann
SAO WMT19 Test Suite: Machine Translation of Audit Reports
Tereza Vojtěchová, Michal Novák, Miloš Klouček, Ondřej Bojar
11:00 Shared Task: Metrics
WMDO: Fluency-based Word Mover’s Distance for Machine Translation Evaluation
Julian Chow, Lucia Specia, Pranava Madhyastha
Meteor++ 2.0: Adopt Syntactic Level Paraphrase Knowledge into Machine Translation Evaluation
Yinuo Guo, Junfeng Hu
YiSi - a Unified Semantic MT Quality Evaluation and Estimation Metric for Languages with Different Levels of Available Resources
Chi-kiu Lo
EED: Extended Edit Distance Measure for Machine Translation
Peter Stanchev, Weiyue Wang, Hermann Ney
Filtering Pseudo-References by Paraphrasing for Automatic Evaluation of Machine Translation
Ryoma Yoshimura, Hiroki Shimanaka, Yukio Matsumura, Hayahide Yamagishi, Mamoru Komachi
11:00 Shared Task: Robustness
Naver Labs Europe’s Systems for the WMT19 Machine Translation Robustness Task
Alexandre Berard, Ioan Calapodescu, Claude Roux
NICT’s Supervised Neural Machine Translation Systems for the WMT19 Translation Robustness Task
Raj Dabre, Kehai Chen, Benjamin Marie, Rui Wang, Atsushi Fujita, Masao Utiyama, Eiichiro Sumita
System Description: The Submission of FOKUS to the WMT 19 Robustness Task
Cristian Grozea
CUNI System for the WMT19 Robustness Task
Jindřich Helcl, Jindřich Libovický, Martin Popel
NTT’s Machine Translation Systems for WMT19 Robustness Task
Soichiro Murakami, Makoto Morishita, Tsutomu Hirao, Masaaki Nagata
JHU 2019 Robustness Task System Description
Matt Post, Kevin Duh
Robust Machine Translation with Domain Sensitive Pseudo-Sources: Baidu-OSU WMT19 MT Robustness Shared Task System Report
Renjie Zheng, Hairong Liu, Mingbo Ma, Baigong Zheng, Liang Huang
Improving Robustness of Neural Machine Translation with Multi-task Learning
Shuyan Zhou, Xiangkai Zeng, Yingqi Zhou, Antonios Anastasopoulos, Graham Neubig
12:30 🍴
14:00 Panel on "Open Problems in Machine Translation"
Alex Fraser (Ludwig-Maximilians-Universität München)
Alon Lavie (Unbabel)
Marcin Junczys-Dowmunt (Microsoft)
Yvette Graham (Dublin City University)
Chair: Ondrej Bojar
15:30 ☕️
16:00 Session 4: Research Papers on Modeling and Analysis
Chair: Matthias Huck
16:00 Saliency-driven Word Alignment Interpretation for Neural Machine Translation
Shuoyang Ding, Hainan Xu, Philipp Koehn
16:15 Improving Zero-shot Translation with Language-Independent Constraints
Ngoc-Quan Pham, Jan Niehues, Thanh-Le Ha, Alexander Waibel
16:30 Incorporating Source Syntax into Transformer-Based Neural Machine Translation
Anna Currey, Kenneth Heafield
16:45 APE at Scale and Its Implications on MT Evaluation Biases
Markus Freitag, Isaac Caswell, Scott Roy
17:00 Generalizing Back-Translation in Neural Machine Translation
Miguel Graça, Yunsu Kim, Julian Schamper, Shahram Khadivi, Hermann Ney
17:15 Tagged Back-Translation
Isaac Caswell, Ciprian Chelba, David Grangier

Day 2

   
9:00 Session 5: Shared Tasks Overview Presentations II
Chair: Mark Fishel
9:00 Findings of the WMT 2019 Shared Tasks on Quality Estimation
Erick Fonseca, Lisa Yankovskaya, André F. T. Martins, Mark Fishel, Christian Federmann
9:20 Findings of the WMT 2019 Shared Task on Automatic Post-Editing
Rajen Chatterjee, Christian Federmann, Matteo Negri, Marco Turchi
9:40 Findings of the WMT 2019 Biomedical Translation Shared Task: Evaluation for MEDLINE Abstracts and Biomedical Terminologies
Rachel Bawden, Kevin Bretonnel Cohen, Cristian Grozea, Antonio Jimeno Yepes, Madeleine Kittner, Martin Krallinger, Nancy Mah, Aurelie Neveol, Mariana Neves, Felipe Soares, Amy Siu, Karin Verspoor, Maika Vicente Navarro
10:00 Shared Task on Similar Languages
10:15 Findings of the WMT 2019 Shared Task on Parallel Corpus Filtering for Low-Resource Conditions
Philipp Koehn, Francisco Guzmán, Vishrav Chaudhary, Juan Pino
10:30 ☕️
11:00 Session 2: Shared Task Poster Session I
11:00 Shared Task: Quality Estimation
RTM Stacking Results for Machine Translation Performance Prediction
Ergun Biçici
Unbabel’s Participation in the WMT19 Translation Quality Estimation Shared Task
Fabio Kepler, Jonay Trénous, Marcos Treviso, Miguel Vera, António Góis, M. Amin Farajian, António V. Lopes, André F. T. Martins
QE BERT: Bilingual BERT Using Multi-task Learning for Neural Quality Estimation
Hyun Kim, Joon-Ho Lim, Hyun-Ki Kim, Seung-Hoon Na
MIPT System for World-Level Quality Estimation
Mikhail Mosyagin, Varvara Logacheva
NJU Submissions for the WMT19 Quality Estimation Shared Task
Hou Qi
Quality Estimation and Translation Metrics via Pre-trained Word and Sentence Embeddings
Elizaveta Yankovskaya, Andre Tättar, Mark Fishel
SOURCE: SOURce-Conditional Elmo-style Model for Machine Translation Quality Estimation
Junpei Zhou, Zhisong Zhang, Zecong Hu
11:00 Shared Task: Automatic Post-Editing
Transformer-based Automatic Post-Editing Model with Joint Encoder and Multi-source Attention of Decoder
WonKee Lee, Jaehun Shin, Jong-Hyeok Lee
Unbabel’s Submission to the WMT2019 APE Shared Task: BERT-Based Encoder-Decoder for Automatic Post-Editing
António V. Lopes, M. Amin Farajian, Gonçalo M. Correia, Jonay Trénous, André F. T. Martins
USAAR-DFKI – The Transference Architecture for English–German Automatic Post-Editing
Santanu Pal, Hongfei Xu, Nico Herbig, Antonio Krüger, Josef van Genabith
APE through Neural and Statistical MT with Augmented Data. ADAPT/DCU Submission to the WMT 2019 APE Shared Task
Dimitar Shterionov, Joachim Wagner, Félix do Carmo
Effort-Aware Neural Automatic Post-Editing
Amirhossein Tebbifakhr, Matteo Negri, Marco Turchi
UdS Submission for the WMT 19 Automatic Post-Editing Task
Hongfei Xu, Qiuhui Liu, Josef van Genabith
11:00 Shared Task: Biomedical Translation
Terminology-Aware Segmentation and Domain Feature for the WMT19 Biomedical Translation Task
Casimiro Pio Carrino, Bardia Rafieian, Marta R. Costa-jussà, José A. R. Fonollosa
Exploring Transfer Learning and Domain Data Selection for the Biomedical Translation
Noor-e- Hira, Sadaf Abdul Rauf, Kiran Kiani, Ammara Zafar, Raheel Nawaz
Huawei’s NMT Systems for the WMT 2019 Biomedical Translation Task
Wei Peng, Jianfeng Liu, Liangyou Li, Qun Liu
UCAM Biomedical Translation at WMT19: Transfer Learning Multi-domain Ensembles
Danielle Saunders, Felix Stahlberg, Bill Byrne
BSC Participation in the WMT Translation of Biomedical Abstracts
Felipe Soares, Martin Krallinger
11:00 Shared Task: Similar Languages
The MLLP-UPV Spanish-Portuguese and Portuguese-Spanish Machine Translation Systems for WMT19 Similar Language Translation Task
Pau Baquero-Arnal, Javier Iranzo-Sánchez, Jorge Civera, Alfons Juan
The TALP-UPC System for the WMT Similar Language Task: Statistical vs Neural Machine Translation
Magdalena Biesialska, Lluis Guardia, Marta R. Costa-jussà
Machine Translation from an Intercomprehension Perspective
Yu Chen, Tania Avgustinova
Utilizing Monolingual Data in NMT for Similar Languages: Submission to Similar Language Translation Task
Jyotsana Khatri, Pushpak Bhattacharyya
Neural Machine Translation: Hindi-Nepali
Sahinur Rahman Laskar, Partha Pakray, Sivaji Bandyopadhyay
NICT’s Machine Translation Systems for the WMT19 Similar Language Translation Task
Benjamin Marie, Raj Dabre, Atsushi Fujita
Panlingua-KMI MT System for Similar Language Translation Task at WMT 2019
Atul Kr. Ojha, Ritesh Kumar, Akanksha Bansal, Priya Rani
UDS–DFKI Submission to the WMT2019 Czech–Polish Similar Language Translation Shared Task
Santanu Pal, Marcos Zampieri, Josef van Genabith
Neural Machine Translation of Low-Resource and Similar Languages with Backtranslation
Michael Przystupa, Muhammad Abdul-Mageed
The University of Helsinki Submissions to the WMT19 Similar Language Translation Task
Yves Scherrer, Raúl Vázquez, Sami Virpioja
11:00 Shared Task: Parallel Corpus Filtering
Dual Monolingual Cross-Entropy Delta Filtering of Noisy Parallel Data
Amittai Axelrod, Anish Kumar, Steve Sloto
NRC Parallel Corpus Filtering System for WMT 2019
Gabriel Bernier-Colborne, Chi-kiu Lo
Low-Resource Corpus Filtering Using Multilingual Sentence Embeddings
Vishrav Chaudhary, Yuqing Tang, Francisco Guzmán, Holger Schwenk, Philipp Koehn
Quality and Coverage: The AFRL Submission to the WMT19 Parallel Corpus Filtering for Low-Resource Conditions Task
Grant Erdmann, Jeremy Gwinnup
Webinterpret Submission to the WMT2019 Shared Task on Parallel Corpus Filtering
Jesús González-Rubio
Noisy Parallel Corpus Filtering through Projected Word Embeddings
Murathan Kurfalı, Robert Östling
Filtering of Noisy Parallel Corpora Based on Hypothesis Generation
Zuzanna Parcheta, Germán Sanchis-Trilles, Francisco Casacuberta
Parallel Corpus Filtering Based on Fuzzy String Matching
Sukanta Sen, Asif Ekbal, Pushpak Bhattacharyya
The University of Helsinki Submission to the WMT19 Parallel Corpus Filtering Task
Raúl Vázquez, Umut Sulubacak, Jörg Tiedemann
12:30 🍴
14:00 Session 7: Invited Talk
Chair: Matt Post
14:00 Semantic, Style and Other Data Divergences in Neural Machine Translation
Marine Carpuat (University of Maryland)
15:30 ☕️
16:00 Session 8: Research Papers on Applications
Chair: Marco Turchi
16:00 Hierarchical Document Encoder for Parallel Corpus Mining
Mandy Guo, Yinfei Yang, Keith Stevens, Daniel Cer, Heming Ge, Yun-hsuan Sung, Brian Strope, Ray Kurzweil
16:15 The Effect of Translationese in Machine Translation Test Sets
Mike Zhang, Antonio Toral
16:30 Customizing Neural Machine Translation for Subtitling
Evgeny Matusov, Patrick Wilken, Yota Georgakopoulou
16:45 Integration of Dubbing Constraints into Machine Translation
Ashutosh Saboo, Timo Baumann
17:00 Widening the Representation Bottleneck in Neural Machine Translation with Lexical Shortcuts
Denis Emelin, Ivan Titov, Rico Sennrich
17:15 A High-Quality Multilingual Dataset for Structured Documentation Translation
Kazuma Hashimoto, Raffaella Buschiazzo, James Bradbury, Teresa Marshall, Richard Socher, Caiming Xiong

Results

General Task

Full results of the shared task: Findings of the 2019 Conference on Machine Translation (WMT19)

The winner systems were listed according to their average score and average z-score.

→ English

The results were determined with a monolingual direct assessment without document context.

Language pair System Average score Average z-score
Gujarati → NEU 64.8 0.210
Lithuanian → GTCOM-Primary 77.4 0.234
Russian → Facebook-FAIR 81.4 0.156
Chinese → Baidu 83.6 0.295
Kazakh → online-B 72.2 0.270
Finnish → MSRA-NAO 78.2 0.285
German → Facebook-FAIR 81.6 0.146

English →

The results were determined with a bilingual direct assessment with document context.

Language pair System Average score Average z-score
→ German Facebook-FAIR 90.3 0.347
→ Gujarati online-B 72.2 0.663
→ Lithuanian tilde-nc-nmt 72.8 0.388
→ Czech CUNI-DocTransformer-T2T 86.0 0.402
→ Russian Facebook-FAIR 88.5 0.506
→ Chinese KSAI 83.0 0.306
→ Kazakh UAlacant-NMT 67.6 0.262
→ Finnish GTCOM-Primary 82.6 0.586

German →

The results were determined with a monolingual direct assessment without document context.

Language pair System Average score Average z-score
→ Czech online-Y 63.9 0.426

French ↔︎ German

The results were determined with a monolingual direct assessment without document context.

Language pair System Average score Average z-score
French → German MSRA-MADL 82.4 0.267
German → French MSRA-MADL 77.0 0.249

Want to learn more about WMT19?


Edit this article →

Machine Translate is created and edited by contributors like you!

Learn more about contributing →

Licensed under CC-BY-SA-4.0.

Cite this article →