Dataset Machine Translation on WMT2014 English-German

Machine Translation on WMT2014 English-German

#
MODEL
REPOSITORY
BLEU SCORE SACREBLEU SPEED
PAPER
ε-REPRODUCES PAPER
1
36.72 36.04 1.7
--
2
36.18 35.56 9.2
--
3
33.89 33.58 8.1
4
29.59 28.90 28.1
5
29.17 28.42 68.6
--
6
29.17 28.62
7
29.10 28.49 57.9
8
28.44 27.80 69.4
--
9
25.70 25.17 76.6
10
25.39 24.77 68.4
Models on Papers with Code for which code has not been tried out yet.
MODEL
PAPER
BLEU SCORE SACREBLEU

This benchmark is evaluating models on the test set of the WMT 2014 English-German news (full) dataset.

Step 1: Evaluate models locally

First, use our public benchmark library to evaluate your model. sotabench-eval is a framework-agnostic library that implements the WMT2014 Benchmark. See sotabench-eval docs here.

Once you can run the benchmark locally, you are ready to connect it to our automatic service.

Step 2: Login and connect your GitHub Repository

Connect your GitHub repository to automatically start benchmarking your repository. Once connected we'll re-benchmark your master branch on every commit, giving your users confidence in using models in your repository and helping you spot any bugs.