Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Navigation Instruction Generation on Partially seen routes 200 instances (test)
Loading...
53.5
Length
reference
52.46
52.73
53
53.27
Dec 30, 2020
Length
BLEU
Landmark Score
SDTW
Success Rate
SNT Score
Updated 3mo ago
Evaluation Results
Method
Method
Links
Length
BLEU
Landmark Score
SDTW
Success Rate
SNT Score
reference
Type=Human Reference
2020.12
53.5
-
2.76
0.728
85.5
0.878
rule based
Type=Heuristic-based
2020.12
53.1
0.71
12.44
0.405
46
0.455
seq2seq
Architecture=Transformer
2020.12
52.9
13.12
1.95
0.139
16
0.206
graph2text
Pre-training=false
2020.12
52.6
18.6
2.41
0.475
54
0.676
g2t+pretrain
Pre-training=true
2020.12
52.5
18.81
2.44
0.471
54
0.537
Feedback
Search any
task
Search any
task