Llama 2
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Watermark Detection | Llama-2-7b-chat-hf 10 samples UMD watermarking (test) | AUROC (t=0)1 | 64 | |
| Attention Operator Latency | LLaMA-2 Chat 7B | Attention Latency (ms)0.075 | 60 | |
| Safety Evaluation | LLaMA-2-7B-CHAT Safety (test) | Safety Score0.55 | 60 | |
| Language Modeling | Llama-2 13B | Perplexity (PPL)4.85 | 32 | |
| Jailbreak Attack Transferability | Llama-2-7b-chat finetuned variants v1 (test) | Transfer Success Rate (TSR)60.4 | 16 | |
| Watermark Attack Success Rate | Llama-2-7b-chat-hf UMD watermarking (10 samples) | ASR100 | 15 | |
| LLM Quantization | Llama-2-70B | GPU Hours (h)2.2 | 13 | |
| LLM Inference Verification | Llama-2 7B | Verification Latency (s)0.17 | 12 | |
| Training Stability Analysis | Llama-2 7B pre-training | Number of Spikes0 | 9 | |
| Hybrid-Dimension Reconfiguration | LLaMA-2 32B | Reconfiguration Time (s)0.253 | 8 | |
| Knowledge Distillation Robustness | Llama-2-7B teacher vs. llama-2-7b-logit-watermark-distill-kgw-k1-gamma0.25-delta2 student (test) | Similarity Score99.98 | 7 | |
| Model Fingerprinting | Llama-2 DPO 7B | Similarity Score99.94 | 7 | |
| Attribute Steering | Llama-2-7b-Chat-hf Open-Ended Generation | Wealth Score2.46 | 7 | |
| Ownership Verification | Llama-2-7B SFT & RLHF | FSR (Anchor)2 | 6 | |
| Ownership Verification | Llama-2-7B Taylor Pruning 5% sparsity | FSR0 | 6 | |
| Ownership Verification | Llama-2-7B Random Pruning, 10% sparsity | FSR0 | 6 | |
| Ownership Verification | Llama-2-7B Random Pruning, 5% sparsity | False Success Rate (FSR)0 | 6 | |
| Decoding Latency | Llama-2-7B 32k sequence length v1 (inference) | Decoding Latency (s)0.062 | 6 | |
| Decoding Latency | Llama-2-7B 16k sequence length v1 (inference) | Decoding Latency (s)0.041 | 6 | |
| Decoding Throughput | Llama 2 7B inference v1.0 | Decoding Throughput (TOK/s)188 | 6 | |
| Language Modeling | LLaMA-2 7B pre-training (val) | Validation Perplexity (40K steps)16.01 | 5 | |
| Model Fingerprinting | LLaMA-2 7B fine-tuned variants | U-test p-value0 | 5 | |
| LLM Inference | LLaMA-2 70B sequence length 2048 | Max Batch Size384 | 5 | |
| Decoding Latency | Llama-2-7B 64k sequence length v1 (inference) | Decoding Latency (s)0.098 | 5 | |
| Decoding Throughput | Llama 2 70B v1.0 (inference) | Throughput (TOK/s)23.5 | 5 |
Showing 25 of 54 rows