SWE-Skills-Bench
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Software Engineering Task Success | SWE-Skills-Bench Hard | Task Success Rate35 | 20 | |
| Software Engineering | SWE-Skills-Bench | Baseline Accuracy98 | 4 | |
| Software Engineering Skill Evaluation | SWE-Skills-Bench (slo-implementation) | Pass Rate (out of 10)60 | 3 | |
| Software Engineering Skill Evaluation | SWE-Skills-Bench python-anti-patterns | Pass Rate4 | 3 | |
| Software Engineering Skill Evaluation | SWE-Skills-Bench implementing-agent-modes | Pass Rate30 | 3 | |
| Software Engineering Skill Evaluation | SWE-Skills-Bench (changelog-automation) | Pass Rate60 | 3 | |
| Software Engineering Skill Evaluation | SWE-Skills-Bench performance-optimization v3 | Pass Rate0 | 3 | |
| Software Engineering Skill Evaluation | SWE-Skills-Bench bazel-build-optimization | Pass Rate10 | 3 | |
| Software Engineering Skill Evaluation | SWE-Skills-Bench python-resilience | Pass Rate30 | 3 | |
| Software Engineering Skill Evaluation | SWE-Skills-Bench gitops-workflow | Pass Rate0 | 3 | |
| Software Engineering Skill Evaluation | SWE-Skills-Bench bash-defensive-patterns | Pass Rate50 | 3 | |
| Software Engineering Skill Evaluation | SWE-Skills-Bench (fix) | Pass Rate2 | 3 | |
| Software Engineering Skill Evaluation | SWE-Skills-Bench python-observability | Pass Count7 | 3 | |
| Software Engineering Skill Evaluation | SWE-Skills-Bench turborepo | Pass Rate30 | 3 | |
| Software Engineering Skill Evaluation | SWE-Skills-Bench python-background-jobs | Pass Rate7 | 3 | |
| Software Engineering Skill Evaluation | SWE-Skills-Bench service-mesh-observability | Pass Rate0 | 3 | |
| Software Engineering Skill Evaluation | SWE-Skills-Bench istio-traffic-management | Pass Rate8 | 3 | |
| Software Engineering Skill Evaluation | SWE-Skills-Bench (tdd-workflow) | Pass Rate4 | 3 | |
| Software Engineering Skill Evaluation | SWE-Skills-Bench distributed-tracing | Pass Rate40 | 3 | |
| Software Engineering Skill Evaluation | SWE-Skills-Bench similarity-search-patterns | Pass Rate0 | 3 | |
| Software Engineering Skill Evaluation | SWE-Skills-Bench prompt-engineering-patterns | Pass Rate30 | 3 | |
| Software Engineering Skill Evaluation | SWE-Skills-Bench (gitlab-ci-patterns) | Pass Rate7 | 3 | |
| Software Engineering Skill Evaluation | SWE-Skills-Bench risk-metrics-calculation | Pass Rate5 | 3 | |
| Software Engineering Skill Evaluation | SWE-Skills-Bench (Aggregate) | Pass Rate65 | 3 |
Showing 24 of 24 rows