A Learning Method with Gap-Aware Generation for Heterogeneous DAG Scheduling
About
Efficient scheduling of directed acyclic graphs (DAGs) is a core problem in large-scale data-intensive computing systems, where query plans, data-processing workloads, and computation graphs consist of dependent tasks competing for limited heterogeneous resource pools. In practice, achieving high-performance execution requires schedulers to adapt across environments with varying resource pools and task types, while generating schedules under tight runtime budgets. We propose WeCAN, an end-to-end reinforcement learning framework for heterogeneous DAG scheduling that addresses task-pool compatibility coefficients and generation-induced optimality gaps. It adopts a two-stage single-pass design: a single forward pass produces task-pool scores and global parameters, followed by a generation map that constructs schedules without repeated network calls. Its weighted cross-attention encoder models task-pool interactions gated by compatibility coefficients, and is size-agnostic to environment fluctuations. Moreover, widely used list-scheduling maps can incur generation-induced optimality gaps from restricted reachability. We introduce an order-space analysis that characterizes the reachable set of generation maps via feasible schedule orders, explains the mechanism behind generation-induced gaps, and yields sufficient conditions for gap elimination. Guided by these conditions, we design a skip-extended realization with an analytically parameterized decreasing skip rule, which enlarges the reachable order set while preserving single-pass efficiency. Experiments on real-world TPC-H query DAGs, resource-intensive workload datasets, and ML-compiler computation graphs demonstrate improved makespan over strong baselines, with inference time comparable to classical heuristics and faster than multi-round neural schedulers.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| DAG scheduling | TPC-H Full Generalization 150 tasks | Makespan2.43e+4 | 16 | |
| DAG scheduling | TPC-H Full Generalization 200 tasks | Makespan3.14e+4 | 16 | |
| DAG scheduling | Erdős-Rényi 500 tasks | Makespan1.01e+4 | 14 | |
| DAG scheduling | Layer Graphs 500 tasks | Makespan1.08e+4 | 14 | |
| DAG scheduling | Stochastic Block 500 tasks | Makespan1.00e+4 | 14 | |
| Heterogeneous Resource Scheduling | TPC-H 30 (3 pools) | Makespan1.90e+4 | 14 | |
| Heterogeneous Resource Scheduling | TPC-H 50 (3 pools) | Makespan3.28e+4 | 14 | |
| Heterogeneous Resource Scheduling | TPC-H-100 (3 pools) | Makespan6.14e+4 | 14 | |
| Computation graph scheduling | TPC-H heavy 30 | Makespan2.41e+4 | 8 | |
| Computation graph scheduling | TPC-H 50 heavy | Makespan3.57e+4 | 8 |