Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

A Learning Method with Gap-Aware Generation for Heterogeneous DAG Scheduling

About

Efficient scheduling of directed acyclic graphs (DAGs) is a core problem in large-scale data-intensive computing systems, where query plans, data-processing workloads, and computation graphs consist of dependent tasks competing for limited heterogeneous resource pools. In practice, achieving high-performance execution requires schedulers to adapt across environments with varying resource pools and task types, while generating schedules under tight runtime budgets. We propose WeCAN, an end-to-end reinforcement learning framework for heterogeneous DAG scheduling that addresses task-pool compatibility coefficients and generation-induced optimality gaps. It adopts a two-stage single-pass design: a single forward pass produces task-pool scores and global parameters, followed by a generation map that constructs schedules without repeated network calls. Its weighted cross-attention encoder models task-pool interactions gated by compatibility coefficients, and is size-agnostic to environment fluctuations. Moreover, widely used list-scheduling maps can incur generation-induced optimality gaps from restricted reachability. We introduce an order-space analysis that characterizes the reachable set of generation maps via feasible schedule orders, explains the mechanism behind generation-induced gaps, and yields sufficient conditions for gap elimination. Guided by these conditions, we design a skip-extended realization with an analytically parameterized decreasing skip rule, which enlarges the reachable order set while preserving single-pass efficiency. Experiments on real-world TPC-H query DAGs, resource-intensive workload datasets, and ML-compiler computation graphs demonstrate improved makespan over strong baselines, with inference time comparable to classical heuristics and faster than multi-round neural schedulers.

Ruisong Zhou, Haijun Zou, Li Zhou, Chumin Sun, Zaiwen Wen• 2026

Related benchmarks

TaskDatasetResultRank
DAG schedulingTPC-H Full Generalization 150 tasks
Makespan2.43e+4
16
DAG schedulingTPC-H Full Generalization 200 tasks
Makespan3.14e+4
16
DAG schedulingErdős-Rényi 500 tasks
Makespan1.01e+4
14
DAG schedulingLayer Graphs 500 tasks
Makespan1.08e+4
14
DAG schedulingStochastic Block 500 tasks
Makespan1.00e+4
14
Heterogeneous Resource SchedulingTPC-H 30 (3 pools)
Makespan1.90e+4
14
Heterogeneous Resource SchedulingTPC-H 50 (3 pools)
Makespan3.28e+4
14
Heterogeneous Resource SchedulingTPC-H-100 (3 pools)
Makespan6.14e+4
14
Computation graph schedulingTPC-H heavy 30
Makespan2.41e+4
8
Computation graph schedulingTPC-H 50 heavy
Makespan3.57e+4
8
Showing 10 of 10 rows

Other info

Follow for update