Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Vision-Based Runtime Monitoring under Varying Specifications using Semantic Latent Representations

About

We study certified runtime monitoring of past-time signal temporal logic (ptSTL) from visual observations under partial observability. The monitor must infer safety-relevant quantities from images and provide finite-sample guarantees, while being \emph{reusable}: once trained and calibrated, it should certify any formula in a target fragment without per-formula retraining. For fragments induced by a finite dictionary of temporal atoms, we prove that the \emph{semantic basis}, the vector of atom robustness scores, is the minimum prediction target within the class of monotone, 1-Lipschitz reusable interfaces: any formula is evaluated by a deterministic decoder derived from the parse tree, and a single conformal calibration pass certifies the entire fragment with no union bound. We also introduce a \emph{rolling prediction monitor} that predicts only current predicate values and reconstructs temporal history online; this is easier to learn but grows conservative at long horizons. On a pedestrian-crossroad benchmark, rolling achieves tighter certified bounds at short horizons while the semantic-basis monitor is up to 4-times tighter at long horizons. We validate the presented monitors on real-world Waymo driving data, where both monitors satisfy the conformal coverage guarantee empirically.

Bardh Hoxha, Oliver Sch\"on, Hideki Okamoto, Lars Lindemann, Georgios Fainekos• 2026

Related benchmarks

TaskDatasetResultRank
Runtime Safety MonitoringCrossroad
q_phi7.47
25
Runtime Safety MonitoringWaymo Open Motion Dataset (WOMD)
q_phi8.11
19
Showing 2 of 2 rows

Other info

Follow for update