2026
Featured ★DAITFO — RL Traffic Signal Control Benchmark
A custom Gymnasium environment wrapping SUMO that benchmarks a PPO agent against fixed-time and queue-based heuristic controllers across 27 controlled runs. The heuristic won every metric — and that honest negative result is the actual finding, not a failure.
The Problem
Fixed-timing traffic signals can't adapt to real-time demand, which wastes fuel and delays emergency vehicles at intersections. The obvious modern answer is reinforcement learning. I wanted to know if that answer actually holds up against a much simpler queue-based heuristic — or if it's just assumed to be better because it's newer.
What I Built
I built a custom Gymnasium environment wrapping SUMO (a standard traffic microsimulator) via its TraCI interface, exposing queue lengths and waiting times per lane as the observation space. I trained a PPO agent (Stable-Baselines3) on this environment and benchmarked it head-to-head against a fixed-time controller and a queue-based heuristic — across 3 controllers × 3 demand levels × 3 seeds, 27 controlled runs in total.
I also built a companion Android app (Kotlin/Compose) with role-based monitoring views for citizens, emergency responders, and traffic inspectors.
SUMO Simulator
├── Fixed-Time Controller
├── Heuristic Controller (queue-based, actuated)
└── PPO Agent (RL)
│
▼
MetricsCollector → wait time, queue length, throughput
│
▼
benchmark_results.csvThe Result
The heuristic won. On every metric, at every demand level.
On medium demand: the heuristic averaged 83s wait time. PPO averaged 136s. Fixed-time was 357s.
That's not a failed project to me — it's the actual finding. I initially had a bug where the heuristic had zero yellow-transition overhead, which made the early comparison look favorable to RL. Once I fixed that and re-ran the full benchmark, the result flipped completely. I kept the negative result instead of quietly reworking the framing.
The more interesting nuance I found: tightening the RL agent's minimum phase duration got one seed out of five to match the heuristic (84.6s) — but two of five seeds catastrophically failed instead, with vehicles getting permanently stuck (max wait times over 199,000 seconds in a simulation that should run 3,600). Shorter minimum phases buy peak performance and cost training reliability. That tradeoff is the real technical takeaway I walked away with.
Key Decisions & Tradeoffs
- I built a custom Gymnasium env instead of using sumo-rl's built-in — I wanted full control over yellow-phase handling, at the cost of maintaining the TraCI integration by hand.
- I used a binary action space (keep/switch) so the comparison to the heuristic would be apples-to-apples. That meant I couldn't express phase-specific durations as a result.
- I trained only on medium demand — simpler experiment design, but low-demand generalization suffered (40s vs. the heuristic's 10.5s).
- I descoped the infrastructure I'd originally planned (Kafka, Neo4j, Redis, an LLM layer) to focus on one defensible, well-measured result instead of a sprawling half-built system.
Highlights
- Heuristic beat PPO on every metric at every demand level — the honest negative result is the finding
- Custom Gymnasium environment wrapping SUMO via TraCI with full yellow-phase control
- 27 controlled runs: 3 controllers × 3 demand levels × 3 seeds
- Discovered fundamental reliability tradeoff: shorter minimum phases buy peak performance but cost training stability
- Companion Android app (Kotlin/Compose) with role-based monitoring for citizens, responders, and inspectors