Skip to content

ci: run MetaDrive simulator test in GitHub Actions#38242

Open
Jonathan-321 wants to merge 16 commits into
commaai:masterfrom
Jonathan-321:bounty-30693-metadrive-ci
Open

ci: run MetaDrive simulator test in GitHub Actions#38242
Jonathan-321 wants to merge 16 commits into
commaai:masterfrom
Jonathan-321:bounty-30693-metadrive-ci

Conversation

@Jonathan-321

@Jonathan-321 Jonathan-321 commented Jun 25, 2026

Copy link
Copy Markdown

Closes #30693.

Summary

This adds a Linux GitHub Actions MetaDrive simulator proof job and hardens the CI-only simulator path enough to run reliably on the free ubuntu-24.04 runner used by forks.

Main changes:

  • install the pinned commaai MetaDrive dependency only for the simulator job
  • add a simulator driving workflow job that runs test_metadrive_bridge.py under Xvfb
  • upload simulator artifacts from each run (qlog, rlog, and camera files)
  • preserve simulator logs during teardown and avoid transient lock-file copy failures
  • run the MetaDrive simulator test without xdist and with CI-scoped simulation hardening
  • address observed CI flakes only: MetaDrive out_of_lane, CI speed on rebased master, modeld crash restart, movement accounting, phantom traffic-free vision leads, Athena upload retry timing, a racy C++ params async timing assertion, and GitHub runner azure-cli apt-source 403s

Validation

Upstream PR checks are green on the current head:

20 consecutive full tests.yaml workflow executions passed on the same commit:

# Run
01 https://github.com/Jonathan-321/openpilot/actions/runs/28184334780
02 https://github.com/Jonathan-321/openpilot/actions/runs/28184786280
03 https://github.com/Jonathan-321/openpilot/actions/runs/28185224998
04 https://github.com/Jonathan-321/openpilot/actions/runs/28185650640
05 https://github.com/Jonathan-321/openpilot/actions/runs/28186084458
06 https://github.com/Jonathan-321/openpilot/actions/runs/28186516203
07 https://github.com/Jonathan-321/openpilot/actions/runs/28186952294
08 https://github.com/Jonathan-321/openpilot/actions/runs/28187383261
09 https://github.com/Jonathan-321/openpilot/actions/runs/28187807799
10 https://github.com/Jonathan-321/openpilot/actions/runs/28188226297
11 https://github.com/Jonathan-321/openpilot/actions/runs/28188650871
12 https://github.com/Jonathan-321/openpilot/actions/runs/28188942315
13 https://github.com/Jonathan-321/openpilot/actions/runs/28189957753
14 https://github.com/Jonathan-321/openpilot/actions/runs/28190396794
15 https://github.com/Jonathan-321/openpilot/actions/runs/28190825910
16 https://github.com/Jonathan-321/openpilot/actions/runs/28191289654
17 https://github.com/Jonathan-321/openpilot/actions/runs/28191747563
18 https://github.com/Jonathan-321/openpilot/actions/runs/28192213775
19 https://github.com/Jonathan-321/openpilot/actions/runs/28192632091
20 https://github.com/Jonathan-321/openpilot/actions/runs/28193082253

Final run simulator proof:

@Jonathan-321 Jonathan-321 force-pushed the bounty-30693-metadrive-ci branch from e89a76d to 7c974de Compare June 25, 2026 15:08
@github-actions

Copy link
Copy Markdown
Contributor

Process replay diff report

Replays driving segments through this PR and compares the behavior to master.
Please review any changes carefully to ensure they are expected.

✅ 0 changed, 66 passed, 0 errors

@Jonathan-321 Jonathan-321 marked this pull request as ready for review June 25, 2026 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Run MetaDrive simulation test in GitHub Actions

1 participant