test: migrate integration tests to jubilant#782
Merged
Conversation
Remove pytest_operator, pytest_asyncio, and types-python-dateutil from lint deps since nothing imports them after the jubilant migration. Add jubilant so mypy can resolve its types.
The tox integration env was renamed from integration-juju3.6 to integration during the jubilant migration, but the workflow files still referenced the old name.
Python 3.12 enforces that typing.Generator takes exactly 3 type arguments (YieldType, SendType, ReturnType). The juju fixture was missing the ReturnType, causing a collection error in CI.
jubilant's juju.run() raises CLIError (not TimeoutError) when the Juju CLI returns non-zero, e.g. when an action times out. Polling loops that call juju.run() need to catch both CLIError and TimeoutError to avoid crashing on transient action failures. Affected: wait_for_runner_ready, _no_runners teardown fixture, and _wait_for_runner_ready in prometheus metrics test.
Juju 3.6 deprecated --series for juju download. Replace --series jammy with --base ubuntu@22.04 to fix the upgrade test.
pytest-operator previously set basetemp to the tox env tmp directory. Without it, pytest defaults to /tmp which the juju snap cannot access, causing juju download to fail with 'no such file or directory'.
…st-operator pytest-operator provided the abort_on_fail marker which skips remaining tests in a module after a failure. Without it, expensive integration tests continue running even when earlier tests fail, wasting CI resources.
wait_for_runner_ready returned as soon as 1 runner was online, even when multiple were requested. This caused test_check_runner to flake when asserting online == 2 before the second runner was ready. Pass num_runners through from set_app_runner_amount so the wait polls until all expected runners are online.
Contributor
There was a problem hiding this comment.
Pull request overview
Migrates the charm’s integration/E2E test harness away from python-libjuju/pytest-operator to the jubilant CLI-driven approach, aiming to reduce flakiness and simplify the test runtime model management.
Changes:
- Replace
pytest-operator/async libjuju model interactions with synchronousjubilantoperations across integration tests and fixtures. - Consolidate tox integration environments into a single
[testenv:integration]and update GitHub Actions workflows to use it. - Refactor integration helpers (
common.py,openstack.py) to operate on Jubilant models, unit/app names (strings), and synchronous polling utilities.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tox.ini | Removes libjuju/pytest-operator integration env matrix; adds unified integration env and tweaks pytest tempdir usage. |
| tests/integration/test_prometheus_metrics.py | Converts the Prometheus/Grafana/COS integration test flow to Jubilant-only operations. |
| tests/integration/test_multi_unit_same_machine.py | Rewrites multi-unit colocated-machine test to use jubilant.status/add_unit/wait and unit-name based helpers. |
| tests/integration/test_e2e.py | Converts E2E workflow dispatch test from async libjuju objects to app-name strings and sync flow. |
| tests/integration/test_charm_upgrade.py | Refactors upgrade test to deploy/integrate/refresh via Jubilant and sync wait utilities. |
| tests/integration/test_charm_runner.py | Converts runner action/workflow tests to Jubilant (run, config) and synchronous waiting. |
| tests/integration/test_charm_no_runner.py | Replaces libjuju relation/secret operations with Jubilant equivalents and sync waits. |
| tests/integration/test_charm_fork_path_change.py | Updates fork-path-change scenario to use app/unit names and Jubilant waits/config. |
| tests/integration/helpers/openstack.py | Refactors OpenStack instance helper to be synchronous and to execute via Jubilant + unit-name strings. |
| tests/integration/helpers/common.py | Reworks core helpers (deploy, exec-in-unit, workflow dispatch, wait loops) to synchronous Jubilant-based APIs. |
| tests/integration/conftest.py | Removes OpsTest/model fixtures; introduces a temp-model Jubilant fixture and updates all dependent fixtures to return app names. |
| tests/conftest.py | Adds --keep-models option and an abort_on_fail marker implementation to skip remaining tests in a module after failure. |
| .github/workflows/integration_test.yaml | Switches CI workflow to run unified tox -e integration. |
| .github/workflows/e2e_test.yaml | Switches CI workflow to run unified tox -e integration for E2E suite. |
cbartz
commented
Apr 15, 2026
cbartz
changed the title
cbartz
requested review from
florentianayuwono,
javierdelapuente,
weiiwang01,
yanksyoon and
yhaliaw
as code owners
yanksyoon
approved these changes
Apr 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Migrate charm integration tests to jubilant.
Most of the PR is
Rationale
Current integration tests are quite flaky, which seems to be related to the usage of python-libjuju.
Also python-libjuju is no longer maintained and jubilant is required for juju 4 .
Checklist
urgent,trivial,complex).github-runner-manager/pyproject.toml.