Add MAUI iOS Inner Loop measurements for CI

[davidnguyen-tech]

Summary

Adds MAUI iOS Inner Loop performance measurements to CI, supporting both iOS simulators and physical devices.

What's included:

• iOSInnerLoopParser.cs — Binlog parser extracting iOS build task/target timings
• Startup.cs/Reporter.cs — Wiring and null-safety fix
• ioshelper.py — iOS simulator and physical device management
• runner.py — IOSINNERLOOP execution branch (build → deploy → measure → incremental)
• Scenario scripts — pre.py, test.py, post.py, setup_helix.py
• maui_scenarios_ios_innerloop.proj — Helix workitem definition (simulator + physical device)
• Pipeline YAML — Job entries in sdk-perf-jobs.yml and routing in run_performance_job.py

Measurements:

• First deploy: full build + install + launch timing via binlog parsing
• Incremental deploy: source edit → rebuild + reinstall + relaunch timing

Targets:

• iOS Simulator (iossimulator-arm64) on macOS Helix machines
• Physical iPhone (ios-arm64) on Mac.iPhone.17.Perf queue

Based on:

• Existing Android Inner Loop CI scenario (working reference)
• Local iOS implementation from feature/measure-maui-ios-deploy branch

[Copilot]

Pull request overview

Adds a new MAUI iOS “inner loop” (build → deploy → cold-startup → incremental repeat) performance scenario and wires it into Helix/CI for both simulator and physical devices.

Changes:

• Added a new binlog parser (iOSInnerLoopParser) to extract iOS build task/target timings and integrated it into ScenarioMeasurement.
• Implemented iOS simulator/device orchestration (workload install, Xcode selection, deploy/startup measurement, reporting merge) via new scenario scripts and a shared iOSHelper.
• Updated Helix/pipeline plumbing to pass iOS RID, route the new run kind, and define the new Helix work items.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
src/tools/ScenarioMeasurement/Util/Parsers/iOSInnerLoopParser.cs	New binlog parser emitting task/target/build duration counters for iOS inner loop.
src/tools/ScenarioMeasurement/Startup/Startup.cs	Adds new MetricType and wires it to the new parser.
src/tools/Reporting/Reporting/Reporter.cs	Makes build timestamp parsing null-tolerant for lab reporting.
src/scenarios/shared/runner.py	Adds iosinnerloop runner flow (build/binlog parse + install/startup timings + report merge + uploads).
src/scenarios/shared/mauisharedpython.py	Extends MAUI workload install helper to support selecting workload manifest IDs + workload ID.
src/scenarios/shared/ioshelper.py	New helper for simulator/device management, install, launch, and (device) watchdog-based startup measurement.
src/scenarios/shared/const.py	Adds IOSINNERLOOP scenario constant and display name mapping.
src/scenarios/mauiiosinnerloop/test.py	New scenario entrypoint for the iOS inner loop test.
src/scenarios/mauiiosinnerloop/setup_helix.py	Helix machine setup (DOTNET_ROOT, Xcode selection, simulator boot/device detect, workload install, restore).
src/scenarios/mauiiosinnerloop/select_xcode.py	Standalone helper for selecting an Xcode matching SDK/workload requirements.
src/scenarios/mauiiosinnerloop/run-local.sh	Local convenience runner for end-to-end measurements and report collection.
src/scenarios/mauiiosinnerloop/pre.py	Creates/normalizes MAUI template for iOS-only build + prepares incremental edit sources; workload install with fallback manifest patching.
src/scenarios/mauiiosinnerloop/post.py	Cleanup script for simulator/device uninstall and workspace cleanup.
src/scenarios/mauiiosinnerloop/results/mono/first-debug-e2e-perf-lab-report.json	Added example/generated report output (should likely not be committed).
src/scenarios/mauiiosinnerloop/results/mono/incremental-debug-e2e-perf-lab-report.json	Added example/generated report output (should likely not be committed).
src/scenarios/mauiiosinnerloop/results/coreclr/first-debug-e2e-perf-lab-report.json	Added example/generated report output (should likely not be committed).
src/scenarios/mauiiosinnerloop/results/coreclr/incremental-debug-e2e-perf-lab-report.json	Added example/generated report output (should likely not be committed).
scripts/send_to_helix.py	Adds ios_rid support and passes it as an MSBuild property for reliable .proj evaluation.
scripts/run_performance_job.py	Propagates run_env_vars into environment for MSBuild evaluation; includes ios inner loop in binlog copying; forwards iOSRid into SendToHelix args.
eng/pipelines/templates/run-performance-job.yml	Enables --runtime-flavor for the new maui_scenarios_ios_innerloop run kind.
eng/pipelines/sdk-perf-jobs.yml	Adds iOS inner loop jobs, but also temporarily disables many unrelated private jobs via if false.
eng/performance/maui_scenarios_ios_innerloop.proj	New Helix work item definition for simulator and device inner loop scenario runs.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

This results/...perf-lab-report.json file appears to be generated output from running the scenario locally. It will likely become stale and add noise to diffs. Consider removing generated results from the repo and keeping only the code/scripts that produce them.

Suggested change

	"tests": [
	{
	"counters": [
	{
	"name": "Install Time",
	"topCounter": false,
	"defaultCounter": false,
	"higherIsBetter": false,
	"metricName": "ms",
	"results": [
	5950.941801071167
	]
	},
	{
	"name": "Cold Startup Time",
	"topCounter": true,
	"defaultCounter": false,
	"higherIsBetter": false,
	"metricName": "ms",
	"results": [
	3416
	]
	}
	]
	}
	]
	"tests": []

[Copilot]

MetricType enum values elsewhere use PascalCase, but iOSInnerLoop starts with a lowercase i. For consistency (and clearer CLI/help output if DragonFruit uses the enum names), consider renaming this to IOSInnerLoop and updating the corresponding parser type name accordingly.

Suggested change

	iOSInnerLoop,
	IOSInnerLoop,

[Copilot]

The parser type is named iOSInnerLoopParser (lowercase leading i), which is inconsistent with the PascalCase naming used by other parsers in this directory (e.g., BuildTimeParser, InnerLoopParser). Consider renaming to IOSInnerLoopParser and updating references so it matches the established type naming conventions.

Suggested change

	public class iOSInnerLoopParser : IParser
	public class IOSInnerLoopParser : IParser

[Copilot]

A large set of existing private jobs is currently wrapped in ${{ if false }} with a [TEMP] note. This effectively disables most of the private perf pipeline (scenarios, affinitized scenarios, Android/iOS release/debug, NativeAOT, etc.) whenever this template is used. If the intent is only to gate iOS inner loop validation, please avoid hard-disabling unrelated jobs here; instead gate the new iOS inner loop jobs behind a parameter/condition, or keep the existing jobs enabled and add the new ones alongside them.

[Copilot]

The comment says the inner loop scenario uses xcrun simctl directly for simulator deploy, but the implementation uses mlaunch --installsim/--launchsim for deploy/launch (with simctl mainly for boot/terminate/uninstall). Please update the comment to match the actual tooling so future maintainers aren’t misled about how deploy is performed.

Suggested change

	<!-- No XHarness needed — iOS inner loop uses xcrun simctl directly for
	simulator deploy and does not use the XHarness test infrastructure. -->
	<!-- No XHarness needed — iOS inner loop uses mlaunch
	(--installsim/--launchsim) for simulator deploy/launch, with xcrun
	simctl used for simulator lifecycle tasks such as boot, terminate,
	and uninstall. -->

[Copilot]

This results/...perf-lab-report.json file appears to be generated output from running the scenario locally. It will likely become stale and add noise to diffs. Consider removing generated results from the repo and keeping only the code/scripts that produce them.

[Copilot]

This results/...perf-lab-report.json file appears to be generated output from running the scenario locally. It will likely become stale and add noise to diffs. Consider removing generated results from the repo and keeping only the code/scripts that produce them.

Suggested change

	"tests": [
	{
	"counters": [
	{
	"name": "Install Time",
	"topCounter": false,
	"defaultCounter": false,
	"higherIsBetter": false,
	"metricName": "ms",
	"results": [
	8044.32487487793
	]
	},
	{
	"name": "Cold Startup Time",
	"topCounter": true,
	"defaultCounter": false,
	"higherIsBetter": false,
	"metricName": "ms",
	"results": [
	2668
	]
	}
	]
	}
	]
	"tests": []

[Copilot]

PERFLAB_BUILDTIMESTAMP is now optional, but the fallback uses DateTime.Now and still uses DateTime.Parse(...) which will throw for invalid formats. For more stable reporting (and to avoid local-time ambiguity), consider using DateTimeOffset.TryParse/TryParseExact with DateTimeStyles.RoundtripKind and falling back to DateTimeOffset.UtcNow (or DateTime.UtcNow) when the env var is missing/invalid.

Suggested change

	var buildTimestamp = !string.IsNullOrEmpty(buildTimestampStr) ? DateTime.Parse(buildTimestampStr, CultureInfo.InvariantCulture) : DateTime.Now;
	var buildTimestamp = DateTimeOffset.TryParse(
	buildTimestampStr,
	CultureInfo.InvariantCulture,
	DateTimeStyles.RoundtripKind,
	out var parsedBuildTimestamp)
	? parsedBuildTimestamp.UtcDateTime
	: DateTimeOffset.UtcNow.UtcDateTime;

[Copilot]

The class docstring says physical-device install/launch uses mlaunch --installdev/--launchdev, but install_app() uses xcrun devicectl device install app for physical devices. Please align the docstring with the current behavior (or switch the implementation to match the stated approach) to avoid confusion when debugging deployment failures.

Suggested change

	Install and launch use mlaunch (the same tool Visual Studio uses for F5)
	to match the real developer inner-loop experience:
	- Simulator: mlaunch --installsim / --launchsim
	- Device: mlaunch --installdev / --launchdev
	Device detection still uses devicectl; simulator management uses simctl.
	The helper uses different platform tools depending on the target:
	- Simulator: mlaunch for app install/launch; simctl for simulator management
	- Device: devicectl for device detection and app install
	This mixed-tool approach reflects the current implementation and is useful
	to keep in mind when debugging deployment failures.

[Copilot]

These checked-in results/...perf-lab-report.json files look like generated output from a local run. They are likely to become stale/noisy and can bloat the repo. Consider removing them from source control and (if needed) documenting how to generate them locally or adding a small sanitized example under a dedicated docs/testdata folder.