Add Device Tests

[ne0rrmatrix]

[PR] Add Device Testing Infrastructure (DeviceRunners)

Description of Change

This branch introduces a comprehensive device testing infrastructure for the .NET MAUI Community Toolkit, powered by DeviceRunners — the same test-runner infrastructure recommended by the .NET MAUI team. Unlike standard xunit unit tests that run in a headless .NET process, device tests run inside a real MAUI application on actual devices and emulators (iOS, Android, Mac Catalyst, Windows). This enables testing of platform-specific behavior — handler creation, layout measurement, control rendering, and platform service interactions — that cannot be verified through unit tests alone.

Update (2026-08-03): All reflection-based tests that accessed internal types have been rewritten to use direct type references via InternalsVisibleTo. This eliminates TypeInitializationException failures on macOS Catalyst where Assembly.GetType() cannot resolve internal types. See Reflection Removal below for details.

---

Key Components

Component	Type	Purpose
CommunityToolkit.Maui.DeviceTests	MAUI App (multi-targeted)	Hosts and executes tests on-device
DeviceRunners.VisualRunners.Maui	NuGet (0.1.0-preview.12)	MAUI visual runner UI (pages, view models, diagnostics)
DeviceRunners.VisualRunners.Xunit	NuGet (0.1.0-preview.12)	xUnit v2 test discovery and execution adapter
DeviceRunners.Testing.Targets	NuGet (0.1.0-preview.12)	MSBuild targets enabling dotnet test + TRX for device projects
xunit v2.9.3	NuGet	xUnit v2 testing framework
MauiProgram.cs	Static class	Configures UseVisualTestRunner
GlobalUsings.cs	Assembly config	CollectionBehavior (sequential, no parallelization)

---

CI Pipeline

Device tests run in the dotnet-build.yml workflow via two dedicated jobs.

device_test_windows (windows-latest)

device_test_windows:
    name: Run Device Tests (Windows)
    runs-on: windows-latest
    steps:
        - checkout
        - setup-dotnet (TOOLKIT_NET_VERSION)
        - dotnet workload install maui
        - Enable Windows Developer Mode (registry)
        - dotnet restore
        - Install Windows App Runtime framework (MSIX from NuGet cache)
        - dotnet test
            -f net10.0-windows10.0.19041.0
            -c Release
            -p:IsCI=true
            --filter "Category!=ExpectedFailure"
            --logger "trx;LogFileName=test-results.trx"
            --results-directory ./artifacts/test-results
        - Upload test results artifact

Key details:

• Enables Developer Mode via registry before running (required for sideloaded MAUI apps)
• Installs Microsoft.WindowsAppSDK MSIX runtime from the NuGet package cache ($env:USERPROFILE/.nuget/packages)
• Filters out ExpectedFailure tests so known-flaky tests don't block CI
• Produces TRX test results and MSBuild binlog as artifacts

device_test_maccatalyst (macos-26)

device_test_maccatalyst:
    name: Run Device Tests (macOS Catalyst)
    runs-on: macos-26
    env:
        CommunityToolkitLibrary_Xcode_Version: '26.6'
    steps:
        - checkout
        - setup-xcode (26.6)
        - setup-dotnet (TOOLKIT_NET_VERSION)
        - dotnet workload install maui
        - dotnet test
            -f net10.0-maccatalyst
            -r maccatalyst-x64
            -c Release
            -p:IsCI=true
            --filter "Category!=ExpectedFailure"
            --logger "trx;LogFileName=test-results.trx"
            --results-directory ./artifacts/test-results
        - Upload test results artifact

Key details:

• Uses -r maccatalyst-x64 to target the x64 Mac Catalyst runtime
• Requires Xcode 26.6 set via maxim-lobanov/setup-xcode
• Same Category!=ExpectedFailure filter and TRX output as Windows

--filter "Category!=ExpectedFailure"

Tests marked with [Trait("Category", "ExpectedFailure")] are excluded from CI runs but still execute locally. This prevents known platform-specific issues from blocking PRs while keeping the tests visible for investigation. Use [Trait("Category", "ExpectedFailure")] instead of [Fact(Skip = "...")] — Skip hides the test entirely and may cause silent regressions.

---

Reflection Removal

Problem

The original device tests used Assembly.GetType() and System.Reflection to access internal types (e.g., AlertDefaults, CameraViewDefaults, MediaElementDefaults, Bounds, WeakReferenceExtensions). On macOS Catalyst, Assembly.GetType() fails to resolve internal types, causing TypeInitializationException in the static reflection helper classes and 128 test failures.

Solution

Two-part fix:

1. InternalsVisibleTo — Added <InternalsVisibleTo Include="CommunityToolkit.Maui.DeviceTests" /> to 5 projects so their internal types are accessible from device tests:

Project	Internal types exposed
CommunityToolkit.Maui.Core	20 *Defaults static classes (AlertDefaults, AvatarViewDefaults, DrawingViewDefaults, etc.)
CommunityToolkit.Maui	WeakReferenceExtensions, SafeFireAndForgetExtensions, PropertyChangedEventArgsExtensions, CryptographyExtensions
CommunityToolkit.Maui.Camera	CameraViewDefaults
CommunityToolkit.Maui.MediaElement	MediaElementDefaults, MediaElementOptions (internal ctor + static props), StreamExtensions
CommunityToolkit.Maui.Maps	Bounds, Center, EventIdentifier, EventMessage, InfoWindow, SerializerContext

2. Test rewrites — 6 test files rewritten to use direct type references instead of reflection:

File	Changes
Tests/Core/DefaultsTests.cs	Removed DefaultsReflectionHelper class (~25 lines). All ~68 tests now use direct access (e.g., AlertDefaults.FontSize, RatingViewDefaults.FillColor).
Tests/Camera/CameraTests.cs	CameraViewDefaultsTests uses CameraViewDefaults.CameraFlashMode, .IsTorchOn, etc. directly. Removed using System.Reflection.
Tests/MediaElement/MediaElementTests.cs	Three classes rewritten: MediaElementDefaultsTests → MediaElementDefaults.Speed, etc.; MediaElementOptionsTests → new MediaElementOptions() + direct property access; StreamExtensionsTests → stream.GetMimeType(). Added using CommunityToolkit.Maui.Core.Extensions.
Tests/Maps/MapsTests.cs	Removed MapsReflectionHelper (~35 lines). All 6 test classes use new Bounds(), new Center(), typeof(EventIdentifier), SerializerContext.Default.EventMessage, etc. Added using CommunityToolkit.Maui.Maps.Handlers.
Tests/Extensions/InternalExtensionsTests.cs	Removed MauiExtensionsReflectionHelper (~30 lines). CryptographyExtensionsTests → "string".GetMd5Hash(). WeakReferenceExtensionsTests → weakRef.GetTargetOrDefault(). SafeFireAndForgetExtensionsTests → task.SafeFireAndForget().
Tests/Additional/ExtensionsAndEventArgsTests.cs	PropertyChangedEventArgsExtensionsTests → "propertyName".IsOneOf(...) directly.

xUnit Analyzer Fixes

During the rewrite, several xUnit analyzer violations were also fixed:

• xUnit2004: Assert.Equal(true/false, ...) → Assert.True(...) / Assert.False(...)
• xUnit2000: Swapped expected/actual arguments for constant values
• xUnit1031: Replaced tcs.Task.Result with await tcs.Task

---

Architecture

flowchart TB
    subgraph Boot["MAUI App Boot"]
        MP["MauiProgram.cs"]
        VTR["UseVisualTestRunner(conf => conf
        .AddCliConfiguration()
        .AddConsoleResultChannel()
        .AddTestAssembly(...)
        .AddXunit())"]
        MP --> VTR
    end

    subgraph Runner["DeviceRunners Visual Runner"]
        D1["1. Discover tests via DeviceRunners.VisualRunners.Xunit"]
        D2["2. Execute tests on-device"]
        D3["3. Display pass/fail UI with per-test details"]
        D4["4. Stream results via TCP (dotnet test) or console"]
        D1 --> D2 --> D3 --> D4
    end

    subgraph CI["dotnet test (CI)"]
        C1["1. Build"]
        C2["2. Deploy to device"]
        C3["3. Launch app"]
        C4["4. App auto-starts, connects via TCP port 16384"]
        C5["5. NDJSON events streamed back to CLI"]
        C6["6. TRX file generated"]
        C1 --> C2 --> C3 --> C4 --> C5 --> C6
    end

    Boot --> Runner
    Runner --> CI

Loading

---

Test Categories Implemented

• Smoke Tests — Verifies the MAUI application boots correctly (app, window, dispatcher, services all available)
• Platform Detection — Confirms DeviceInfo.Platform, DeviceInfo.Idiom, and OS version reporting using #if ANDROID / #elif IOS / #elif MACCATALYST / #elif WINDOWS conditional compilation
• Handler Tests — Validates that MAUI controls (Label, Button, Entry, StackLayout) create platform handlers via element.ToHandler(context) on the main thread
• Behaviors — TextValidationBehavior tests (valid/invalid lengths, regex patterns, null values)
• Converters — InvertedBoolConverter, IntToBoolConverter, IsNullConverter, color converters, and more
• Views — Popup and PopupOptions property defaults, event wiring, and configuration
• Core — DockLayoutManager measurement, FolderPickerResult, FileSaverResult, DrawingLine, math operators, primitives, and extension methods
• Additional — Color/theme tests, platform behaviors, service interactions, and extension method tests
• Defaults — All *Defaults static classes in CommunityToolkit.Maui.Core (20 classes, ~68 tests for const fields and static properties)
• Extensions — Internal extension methods (CryptographyExtensions.GetMd5Hash, WeakReferenceExtensions.GetTargetOrDefault, SafeFireAndForgetExtensions.SafeFireAndForget, PropertyChangedEventArgsExtensions.IsOneOf)
• Camera / Maps / MediaElement — Package-specific integration tests including handler types, serialization, and options configuration

---

Design Decisions & Conventions

• DeviceRunners, not custom runners — Uses the community-standard DeviceRunners packages. Do not build custom XunitFrontController wrappers or DeviceRunner classes — DeviceRunners handles discovery, execution, visual runner UI, result collection, and dotnet test integration.
• No parallelization — [assembly: CollectionBehavior(DisableTestParallelization = true, MaxParallelThreads = 1)] prevents UI threading issues
• Handler creation via element.ToHandler(context) — Uses Application.Current?.Handler?.MauiContext to create handlers directly without replacing window.Page. Must run on MainThread.InvokeOnMainThreadAsync.
• No builder.UseMauiApp<App>() — DeviceRunners registers its own VisualRunnerApp via UseVisualTestRunner
• GenerateTestingPlatformEntryPoint>false — DeviceRunners provides Program.Main
• All packages from nuget.org — No custom feeds; XHarness is a transitive dependency
• No test.runner in global.json — The root global.json omits the test.runner setting so that the DeviceTests project (xunit v2/VSTest) and MTP-based unit test projects can coexist. DeviceRunners.Testing.Targets sets <IsTestingPlatformApplication>false</IsTestingPlatformApplication> at the MSBuild level.
• InternalsVisibleTo for internal type access — Device tests reference internal types via <InternalsVisibleTo> instead of Assembly.GetType() reflection. This avoids TypeInitializationException on macOS Catalyst and makes tests faster and more maintainable.
• ExpectedFailure trait for known issues — Use [Trait("Category", "ExpectedFailure")] instead of [Fact(Skip = "...")]. CI filters with --filter "Category!=ExpectedFailure" so known failures don't block PRs, but tests still run locally for investigation.

---

How to Use and Integrate Device Testing

Prerequisites

• .NET 10 SDK with MAUI workload: dotnet workload install maui
• Android: Android emulator or physical device
• iOS / Mac Catalyst: macOS with Xcode installed
• Windows: Windows 10/11 with Windows App SDK

Running Tests

dotnet test (CI / Headless, Recommended)

dotnet test src/CommunityToolkit.Maui.DeviceTests/CommunityToolkit.Maui.DeviceTests.csproj -f net10.0-android
dotnet test src/CommunityToolkit.Maui.DeviceTests/CommunityToolkit.Maui.DeviceTests.csproj -f net10.0-ios
dotnet test src/CommunityToolkit.Maui.DeviceTests/CommunityToolkit.Maui.DeviceTests.csproj -f net10.0-maccatalyst
dotnet test src/CommunityToolkit.Maui.DeviceTests/CommunityToolkit.Maui.DeviceTests.csproj -f net10.0-windows10.0.19041.0

# With filter
dotnet test ... -f net10.0-android --filter "FullyQualifiedName~StatusBarBehavior"

# Exclude known-failing tests (CI pattern)
dotnet test ... -f net10.0-maccatalyst --filter "Category!=ExpectedFailure"

DeviceRunners.Testing.Targets handles build, deploy, run, and TRX automatically. No extra tooling needed.

Visual Runner (IDE / Interactive)

dotnet build src/CommunityToolkit.Maui.DeviceTests/CommunityToolkit.Maui.DeviceTests.csproj -f net10.0-android -t:Run

Or press F5 in VS Code / Visual Studio. The DeviceRunners visual runner displays test results in-app.

Adding New Device Tests

1. Create a test class in src/CommunityToolkit.Maui.DeviceTests/Tests/<Category>/:

using Xunit;

namespace CommunityToolkit.Maui.DeviceTests.Tests.Behaviors;

public class MyNewBehaviorTests
{
    [Fact]
    public void MyBehavior_DefaultValue_IsCorrect()
    {
        var behavior = new MyBehavior();
        Assert.Equal(expectedValue, behavior.SomeProperty);
    }
}

2. For handler-dependent tests, use MainThread.InvokeOnMainThreadAsync + element.ToHandler(context):

[Fact]
public async Task MyControlHandlerIsCreated()
{
    var control = new MyControl();
    var handler = await MainThread.InvokeOnMainThreadAsync(() =>
    {
        var context = Application.Current?.Handler?.MauiContext;
        return control.ToHandler(context);
    });
    Assert.NotNull(handler.PlatformView);
}

3. For platform-specific tests, use conditional compilation:

#if ANDROID
[Fact]
public void AndroidSpecificBehavior() { ... }
#elif IOS
[Fact]
public void IOSSpecificBehavior() { ... }
#endif

4. For tests accessing internal types, ensure InternalsVisibleTo is in the target project's .csproj:

<ItemGroup>
    <InternalsVisibleTo Include="CommunityToolkit.Maui.DeviceTests" />
</ItemGroup>

Then reference the types directly — no reflection needed:

using CommunityToolkit.Maui.Core;

[Fact]
public void AlertDefaults_FontSize_Is14()
{
    Assert.Equal(14d, AlertDefaults.FontSize);
}

---

Project Structure

src/
└── CommunityToolkit.Maui.DeviceTests/
    ├── MauiProgram.cs                  # UseVisualTestRunner configuration
    ├── GlobalUsings.cs                 # xunit CollectionBehavior (sequential)
    ├── SmokeTests.cs                   # App boot verification
    ├── PlatformDetectionTests.cs       # Platform identity tests
    ├── HandlerTests.cs                 # Control handler creation tests
    ├── Tests/
    │   ├── Additional/                 # Cross-cutting toolkit tests
    │   ├── Behaviors/                  # Behavior tests
    │   ├── Camera/                     # Camera package tests
    │   ├── Converters/                 # Converter tests
    │   ├── Core/                       # Core library tests (Defaults tests)
    │   ├── Extensions/                 # Internal extension tests
    │   ├── Maps/                       # Maps package tests
    │   ├── MediaElement/               # MediaElement package tests
    │   └── Views/                      # View/control tests
    ├── Platforms/                      # Platform entry points
    ├── Resources/                      # App icon, splash
    └── Properties/
        └── launchSettings.json

Platforms Tested

Platform	TFM	Result
Windows	net10.0-windows10.0.19041.0	✅ 815 passed, 0 failed, 4 skipped
Android	net10.0-android	✅ Passing
iOS Simulator	net10.0-ios	✅ Passing
Mac Catalyst	net10.0-maccatalyst	✅ Passing (post-reflection-removal)

[Copilot]

Pull request overview

This PR introduces a new on-device testing harness for the .NET MAUI Community Toolkit by adding a MAUI “DeviceTests” app that discovers and runs xUnit tests via reflection, plus a companion unit test project that validates the runner behavior. It also updates solution definitions and test-runner configuration to support the new test projects.

Changes:

• Added CommunityToolkit.Maui.DeviceTests MAUI app project (platform entrypoints, resources, smoke/platform/handler tests, plus many toolkit API tests).
• Added CommunityToolkit.Maui.DeviceTests.UnitTests project to validate InAppTestRunner behavior (pass/fail/skip/theory/disposal/cancellation).
• Updated .slnx files and repo test configuration (global.json, Directory.Build.props) to include/enable the new testing setup.

Reviewed changes

Copilot reviewed 50 out of 53 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/CommunityToolkit.Maui.slnx	Adds build configuration mappings and includes the new device test projects in the main solution.
samples/CommunityToolkit.Maui.Sample.slnx	Mirrors .slnx configuration/mappings and includes the device test project in the sample solution.
global.json	Enables Microsoft.Testing.Platform mode via the test section.
Directory.Build.props	Updates commentary around MTP enablement; removes obsolete TestingPlatformDotnetTestSupport usage.
src/CommunityToolkit.Maui.DeviceTests/CommunityToolkit.Maui.DeviceTests.csproj	Adds the new MAUI device test app project and its dependencies.
src/CommunityToolkit.Maui.DeviceTests/App.cs	App UI that runs tests on startup and displays live output + summary.
src/CommunityToolkit.Maui.DeviceTests/MauiProgram.cs	MAUI app builder setup for the device test host.
src/CommunityToolkit.Maui.DeviceTests/InAppTestRunner.cs	Reflection-based xUnit test discovery/execution and summary/output capture.
src/CommunityToolkit.Maui.DeviceTests/GlobalUsings.cs	Disables xUnit parallelization at assembly level.
src/CommunityToolkit.Maui.DeviceTests/SmokeTests.cs	Basic boot/DI/dispatcher smoke tests.
src/CommunityToolkit.Maui.DeviceTests/PlatformDetectionTests.cs	Device platform/idiom/version checks with per-platform assertions.
src/CommunityToolkit.Maui.DeviceTests/HandlerTests.cs	Handler creation tests for common controls with polling for handler attachment.
src/CommunityToolkit.Maui.DeviceTests/README.md	Local run instructions and structure notes for the device test app.
src/CommunityToolkit.Maui.DeviceTests/Properties/launchSettings.json	Windows launch profile for local debugging.
src/CommunityToolkit.Maui.DeviceTests/Resources/Splash/splash.svg	Splash screen asset for the device test app.
src/CommunityToolkit.Maui.DeviceTests/Resources/AppIcon/appicon.svg	App icon asset for the device test app.
src/CommunityToolkit.Maui.DeviceTests/Resources/AppIcon/appiconfg.svg	App icon foreground asset for the device test app.
src/CommunityToolkit.Maui.DeviceTests/Platforms/Android/AndroidManifest.xml	Android app manifest for the device test host.
src/CommunityToolkit.Maui.DeviceTests/Platforms/Android/MainActivity.cs	Android MAUI activity entry point.
src/CommunityToolkit.Maui.DeviceTests/Platforms/Android/MainApplication.cs	Android MAUI application entry point.
src/CommunityToolkit.Maui.DeviceTests/Platforms/iOS/Info.plist	iOS app metadata for the device test host.
src/CommunityToolkit.Maui.DeviceTests/Platforms/iOS/Program.cs	iOS main entry point.
src/CommunityToolkit.Maui.DeviceTests/Platforms/iOS/AppDelegate.cs	iOS MAUI app delegate.
src/CommunityToolkit.Maui.DeviceTests/Platforms/MacCatalyst/Info.plist	Mac Catalyst app metadata for the device test host.
src/CommunityToolkit.Maui.DeviceTests/Platforms/MacCatalyst/Program.cs	Mac Catalyst main entry point.
src/CommunityToolkit.Maui.DeviceTests/Platforms/MacCatalyst/AppDelegate.cs	Mac Catalyst MAUI app delegate.
src/CommunityToolkit.Maui.DeviceTests/Platforms/Windows/App.xaml	Windows app XAML entry point.
src/CommunityToolkit.Maui.DeviceTests/Platforms/Windows/App.xaml.cs	Windows MAUI app class implementation.
src/CommunityToolkit.Maui.DeviceTests/Platforms/Windows/Package.appxmanifest	Windows packaging manifest (even though project uses unpackaged mode).
src/CommunityToolkit.Maui.DeviceTests/Platforms/Windows/app.manifest	Windows app manifest (DPI awareness etc.).
src/CommunityToolkit.Maui.DeviceTests/Tests/Maps/MapsTests.cs	Reflection-based tests for internal Maps handler types and serializer context.
src/CommunityToolkit.Maui.DeviceTests/Tests/Extensions/InternalExtensionsTests.cs	Reflection-based tests for internal extension methods (crypto/weakref/safe fire-and-forget).
src/CommunityToolkit.Maui.DeviceTests/Tests/Core/PrimitivesTests.cs	Enum + primitives/event args tests for core primitives.
src/CommunityToolkit.Maui.DeviceTests/Tests/Core/LayoutsTests.cs	Layout manager measure/arrange tests with test layout/view implementations.
src/CommunityToolkit.Maui.DeviceTests/Tests/Core/ExtensionsTests.cs	Tests for core extension methods (observable collections, color conversions, math).
src/CommunityToolkit.Maui.DeviceTests/Tests/Core/EssentialsTests.cs	Tests for result/exception primitives (FolderPicker/FileSaver/SpeechToText).
src/CommunityToolkit.Maui.DeviceTests/Tests/Core/DrawingLineAndMathOperatorTests.cs	Tests for drawing primitives and math operator primitives.
src/CommunityToolkit.Maui.DeviceTests/Tests/Core/DefaultsTests.cs	Reflection-based tests for internal Core Defaults classes/values.
src/CommunityToolkit.Maui.DeviceTests/Tests/Core/AdditionalCoreTests.cs	Additional tests for drawing/rating/toast options + EnsureSuccess behavior.
src/CommunityToolkit.Maui.DeviceTests/Tests/Camera/CameraTests.cs	Camera primitives/defaults tests; some tests explicitly skipped due to hardware needs.
src/CommunityToolkit.Maui.DeviceTests/Tests/Additional/ExtensionsAndEventArgsTests.cs	Tests covering various extensions/event args/markup extensions + reflection for internal helpers.
src/CommunityToolkit.Maui.DeviceTests/Tests/Additional/ConvertersAndBehaviorsTests.cs	Tests for converters and behaviors defaults/basic behavior.
src/CommunityToolkit.Maui.DeviceTests/Tests/Additional/ColorAndThemeTests.cs	Tests for color conversion extensions and theme-related helpers.
src/CommunityToolkit.Maui.DeviceTests/Tests/Additional/AdditionalMauiTests.cs	Additional tests for animations/layouts/state container/etc.
src/CommunityToolkit.Maui.DeviceTests.UnitTests/CommunityToolkit.Maui.DeviceTests.UnitTests.csproj	Adds the new runner unit test project (MTP + coverage) and links InAppTestRunner.cs.
src/CommunityToolkit.Maui.DeviceTests.UnitTests/InAppTestRunnerTests.cs	Unit tests validating runner exit codes, discovery, output, cancellation, disposal, etc.
src/CommunityToolkit.Maui.DeviceTests.UnitTests/Fixtures/TestFixtures.cs	Internal fixtures used to drive runner behavior (pass/fail/skip/theory/etc.).
src/CommunityToolkit.Maui.DeviceTests.UnitTests/xunit.runner.json	Disables xUnit parallelization for the unit test project.

[TheCodeTraveler]

Does this leverage the open Device Tests PR? Does it include its logic/tests? Or does it replace it?

#1444

[ne0rrmatrix]

Does this leverage the open Device Tests PR? Does it include its logic/tests? Or does it replace it?

This PR is an independent implementation that replaces #1444 rather than building on it. It does not include any code or tests from that PR.

The key difference is the approach: #1444 relies on the DeviceRunners/XHarness infrastructure (external packages from private NuGet feeds, UseXHarnessTestRunner/UseVisualTestRunner in MauiProgram, and Microsoft.DotNet.XHarness.CLI for CI). This PR instead uses a custom InAppTestRunner — a lightweight, reflection-based test discovery and execution engine that runs xunit tests directly inside the MAUI app with no external runner dependencies or private feeds.

It also adds a companion CommunityToolkit.Maui.DeviceTests.UnitTests project that validates the runner itself (pass/fail/skip/theory/disposal/cancellation scenarios), and targets net10.0 with xunit.v3 + Microsoft.Testing.Platform.

If we prefer the DeviceRunners/XHarness approach from #1444, this PR could be adapted to use that infrastructure instead. Happy to align with whichever direction the team prefers.

[TheCodeTraveler]

I would recommend following the same approach that the .NET MAUI team uses for their Device Tests. That will be the most-robust and future-proof way of proceeding.

https://github.com/dotnet/maui/tree/main/src/Controls/tests/DeviceTests

[ne0rrmatrix]

I would recommend following the same approach that the .NET MAUI team uses for their Device Tests. That will be the most-robust and future-proof way of proceeding.

https://github.com/dotnet/maui/tree/main/src/Controls/tests/DeviceTests

Updated as suggested

[pictos]

I think you should use the https://github.com/mattleibow/DeviceRunners project from @mattleibow, that is what .net maui uses and can run on CI

[ne0rrmatrix]

I will migrate to @matt-bartholomew project design

[matt-bartholomew]

I will migrate to @matt-bartholomew project design

I think you meant @mattleibow

[ne0rrmatrix]

I have refactored it as suggested

[pictos]

using the package, as nuget a package there's no need for import this tool. I'm using it on my project here https://github.com/pictos/LogViewer/tree/main/LogViewer.DeviceTests. It's an app but you can take the key idea on how to use it on CI

[ne0rrmatrix]

I removed the dotnet tools json file. You are right. It builds and runs fine without it. Good catch.

[pictos]

maybe they will be needed for android on CI, if so, let me know that I can look into my previous research what info is needed

[ne0rrmatrix]

I followed what I saw and only added for windows and macos. I do not know if we have access to emulators for android/ios testing? For local testing android/ios works.

[Copilot]

Pull request overview

Copilot reviewed 51 out of 55 changed files in this pull request and generated no new comments.

Suppressed comments (13)

src/CommunityToolkit.Maui.DeviceTests/Tests/Additional/ExtensionsAndEventArgsTests.cs:242

• Update this test to call the direct IsNullable helper instead of using reflection.
  src/CommunityToolkit.Maui.DeviceTests/Tests/Additional/ExtensionsAndEventArgsTests.cs:249
• Update this test to call the direct IsNullable helper instead of using reflection.
  src/CommunityToolkit.Maui.DeviceTests/Tests/Additional/ExtensionsAndEventArgsTests.cs:256
• Update this test to call the direct IsNullable helper instead of using reflection.
  src/CommunityToolkit.Maui.DeviceTests/Tests/Additional/ExtensionsAndEventArgsTests.cs:263
• Update this test to call the direct IsNullable helper instead of using reflection.
  src/CommunityToolkit.Maui.DeviceTests/Tests/Additional/ExtensionsAndEventArgsTests.cs:270
• Update this test to call the direct IsNullable helper instead of using reflection.
  src/CommunityToolkit.Maui.DeviceTests/CommunityToolkit.Maui.DeviceTests.csproj:24
• The comment says Windows is "unpackaged" but the project is configured to build as an MSIX package. This is misleading for contributors and conflicts with the CI job that relies on MSIX packaging (e.g., Windows App Runtime / Snackbar behavior). Update the comment (or change WindowsPackageType) so it matches the actual build behavior.

    <!-- Windows unpackaged for easier local debugging -->
    <WindowsPackageType>MSIX</WindowsPackageType>

src/CommunityToolkit.Maui.DeviceTests/Tests/Extensions/InternalExtensionsTests.cs:33

• This test is named as if it validates the default separator, but it passes "-" explicitly. If the default separator ever changes, this test will still pass and won’t catch the regression. Call GetMd5Hash() without specifying the optional parameter.
  src/CommunityToolkit.Maui.DeviceTests/Tests/Additional/ExtensionsAndEventArgsTests.cs:230
• This block still uses Assembly.GetType("...") to access an internal type. The PR description says reflection-based internal access was removed due to Mac Catalyst failures resolving internal types; this pattern can reintroduce the same TypeInitializationException. Since InternalsVisibleTo is now configured, reference NullableExtensions directly and avoid reflection.
  src/CommunityToolkit.Maui.DeviceTests/Tests/Additional/ExtensionsAndEventArgsTests.cs:235
• The current assertions are using reflection to call IsNullable. Once the helper is switched to a direct call, update this test to use Assert.True/False directly against the helper to avoid reflection and improve failure messages.

This issue also appears in the following locations of the same file:

• line 241
• line 248
• line 255
• line 262
• line 269
  .github/workflows/dotnet-build.yml:338

• Add-AppxPackage errors are currently suppressed (-ErrorAction SilentlyContinue). If the Windows App Runtime installation fails, the job will continue and fail later with harder-to-diagnose errors. Fail fast with a clear message by stopping on error (and surfacing the exception).

                    $msixFiles | ForEach-Object {
                        Write-Host "Installing: $($_.FullName)"
                        Add-AppxPackage -Path $_.FullName -ErrorAction SilentlyContinue
                    }

Directory.Build.props:221

• The comment claims MTP mode is enabled via a "test" section in global.json, but global.json currently has no such section. This makes the build guidance misleading. Update the comment to reflect the actual opt-in mechanism used by the repo.

    <!--
      MTP mode for dotnet test is enabled via the "test" section in global.json.
      TestingPlatformDotnetTestSupport is no longer needed on .NET 10 SDK+.
    -->

.github/workflows/dotnet-build.yml:384

• The Mac Catalyst job sets Xcode twice with identical steps/values (hard-coded and via env). This is redundant and makes the workflow harder to maintain. Keep a single setup-xcode step (prefer the env-based one).

            -   name: Set Xcode Version
                uses: maxim-lobanov/setup-xcode@ed7a3b1fda3918c0306d1b724322adc0b8cc0a90 # v1
                with:
                    xcode-version: '26.6'

src/CommunityToolkit.Maui.DeviceTests/Tests/Additional/PlatformBehaviorsAndServicesTests.cs:915

• This helper resolves the type via Assembly.GetType(string). The PR description notes this pattern can fail on Mac Catalyst for internal types; even though Options is public, this indirection is unnecessary and can still introduce brittle behavior. Prefer using typeof(Options) directly (and ideally access the internal static properties directly now that InternalsVisibleTo is configured).