Skip to content
BranchesTags

Repository files navigation

moq-kit

Native Swift and Kotlin SDKs for publishing and playing low-latency media streams over QUIC.

License Platform: iOS Platform: Android Swift Package Manager Maven Central

moq-kit gives iOS and Android apps platform-native APIs for Media over QUIC-style live streaming: connect to a relay, discover broadcasts, publish camera/microphone/screen tracks, play catalog-described streams, and send or receive raw data tracks.

It is built on the published platform packages backed by UniFFI bindings generated from moq-ffi, the Rust bindings from Luke Curley's moq-dev/moq project.

For the repository codemap, layer boundaries, and invariants, see ARCHITECTURE.md.

Table of contents

Quick Start

Get a playing demo stream in two terminals:

  1. Clone the repository with submodules (the Rust vendor/moq checkout is a submodule):

    git clone --recurse-submodules https://github.com/software-mansion-labs/moq-kit.git

  2. Install mise — the project task runner used for every build and run command.

  3. Start a local moq-lite relay in one terminal:

    mise run relay:run

  4. Run the iOS or Android demo in another terminal:

    mise run ios:run --simulator    # or: mise run android:run

For SDK integration, jump to Installation and Usage. See Prerequisites for the toolchains each workflow needs.

Prerequisites

The repository's mise.toml is intentionally minimal and does not auto-install language toolchains. Install the tools below manually before running the build commands.

Consumers integrating the published Swift Package or Maven Central artifact do not need Rust, the Android NDK, or cargo-ndk.

All platforms

  • mise-en-place — task runner used for every build and run command.
  • Rust toolchain (rustup, stable channel) — required for vendor/moq tasks such as running the local relay or Rust checks. Skip if you only consume the published packages.
  • Git with submodule support (vendor/moq is a submodule).

iOS development

  • Xcode 16+ with Command Line Tools.
  • Deployment target: iOS 16+ / macOS 13+.

Android development

  • Android Studio (Hedgehog or newer).
  • Android SDK with API 35 (compileSdk) and platform 29+ (minSdk).
  • Java 11+.

Protocol

moq-kit currently targets moq-lite rather than the fast-changing IETF moq-transport draft wire format.

MoQ is mainly a transport protocol. Media behavior is application-defined on top of that transport, and moq-kit uses media catalogs and track conventions to describe codecs, renditions, and app-level streams.

The main MOQ draft is still moving quickly and can break compatibility between versions. moq-kit focuses on practical mobile functionality first: native capture, native playback, catalogs, relay workflows, and usable app-level APIs. Luke Curley's moq-lite work is a good fit for that direction because it emphasizes real deployments and concrete media use cases.

When the main MOQ draft stabilizes further, moq-kit may consider moving toward it.

What moq-kit supports

  • Native SDKs for iOS Swift and Android Kotlin.
  • Relay sessions for publishing and consuming through the same MoQ relay.
  • Broadcast discovery with catalog-driven track selection.
  • Publishing from camera, iOS and Android multi-camera capture, microphone, iOS ReplayKit screen capture, Android screen capture, and raw data tracks.
  • Playback with native low-latency renderers, dynamically adjustable target latency, track switching, and playback stats.
  • Data tracks for app-defined payloads such as JSON chat messages.

The best-tested media path today is H.264 video with AAC or Opus audio.

Codec iOS publish iOS playback Android publish Android playback
H.264 / AVC ✅ ✅ ✅ ✅
H.265 / HEVC ✅ ✅ ✅ ✅
AAC ✅ ✅ ✅ ✅
Opus ✅ ✅ ✅ ✅
AV1 Not yet ✅* Not yet ✅*

* AV1 playback depends on the platform decoder available at runtime. On Apple devices, Apple documents AV1 playback for iPhone 15 Pro and says A17 Pro includes a dedicated AV1 decoder. moq-kit does not currently expose AV1 publishing, and iOS AV1 playback should be treated as iPhone 15 Pro-class device support rather than broad Apple platform support. See Apple's iPhone 15 Pro tech specs and A17 Pro announcement for more context.

Installation

moq-kit is in active preview. APIs, package versions, and relay compatibility may still change.

iOS

Add the Swift package and depend on the MoQKit product:

.package(
    url: "https://github.com/software-mansion-labs/moq-kit",
    from: "0.4.1"
)
.product(name: "MoQKit", package: "moq-kit")

The Swift package depends on https://github.com/moq-dev/moq-swift from 0.4.5 for its Swift abstractions and moq-swift-ffi from 0.3.16 (within 0.3.x) for generated UniFFI bindings and the prebuilt XCFramework.

The iOS SDK does not add permissions, entitlements, or audio-session configuration for you. Camera publishing requires NSCameraUsageDescription. Microphone publishing requires NSMicrophoneUsageDescription, and your app is responsible for configuring AVAudioSession before starting MicrophoneCapture. ReplayKit Broadcast Upload integrations also need a Broadcast Upload Extension target plus an App Group shared by the host app and the extension.

Android

Add Maven Central and the Android artifact:

repositories {
    google()
    mavenCentral()
}

dependencies {
    implementation("com.swmansion.moqkit:moqkit:0.4.0")
}

The Android SDK includes Kotlin APIs backed by the upstream dev.moq:moq Maven package, using its idiomatic Kotlin facade and aliases. That package resolves dev.moq:moq-ffi transitively for the generated UniFFI bindings and JNI libraries.

Android apps must declare the permissions they use. Typical integrations need INTERNET. Camera publishing needs CAMERA, microphone publishing needs RECORD_AUDIO, and screen capture needs the Android MediaProjection permission flow plus a foreground service with the mediaProjection service type on Android versions that require it. The library does not add these permissions transitively.

Usage

Everything in moq-kit starts with a Session. A session owns one QUIC connection to a relay and is the starting point for subscribing to broadcasts, publishing tracks, and sharing relay state across app workflows.

Play a broadcast in Swift

import MoQKit

let session = Session(url: "http://localhost:4443/anon")
try await session.connect()

let subscription = try session.subscribe(prefix: "live")

for await broadcast in subscription.broadcasts {
    for await catalog in broadcast.catalogs() {
        let videoTrack = catalog.playableVideoTracks.first?.name
        let audioTrack = catalog.playableAudioTracks.first?.name
        guard videoTrack != nil || audioTrack != nil else { continue }

        let player = try await MainActor.run {
            try Player(
                catalog: catalog,
                videoTrackName: videoTrack,
                audioTrackName: audioTrack,
                targetBuffering: .milliseconds(100)
            )
        }

        try await player.play()

        let diagnosticsTask = Task {
            for await event in await player.diagnostics() {
                // Detailed, non-replayed pipeline events for logging or telemetry.
                print(event)
            }
        }

        // Keep a strong reference to the player for as long as playback should continue.
        // Cancel diagnosticsTask when that observation is no longer needed.
    }
}

Player.diagnostics() creates a bounded stream per call. Slow consumers receive the newest events without blocking playback. Use it for typed drop, discontinuity, buffering, switch, recovery, clock, latency, and stall details; use Player.stats or subscribeStats(_:) for aggregate UI metrics.

On iOS, the PipelineContext.dropDiagnostics value on frameDropped events caused by staleVsPlayback or backlogOverflow captures the playhead, the timestamp reference and exact microsecond delta used by the decision, buffer occupancy before and after the drop, and applicable buffer limits. The typed event retains exact values for telemetry.

Publish camera and microphone in Swift

import AVFoundation
import MoQKit

let audioSession = AVAudioSession.sharedInstance()
try audioSession.setCategory(
    .playAndRecord,
    mode: .videoRecording,
    options: [.defaultToSpeaker, .allowBluetoothHFP]
)
try audioSession.setActive(true)

let session = Session(url: "http://localhost:4443/anon")
try await session.connect()

let camera = CameraCapture(camera: Camera(position: .front))
let microphone = MicrophoneCapture()

try await camera.start()
try await microphone.start()

let publisher = try Publisher()
try publisher.addVideoTrack(name: "camera", source: camera)
try publisher.addAudioTrack(name: "mic", source: microphone)

try await session.publish(path: "live/ios", publisher: publisher)
try await publisher.start()

// Mute/unmute during the broadcast without stopping capture or the audio track:
// microphone.isMuted = true
// microphone.isMuted = false

// When the broadcast ends:
// await publisher.stop()
// await camera.stop()
// await microphone.stop()
// await session.close()

The name passed to addVideoTrack and addAudioTrack is a local SDK label used by PublishedTrack and publisher events. Media catalog track names are generated by the underlying muxer, so subscribers should discover actual media track names from Catalog.videoTracks and Catalog.audioTracks.

On iOS, capture and publication have separate controls. microphone.isMuted = true sends silence while keeping capture and publication active. Audio/video registration returns a PublishedMediaTrack: await setEnabled(false) to release its encoder and remove its catalog entry while camera preview continues. Await capture stop() to release hardware and suspend preview; start() resumes the same capture and any enabled track. Enabling publication never starts capture. Both controls are async.

Capture close(), track stop(), and publisher stop() are terminal. A publisher with no active media remains open. Capture restart republishes under fresh wire track names, so players must follow catalog updates. One camera/microphone supports one publication attachment, and a camera supports one native preview alongside it.

Publish front and back cameras in Swift

import MoQKit

guard MultiCameraCapture.isSupported else {
    throw SessionError.invalidConfiguration("Multi-camera capture is not supported")
}

let session = Session(url: "http://localhost:4443/anon")
try await session.connect()

let cameras = MultiCameraCapture(
    front: Camera(position: .front, width: 720, height: 1280),
    back: Camera(position: .back, width: 720, height: 1280),
    maxFrameRate: 30
)
try await cameras.start()

let videoConfig = VideoEncoderConfig(width: 720, height: 1280, bitrate: 900_000)

let publisher = try Publisher()
try publisher.addVideoTrack(name: "front-camera", source: cameras.frontSource, config: videoConfig)
try publisher.addVideoTrack(name: "back-camera", source: cameras.backSource, config: videoConfig)

try await session.publish(path: "live/ios-multicam", publisher: publisher)
try await publisher.start()

// When the broadcast ends:
// await publisher.stop()
// cameras.stop()
// await session.close()

Play a broadcast in Kotlin

lifecycleScope.launch {
    val session = Session(
        url = "http://localhost:4443/anon",
        parentScope = lifecycleScope,
    )

    session.connect()
    val subscription = session.subscribe(prefix = "live")

    subscription.broadcasts.collect { broadcast ->
        broadcast.catalogs().collect { catalog ->
            val videoTrack = catalog.playableVideoTracks.firstOrNull()?.name
            val audioTrack = catalog.playableAudioTracks.firstOrNull()?.name
            if (videoTrack == null && audioTrack == null) return@collect

            val player = Player(
                catalog = catalog,
                videoTrackName = videoTrack,
                audioTrackName = audioTrack,
                targetBuffering = java.time.Duration.ofMillis(100),
                parentScope = lifecycleScope,
            )

            player.setSurface(surfaceView.holder.surface)
            player.play()

            // Keep the player while the screen is active, then call player.close().
        }
    }
}

Publish camera and microphone in Kotlin

lifecycleScope.launch {
    val session = Session(
        url = "http://localhost:4443/anon",
        parentScope = lifecycleScope,
    )
    session.connect()

    val camera = CameraCapture(position = CameraPosition.Front)
    camera.start(context, lifecycleOwner)

    val microphone = MicrophoneCapture(sampleRate = 48_000)
    microphone.start()

    val publisher = Publisher()
    publisher.addVideoTrack(name = "camera", source = camera)
    publisher.addAudioTrack(name = "mic", source = microphone)

    session.publish(path = "live/android", publisher = publisher)
    publisher.start()

    // When the broadcast ends, stop the publisher, captures, and session.
    // publisher.stop()
    // camera.stop()
    // microphone.stop()
    // session.close()
}

Publish front and back cameras in Kotlin

lifecycleScope.launch {
    if (!MultiCameraCapture.isFrontBackSupported(context)) {
        error("Multi-camera capture is not supported")
    }

    val session = Session(
        url = "http://localhost:4443/anon",
        parentScope = lifecycleScope,
    )
    session.connect()

    val cameras = MultiCameraCapture(
        front = CameraStreamConfig(
            position = CameraPosition.Front,
            width = 1280,
            height = 720,
            frameRate = 30,
        ),
        back = CameraStreamConfig(
            position = CameraPosition.Back,
            width = 1280,
            height = 720,
            frameRate = 30,
        ),
    )
    cameras.start(context, lifecycleOwner)

    val videoConfig = VideoEncoderConfig(width = 1280, height = 720, bitrate = 900_000)

    val publisher = Publisher()
    publisher.addVideoTrack(name = "front-camera", source = cameras.frontSource, config = videoConfig)
    publisher.addVideoTrack(name = "back-camera", source = cameras.backSource, config = videoConfig)

    session.publish(path = "live/android-multicam", publisher = publisher)
    publisher.start()

    // When the broadcast ends:
    // publisher.stop()
    // cameras.stop()
    // session.close()
}

Use VideoEncoderConfig.isSupported, AudioEncoderConfig.isSupported, and the supportedCodecs() helpers before offering codec choices in UI. Use Catalog.playableVideoTracks and Catalog.playableAudioTracks when selecting tracks for playback. On iOS, video track playability is based on codec families recognized by MoQKit's renderer; actual decode/render support is still determined by AVFoundation at runtime. For app-defined messages or telemetry, add a DataTrackEmitter with Publisher.addDataTrack and read it with Broadcast.subscribeTrack.

On iOS, camera and microphone publishing are app-owned integrations: your app handles the privacy usage strings and AVAudioSession setup. On Android, your app declares and requests camera and audio permissions. Multi-camera capture requires hardware support reported by MultiCameraCapture.isSupported on iOS. On Android, MultiCameraCapture.isSupported(context) is the fast platform feature check, while MultiCameraCapture.isFrontBackSupported(context) verifies that CameraX exposes an actual concurrent front/back pair. For screen publishing, use ScreenCapture when in-app capture is enough, and use the ReplayKit Broadcast Upload flow for full-device iOS capture that survives app switches. The iOS demo shows the App Group and extension wiring for that path.

For complete app-shaped code, use the native demos.

Demo apps

The demo apps are the best integration references in this repository:

  • Player: connect to a relay, discover broadcasts, select tracks, and play streams.
  • Publisher: publish camera, microphone, and screen capture streams.
  • Chat: publish and receive JSON messages over raw MoQ data tracks.

Native demo paths:

The iOS demo also includes Luke's experimental Boy demo with announced games, live playback, and viewer controls.

Local development

Use mise-en-place (mise) as the task runner for local development.

mise tasks

Build platform SDKs

iOS resolves moq-swift through Swift Package Manager and compiles the Swift SDK:

mise run ios:build
mise run ios:test

Android resolves the upstream dev.moq:moq Maven package and assembles the Kotlin SDK:

mise run android:build

The task scripts in mise-tasks are the source of truth for exact build outputs. Android generated bindings and JNI libraries come from the transitive dev.moq:moq-ffi package beneath dev.moq:moq. iOS generated bindings come from the resolved moq-swift package. See ARCHITECTURE.md for binding boundaries and invariants.

For local SDK development, the demo apps are usually the fastest feedback loop. Prefer wiring demos to local Swift and Android modules instead of published package versions when you are iterating on moq-kit itself. The Android demo already uses Gradle substitution for the local android/moqkit build.

Run a local relay and test streams

For development, the regular flow is to run the local moq-lite relay and publish a single prepared media file into it. Start by checking the task list:

mise tasks

The main streaming tasks are:

  • relay:run — run the local moq-lite relay from this checkout.
  • media:to-fmp4 — convert a source video to CMAF fragmented MP4.
  • stream:file — loop one or more CMAF fragmented MP4 file streams into a relay.
  • stream:obs — control OBS through obs-cmd and publish an OBS stream.

Use mise task <task> to see what a task does and which flags it accepts before running it:

mise task relay:run
mise task stream:file
mise task media:to-fmp4
mise task stream:obs

stream:file expects a CMAF fragmented MP4 input. Prepare local media with media:to-fmp4 first, then run relay:run and publish the file with stream:file.

Run the demos with:

mise run ios:run --simulator
mise run android:run

You can also split the Android demo flow with mise run android:install and mise run android:launch.

Status

moq-kit is an active preview. It is suitable for demos, prototypes, and early integrations, but the public APIs, packages, codec coverage, and protocol compatibility can still evolve.

Protocol compatibility relies on what moq-lite provides. For more detail, see the moq-lite draft and Luke Curley's moq-dev/moq repository.

License

Apache License, Version 2.0

Acknowledgments

Built on top of moq-dev/moq by Luke Curley and contributors.

MoqKit is created by Software Mansion

Since 2012 Software Mansion is a software agency with experience in building web and mobile apps. We are Core React Native Contributors and experts in dealing with all kinds of React Native issues. We can help you build your next dream product - Hire us.

swm

Copyright 2026, Software Mansion

About

No description, website, or topics provided.

Resources

Readme
Apache-2.0 license
Activity
Custom properties

Stars

31 stars

Watchers

5 watching

Forks

6 forks
Report repository

Releases

Packages

Contributors

Languages