I ship an iOS app that turns audiobooks into study material — read along with live captions, highlight a passage, snip the exact spoken words into a note. Captions mean speech-to-text, and speech-to-text on a 20–60 minute chapter has an obvious, fatal UX problem: if you transcribe the whole chapter before showing anything, the user stares at a spinner for a minute. And it all has to run on-device — the audio never leaves the phone.

The fix isn’t a faster recognizer. It’s transcribing the right part first: a short window at the playhead, streamed so the first caption lands in about a second, then expanding outward in the background. Here’s how that’s built on iOS 26’s SpeechAnalyzer, including the concurrency that makes it fast and the merge that keeps it correct.

The governing design is “transcribe once, sync forever”: recognition runs a single time per chapter, off the main thread, and the result is persisted. Playback never re-runs it — it just reads the stored, timestamped segments.

The recognizer: SpeechAnalyzer with volatile results

SpeechAnalyzer (iOS 26) runs one or more modules; for captions that’s a SpeechTranscriber. Two options matter: .volatileResults so it emits in-progress hypotheses (the live, still-changing line), and .audioTimeRange so every result carries timestamps — without those you can’t sync text to audio:

let transcriber = SpeechTranscriber(
    locale: resolved,
    transcriptionOptions: [],
    reportingOptions: [.volatileResults],
    attributeOptions: [.audioTimeRange]
)
try await prepareModel(for: transcriber, localeID: resolved.identifier, onModelState: onModelState)

let analyzer = SpeechAnalyzer(modules: [transcriber])
guard let analyzerFormat = await SpeechAnalyzer.bestAvailableAudioFormat(compatibleWith: [transcriber]) else {
    throw TranscriberError.recognitionFailed("no compatible audio format")
}

prepareModel handles a subtlety: the first transcription in a given language may have to download an on-device model. That’s a distinct, one-time cost, so it’s surfaced to the UI as its own state (.downloading → .ready) — the user understands why the first run in French is slow and every later run is instant, instead of blaming recognition.

The concurrency: a feeder and a consumer, racing

This is the part that makes it feel fast. There are two tasks, both off the main actor, running at the same time.

The consumer loops over transcriber.results as they arrive. A result is either volatile (a guess that will change) or final. It rebases each result’s time into the file’s absolute timeline, and the only time it touches the main actor is to paint the caption:

let consumer = Task { () -> [TranscriptSegment] in
    var finalized: [TranscriptSegment] = []
    var volatile: TranscriptSegment?

    for try await result in transcriber.results {
        let start = CMTimeGetSeconds(result.range.start) + windowStart
        let end   = CMTimeGetSeconds(result.range.end) + windowStart
        guard start.isFinite, end.isFinite, end > start else { continue }
        let text = String(result.text.characters).trimmingCharacters(in: .whitespacesAndNewlines)
        guard !text.isEmpty else { continue }
        let segment = TranscriptSegment(text: text, start: start, end: end)

        if result.isFinal {
            finalized.append(segment)
            if let v = volatile, v.start < end { volatile = nil }
        } else {
            volatile = segment
        }

        let snapshot = streamSnapshot(finalized: finalized, volatile: volatile)
        await MainActor.run { onPartial(snapshot) }   // the ONLY main-actor hop
    }
    return finalized
}

The feeder pushes audio in. SpeechAnalyzer can take an AsyncStream<AnalyzerInput>, so I make one and feed it short PCM chunks from the window on a separate task. Crucially the feeder runs off the main actor, so the consumer above can race ahead and surface results while the feeder is still enqueuing audio — that overlap is where the low latency comes from:

let (inputSequence, inputBuilder) = AsyncStream<AnalyzerInput>.makeStream()

let feeder = Task<Void, Error> {
    defer { inputBuilder.finish() }
    try Self.feedWindow(fileURL: fileURL, windowRange: windowRange,
                        analyzerFormat: analyzerFormat, into: inputBuilder, progress: progress)
}

do {
    try await analyzer.start(inputSequence: inputSequence)
    try await feeder.value                         // wait until every chunk is enqueued
    try await analyzer.finalizeAndFinishThroughEndOfInput()  // then drain
} catch {
    feeder.cancel(); consumer.cancel()
    throw TranscriberError.recognitionFailed(error.localizedDescription)
}

let out = try await consumer.value
return out.sorted { $0.start < $1.start }

Feeding only the window (seek, don’t decode)

feedWindow is where “playhead-first” actually happens. It seeks with AVAudioFile.framePosition so it never decodes the audio before the window, reads ~0.5s chunks (so the very first AnalyzerInput is yielded almost immediately), and converts each chunk to the analyzer’s required format:

let startFrame = max(0, min(totalFrames, AVAudioFramePosition(windowRange.lowerBound * sampleRate)))
audioFile.framePosition = startFrame

// ~0.5s chunks so the first AnalyzerInput is yielded almost immediately.
let chunkFrames = AVAudioFrameCount(max(1, sampleRate * 0.5))
let converter = AVAudioConverter(from: fileFormat, to: analyzerFormat)

var remaining = framesToRead
while remaining > 0 {
    if Task.isCancelled { return }
    // read a chunk, convert to analyzerFormat, builder.yield(AnalyzerInput(...))
}

Small chunk size is a deliberate latency lever: half-second chunks mean the recognizer starts working almost immediately instead of waiting for a big buffer. Task.isCancelled in the loop means if the user scrubs away, the in-flight window stops promptly.

Correctness: merging windows that arrive out of order

Because I transcribe the playhead window first and then backfill forward and backward, results don’t arrive in reading order, and adjacent windows overlap so the boundary line gets recognized twice. The stored transcript has to be sorted and de-duplicated regardless of arrival order. That logic lives in a pure function (so it’s trivially unit-testable, no audio required):

public static func merge(existing: [TranscriptSegment], incoming: [TranscriptSegment]) -> [TranscriptSegment] {
    var out = existing
    for new in incoming {
        if let i = out.firstIndex(where: { isSameLine($0, new) }) {
            out[i] = preferred(existing: out[i], incoming: new)  // collision → keep the better copy
        } else {
            out.append(new)                                       // new line → insert
        }
    }
    return dedupeAdjacent(stableSortByStart(out))
}

public static func isSameLine(_ a: TranscriptSegment, _ b: TranscriptSegment) -> Bool {
    guard abs(a.start - b.start) <= startTolerance else { return false }
    return textsNearIdentical(a.text, b.text)
}

“Same line” is starts-within-tolerance and near-identical text, and when two copies collide the longer one wins — a finalized line is usually longer and cleaner than the mid-word volatile it replaces. Keeping this out of the audio path means the tricky part is deterministic and tested.

Measuring it, and the legacy path

“First caption in ~1–2s” is a claim, so it’s instrumented. OSSignposter marks the key moments — window feed start, first partial, first final — so time-to-first-caption shows up as an interval in Instruments. And it has to be measured on a real device: the iOS Simulator ships no on-device speech models, so this simply doesn’t run there.

static let signpost = OSSignposter(subsystem: "com.almokhtar.recall", category: .pointsOfInterest)
// ...
signpost.emitEvent("window-feed-start", id: signpostID)
signpost.emitEvent("first onPartial", id: signpostID)
signpost.emitEvent("first isFinal", id: signpostID)

There’s also a legacy path for iOS 17–25 built on SFSpeechRecognizer (with requiresOnDeviceRecognition), and a degradation ladder — manual stream → trimmed clip → whole file — so a device or locale that can’t do the fancy version still gets captions. The modern path is the one worth studying; the fallbacks are there so nobody gets a blank screen.

The takeaway

Latency problems are usually ordering problems. Nothing here made recognition faster — it made the app transcribe the part the user is looking at first, stream it, and do the rest in the background. The tools that made that clean in Swift: an AsyncStream to feed audio, two tasks (feeder + consumer) racing off the main actor with a single main-actor hop to paint, framePosition to seek instead of decode, and a pure merge function so out-of-order results still produce a correct transcript. On-device the whole way, so the audio never leaves the phone.

This powers the captions in Recall, my offline-first audiobook study app — live on the App Store, web version in progress. If you’re doing on-device speech or tricky Swift concurrency, get in touch. Related: the spaced-repetition scheduler behind its flashcards.