CameraX and ML Kit: Build Real-Time Image Analysis Without Frame Backlogs

Real-time camera features often fail in a surprisingly ordinary way: the camera produces frames faster than the machine-learning model can consume them. The preview may still look smooth, but analysis latency grows, results arrive for stale frames, memory pressure increases, or the analyzer appears to freeze.

The practical fix is to treat camera analysis as a latest-state pipeline, not a queue. With CameraX, keep ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST, convert each delivered ImageProxy using its rotation metadata, send it to ML Kit, and close the proxy exactly once after processing completes. For many live scanning features, that architecture matters more than trying to process every frame.

This guide focuses on the reusable pipeline around ML Kit rather than one detector. The same ownership rules apply to text recognition, barcode scanning, object detection, and similar vision tasks.

Why processing every frame is usually the wrong goal

A camera can produce frames at a rate that exceeds the latency of an ML model. If inference takes longer than the interval between frames, a queue creates work that is already obsolete by the time it runs.

Suppose the camera delivers 30 frames per second, roughly one frame every 33 milliseconds, while a detector takes 80 milliseconds. Processing every frame cannot catch up. After one second, the application has accumulated more analysis work than it can finish.

For a live UI, the user normally cares about what the camera sees now, not what it saw several hundred milliseconds ago. Dropping superseded frames is therefore a correctness decision as much as a performance optimization.

Google’s ML Kit guidance explicitly recommends ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST for CameraX real-time analysis. When the analyzer is busy, newer frames replace older pending work instead of building a backlog.

Build the pipeline around ownership

A stable design has four owners:

1. CameraX owns capture and frame delivery.
2. The analyzer owns conversion from ImageProxy to ML Kit input.
3. ML Kit owns asynchronous inference.
4. The UI layer owns presentation of the latest accepted result.

Keep those boundaries visible in code. Do not let a detector callback retain camera frames, and do not let the UI decide when an ImageProxy should be released.

A minimal CameraX setup can make the backpressure policy explicit:

val analysis = ImageAnalysis.Builder()
    .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
    .build()

analysis.setAnalyzer(cameraExecutor) { imageProxy ->
    analyzeFrame(imageProxy)
}

This is a better default than inventing an application-level frame queue. A queue is justified only when every frame is independently valuable, which is uncommon for interactive scanning.

Preserve rotation when creating the ML Kit input

The raw camera buffer does not necessarily have the same orientation as the preview. ML Kit needs the image plus the rotation required to interpret it correctly.

With CameraX, read imageProxy.imageInfo.rotationDegrees and pass it when creating InputImage:

private fun analyzeFrame(imageProxy: ImageProxy) {
    val mediaImage = imageProxy.image

    if (mediaImage == null) {
        imageProxy.close()
        return
    }

    val input = InputImage.fromMediaImage(
        mediaImage,
        imageProxy.imageInfo.rotationDegrees
    )

    processWithMlKit(input, imageProxy)
}

Avoid copying the frame into a bitmap unless another part of the feature genuinely requires a bitmap. ML Kit accepts camera-backed image data directly for common vision APIs, which avoids unnecessary conversion work in the hot path.

Close ImageProxy after asynchronous processing

The most important lifecycle rule is simple: every delivered ImageProxy must eventually be closed.

The subtle part is timing. ML Kit processing is asynchronous, so closing the proxy immediately after calling process() can release the underlying image before inference finishes. Never closing it is worse because CameraX eventually stops delivering usable frames.

Use a completion callback that runs on both success and failure:

private fun processWithMlKit(
    input: InputImage,
    imageProxy: ImageProxy
) {
    textRecognizer.process(input)
        .addOnSuccessListener { result ->
            publishLatestResult(result.text)
        }
        .addOnFailureListener { error ->
            reportAnalysisFailure(error)
        }
        .addOnCompleteListener {
            imageProxy.close()
        }
}

addOnCompleteListener makes frame release independent from the detector outcome. Keep the close operation in one place so a future error branch does not accidentally leak the frame or close it twice.

This is the same kind of ownership discipline that helps when preventing memory leaks in Android apps: resources with lifecycle boundaries need an explicit release owner.

Do not run inference on the main thread

Camera analysis and UI rendering have different latency budgets. The analyzer should use a dedicated executor, while only the small result needed for presentation crosses back to the UI layer.

That separation prevents camera work from competing with Compose or View rendering. It also makes overload easier to diagnose because the analysis path has its own execution boundary.

For a ViewModel-based application, convert detector output into a compact domain result before updating state:

data class ScanResult(
    val value: String,
    val observedAtMillis: Long
)

private fun publishLatestResult(value: String) {
    viewModel.onScanResult(
        ScanResult(
            value = value,
            observedAtMillis = System.currentTimeMillis()
        )
    )
}

Do not pass ImageProxy, media.Image, detector objects, or camera executors into the ViewModel. Those are infrastructure details tied to the camera lifecycle.

Choose bundled or downloaded models deliberately

Some ML Kit APIs offer both bundled and Google Play services based model delivery. The choice changes startup behavior.

A bundled model increases application size but is available immediately. A dynamically delivered model keeps the application smaller, but the first attempt may need to wait for the model to become available. For an offline-first scanner or a workflow that must work immediately after installation, that trade-off can matter more than a few megabytes.

Treat model readiness as a UI state rather than assuming the detector is instantly usable:

Camera ready
    |
    +-- Model ready ------> Analyze latest frame
    |
    +-- Model unavailable -> Show preparing state
                              and retry when ready

This also corrects a common architectural misconception: ML Kit is not one uniform “cloud inference” product. Several vision APIs perform inference on device, while some unbundled variants obtain model components through Google Play services. Design from the behavior of the specific API you selected.

Keep overlays aligned with the preview

Detection is only half the job when the UI draws bounding boxes or highlights over a live preview. Model coordinates belong to the analyzed image, while the user sees a preview that may be cropped, scaled, mirrored, or rotated.

Avoid scattering coordinate math across composables or custom views. Create one transformation boundary that maps analysis coordinates into preview coordinates.

If your feature is mostly CameraX plus ML Kit, CameraX’s higher-level controller integration can reduce this work because its ML Kit integration can map model output coordinates to PreviewView. If you need custom surfaces or unusual camera control, a lower-level CameraProvider setup gives more flexibility but leaves more transformation responsibility in your code.

Pick the abstraction based on how much camera control the product actually needs.

Failure modes worth testing on a real device

A camera feature can pass unit tests and still fail under device pressure. Test the pipeline as a state machine, not only as a happy-path detector call.

Failure mode	What it looks like	What to verify
Proxy not closed	Analysis stops after several frames	Every analyzer path reaches one close
Frame queue grows	Results lag behind the preview	Backpressure is KEEP_ONLY_LATEST
Wrong rotation	Recognition works only in one orientation	Rotation comes from imageInfo.rotationDegrees
Model not ready	First scan returns nothing	Startup exposes model readiness and retry
Heavy conversion	Preview stutters or device heats up	Avoid bitmap copies in the hot path
UI receives stale results	Overlay jumps backward	UI accepts only the latest relevant observation
Analyzer survives screen exit	Camera or detector leaks	Unbind camera and close detector with lifecycle owner

Test slow devices as well as fast ones. Backpressure bugs often hide on flagship hardware because inference completes quickly enough to mask the architecture problem.

If device communication itself is unreliable during testing, separate that issue from the camera pipeline. The Android wireless debugging disconnect guide shows how to distinguish ADB transport problems from application behavior.

A reusable implementation checklist

Before calling a CameraX plus ML Kit feature production-ready, verify these invariants:

• ImageAnalysis uses an intentional backpressure strategy.
• The analyzer runs away from the main thread.
• Rotation metadata is preserved.
• Each ImageProxy has exactly one release path.
• Detector success and failure both release the frame.
• Model availability is handled explicitly when delivery is dynamic.
• Camera objects do not leak into ViewModel or domain layers.
• Overlay coordinates have one documented transformation path.
• The detector is closed when its lifecycle owner is destroyed.
• Slow-device and first-install behavior are tested.

The key idea is not specific to OCR or barcode scanning. A real-time vision feature is a bounded streaming system. Once you model it that way, frame dropping, resource ownership, lifecycle cleanup, and UI freshness become explicit design choices instead of intermittent bugs.