PagingSource Random API Duplicate Empty Batch Refresh

When integrating an API that returns randomized items on every request, standard pagination assumptions break down immediately. A random batch endpoint has no stable continuation contract merely because the client supplies an integer key. Bound refill attempts, deduplicate within a generation, and document what a short or empty batch means.

Quick solution:

class RandomItemPagingSource(
    private val api: RandomItemApi
) : PagingSource<Int, Item>() {
    override suspend fun load(params: LoadParams<Int>): LoadResult<Int, Item> {
        val position = params.key ?: 1
        return try {
            val response = api.getItems(page = position, size = params.loadSize)
            val data = response.items
            
            if (data.isEmpty() && position > 3) {
                return LoadResult.Page(data = emptyList(), prevKey = null, nextKey = null)
            }
            
            LoadResult.Page(
                data = data,
                prevKey = if (position == 1) null else position - 1,
                nextKey = if (data.isEmpty()) null else position + 1
            )
        } catch (e: Exception) {
            LoadResult.Error(e)
        }
    }
    
    override fun getRefreshKey(state: PagingState<Int, Item>): Int? {
        return state.anchorPosition?.let { anchor ->
            state.closestPageToPosition(anchor)?.prevKey?.plus(1)
        }
    }
}

When working with unstable datasets, developers frequently encounter infinite retry loops or duplicate items across pages. Official documentation for Android Paging data sources and refresh outlines how the library expects deterministic keys and sequential boundaries. When an API violates this contract by serving randomized rows, you must manage expectations explicitly in client code. For developers managing local system states alongside network layers, auditing cached items can reveal unexpected behaviors as seen when you audit macos system data before deleting.

Does an empty random batch really name end-of-data?

In standard deterministic paging, an empty list returned by a data source is an unambiguous signal that the end of the collection has been reached. For a random API, however, an empty batch can simply mean transient shuffling, sparse index selection, or a server-side filter collision.

Treating every empty response as a definitive end-of-data signal leads to premature termination of the list. If the backend selects random records from a sparse table, a single query might yield zero matching rows while subsequent queries find plenty. To solve this safely, you must establish an explicit product policy for transient empty batches. For instance, if an empty response occurs before reaching a maximum retry threshold or a predefined page ceiling, the client should treat it as a skip rather than an exhaustion marker.

What is reset by a new PagingSource generation?

Every time a user triggers a refresh action or an invalidation event occurs, the Paging library discards the existing PagingSource instance and requests a brand new one. This lifecycle boundary resets several internal states.

When a new generation begins, the anchor position is evaluated against the current UI state to determine where to place the initial load of the new source. However, because a random API changes its dataset ordering on every query, the items present during the previous generation will likely not align with the new generation. This means that invalidating a random PagingSource does not preserve scroll position context reliably. Developers must accept that refreshing a randomized feed will reshuffle the entire viewport rather than appending or patching in place.

How is the refill request ceiling tested?

Testing a PagingSource that interacts with a random endpoint requires explicit simulation of edge cases such as duplicate-heavy payloads and repeated empty responses. Without rigorous bounds, a client can easily get trapped in excessive refill cycles trying to satisfy the requested load size.

To verify your implementation, apply the following execution checklist in your test suite:

1. Feed a sequence of duplicate-only items into the load function and verify that the pagination layer reaches a bounded stop instead of looping infinitely.
2. Trigger a manual refresh while mocking a randomized backend response to confirm that ordering instability does not crash the adapter.
3. Return a transient empty batch followed by valid data items to ensure the policy handles short-circuiting correctly.
4. Measure memory consumption and query counts during rapid scrolling to confirm the refill request ceiling prevents excessive network calls.