Optimizing External API Consumption with Negative Caching

1. The Trigger: A 98% Error Rate

Our application features a social timeline tab where users can view posts. To support blocking, the backend must check a third-party chat platform's API (GET /v3/users/{user_id}/block) to filter out posts from blocked users.

During an audit of our APM dashboards, we noticed a massive spike in outbound API errors. Over 98.4% of calls (roughly 65,000 requests per day) to the chat platform were failing with an HTTP 400 Bad Request. Digging into the details, the API response was always:

Because the application caught this exception and fell back to returning an empty block list, the front-end never saw the failure. The system continued to function, but every check for an unregistered user paid a 56ms latency tax, accounting for 71% of our total timeline response time.

2. The Cause: Lazy User Registration

Why were the users not found?

Our system registers users on the third-party chat platform **lazily**. A user is only registered when they physically tap on the chat tab for the first time. However, the social timeline tab is open to all logged-in users. Since 98% of our active users had never opened the chat tab, they did not exist in the chat database. Checking their block list was guaranteed to fail.

3. The Solution: Negative Caching

The obvious fix was to call a "check user existence" API first. But that is still an outbound network call, which would not solve the latency issue.

Instead, we implemented Negative Caching (caching the non-existence of a resource).

The flow works as follows:

1. Before calling the external API, check our local Redis cache for a negative marker: unregistered:{userId}.
2. If the marker exists, skip the API call and immediately return an empty block list (saving 50ms+).
3. If the marker does not exist, call the external API.
4. If the API returns a 400301 error, write the negative marker to Redis with a TTL of 7 days.

Here is the basic implementation in Kotlin:

fun findBlockedUsers(userId: String): List<String> {
    if (redisTemplate.hasKey("chat:unregistered:$userId")) {
        return emptyList()
    }

    return try {
        chatClient.getBlockedUsers(userId)
    } catch (e: ChatApiException) {
        if (e.errorCode == 400301) {
            redisTemplate.opsForValue().set(
                "chat:unregistered:$userId",
                "true",
                Duration.ofDays(7)
            )
            return emptyList()
        }
        throw e
    }
}

4. Cache Eviction and Consistency

Negative caching introduces consistency risks: what happens if a user is created in the chat system, but their negative marker is still active in Redis? They will be unable to block or be blocked correctly for 7 days.

To maintain consistency, we must evict the negative cache key during user registration events:

• Chat Connection: When the client SDK successfully connects to the chat platform (the primary registration path), evict the Redis key.
• Block Command: If a user attempts to block another user, clear both users' negative cache keys before executing the command.
• TTL Fallback: The 7-day TTL acts as a safety net in case of failed evict commands.

Scale Warning: Under ultra-high traffic, hitting Redis for every timeline request can introduce Redis network overhead. To scale this further, consider implementing a 2-Tier Caching strategy (using a short-lived local In-Memory cache like Caffeine before fallback to Redis) or storing a registration status flag (e.g., is_registered_on_chat) directly in your primary User Database.