From 2s to 10ms: Caching Search

1. A Real-World Caching Story

A while ago, we were building a search module for a hotel booking application. The requirements were straightforward: users would input their travel dates, destination, and guest counts, and the system would return a list of available hotels, pricing matrices, and amenities.

Under the hood, every single user search triggered a query to our live database to fetch hundreds of hotel records, join them with active pricing databases, and evaluate real-time availability. While this worked fine in our local testing environments with a few dozen rows, in production under load, it became a significant bottleneck.

As traffic grew, the network call latency began to climb. What used to take 100 milliseconds was now taking upwards of 2 seconds. Every time a user toggled a filter, changed dates, or clicked back to the search results page, the frontend had to make another expensive API round-trip to the live database. It was clear that query execution and network round-trips were hurting the user experience.

To optimize this, we introduced the concept of caching. Instead of hitting the live database on every single duplicate request, we stored the search results in a fast cache layer. If a user searched for hotels in the same city for the same dates, or simply went back to a search they had made moments ago, we served the results instantly from the cache in under 10 milliseconds. This drastically reduced the network latency, relieved our database from redundant queries, and made the entire booking flow feel instantaneous.

This experience introduced me to the power of caching. I realized that caching is not just an optional optimization; it is a fundamental architectural requirement for any scale. In this article series, we will explore why we need caching, where caching does not help, and the different strategies we can use to implement it effectively.