Why Netflix, Instagram, and Twitter Pick Different Databases

In the world of system design, there is no "perfect" database. Engineering teams at tech giants don't choose a database based on popularity; they choose based on the specific data access patterns and guarantees their unique features require.

Here is a breakdown of why Netflix, Instagram, and Twitter (X) ended up with vastly different storage architectures.

1. Netflix: High Availability and Global Scale

Primary Choice: Apache Cassandra (NoSQL / Wide-Column Store)

The Challenge

Netflix operates in nearly every country. When you press "play," the system needs to know your viewing history and preferences instantly. If the database goes down, Netflix effectively stops working.

Why Cassandra?

• Availability over Consistency (AP): Netflix follows the CAP theorem's AP model. It’s better for a user to see a slightly outdated "Continue Watching" list than for the app to crash entirely.
• Masterless Architecture: Cassandra has no single point of failure. Data is replicated across multiple "nodes" and regions. If one AWS data center goes dark, the others pick up the slack without a hiccup.
• Write Throughput: Every time you pause, skip, or watch a show, Netflix logs data. Cassandra is designed to handle massive volumes of concurrent writes across a distributed cluster.

2. Instagram: Relationships and Reliability

Primary Choice: PostgreSQL (Relational / SQL)

The Challenge

Instagram is built on complex relationships: Users follow accounts, accounts post photos, photos have likes, and likes belong to users. This "social graph" is inherently relational.

Why PostgreSQL?

• Data Integrity (ACID): When you like a photo, that transaction must be permanent and accurate. PostgreSQL ensures that your data remains consistent.
• Sharding for Scale: While PostgreSQL is a traditional SQL database, Instagram engineers built a custom "sharding" layer on top of it. This allows them to split their massive dataset across thousands of smaller PostgreSQL instances.
• Structured Queries: SQL allows Instagram to perform complex joins and filters easily, which is essential for serving a personalized feed based on who you follow and what you’ve interacted with.

3. Twitter (X): Speed and Real-Time Feeds

Primary Choice: Redis & Manhattan (In-Memory & Distributed Key-Value)

The Challenge

Twitter’s primary product is the "Timeline." When a celebrity with 100 million followers tweets, that message needs to appear on 100 million screens in near real-time. This is a massive "fan-out" problem.

Why Redis?

• In-Memory Speed: For the active "Home Timeline," Twitter uses Redis. Because Redis stores data in RAM rather than on a disk, the read speeds are incredibly low (sub-millisecond).
• The Timeline Cache: Twitter pre-computes your timeline. Instead of searching a database when you log in, Twitter looks at a "list" already waiting for you in Redis.
• Manhattan for Persistence: For long-term storage of tweets, Twitter built "Manhattan," a distributed key-value store designed for low latency and high scale, similar in spirit to Amazon’s DynamoDB.

Summary Comparison

Conclusion

The "best" database depends on what you are willing to sacrifice.
* Netflix sacrifices strict consistency for global uptime.
* Instagram sacrifices horizontal simplicity for relational integrity.
* Twitter sacrifices disk-storage economy for raw RAM-based speed.

When building your own applications, start by asking: Is my data a list, a graph, or a document? The answer to that will lead you to your database.