[System Design Tech Case Study Pulse #25] 500+ Million Users Daily : How Instagram Stories Actually Work

Tech you must know....with flowchart and explanations how it works..

Nov 01, 2024

Hi All,

Instagram Stories is a tech marvel, capable of delivering ephemeral content to 500 million daily active users in real-time.

This system forms the backbone of one of Instagram's most popular features, enabling users to share and consume short-lived photos and videos seamlessly.

Let's dive deep into how this system works, following the journey of a Story from creation to viewer consumption, and exploring the impressive metrics behind each step .

[System Design Tech Case Study Pulse #2] How Lyft Handles 2x Traffic Spikes during Peak Hours with Auto scaling Infrastructure..

System Overview

Before we begin our journey, let's look at some key overall metrics of Instagram Stories:

- Daily active users: 500 million

- Stories created daily: Over 1 billion

- Peak creation rate: Over 100,000 Stories per second

- Average Story view time: < 1 second

- Content types: Photos, videos, boomerangs, live streams

- Maximum Story duration: 15 seconds per segment

- Story lifetime: 24 hours

- Global reach: Available in 150+ countries

- Supported devices: iOS, Android, Web

- System availability: 99.99% uptime achieved in 2023

How Real World Scalable Systems are Build — 200+ System Design Case Studies:

System Design Den : Must Know System Design Case Studies

[System Design Tech Case Study Pulse #15] 80 Million Photos Daily : How Instagram Achieves Real Time Photo Sharing

[System Design Tech Case Study Pulse #20] Serving 1 Trillion Edges in Social Graph with 1ms Read Times : How Facebook TAO works

[System Design Tech Case Study Pulse #9] Facebook’s News Feed Algorithm Marvel: How it Serves 2.9 Billion Daily Active Users Using PyTorch and Cassandra

[System Design Tech Case Study Pulse #2] How Lyft Handles 2x Traffic Spikes during Peak Hours with Auto scaling Infrastructure..

[System Design Tech Case Study Pulse #12] 8+ Billion Daily Views: How Facebook’s Live Video Ranking Algorithm Processes Daily Views Using Storm and Memcache

[System Design Tech Case Study Pulse #18] Tinder 1.5 Billion Swipes per Day : How Tinder Real Time Matching Actually Works

[System Design Tech Case Study Pulse #17] How Discord’s Real-Time Chat Scales to 200+ Million Users

[System Design Tech Case Study Pulse #13] 20 Billion Messages Daily: How Facebook Messenger Actually Works

[System Design Tech Case Study Pulse #16] How Facebook’s Live Video Ranking Algorithm Works to Process 8+ Billion Daily Views

[System Design Tech Case Study Pulse #16] 100+ Million Requests per Second : How Amazon Shopping Cart Actually Works

[System Design Tech Case Study Pulse #11] Serving 132+ Million Users : Scaling for Global Transit Real Time Ride Sharing Market at Uber

[System Design Tech Case Study Pulse #8] Processing 2 Billion Daily Queries Using Unicorn and Apache Thrift : Facebook’s Graph Search Secret Sauce

[Tuesday Engineering Bytes] How Netflix handles millions of memberships efficiently?

[Friday Engineering Bytes] The Billion-Dollar Question — What’s My ETA? How Uber Calculates ETA…

[Saturday Engineering Bytes] What happens Once You Press Play button on Netflix..

[Monday Engineering Bytes] FAANG level — How to Write Production Ready Code ?

[Tuesday Engineering Bytes] Impressive Tech Behind YouTube’s Scaling to 3 Billion Users: Inside Vitess Database Clustering

[Thursday Engineering Bytes] Binge-worthy Backend: Netflix’s 99.99% Uptime Serves 230Million Users Globally!

[Friday Engineering Bytes] At Amazon How 310 Million Users Experience Lightning-Fast Load Times

[Monday System Design Bytes] — How to solve any system design question ( with detailed case study example)

[Tuesday Engineering Bytes] How PayPal Manages Over 400 Million Active Accounts Seamlessly?

Now, let me walk through the Story creation and consumption process and see how these numbers translate to real-world performance.

How things Work —

1. Story Creation and Upload

The journey begins when a user creates a new Story:

Content Capture and Processing

1. The user captures or selects content within the Instagram app.
2. Client-side processing begins immediately:
- Applies filters and effects in real-time
- Compresses content for efficient upload
- Generates a preview for the user
Key metrics for this process:
- Filter application time: < 100ms for photos, < 500ms for videos
- Compression ratio: 70-80% size reduction without noticeable quality loss
- Preview generation time: < 200ms
3. Once the user is satisfied, they initiate the upload.

Upload and Initial Processing

1. The content is sent to Instagram's nearest edge server:
- Uses a globally distributed CDN for low-latency uploads
- Employs resumable upload protocols for reliability
2. Upon receipt, the Story Ingestion Service:
- Validates the content format and user permissions
- Generates multiple resolutions for different device types
- Extracts metadata (location, mentioned users, hashtags)
Key performance indicators for this process:
- Average upload time: < 2 seconds for 90% of Stories
- Content validation time: < 50ms
- Transcoding time: < 5 seconds for 15-second video Stories
- Metadata extraction time: < 100ms

2. Story Indexing and Distribution

Once a Story is uploaded and processed:
1. The Story Indexing Service takes over:
- Assigns a unique identifier to the Story
- Updates the user's Story queue
- Indexes the Story for discoverability (if public)
2. The Content Distribution Service then:
- Determines the initial set of potential viewers (followers, close friends)
- Pre-warms caches in relevant geographic regions
- Updates the Story ring UI for the creator's followers
3. For Stories with location tags or hashtags, additional indexing occurs:
- Updates global and local Story maps
- Adds to relevant discovery feeds
Indexing and distribution metrics:
- Indexing time: < 100ms
- Initial distribution list generation: < 200ms for accounts with up to 1 million followers
- Cache pre-warming time: < 1 second for 99% of Stories
- Story ring UI update propagation: < 5 seconds for 95% of followers

3. Real-time Content Delivery

When a user opens Instagram to view Stories:
1. The Story Fetching Service springs into action:
- Retrieves the list of available Stories for the user
- Prioritizes based on relevance algorithms and viewing history
2. As the user taps to view a Story:
- The nearest CDN edge node serves the content
- Adaptive bitrate streaming is used for video Stories
- Prefetching begins for the next few Stories in the queue
3. View statistics are recorded in real-time:
- Updates view counts and viewer lists
- Triggers any relevant notifications (e.g., for close friends views)
Content delivery metrics:
- Story list retrieval time: < 100ms
- Time to first frame: < 300ms for 99% of Story views
- Adaptive bitrate switch time: < 200ms
- View statistic update time: < 50ms

4. Fleeting Content Management

Managing the 24-hour lifespan of Stories requires careful system design:
1. The Expiration Management Service constantly monitors Story lifetimes:
- Schedules deletion of expired Stories
- Updates indexes and caches to remove expired content
2. The Storage Optimization Service:
- Moves aging Stories to cooler storage tiers
- Implements intelligent caching based on view patterns
3. For Stories marked as "Highlights" by users:
- Content is moved to long-term storage
- Indexes are updated to reflect the new status
Ephemeral management metrics:
- Expiration check interval: Every 5 minutes
- Deletion propagation time: < 1 minute across all global caches
- Storage tier transition time: < 10 minutes for 99% of Stories
- Highlight conversion time: < 2 seconds

Behind the Scenes: System Interactions and Performance

Throughout all these processes, Instagram Stories' components are constantly interacting:

1. Client App <-> Edge Servers :

- Handles millions of uploads and billions of views per hour

- Uses protocol optimizations for faster data transfer

- Average request routing time: < 20ms

2. Story Ingestion Service <-> Transcoding Farm :

- Processes incoming content in parallel

- Utilizes GPU acceleration for video processing

- Peak transcoding capacity: 500,000 Stories per minute

3. Indexing Service <-> Distributed Database :

- Maintains real-time indexes of active Stories

- Uses a combination of in-memory and persistent storage

- Index update latency: < 50ms

4. Content Distribution Service <-> Global CDN :

- Coordinates content placement across thousands of edge nodes

- Employs predictive algorithms for optimal content distribution

- Global propagation time: < 30 seconds for 99% of Stories

5. Story Fetching Service <-> Relevance Ranking Engine :

- Personalizes Story order for each user in real-time

- Incorporates machine learning models for relevance scoring

- Ranking computation time: < 50ms per user

6. Monitoring System <-> All Components :

- Collects trillions of data points daily

- Uses anomaly detection for proactive issue identification

- Alert generation time: < 10 seconds for critical issues

Handling Scale and Efficiency

To deliver stories content to 500 million daily users in real-time, Instagram Stories employs several advanced techniques:

1. Dynamic Content Encoding :

- Adapts encoding parameters based on content type and network conditions

- Reduces bandwidth usage by up to 50% without quality loss

- Encoding decision time: < 10ms per segment

2. Intelligent Content Prefetching :

- Predicts user viewing patterns to preload content

- Uses machine learning models trained on billions of user interactions

- Improves Story transition smoothness by 40%

3. Distributed Caching Hierarchy :

- Employs a multi-tiered caching strategy (device, edge, regional, global)

- Dynamically adjusts caching policies based on content popularity

- Achieves 95% cache hit rate for popular Stories

4. Real-time Analytics Processing :

- Processes view events using a stream processing framework

- Updates metrics and triggers notifications in near real-time

- Handles peak loads of over 10 million events per second

5. Content-Aware Networking :

- Optimizes network paths based on content type and user location

- Reduces average latency by 30% compared to traditional CDN routing

- Path optimization time: < 5ms

6. Storage Optimization :

- Implements a tiered storage system optimized for short-lived content

- Reduces storage costs by 40% compared to traditional persistent storage

- Achieves 99.999% data durability for the 24-hour Story lifetime

That’s all for Instagram Stories!

If you liked this article, like and share.

Thank you for reading Ignito. This post is public so feel free to share it.

Learn system design pulses -

[System Design Pulse #1] Understanding Latency and Throughput: Critical Factors in System Design and Performance Tuning

[System Design Pulse #2] Striking the Right Balance: Optimizing Availability and Consistency in Distributed Architectures

[System Design Pulse #3] THE theorem of System Design and why you MUST know it - Brewer theorem

[System Design Pulse #4] How Distributed Message Queues Work?

[System Design Pulse #5] Breaking It Down: The Magic Behind Microservices Architecture

[System Design Pulse #6] Why Availability Patterns Are So Crucial in System Design?

[System Design Pulse #7] How Consistency Patterns helps Design Robust and Efficient Systems?

[System Design Pulse #8] Caching is Crucial : How Caching Slashes Latency and Supercharges Performance

[System Design Pulse #9] Why these Key Components are Crucial for System Design.

Master System Design

More system design case studies coming soon! Follow - Link

Things you must know in System Design -

System design basics : https://bit.ly/3SuUR0Y
Horizontal and vertical scaling : https://bit.ly/3slq5xh
Load balancing and Message queues: https://bit.ly/3sp0FP4
High level design and low level design, Consistent Hashing, Monolithic and Microservices architecture : https://bit.ly/3DnEfEm
Caching, Indexing, Proxies : https://bit.ly/3SvyVDc
Networking, How Browsers work, Content Network Delivery ( CDN) : https://bit.ly/3TOHQRb
Database Sharding, CAP Theorem, Database schema Design : https://bit.ly/3CZtfLN
Concurrency, API, Components + OOP + Abstraction : https://bit.ly/3sqQrhj
Estimation and Planning, Performance : https://bit.ly/3z9dSPN
Map Reduce, Patterns and Microservices : https://bit.ly/3zcsfmv
SQL vs NoSQL and Cloud : https://bit.ly/3z8Aa49

Github for System Design Interviews with Case Studies

Master Data Structures and Algorithms

Topics that are important in Data Structures and Algorithms : https://bit.ly/3EAud36
Complexity Analysis : https://bit.ly/3fSMChP
Backtracking : https://bit.ly/3TazwL3
Sliding Window : https://bit.ly/3ywJezP
Greedy Technique : https://bit.ly/3rMgb7m
Two pointer Technique : https://bit.ly/3yvVqRc
1- D Dynamic Programming : https://bit.ly/3COFU5s
Arrays : https://bit.ly/3MqxuEK
Linked List : https://bit.ly/3rIwBxI
Strings : https://bit.ly/3MmIH96
Stack : https://bit.ly/3ToikSB
Queues : https://bit.ly/3yHSssX
Hash Table/Hashing : https://bit.ly/3ew8oYm
Binary Search : https://bit.ly/3yK9R4l
Trees : https://bit.ly/3g1og5u
Heap/Priority Queue : https://bit.ly/3rZb9EI
Divide and Conquer Technique : https://bit.ly/3esYWF3
Recursion : https://bit.ly/3yvPbwN

Curated Question List 1 : https://bit.ly/3ggSDFq
Curated Question List 2 : https://bit.ly/3VrUqrj

Ignito