[System Design Tech Case Study Pulse #103] 8+ Billion Daily Views: How Facebook's Live Video Ranking Algorithm Works
With detailed engineering explanation and flow chart....
Hi All,
Facebook's Live Video Ranking Algorithm is a tech marvel, capable of processing 8 billion daily views using Apache Storm for real-time stream processing and Memcached for high-speed caching. This sophisticated system forms the backbone of Facebook's live video experience, ensuring that users discover the most relevant and engaging live content in real-time.
In this post, I’ll dive deep into how this system works, exploring the key components, technologies, and processes that enable such massive-scale, low-latency live video ranking.
Learn how to System design —Design Lyft
System Overview
Daily live video views: 8+ billion
Concurrent live streams: 100,000+
Peak viewers for a single stream: 5 million+
Live video sessions started per day: 50 million+
Average ranking latency: < 100ms
Storm clusters: 50+
Storm workers: 100,000+
Memcached nodes: 500,000+
Features considered per video: 10,000+
ML models in production: 100+
Global data centers: 20+
Edge locations: 150+
System availability: 99.99%
Learn system design pulses -
[System Design Pulse #3] THE theorem of System Design and why you MUST know it - Brewer theorem
[System Design Pulse #4] How Distributed Message Queues Work?
[System Design Pulse #5] Breaking It Down: The Magic Behind Microservices Architecture
[System Design Pulse #6] Why Availability Patterns Are So Crucial in System Design?
[System Design Pulse #7] How Consistency Patterns helps Design Robust and Efficient Systems?
[System Design Pulse #9] Why these Key Components are Crucial for System Design.
How it works —
1. Broadcaster initiates a live stream through the Facebook App or Website.
2. Live Video Ingestion Service receives and processes the incoming stream.
3. Transcoding Farm generates multiple quality versions of the stream.