Design a Video Player (YouTube / Netflix style)

A frontend system design guide for a web video player. You will learn how the browser turns small media segments into smooth playback. You will also learn how the player picks quality and how to build controls that everyone can use.

Frontend system design · Media and real-time UI

Design a player that starts fast and rarely stalls. It adapts quality to the network and works with a keyboard and a screen reader.

Correctness contract: The player never shows a frame from the wrong video and never plays protected content without a valid license. The UI state always matches the real media element. The user always has a working way to pause, seek, and read captions.

You know basic HTML, JavaScript, and HTTP. You do not need to know video codecs. Products like YouTube and Netflix are used only to frame the problem. This is a teaching design, not a claim about how those products work inside. All numbers are rough estimates for reasoning, not benchmarks.

A video player looks like one HTML tag. In practice it is a small streaming system that runs inside the browser. It downloads a manifest and picks a quality level. It fetches media segments one by one, feeds them to the decoder, and watches the buffer closely. At the same time it must draw controls, show captions, report metrics, and react to the user within one frame. The interviewer wants to see that you understand both halves. The first half is the media pipeline: Media Source Extensions, adaptive bitrate, and buffer management. The second half is the product layer: controls, accessibility, thumbnails, analytics, and preloading. This guide builds both halves and shows where they meet.

Contents

  1. Concepts to understand first
  2. 1. Clarifying questions and requirements
  3. 2. Scope and budgets
  4. 3. High-level architecture
  5. 4. Data model
  6. 5. Interface and API
  7. 6. Deep dive: adaptive bitrate on the client
  8. 7. Deep dive: buffer management and startup time
  9. 8. Deep dive: seeking and thumbnail previews
  10. 9. Deep dive: captions and DRM
  11. 10. Deep dive: custom controls, accessibility, and picture-in-picture
  12. 11. Deep dive: QoE analytics and preloading the next video
  13. 12. Failure modes
  14. 13. Trade-offs
  15. 14. 45-minute interview walkthrough
  16. 15. Summary
  17. 16. Common mistakes
  18. Worked example: a viewer on a train
  19. Code lab: a buffer-aware ABR rule
  20. Check yourself
  21. Key terms
  22. Further reading

Concepts to understand first

Media Source Extensions (MSE)

MSE lets JavaScript build the media stream for a <video> element. You create a MediaSource, attach it with URL.createObjectURL, and push bytes into one or more SourceBuffer objects with appendBuffer(). The browser still decodes and renders. Your code only decides which bytes to fetch and when.

Example. The player fetches a 4 second video segment at 720p, appends it, then fetches the next one at 1080p because the network got faster. The viewer sees one smooth video.

Limit. MSE support differs by platform. Older iPhone Safari had no MSE and only played HLS natively. Newer iOS versions offer ManagedMediaSource, which can ask your code to release memory.

Adaptive bitrate (ABR)

The server encodes each video at several quality levels called renditions. Each rendition is cut into short segments with aligned keyframes. The client picks a rendition for every segment, based on measured throughput and how much video is already buffered.

Example. A ladder has 240p at 400 kbps, 480p at 1.2 Mbps, 720p at 3 Mbps, and 1080p at 6 Mbps. On a 4 Mbps link the player settles on 720p.

Limit. Throughput estimates are noisy. A rule that only follows throughput switches too often. A rule that only follows the buffer reacts slowly at startup.

HLS and DASH manifests

A manifest is a text file that lists renditions, codecs, and segment URLs. HLS uses .m3u8 playlists. DASH uses an XML file called an MPD. Both point to the same kind of fragmented MP4 (CMAF) segments in modern setups.

Example. A master .m3u8 lists four variant playlists. Each variant playlist lists 300 segments of 4 seconds for a 20 minute video.

Limit. The manifest is only a promise. A segment can still be missing, slow, or corrupt, so the player must handle per-segment errors.

Forward buffer

The forward buffer is the number of seconds of decoded-ready media ahead of currentTime. If it reaches zero, playback stalls and the element fires waiting. The player reads it from video.buffered, which is a list of time ranges.

Example. currentTime is 62 s and buffered holds the range 40 s to 90 s. The forward buffer is 28 seconds.

Limit. A large buffer costs memory and wasted bytes if the user leaves. Browsers also cap SourceBuffer size and throw QuotaExceededError when you push too much.

Encrypted Media Extensions (EME)

EME is the browser API for DRM. Encrypted segments go into MSE as usual. The browser asks a Content Decryption Module (CDM) to decrypt them. The CDM needs a license, which your code fetches from a license server and passes back with MediaKeySession.update().

Example. Chrome uses Widevine, Safari uses FairPlay, and Edge supports PlayReady. One CMAF file encrypted with CENC can serve several of them.

Limit. DRM stops casual copying, not a determined attacker with a camera. It also adds a network round trip before the first frame.

1. Clarifying questions and requirements

Start by asking questions. Each answer removes a branch from the design. Good questions for a video player are these.

Functional requirements

Non-functional requirements

Interview

Say out loud which parts you treat as server work. Transcoding, packaging, and the CDN are backend topics. Your job here is everything from the manifest URL to the pixels and the analytics beacon.

2. Scope and budgets

Numbers turn vague goals into design choices. Work them out on the board.

Network and bitrate

Startup budget

A 1 second startup target leaves little room. Split it like this on a 30 ms round trip.

The total is about 520 ms without DRM. That fits, but only if the player JavaScript is already parsed. Downloading and parsing a 300 KB player bundle on a mid-range phone can cost 600 ms by itself.

Memory and buffer

Frame and bundle budgets

3. High-level architecture

Split the player into layers with one-way data flow. The media element is the source of truth for time and readiness. The engine is the source of truth for what to fetch. The store mirrors both so the UI can render.

Video player architecture: watch page and controls read a player state store, the playback engine fetches from the CDN and appends to MediaSource, EME talks to a DRM license server, and QoE beacons go to the player API.
Figure 1. Player layers. The engine owns fetching, the video element owns decoding, and the store feeds the UI.

Layers

Why keep the engine framework-free

Media events fire often. timeupdate fires about 4 times per second and progress fires on every chunk. If each event triggers a React render of the whole page, you burn the frame budget. The engine updates its own state and the store publishes only the fields that changed. The seek bar can even skip the store and update its position in a requestAnimationFrame loop that reads video.currentTime directly.

Intuition

Think of the engine as a small backend that happens to live in the tab. It has a queue (segments to fetch), a scheduler (when to fetch), a policy (which quality), and a storage limit (the SourceBuffer quota).

4. Data model

Keep three kinds of state apart. Server data describes the video. Engine state describes fetching. UI state describes what the user sees. Normalize renditions and tracks by ID so the quality menu and the engine refer to the same objects.

type PlayInfo = {
  videoId: string
  title: string
  durationSec: number
  manifestUrl: string            // master .m3u8 or .mpd
  drm?: { keySystem: 'com.widevine.alpha' | 'com.apple.fps' | 'com.microsoft.playready'
          licenseUrl: string; token: string }
  thumbnails: { spriteUrlTemplate: string; cols: number; rows: number
                intervalSec: number; width: number; height: number }
  textTracks: TextTrackInfo[]
  resumeAtSec?: number
  nextVideoId?: string
}

type Rendition = { id: string; height: number; bandwidth: number
                   codecs: string; segmentUrls: string[]; segmentSec: number }

type PlayerState = {
  status: 'idle' | 'loading' | 'playing' | 'paused' | 'buffering' | 'ended' | 'error'
  currentTime: number
  duration: number
  buffered: Array<[start: number, end: number]>
  renditions: Record<string, Rendition>
  renditionOrder: string[]        // sorted by bandwidth
  activeRenditionId: string
  qualityMode: 'auto' | { fixed: string }
  estimateKbps: number
  textTrackId: string | null
  volume: number; muted: boolean; rate: number
  pip: boolean; fullscreen: boolean
  error?: { code: 'NETWORK' | 'DECODE' | 'DRM' | 'UNSUPPORTED'; retryable: boolean }
}

type QoeEvent = {
  sessionId: string; videoId: string; t: number
  kind: 'start' | 'stall_begin' | 'stall_end' | 'switch' | 'seek' | 'error' | 'heartbeat'
  data: Record<string, number | string>
}

Two rules matter. First, currentTime in the store is a mirror, never the source of truth. Read the real value from the element when accuracy matters, for example when you compute the seek bar position or a resume point. Second, every async result carries the videoId and a load generation number. When the user switches video, the generation goes up, and late results from the old video are dropped. This is how you keep the contract of never showing a frame from the wrong video.

5. Interface and API

Server endpoints

Engine interface

interface PlaybackEngine {
  load(info: PlayInfo, opts: { startAt?: number; preferLowStart?: boolean }): Promise<void>
  play(): Promise<void>        // may reject with NotAllowedError (autoplay policy)
  pause(): void
  seek(timeSec: number): void
  setQuality(mode: 'auto' | { fixed: string }): void
  setTextTrack(id: string | null): void
  destroy(): void                // abort fetches, revoke object URL, close key sessions
  subscribe(fn: (s: PlayerState, changed: Set<keyof PlayerState>) => void): () => void
}

Component props

<VideoPlayer
  videoId="abc123"
  autoplay="muted"               // 'off' | 'muted' | 'on'
  startAt={42}
  onEnded={() => goToNext()}
  onQoe={(e) => analytics.push(e)}
/>

The player exposes a small, stable surface. Everything else stays internal so you can swap the engine later. For embeds, wrap the player in an <iframe> and expose the same commands through postMessage with an origin check.

6. Deep dive: adaptive bitrate on the client

ABR is the heart of the player. It answers one question before each segment: which rendition should I fetch next? A wrong answer either wastes quality or causes a stall. A stall hurts far more.

ABR loop: the video element reports buffer level, a bandwidth estimator and the buffer model feed the ABR rule, the scheduler fetches a segment from the CDN with fetch and AbortController, and appends it to a SourceBuffer.
Figure 2. The ABR control loop runs before every segment request.

Throughput-based rules

Measure how fast each segment downloaded. Divide bytes by seconds. Smooth the samples with an exponentially weighted moving average (EWMA). Weight each sample by its download time, so a long slow download counts more. Keep two averages, a fast one with a half life of 2 seconds and a slow one with 5 seconds. Use the lower of the two. This reacts fast to drops and slowly to gains. Then pick the highest rendition whose bitrate is below about 80 percent of the estimate. The 20 percent margin absorbs noise.

Ignore very small or cached responses. A segment served from the HTTP cache in 2 ms would make the estimate jump to gigabits.

Buffer-based rules

A buffer-based rule like BOLA or BBA maps the buffer level to a quality. With under 10 seconds buffered, use the lowest rendition. Between 10 and 30 seconds, scale up linearly. Above 30 seconds, use the highest. It ignores throughput noise and produces fewer switches. But at startup the buffer is empty, so it always starts at the lowest quality.

Hybrid rule

Most real players combine both. Use throughput at startup and after a seek, when the buffer is empty. Switch to buffer-based logic once the buffer is above a threshold, for example 10 seconds. Add three guards.

Abandoning a slow request

Sometimes the network drops in the middle of a 1080p segment. Waiting for it would drain the buffer. Each fetch carries an AbortController. A watchdog checks progress every 250 ms. The buffer may run dry before the projected finish time. In that case the engine aborts and refetches the same segment at a lower rung. This is called emergency switching.

Tip

Read the response with response.body.getReader() and count bytes as they arrive. You get a bandwidth sample every chunk, not only at the end of a segment. That makes the abandon decision fast and accurate.

Switching without glitches

Renditions share keyframe positions, so a new segment can start cleanly after the old one. When codecs change, for example from H.264 to AV1, call sourceBuffer.changeType() first. When the user picks a fixed quality from the menu, there are two options. The fast switch clears the buffer ahead and refetches, so the change is visible in 1 to 2 seconds. The smooth switch keeps the buffered segments and lets the new quality take over later. Use the smooth switch for automatic changes and the fast switch for manual ones, because the user expects to see the change.

7. Deep dive: buffer management and startup time

Buffer goals

The engine keeps a forward buffer target of about 30 seconds. It also keeps a back buffer of about 20 seconds. It stops fetching when the forward buffer is full and starts again when it drops below the target minus one segment. This hysteresis avoids fetching on every tick. The goal shrinks with bitrate: at 4K, hold 15 seconds instead of 30 to stay inside the quota.

Appending and evicting

A SourceBuffer accepts one operation at a time. While sourceBuffer.updating is true, a second appendBuffer() throws InvalidStateError. So the engine keeps a small operation queue per buffer and runs the next item on the updateend event. Eviction is just another queue item: remove(0, currentTime - backBufferSec). If an append throws QuotaExceededError, evict more back buffer, lower the forward goal, and retry the same segment once.

Mistake

Calling appendBuffer() from several async callbacks at once. It works on a fast desktop and fails at random on slow devices. Always serialize SourceBuffer operations through one queue.

Making startup fast

Startup time is the time from the play intent to the first rendered frame. Measure it with the playing event, or better, with video.requestVideoFrameCallback(), which fires when a frame is actually presented. Tactics that cut startup time:

Autoplay rules

Browsers block autoplay with sound unless the user has interacted with the site or has a high media engagement score. video.play() returns a promise that rejects with NotAllowedError. Handle it. A good pattern is to try unmuted, catch the rejection, then retry muted and show a clear "Tap to unmute" button. Never leave the user staring at a spinner because a promise was ignored.

8. Deep dive: seeking and thumbnail previews

Seeking

When the user seeks to 8:20, the engine must find the segment that contains 500 seconds. With 4 second segments that is segment 125. The steps are as follows.

  1. Abort all in-flight segment fetches with their AbortController.
  2. Check video.buffered. If 500 seconds is already inside a range, just set currentTime. This is an instant seek.
  3. Otherwise, keep the old buffer if quota allows. It helps if the user seeks back.
  4. Reset ABR into startup mode and fetch segment 125 at a safe rendition.
  5. Set video.currentTime = 500. The element fires seeking, then seeked once it can show a frame.

Keyframes matter here. A decoder can only start at a keyframe. If keyframes are 4 seconds apart, a seek to 501 seconds decodes from 500 and drops frames until 501. That costs CPU but is usually invisible. Some players snap the seek to the nearest keyframe for speed.

Scrubbing and the seek bar

While the user drags the thumb, do not seek on every pointer move. That would fire dozens of requests per second. Update only the preview and the time label during the drag. Seek once on pointerup. On a keyboard, arrow keys seek 5 seconds right away. Debounce the network part by 150 ms so a held key does not flood the CDN.

Thumbnail sprite sheets

The preview above the seek bar shows a small image of the frame at the hover position. Fetching one image per second would be thousands of requests. Instead, the server builds sprite sheets. Each sheet is a grid of small frames, for example 10 by 10 thumbnails of 160 by 90 pixels, one every 5 seconds. One sheet covers 100 times 5, which is 500 seconds. A 2 hour film needs 15 sheets of about 60 KB each in WebP.

To show the preview for time t, compute the index i = floor(t / interval). The sheet is floor(i / 100). The cell is i % 100, so the column is cell % 10 and the row is floor(cell / 10). Show the sheet as a CSS background with background-position: -col*160px -row*90px. Prefetch the sheet for the current position and its neighbours when the user first hovers the bar. Some services also describe thumbnails as a special image track in the DASH or HLS manifest.

Intuition

A sprite sheet is the same trick games use for animation frames. One request, many small pictures, and the browser only crops.

9. Deep dive: captions and DRM

WebVTT captions

WebVTT is the text format the web uses for captions. Each cue has a start time, an end time, optional position settings, and text. The simple path is to add <track kind="subtitles" src="en.vtt" srclang="en" label="English"> inside the video element. The browser parses it and renders cues. Change the active track through video.textTracks[i].mode = 'showing'.

Native rendering has limits. Styling options differ by browser, and native cues are hidden in some custom fullscreen layouts. So many players set the track mode to 'hidden', listen for cuechange, and render the active cues in their own overlay. This keeps full control over font size, background, and position. It also lets the overlay move up when the controls are visible, so captions never hide under the seek bar.

DRM with EME, at a high level

The flow has six steps. You do not need byte-level detail in an interview, but you should know the order.

  1. Call navigator.requestMediaKeySystemAccess(keySystem, configs) to check support and get a MediaKeySystemAccess. Try Widevine, then PlayReady, then FairPlay.
  2. Create MediaKeys and attach them with video.setMediaKeys().
  3. When encrypted data appears, the element fires encrypted with init data. Some players skip this event and read the key ID from the manifest instead.
  4. Create a MediaKeySession and call generateRequest(). The session fires message with a license challenge.
  5. Send the challenge and a short-lived auth token to your license server. It checks entitlement and returns a license.
  6. Call session.update(license). The CDM can now decrypt, and playback continues.

Three design points matter for the frontend. Request the license in parallel with the first segment. Cache the MediaKeySystemAccess across videos. Close sessions in destroy(). Higher security levels, like Widevine L1, decrypt in hardware and may forbid screenshots and some picture-in-picture modes. Some licenses cap resolution at 720p on software-only devices. The ABR rule must respect that cap, or it will fetch 1080p segments that cannot play.

Interview

Do not overclaim DRM. Say that it raises the cost of copying and enforces license rules like resolution caps and rental windows. It does not make leaks impossible. Mention watermarking as a separate server-side tool.

10. Deep dive: custom controls, accessibility, and picture-in-picture

Why custom controls

The native controls attribute is accessible and free, but it looks different in every browser. It also cannot show a quality menu, thumbnails, or chapters. So most products build their own. Once you hide native controls, you own all of their accessibility. That is the most common place where players fail audits.

Semantic structure

Keyboard model

Follow the shortcuts users already know from large players. Space or K toggles play. J and L seek back and forward 10 seconds. Left and right arrows seek 5 seconds. Up and down arrows change volume. M mutes, C toggles captions, and F toggles fullscreen. Digits 0 to 9 jump to 0 to 90 percent.

Scope the shortcuts. Attach the listener to the player container, not document, so typing in the comment box does not pause the video. Ignore keys when an input or contenteditable is focused. Do not capture Tab. Keep a visible focus ring with :focus-visible. When controls auto-hide after 3 seconds of no movement, never hide them while focus is inside them.

Announcing state changes

Keep one visually hidden aria-live="polite" region. Write short messages to it: "Muted", "Captions on, English", "Quality 720p". Do not announce time updates. They would flood the user.

Picture-in-picture and fullscreen

Tip

Respect prefers-reduced-motion. Turn off animated progress glows and auto-hiding slide animations for users who ask for less motion.

11. Deep dive: QoE analytics and preloading the next video

What to measure

How to send it

Buffer events in memory. Flush a batch every 30 seconds as a heartbeat, and on visibilitychange to hidden. Use navigator.sendBeacon('/api/qoe', blob) or fetch(url, { keepalive: true }) so the request survives page unload. Do not rely on unload. It is unreliable on mobile and breaks the back-forward cache. Give each event an ID so retries do not double count. Sample heavy debug events, like per-segment timing, at 1 to 5 percent of sessions. Always send core QoE events.

Mistake

Starting the startup timer when the player mounts. If the video is below the fold and the user scrolls for 20 seconds, you will report a 20 second startup. Start the timer at the play intent.

Preloading the next video

Autoplay of the next item feels instant only if bytes are ready. The plan is to preload a little, late, and cheaply.

For feeds of short videos, like a vertical swipe feed, keep a pool of 2 or 3 video elements. Use an IntersectionObserver to pick the visible item. Preload only the next item and release the one that went off screen by calling destroy() and clearing its src. Never keep 20 MediaSources alive.

12. Failure modes

13. Trade-offs

Native HLS versus MSE engine

Large buffer versus small buffer

Short segments versus long segments

Aggressive preloading versus data cost

14. 45-minute interview walkthrough

  1. Minutes 0 to 5. Ask about on demand versus live, devices, DRM, and embeds. Write the functional and non-functional lists.
  2. Minutes 5 to 10. Do the numbers: ladder, segment size, startup budget, buffer memory.
  3. Minutes 10 to 17. Draw the layers: page, player component, engine, MediaSource, video element, store, controls, analytics. Explain native versus MSE.
  4. Minutes 17 to 22. Show the data model and the engine interface. Mention the load generation guard.
  5. Minutes 22 to 30. Deep dive on ABR and buffer management. Cover hybrid rules, abandonment, and the SourceBuffer queue.
  6. Minutes 30 to 36. Seeking, sprite thumbnails, and startup tactics.
  7. Minutes 36 to 41. Accessibility of custom controls, captions, and DRM at a high level.
  8. Minutes 41 to 45. QoE metrics, preloading, failure modes, and what you would build next, such as live low latency.

15. Summary

16. Common mistakes

Worked example: a viewer on a train

A viewer opens a 12 minute video on a phone. The network starts at 8 Mbps, drops to 1 Mbps in a tunnel, then recovers.

  1. The page head already has preconnect to the CDN. The user taps play. The player fetches play info in 70 ms. It includes the variant playlist inline and a last known estimate of 5 Mbps from localStorage.
  2. The ABR is in startup mode. 80 percent of 5 Mbps is 4 Mbps, but the player is 360 pixels wide at 3x density, so the cap is about 1080p. It still starts one step lower, at 480p, for safety.
  3. The init segment and a 2 second first segment of 300 KB arrive in 320 ms. The engine appends them through its queue. At 2 seconds buffered it calls play(). The first frame shows at 610 ms. The QoE module logs start time 610 ms.
  4. Segments 2 to 5 download fast. The EWMA estimate settles at 7.5 Mbps. The buffer grows to 14 seconds. ABR steps up one rung at a time to 720p, then 1080p.
  5. At minute 3 the train enters a tunnel. A 1080p segment of 3 MB is half downloaded after 2 seconds. The watchdog projects 10 more seconds. The buffer holds 9 seconds. It aborts the fetch.
  6. The fast EWMA drops to 900 kbps. The panic rule picks 240p at 400 kbps. The same segment is refetched at 240p in 0.9 seconds. The buffer never empties. The viewer sees a softer picture but no spinner.
  7. Out of the tunnel, throughput jumps to 8 Mbps. The slow EWMA lags, so the player steps up over 3 segments instead of one. No oscillation.
  8. At minute 10, the next video's play info, manifest, and first 2 segments are prefetched, about 450 KB.
  9. The video ends. The player calls destroy(), bumps the load generation, and loads the next video from the warm cache. Start time is 180 ms.

Code lab: a buffer-aware ABR rule

This snippet builds a dual EWMA bandwidth estimator. It adds a hybrid ABR rule with a viewport cap and a panic rule. It has no browser dependencies, so you can run it with Node. Save it as abr.mjs and run node abr.mjs. It prints the chosen rendition for a simulated network trace with a tunnel. It also shows one abandoned request.

const LADDER = [{ h: 240, kbps: 400 }, { h: 480, kbps: 1200 }, { h: 720, kbps: 3000 }, { h: 1080, kbps: 6000 }]

// EWMA weighted by download time (at least 1 s), so slow downloads count more
class Ewma {
  constructor(halfLifeSec) { this.half = halfLifeSec; this.v = 0; this.w = 0 }
  add(sec, x) {
    const alpha = 1 - Math.pow(0.5, Math.max(sec, 1) / this.half)
    this.v = alpha * x + (1 - alpha) * this.v
    this.w = alpha + (1 - alpha) * this.w
  }
  get() { return this.v / this.w }
}
class Abr {
  constructor({ maxHeight, lastKnownKbps }) {
    this.fast = new Ewma(2)
    this.slow = new Ewma(5)
    this.allowed = LADDER.map((r, i) => i).filter((i) => LADDER[i].h <= maxHeight)
    this.seed = lastKnownKbps
    this.current = 0
  }
  sample(bytes, ms) {
    if (bytes < 16000 || ms < 20) return       // ignore tiny or cached responses
    this.fast.add(ms / 1000, (bytes * 8) / ms)
    this.slow.add(ms / 1000, (bytes * 8) / ms)
  }
  estimate() { return this.fast.w === 0 ? this.seed : Math.min(this.fast.get(), this.slow.get()) }
  highestUnder(kbps) { return this.allowed.filter((i) => LADDER[i].kbps <= kbps).pop() ?? 0 }
  choose(bufferSec) {
    const est = this.estimate() * 0.8
    let best = this.highestUnder(est)
    if (bufferSec < 4) best = this.highestUnder(est / 2)               // panic: download in half the play time
    else if (best > this.current) best = bufferSec >= 10 ? this.current + 1 : this.current
    this.current = best
    return LADDER[best]
  }
  emergency(measuredKbps) { this.current = this.highestUnder(measuredKbps / 2); return LADDER[this.current] }
}

// Simulate 4 s segments. Each entry is the network speed in kbps.
const trace = [8000, 8000, 8000, 8000, 8000, 900, 900, 900, 900, 8000, 8000, 8000]
const abr = new Abr({ maxHeight: 1080, lastKnownKbps: 5000 })
let buffer = 0, stall = 0
for (const net of trace) {
  let r = abr.choose(buffer)
  let ms = (r.kbps * 4 * 1000) / net
  if (ms / 1000 > buffer && buffer > 0 && r !== LADDER[0]) {
    abr.sample((net * 1000) / 8, 1000)             // watchdog aborts after 1 s of slow bytes
    buffer -= 1
    r = abr.emergency(net)
    ms = (r.kbps * 4 * 1000) / net
    console.log('  abandoned segment, refetching lower')
  }
  abr.sample((r.kbps * 1000 * 4) / 8, ms)
  stall += Math.max(0, ms / 1000 - buffer)
  buffer = Math.max(0, buffer - ms / 1000) + 4
  console.log(`net=${net} pick=${r.h}p buffer=${buffer.toFixed(1)}s est=${abr.estimate().toFixed(0)}`)
}
console.log(`stall time including startup: ${stall.toFixed(1)} s`)

Try changing maxHeight to 480 to see the viewport cap. Try removing the slow EWMA to see how the player oscillates after the tunnel.

Check yourself

  1. Why does a robust player use both throughput and buffer level for ABR?
    Answer

    Throughput is the only signal at startup and after a seek, when the buffer is empty. But throughput is noisy, and following it alone causes many switches. Buffer level is stable and directly tied to stall risk. A hybrid uses throughput early and buffer level once there is a cushion.

  2. What happens if you call appendBuffer() while sourceBuffer.updating is true, and how do you prevent it?
    Answer

    It throws InvalidStateError. Prevent it with a per-buffer operation queue that starts the next append or remove only after the updateend event.

  3. A 2 hour film has a thumbnail every 5 seconds in 10 by 10 sprite sheets. How many sheets do you need, and which sheet and cell show time 1,234 seconds?
    Answer

    7,200 divided by 5 is 1,440 thumbnails, so 15 sheets of 100. Index is floor(1234 / 5), which is 246. Sheet is 2 (counting from 0), cell is 46, so row 4 and column 6.

  4. Why should keyboard shortcuts be attached to the player container instead of document?
    Answer

    Global shortcuts fire while the user types in a search box or comment field, which pauses or seeks the video by accident. Scoping them to the player, and ignoring events from inputs, avoids that.

  5. Name two ways to cut startup time without changing the server encoding ladder.
    Answer

    Any two of these. Preconnect to the CDN. Inline the variant manifest in the play info. Start at a low rendition. Fetch the first segment with high priority. Request the DRM license in parallel. Call play() after about 2 seconds of buffer.

  6. Why is unload a bad place to send QoE data, and what should you use instead?
    Answer

    unload often does not fire on mobile and it blocks the back-forward cache. Flush on visibilitychange to hidden and on pagehide, using navigator.sendBeacon or fetch with keepalive: true.

  7. A DRM license caps resolution at 720p on this device. What must the frontend do?
    Answer

    Feed the cap into the ABR as a hard maximum, hide higher qualities in the menu, and never fetch segments above 720p. Otherwise those segments would fail to decrypt or show a black screen.

Key terms

Rendition
One encoded version of a video at a given resolution and bitrate.
Bitrate ladder
The full list of renditions offered for one video, sorted by bitrate.
Segment
A short chunk of media, often 2 to 6 seconds, fetched as one HTTP request.
CMAF
Common Media Application Format, a fragmented MP4 layout shared by HLS and DASH.
SourceBuffer
The MSE object that receives media bytes for one track type.
Forward buffer
Seconds of media buffered ahead of the playhead.
Rebuffer ratio
Stall time divided by watch time, a core QoE metric.
EWMA
Exponentially weighted moving average, used to smooth bandwidth samples.
CDM
Content Decryption Module, the browser component that decrypts protected media.
WebVTT
The Web Video Text Tracks format for captions and subtitles.
Sprite sheet
One image containing a grid of small thumbnails.

Further reading

Primary references