The Bitmovin Playback Score

Overview

The Playback Score is a comprehensive metric that rates the overall quality of video playback for your viewers. It combines quality of experience indicators into a single score, enabling you to monitor and optimize video playback.

Core Metrics

The Playback Score is calculated from three key metrics:

Startup Score
Measures how quickly videos begin playing for viewers.

Smoothness Score
Evaluates the continuity of playback, tracking buffering events.

Success Factor
Represents successful playback without critical errors.

Calculation

Playback Score = AVG(Startup Score, Smoothness Score) × Success Factor

The formula takes the average of the Startup and Smoothness scores, then multiplies by the Success Factor to ensure that playback failures are appropriately considered in the overall score.

Interpretation

A higher Playback Score indicates a better quality of experience. The score reflects both the quality of successful playbacks and the reliability of your delivery infrastructure.

Startup Score

The Startup Score rates how long the user has to wait until playback begins.

The video startup time is normalised to a range of 300ms to 20,000ms:

• Below 300ms: Perfect score of 100
• 300ms to 20,000ms: Score decreases quadratically as startup time increases
• Above 20,000ms: Score of 0
  This limit exists because once startup time exceeds 20 seconds, the viewing experience is already considered poor, and additional delay no longer meaningfully changes user perception.

Formula

Startup Score = 100 * (1.0 - (VideoStartupTime - 300) / VideoStartupTimeLimit)²

with VideoStartupTimeLimit = (20000 - 300)

The quadratic formula penalizes longer startup times more heavily, reflecting the significant impact of delays on user experience. A video that starts in under 300ms provides optimal viewer satisfaction, while startups exceeding 20 seconds indicate critical performance issues.

The Startup Score is first calculated per session and then averaged over all sessions.

Examples:

Smoothness Score

The smoothness score rates the viewer experience in terms of buffering during playback. There are two dimensions that are taken into consideration:

1. How long playback was interrupted due to buffering
2. How often buffering occurred

Playing Percentage = played time / (played time + buffer time)

The Playing Percentage measures how much of the total playback time was actually spent playing video, as opposed to buffering. It's calculated as the ratio of played time to the sum of played time and buffer time. A higher value means the viewer experienced fewer interruptions, indicating smoother playback.

Rebuffer Events Factor  = e^(-0.2 * buffer events))

The Rebuffer Events Factor reflects how often playback was interrupted. It decreases exponentially with each buffering event. This means that every additional buffering event has a progressively smaller negative impact on the score.

Smoothness Score = (Playing Percentage^3 * Rebuffer Events Factor) * 100

We combine the effects of both buffering duration and frequency into a single measure of playback quality. It emphasizes uninterrupted playback by giving more weight to sessions where most of the time was spent playing smoothly, while still accounting for how often buffering occurred. As a result, even short interruptions can noticeably lower the score, reflecting the importance of consistent, continuous playback for a good viewer experience.

Similar to the Startup Score, the Smoothness Score is first calculated per session and then averaged over all sessions.

Examples:

Success Factor

The Success Factor rates errors during playback with the following conditions:

As opposed to the other Scores, the Success factor is only calculated over all sessions the following formula:

Success Factor = (AVG(points per session)/100)²

This is to ensure that errors are penalised more heavily.

Example

100 streams are played.
5 of these fail during startup. These are given 0 points.
The other 95 successfully play without critical Errors receiving 100 points

AVG(points per session) = 95

Success Factor = (95/100)² = 0.9025