Live packaging refers to the process of converting live encoded audio and video streams into specific formats suitable for delivery over different networks and devices. It involves fragmenting the media, applying protocols, and adjusting the stream to ensure compatibility with various platforms and viewers. Live packaging is critical for adaptive streaming and ensuring a smooth user experience on a wide range of devices and network conditions.
Let’s break down the live packaging process into key components, technologies, challenges, and industry-leading products.
Live packaging is the step where an encoded media stream (video/audio) is broken into smaller chunks and formatted into streaming protocols like HLS (HTTP Live Streaming), DASH (Dynamic Adaptive Streaming over HTTP), or CMAF (Common Media Application Format). The packaged media is then sent to content delivery networks (CDNs) or end-user devices for playback.
Live packaging serves a few important purposes:
The live packaging process involves several key steps to transform an encoded stream into a format that can be delivered to viewers in real-time.
Segmenting the Stream: The first step in live packaging is splitting the stream into small, time-bound fragments (typically 2-10 seconds). These fragments are individual video and audio files (e.g., .ts files for HLS or .m4s files for DASH).
Fragment Duration: The length of each fragment can be adjusted based on factors like latency and network conditions. Shorter fragments reduce latency but require more frequent requests to the server.
Multiple Quality Levels: To enable ABR, the stream is packaged into multiple different quality levels, each with its own bitrate and resolution. For example, a stream might be packaged at 1080p (5 Mbps), 720p (3 Mbps), and 480p (1 Mbps). These different bitrates are created in the encoding stage but must be handled correctly during packaging.
Manifest Files (Playlists): A master manifest file is generated to list the available versions of the stream. The manifest file contains references to each of the fragmented segments and bitrates. For HLS, the manifest file is typically an .m3u8 playlist, while for DASH, it’s an .mpd file.
Depending on the target platforms, live packaging will format the stream for specific streaming protocols:
HLS (HTTP Live Streaming): Developed by Apple, HLS is widely used across platforms, especially iOS devices. The fragmented video and audio files are packaged into .ts files, with a master .m3u8 playlist file.
DASH (Dynamic Adaptive Streaming over HTTP): An open-source alternative to HLS. It fragments video and audio into .m4s files, with a corresponding .mpd manifest file.
CMAF (Common Media Application Format): A newer format that seeks to unify HLS and DASH under a single standard. CMAF uses the .m4s format for video and audio fragments, reducing duplication of encoding and packaging efforts when targeting both HLS and DASH.
For premium content, live packaging may include encryption using DRM (Digital Rights Management) technologies to protect against unauthorized access and piracy. Encryption is applied during packaging, and manifest files point to the decryption keys needed for playback.
Distribution: Once the stream is packaged into the desired formats, it’s distributed to a CDN or directly to the end-user. The CDN caches the packaged fragments at edge locations to minimize latency and serve content more efficiently to viewers.
Player: On the client side, media players request the manifest file and select the appropriate fragments to play based on the viewer’s bandwidth and device capabilities. For example, a player might select a 1080p fragment for a viewer with a strong connection but switch to 720p if bandwidth drops.
Live packaging involves a combination of protocols, formats, and tools to ensure a seamless streaming experience.
HLS (HTTP Live Streaming): Apple’s protocol for adaptive bitrate streaming. HLS is one of the most widely used formats, supported by almost all devices and platforms. HLS delivers .ts files over HTTP and uses an .m3u8 playlist for manifests.
DASH (Dynamic Adaptive Streaming over HTTP): An open-source alternative to HLS. DASH is widely used in web applications and on Android devices. It uses the .mpd manifest and delivers fragments in the .m4s format.
CMAF (Common Media Application Format): CMAF is a relatively new standard that unifies HLS and DASH, reducing the need for separate packaging workflows. It uses fragmented MP4 containers and is designed to minimize latency while maintaining compatibility across different devices and platforms.
Smooth Streaming: Developed by Microsoft, Smooth Streaming was one of the early adaptive bitrate streaming protocols but has largely been replaced by DASH and HLS.
TS (Transport Stream): Used in HLS, TS files are a common format for transporting audio, video, and metadata.
FMP4 (Fragmented MP4): Used in DASH and CMAF, fragmented MP4 files offer more efficient compression and compatibility across devices.
For content protection, live packaging can include DRM solutions:
Widevine: Google’s DRM system, commonly used for DASH streaming on Android devices and Chrome browsers.
FairPlay: Apple’s DRM technology, used for HLS streaming on iOS devices.
PlayReady: Microsoft’s DRM, often used in Smooth Streaming and supported by various devices.
Live packaging integrates with CDNs to ensure efficient delivery of the fragmented content. Popular CDNs include:
There are both cloud-based and on-premises solutions for live packaging. The choice of solution depends on the scale of the event, the audience size, and the specific requirements for compatibility and latency.
AWS MediaPackage: A highly scalable cloud-based live packaging solution that supports HLS, DASH, and CMAF formats. AWS MediaPackage works with AWS Elemental MediaLive for encoding and integrates with AWS CloudFront for CDN delivery.
Microsoft Azure Media Services: Azure offers a cloud-based live encoding and packaging solution that supports HLS, DASH, and Smooth Streaming. It also provides DRM protection and integrates with the Azure CDN.
Wowza Streaming Cloud: A cloud-based platform that provides live encoding, packaging, and distribution, supporting multiple streaming formats like HLS, DASH, and RTMP. Wowza also offers low-latency streaming and multi-bitrate packaging for ABR.
Wowza Streaming Engine: A self-hosted live streaming server that includes both encoding and packaging capabilities. It can output streams in various formats, including HLS and DASH.
Zixi: A professional-grade streaming platform with on-premises solutions for live encoding and packaging. Zixi offers low-latency transport with adaptive bitrate capabilities and packaging for HLS, DASH, and CMAF.
Bitmovin: Bitmovin offers a live encoding and packaging solution that supports HLS, DASH, and CMAF with integrated DRM and ABR.
Elemental Live: AWS Elemental Live is a hardware and software solution for live video encoding and packaging, supporting HLS, DASH, and CMAF with integrated DRM options.
Live packaging introduces some delay as the stream is fragmented and reassembled into packets for delivery. Reducing this latency while maintaining a high-quality stream is a significant challenge, especially for live events like sports or gaming where real-time interaction is critical. Shorter fragment sizes (2-4 seconds) can reduce latency but increase the load on the network.
Different platforms (e.g., iOS, Android, web) require different packaging formats (HLS vs. DASH), which adds complexity to the live packaging process. While CMAF attempts to unify these formats, legacy devices still require separate packaging for HLS and DASH.
As the number of viewers increases, the live packaging solution must scale to handle thousands or even millions of concurrent viewers. Cloud-based services like AWS MediaPackage can scale dynamically, but on-premises solutions may struggle with large-scale events.
ABR packaging relies on the ability to switch between different bitrate streams based on network conditions. However, fluctuations in bandwidth can cause buffering or quality drops, especially for viewers with unstable internet connections.
Securing live streams with DRM adds complexity to the live packaging workflow. Integrating multiple DRM systems (e.g., Widevine, FairPlay, PlayReady) requires careful handling during packaging to ensure the stream can be securely delivered to all platforms.
Here’s a list of some industry-leading products and services used in live packaging:
Dive deeper into the concepts