Sign up to our Newsletter
Let’s steer back to where we came from, our Streaming Mini Series, and our fifth, brand new Article on Microsoft’s Smooth Streaming (a.k.a. “Silverlight”): Microsoft’s Standards and answer to Adobe and Apple in the HTTP Streaming arena, though it did not meet the widest Audience inception (as opposed to HLS): it did, like HDS, get incepted as a “second best” set of Standards, although not essential as HLS (due to iDevices natively only “understanding” it). We will have a look into its essentials from a technologic – and business level – standpoint, and then draw some conclusions as related to HDS and HLS.
Microsoft SmoothStreaming
Microsoft has been offering Streaming Solutions since 1998 and is probably one of the oldest Players in this field. However, the initial offering was primitive in comparison to the modern Streaming Technologies. Although Microsoft brought out many modifications, it still suffered from limitations. Switching between Streams due to conditions at the receiving – or display – end could not be possible because the streams were not aligned. As a result, user experience became a problem back in the day.
How One AI-Driven Media Platform Cut EBS Costs for AWS ASGs by 48%
Meanwhile, around 2010, other Players in the Market created or designed their own Standards for Streaming. One of the outcomes of this churning was that Streaming of Video or Audio could be considered as delivering any other Content and could be transmitted using HTTP Protocol. As such, HTTP Protocol involves dividing data into small Segments and assembling or restoring data at the Client Player’s end. This adaptation of HTTP Protocol for Video/Audio Streaming revolutionized this panorama.
The advantages of using HTTP as a delivery Protocol are several. A Video Streaming Protocol has to use a different Port Number, for example. Firewalls and routers are designed to use port 80 for HTTP downloads and therefore Video Streaming through a different Port may look like an attack on Security and be blocked. Such one issue is out of the equation when HTTP is used for Video Streaming. Microsoft followed the trends and released its own SmoothStreaming – or Silverlight – Streaming specifications.
SmoothStreaming Technology
Like we discussed earlier, HTTP Streaming permits flexibility in transmission and leads to better efficiency. Progressive Downloads are quite popular for Streaming and are used by many big Players like YouTube. The limitation of Progressive Downloads, though, is the loss of efficiency. Imagine that you download a Video of ninety minutes and you decide after ten minutes that you no longer want to watch the Video. This is a colossal waste of resources.
Alternatively, you can decide to offer Progressive Downloads of only a small part of the Video. This is technically called segmentation. With Microsoft’s SmoothStreaming, a Video Stream is cut into small Segments of 2 seconds by the Encoder and sent sequentially to the Decoder (or Client Player, let alone you are using a CDN for delivering your traffic). At the Receiver’s end, these small Segments are again merged together to provide a smooth Video output. What happens when the user environment deteriorates and cannot playback Video at a specific Bitrate? The problem is neatly resolved when a Segmenter creates and transmits Segments at various bitrates. At the beginning of the playback, the minimum Bitrate Segments are delivered. Later on, depending on the Client’s Bandwidth and Processor spare cycles, the Bitrate is enhanced to an optimal level.
The Streaming Process
SmoothStreaming is based on MP4 as a Source File Format. Microsoft adopted this Format because it is a widely used standard and makes third party support easier. MP4 is architected with H.264 Video Codec which is probably the most popular and widely used Video compression Standard.
Let us now look deeper into the architecture of SmoothStreaming. To begin with, we must understand that the design or architecture of this Streaming Technology is elegant and makes use of the native MP4 capabilities.
SmoothStreaming consists of two basic Formats – Disk File Format and Wire Format. A Video is recorded in one contiguous or continuous Stream and stored on the Disk as a single file. There are different files for each encoded Bitrate. The Disk File Format defines the structure of these files which reside on the Disk.
The encoded files are divided into small segments and then transmitted through the Networks. Each Wire Format defines the structure of the corresponding fragments. Therefore, for a single Disk File Format we will have several Wire Formats.
The Disk file consists of higher level information (metadata) about the video, while the Wire Format contains the fragment metadata data as well as the media.
Therefore, we are now looking at the following files –
1) MP4 files containing Video/Audio
.ismv – Contains Video and Audio, or only Video. There is one .ismv file per encoded Video Bitrate.
.isma – Contains only Audio.
2) Server Manifest File
.ism – This Manifest File is used by the Server to describe the relationship between the media tracks, Bitrates and files on Disk.
3) Client Manifest File
.ismc – This Manifest isused by the Clientto describe the available Streams – such as Codecs used, Bitrates encoded, Video resolutions, markers and captions.
Both the Server and Client Manifest Files are in XML format.
Using these files, SmoothStreaming creates a smooth Streaming experience with the following sequence of events :
The Client receives its .ismc or Client Manifest File first. The Manifest tells it which codecs were used to compress the content, which Bitrates and resolutions are available, and a list of the available chunks.
The Server now looks at this information and amps it to a physical *.ismv or *.isma file on Disk. It then reads the appropriate MP4 File. It extracts the fragment box and sends it over to the Client as a standalone file.
The beauty and elegance of SmoothStreaming becomes evident here. The Bitrate switching is handed over completely to the Client and the Server has no role to play in it. The Client-side code monitors various Network parameters and calls for higher or lower Bitrates from the Server.
Advantages of SmoothStreaming
From the viewpoint of deployment, SmoothStreaming has several advantages. Use of HTTP Protocol means you can use commodity Web Servers which is much cheaper than using specialized Servers (but hey! Plan for capacity and scalability by exploring CDNs as a delivery option). It provides adaptability by adjusting Bitrates to available conditions at the User’s end. Since the Player receives segments from the nearest node, it receives Video at ideal rate. The Provider need not keep track of every User, since the Network adapts itself automatically to the User environment and availability of Bandwidth.
From an End User’s standpoint, SmoothStreaming leads to excellent playback without any stutter. There is no need for buffering. Faster start up and seek times can be experienced by starting at lower Bitrates and moving to higher rates depending on Network conditions.
Conclusions
SmoothStreaming is a proven technology, which wasn’t otherwise as adopted as HLS Streaming. It is versatile, simple to use and has an elegant process, however as you may know money is what makes the world go round and Microsoft, after several patches and updates to SmoothStreaming, decided to discontinue its support in time. If you look at the future, SmoothStreaming is going to be replaced by other Streaming Formats, MPEG DASH first and foremostly and we’ll cover it as we move forward.
