How We Test Streaming Services

Streaming services are household utilities for millions around the world. For many, using one is now as common as turning on a water faucet. Whether you’re solely a cord-cutter or supplement your cable or over-the-air entertainment with streaming platforms, these services are how many of us watch TV or listen to music. CNET provides hands-on reviews and evaluations of streaming services to help you choose which ones best suit your needs. 

Not every streaming service operates with the same purpose. For example, a live TV service may be a suitable replacement for cable and a way to watch live sports, while a lower-priced on-demand platform may be better for watching content ad-free if you want to use parental controls. Music streaming platforms allow you to customize your listening experience and take music with you wherever you go. That’s why we test these services on myriad devices. Here’s a look into our process.

How we test streaming services

We use our own media devices at home or in CNET’s TV lab in New York to try out every streaming service, and we spend hours on these apps for each subscription tier that is offered. That means we measure how many ads play — and for how long — during a TV show or movie for ad-supported plans and check to see if ad-free truly means “ad-free.” 

We look at how kids’ profiles are set up and how parents can tweak ratings or curate any included content. Because features and playback are just as important as a service’s catalog, we examine functionality to make sure an app is easy to navigate, works across multiple device types and offers what users need for things like search, playback, accessibility, DVR and watchlists.

To test video streaming services, we use the app on mobile phones, tablets, web browsers and smart TVs to see how they perform. If we come across a glitch that’s unique to a phone versus the TV viewing experience, we note that in our reviews. Similarly, if a platform has a well-designed user interface that’s visually appealing and easy to use, we highlight that. Often we compare the home screen design among streaming services to see how each one stacks up against their rivals. Are there Top 10 rows? How are watchlists and menus organized? How well do the recommendation engines work? Are PINs available to lock accounts or profiles? All of these and more are considered.

We try out features like downloads, playback, DVR and search to ensure these functions are smooth and useful, but we make comparisons based on the device and how each streamer stacks up against its rivals. 

The process for music streaming is similar, and our tests include device compatibility, audio quality, user experience and library analysis. Most of these services offer music, podcasts and audiobooks, and we comb through all the catalog offerings to check the variety and the recommendation algorithm. 

The content and price of any streaming service we test are equally important to the overall user experience.

How we rate streaming services

Our streaming service reviews include ratings on a 0-10 scale, with 10 being the highest. We assign ratings based on several main factors, including library, cost/value and features. Here’s a look at how we measure:

Content 

What do you want to watch? Live TV streaming services that operate similarly to cable and satellite providers should have comparable channel lists with the option to use DVR. Sports fans will want access to local and national games, and we consider whether live TV platforms have a good mix of lifestyle, news, entertainment and sports programming. Our channel comparison guide breaks it all down. 

On-demand services like Netflix, Hulu, Peacock and Max have original releases, but some services also carry licensed titles, live content or large back catalogs of network shows and theatrical films. We examine how much content is offered, how often fresh titles are added to a platform and what types of programming are available for all demographics. 

• Is there enough family-friendly content for all ages? 
• How extensive is the catalog — full seasons/series/albums, etc.?
• Are there diverse stories available? 
• How’s the selection of genre titles, international releases, live content or new films? 
• Can you watch network TV shows at the same time or next day?
• Is there a lack of new content dropping on a consistent basis?
• What is the quality of the content?

The same principles are applied to music streaming services, and we analyze library size for songs, albums, books and podcasts and recommendations that line up with consumers’ — or your favorite artist’s — taste. With some services slowly adding video content to their platforms, we will continue to update how we evaluate music streamers. 

Value 

Only you know how much you’re willing to pay for your streaming subscriptions, but you still want to know what you get for your money. We provide a look at each subscription type and what comes with it for a monthly or annual price. Live TV services may offer one straightforward price or several packages, while other services like Netflix and Spotify offer different prices depending on how many people are using an account. There may also be optional add-ons that will affect how much you pay. We recommend using a monthly plan to make it easier to cancel or pause your subscription, but if you’re able to snag a discount on an annual plan, it’s often worth the savings. 

Here are the guidelines we use when considering value:

• Overall affordability 
• How many simultaneous streams come with each price tier
• Pricing and tier offerings compared to competitors
• Bundle package availability
• Do customers get additional perks with their subscriptions (e.g. Crunchyroll offers merch, early event access and store discounts)?
• Do features like downloads or 4K cost extra?
• Where is the service available: select regions or globally?
• Are portions of a library inaccessible for lower-priced plans?
• Is the catalog worth the monthly or yearly price?

App design and features

We spend much of our time hopping from device to device to test and review each streaming service to find its pros and cons. As we evaluate performance, we take notes, shoot pictures and videos, and go through the process of creating an account and setting up profiles. The ability to tweak parental controls and audio and video settings or remove the listening and viewing history is as important as being able to find what you’re looking for and adjusting any default configurations. 

Our criteria are listed here:

• Can you use parental controls or customize content ratings by age?
• Display and navigation design across devices
• What kinds of accessibility features are available?
• How user-friendly is the app?
• Does the search function work properly or is it confusing?
• Are there any standout or cool features?
• Is it easy to cancel or sign up for the service?

We test for reliability, too

Like you, we don’t want our streaming apps to crash or experience glitches while we use them. An individual reviewer will stream multiple titles and types of content on a service to ensure playback is smooth and that the app doesn’t drop out while viewing. That goes for live feeds or on-demand content as well. If we notice a pattern of freezing, crashing or issues with log-in, we note that in our reviews.