hidef

Every day we become more connected to the rich media around us, thanks in part to the proliferation of fast and reliable internet connections to more and more people. In a given day you’re streaming videos from YouTube, catching the most recent episode of your favorite sitcom streamed from Netflix, and listening to Internet radio to your desk all day long. These services are built on a reliable media distribution network to ensure that valuable content is delivered successfully to us.

If you’re like most organizations, over time you’ve collected a rat’s nest of files in which nobody has a great idea which version of the file is which. Perhaps even knowing where certain files are available is difficult to track down. A well-designed distribution network can make a world of difference in how efficient your organization is and how quickly it can react to changing conditions and new technologies.

HiDef has built such large-scale media distribution networks, so we’d like to share some insight into how your organization can capitalize on the great tools and knowledge available. Regardless of whether your content is audio, video, or even text, we can show you how to build the network you need to reach your audience wherever they are.

Performance

First and foremost, access to content needs to be fast. There are a variety of options available when evaluating the distribution of your media. Let’s take a look at a few considerations.

Content Delivery Network (CDN)

One of the options is going with a traditional Content Delivery Network (CDN) that has built the infrastructure to geographically distribute your content where your audience is. This traditionally is done by establishing servers in different parts of the world and distributes your content across these servers. Thus, when a user makes a request in Europe, a server in close geographic proximity to the user delivers your content. This serves to reduce the number of networks that must be traversed and subsequently reduces the time it takes for the user to start experiencing your content. Just hint, but this makes your users very happy. The biggest player in this market is Akamai, but many others exist in this space as well (including CacheFly, EdgeCast, CDNetworks to name just a few).

Instantly Provisioned CDN’s

One of the most exciting developments recently is what I call “instantly provisioned CDNs.” Amazon’s CloudFront and Microsoft’s Azure CDN are the notable players in this market. The primary advantage of utilizing these services is the reduction of overall cost and the convenience of controlling the distribution and access to your content in easy to use ways. These instantly provisioned CDNs operate in a very similar way to traditional CDNs, but tie natively into your potentially existing content and are ready to integrate into your existing products and tools effortlessly.

Customizable vs. Off-the-shelf

Another important consideration in building a high-performance distribution network is custom tailoring it for your needs. Generally speaking utilizing off-the-shelf services like I’ve described above will achieve some obvious benefits, but what we’ve found is that sometimes building a custom solution makes more sense for your audience and your content. We have experience building large-scale custom distribution networks built on Squid, which is an open-source web caching product.

The principle behind web caching is keeping frequently requested content readily and quickly available, so that you maintain a high level of service for your most popular content. If you pair this technique with geographic distribution of content and a host of other techniques, it’s possible to build a highly-tuned custom distribution network that is extremely cost efficient, easy to maintain, and perfectly appropriate to meet your organization’s goals.

Keeping your Content Accessible

Your content isn’t valuable if people can’t get to it! One of the pressing matters in building a media distribution network is building one that performs reliably day in and day out. The key to building a reliable distribution network boils down to one word: redundancy. When you are choosing a CDN, it’s important to consider how failure is handled within the network. It’s not simply enough to distribute your content to hundreds of servers around the world.

Consider the scenario where the server that delivers content to your audience on the east coast of the United States is destroyed by a flood. If your CDN does not properly handle the failure of that machine by instantly routing that traffic to the next closest content server, you will have a lot of unhappy customers. These failures can and often do happen, so it’s best to be prepared for the worst.

Different services handle this scenario differently using various technologies (Geo-aware DNS, and TCP Anycast are popular options). The important consideration is picking a service that can handle that failure quickly enough to serve your organization and its requirements appropriately.

Another piece of the puzzle when ensuring your clients have access to your content is having a fall-back option for them. There are times when your content delivery network simply is not available or there are technical problems that are causing slowness or other issues. In these cases it may be important to have a second option to fall back on. For some of our clients we’ve implemented a simple HTTP option that is available in the case of failures in other parts of the system. In this case, if the CDN fails to deliver the content the user’s application can simply request the content from a standard HTTP server that has a duplicate of all the content available in the CDN. The user’s experience in terms of speed won’t be as great as it would be with a fully working CDN but at least it’ll work!

Scalability

It’s hard to predict how popular your content is going to be, especially on today’s socially connected networks. At any moment your content could explode in popularity leaving you no time to react. It’s a good idea to be prepared for sudden spikes, whether they last 10 minutes or 2 weeks during conference season.

The great news here is that most commercially available Content Delivery Networks are built upon robust and dependable hardware. Most networks make service level agreements committing themselves to supporting your traffic needs. The biggest determination here is really cost; some of the larger CDNs have fairly rigid tiers for which exceeding the negotiated traffic in a given month can get very expensive.

The newest players in the CDN market that I’ve discussed above like Amazon’s CloudFront and Microsoft’s Azure CDN have reasonably increasing tiers based on your traffic usage that is upfront and communicated fairly clearly. Most CDNs give you the tools to monitor your traffic usage to understand what content you are delivering and where you are delivering it to.

Future Proof your Content

There are a lot of ways to deliver a single piece of content these days. It’s important to understand the advantages and disadvantages of each. For delivering video content, you can deliver the content via RTSP, RTMP, Progressive HTTP Download and many others. The same goes for audio as well. To further complicate matter is the format of your content. Depending on where and to whom you want to distribute your content, you have to consider the formats appropriate for your customers.

There are a wide variety of encoding formats to support mobile phones, web based technologies, Flash technologies, and so on. These are constantly changing, and depending on your needs you may even want to investigate on-the-fly conversion between formats for your system. The important question to understand and evaluate is how you want to provide access to your content. Do you want it available via mobile phones, only via the web, in a custom desktop application or all of the above? Each delivery method has its own idiosyncrasies. We can help you sort through the acronyms and build a network that works with your organization and its content.

Management

One of the most overlooked aspects of designing a media delivery network is the tools that make living with your expensive investment a joy day-to-day. Inevitably, you’ll want to learn about what your content is doing, you’ll want to update the network with new content, you’ll want to restrict access to certain content, and you’ll want to generally take control of your network. Your organization is unique and your tools and processes around your delivery network are crucial to ensuring you get the most out of it. One of the most important parts of ensuring that a CDN is a success is investing in the tools you need to manage it.

Think about your production process today. Introducing a robust, high performance CDN into the delivery component of this process is bound to necessitate the creation of tools to manage conversion of the content to compatible formats, reporting on usage across the network, enforcing security on access to the network, managing the vast amount of information surrounding the content (including author information, descriptions, titles, etc), and generally providing the business specific tools you need to get your content out there. We’ve helped a variety of organizations understand their needs, and we can help you understand yours.

The Final Word

As you can see, setting out to design your own CDN is certainly not a simple exercise. You need a partner that has been there before and can walk you through these important decisions. HiDef is excited about helping deliver your content to thousands and thousands of users, and want to bring our years of experience to your organization so that you do it right the first time.