Some prospective customers challenge us as to why we think it’s important to have as many transit providers as we do. They often state that a competitor they are comparing us to has four or five providers, which is more than adequate for the sake of redundancy, and more than most other competitors. Actually, we would agree, but only with the fact that four or five providers is sufficient for redundancy. The likelihood of all five transit providers failing and causing a complete outage is highly unlikely (unless, of course, they all enter the building via the same conduit or fiber).  But redundancy is only one reason to have multiple providers. After ten years in the data center business, we've come to the conclusion that having a connection to every provider offers a superior blended Internet product for significant reasons.

Prior to vastly increasing the number of transit providers in our bandwidth mix, our clients would periodically ask if we could improve routing from point 'A' to point 'B' with the hope of shaving a few milliseconds off a route or with the goal of getting around latency deep within a network that was outside of our control.  Adding additional transit providers offered us three unique advantages with greater control over our network and dramatically improved performance for our customers. Let me explain each of these benefits in more detail.

More transit providers means better physical fiber routes
Not every provider uses the same physical fiber path to go from one city to another or even from one country to another. Fiber generally follows power line right-of-ways, railroads, interstate highways and other convenient paths. And while it is true that smaller providers lease capacity from larger providers to build their networks, there are numerous providers who have spent great deals of money running physical fiber around the country and around the world. Additionally, there are numerous regional fiber providers who have run fiber spanning multiple states. These providers often enter into agreements with other regional providers to create larger, multi-regional networks that can offer seamless capacity often spanning multiple states.

Here's a simple scenario to show you what I'm talking about. Let's take Level(3) and AboveNet , both of which are part of our 15+ provider Internet blend. As shown in the thumbnail images below (click for a larger image), you'll see that from Atlanta to Chicago, Level(3) has a direct path where AboveNet does not. AboveNet would have to get there via Houston or Washington D.C. Obviously, going via Houston or D.C. adds several thousand miles to the trip; and for every 100 miles, approximately 1 millisecond of latency is added. So, if we had to make a routing decision based solely on latency, Level(3) would most definitely get the traffic.

AboveNet (click to enlarge)	Level(3) (click to enlarge)

Of course, the opposite would be true if we were sending traffic to a client in Houston in which case AboveNet has a more direct path where Level(3) does not. So using this example, if we did not have a connection to Level(3), customers with servers in Atlanta and clients in Chicago would probably get upset with the high latency between the two cities.  Having a connection to both of these providers creates a better performing bandwidth product than having just one. Imagine how infinitely better it can be with 15 providers.

Sending traffic overseas is even more sensitive due to the large distances that need to be covered. Again, with the Level(3) and AboveNet example, if a client were in London and the server in Atlanta, AboveNet would route up to Washington D.C. and then out under the ocean to London where Level(3) would take it up to New York. In that scenario, it may be possible to shave 10ms off of the round trip, which can be significant for VoiP or other latency sensitive traffic. Furthermore, with the frequency of undersea cable cuts affecting global traffic in a significant way, having as many routes as possible makes the network less susceptible to these types of issues.

More transit providers improves logical routing
When traffic traverses only one network from client to server, things generally run more smoothly. In the past we've received the occasional call from a customer with a trace route in hand to show that a bottleneck lies between provider 'A' (which we have a connection to) and provider 'B' (which the end customer has a connection to). The trace route suggests that, at a certain hop when the traffic moves from our provider's network to another provider's network, latency increases dramatically.

Phone calls to either provider are most often fruitless. Both providers tend either to blame ICMP deprioritization and suggest the trace route is irrelevant or to point the finger at the other provider suggesting a bottleneck somewhere within their network. The truth is, as soon as their network engineers see that the perceived bottleneck is outside of the network they have access to, they take the position that the issue is not theirs.

Our method of getting around this is to get a direct connection to as many providers as we can in as many peering locations as possible. Our ultimate objective is to have a connection to the same provider that almost every one of the clients have who connects to infrastructure at our data center. As you might imagine, that is no small undertaking, but it is the easiest way to avoid finger pointing. If the traffic enters network 'A' at the datacenter and goes all the way to the client on the same network, there is no one else to blame but network 'A' when problems with latency or packet loss arise. A single network generally improves performance as well. Network 'A' will always optimize traffic within their own network before optimizing or upgrading links to other networks, so it is win-win for us and for our customers.

Another issue to consider is switching delay and queuing delay. Providers who have numerous router hops along a path to get from point 'A' to point 'B' add additional latency. Every router or switch that needs to process packets to forward it along to the destination adds a small delay. So a provider who has end-to-end fiber that traverses fewer routers or switches will most likely offer slightly lower latency, assuming the physical fiber path is identical. Of course, today's high-capacity routers and switches add very little delay, usually in the neighborhood of 150 to 200 microseconds (1/5 of a millisecond) compared to many years ago. But it is important to note especially for long routes that even today's best routers could add a couple milliseconds on a several-thousand mile trip.

More transit providers mitigate peering disputes
We've all heard how Cogent and other providers disagree when it comes to peering. Cogent has had peering disputes that resulted in serious Internet issues with at least 5 providers in as many years. From what we've read in the news, each of these disputes resulted in Cogent customers being unable to reach the customers of the peering partner and vice-versa. 

To illustrate how this could impact our customers, let's use the Cogent-Telia dispute. If we had only a connection to Telia and not Cogent, then during the dispute we would be unable to reach Cogent customers. During such a dispute, customers don't care who is at fault, they just want the issue resolved so they can resume business. If they were unaware of the dispute, they would most likely blame the provider or the data center that they were trying to reach.

By Netriplex having an alternate path to reach Cogent customers, traffic can move over that alternate route. Sure, it may be less optimal than the direct route, but it accomplishes the goal of ensuring redundancy and reachability. Of course my analogy here assumes that any other providers we have do not peer with Cogent, which would be slim. But the fewer providers a data center has in their mix, the greater the chance that this type of issue could affect them.

In conclusion, from what we've seen over the years we've been in business, more providers equals a highly superior connection. We will continue to make it our goal to peer with as many providers as we can to always improve the connectivity we provide to our customers.

There are a lot of critics out there and some of them are voicing their opinions about our business model. I’m all for ‘freedom of speech’, but making assumptions and asserting them as fact is rarely a wise thing to do.

We’ve seen posts on various forums where our critics claim the following:

1. It is impossible to purchase bandwidth for $3.99/mbps
2. It is impossible to back-haul bandwidth to our flagship data center in an affordable way
3. Our business model is doomed as a result of our ‘buying high’ and ‘selling low’ and therefore Uber Bandwidth won’t be around for very long

Please allow me to clarify how we can not only offer bandwidth for $3.99/mbps, but how we offer first-rate bandwidth for this price by addressing the three claims above with as much detail as our CEO and network security folks allow.

1. It is impossible to purchase bandwidth for $3.99/mbps.

That was true a number of years ago, but it is no longer the case. I remember when I first started out in the business back in 1999, bandwidth could be purchased in ‘wholesale’ quantities (usually at least a fully committed GigE) for $65/mbps. We thought that was a very competitive deal. Today, that is definitely not the case… especially when several things are in place.

   a) We pick up bandwidth at the source,
   b) We purchase a very large quantity and agree to burst at a premium rate.

By simply purchasing a ‘port’ from the provider, they don’t have to run any fiber. That saves them money. By committing to a large quantity of bandwidth (usually at least a GigE, but in our case as much as 10G) you actually can get a very low price.

It might be hard to prove that we actually pay less than $3.99/mbps without showing you an invoice (which I’m not permitted to do), but here is an example. If you go to Cogent’s website (http://cogentco.com), you’ll see right on their home page that they offer bandwidth for $4/mbps. We don’t use Cogent right now (that’s a topic for another discussion), but they are considered a major player. Other providers, such as Level(3) or XO, are happy to match or even beat a price on an equal commit.  We’re finding that no provider wants to lose business to another, and that’s helping us pass significant savings to our customers. The key is getting a ‘port’. As we move more traffic over the coming months and years, prices will only go down more as the industry changes and our buying power increases.

2. It is impossible to back-haul bandwidth to our flagship data center in an affordable way.

It is true that our flagship data center is not close to any major carrier hotels. That was a design requirement in order to increase the geographic security of our facility.  This is why we already contracted for fiber from 56 Marietta St. in Atlanta to our location (about 200 miles). We’re actively doing the same with other carrier hotels north of us too in order to create network diversity, but we’ll discuss that in another post later.

Some of our detractors state that it can cost as much as an additional $5/mbps to lease dark fiber and that a wavelength is also very expensive.  To the former (dark), absolutely. There are very few providers today who will give you dark fiber, and for good reason. Once you give a strand of fiber away, it’s hard to get back. And, the reason they don’t want to give it away is that they can make a lot more money if they put electronics on it and sell you a wavelength (10G). This is due to the fact that advancements in transmission technology such as DWDM (Dense Wavelength Division Multiplexing) increase the capacity of existing fiber by 40 or 80 times. What a fiber provider could sell as a single as ‘dark fiber’ can now be sold over and over again as ‘lit fiber’ in ‘wavelengths’ which means prices can be lowered to gain more market share and acceptance. This is what we buy, and with 96 strands of fiber into our building, a number of them go to 56 Marietta St. where we can pick up transit. Again, I can’t show you an invoice, but with the surprising number of fiber providers in our area competing for our long-haul business, we’ve been able to bring the cost of straight transport to well below $1/mbps. This number will go down as well as we move more traffic.

3. Our business model is doomed as a result of our ‘buying high’ and ‘selling low’ and therefore Uber Bandwidth won’t be around for very long

I hope I’ve given you enough information in the above two points to help you conclude that we’re buying and transporting bandwidth at very competitive rates at least equal to what we’re selling it for. Let me throw some more information at you.

Our parent company, Netriplex, has profitably operated datacenters in the U.S. and U.K. for many years. We have some experience under our belt and we really do know what we’re doing. We make very careful decisions when assessing growth or new opportunities. Any larger data center operator will tell you that when it comes to bandwidth, every customer doesn’t use what they have fully committed to at the same time. This isn’t as much the case with smaller facilities, but when you move more than 5G of traffic, this does become very apparent.

For example, if I sold 50 customers a fully committed 100mbps connection, would I be moving 5G of traffic all the time? Definitely not. In fact, our almost 9 years of experience has given us some data that actually indicates that with the above example, we might (on average) actually peak out at less than 2G. Uber Bandwidth will probably make that number higher since we attract high bandwidth users, but we’re pretty sure it will remain true to some degree.

Does this mean we have oversold our network? Not in the least! (I thought I better pre-empt that question.) It does mean we’ve committed to purchase less transit from our providers which reduces our bandwidth costs, but just like many of our customers, we have high burst capability so that if for some bizarre reason every customer who ordered 100mbps (or 1000mbps, for that matter) used it to the fullest at the same time, we’d have plenty of room to give everyone what they paid for (and then some!).

Of course, you’ve now figured that if we actually use less than half of what we sell, we stand to make at least $8/mbps instead of $3.99. While that is actually quite true (and while you might now feel upset that other data centers are making more than you thought), there is a lot that goes behind the scenes to make the network run smoothly. Redundant Cisco gear, IPS and FCP devices all cost hundreds of thousands of dollars to purchase. They also require us to have pretty brilliant (and expensive) geeks on hand to keep them purring, so in the end, you’re still getting an Uber deal.

Everyone wants to know, “How Uber is Uber?” And that’s a great question that we love to answer!

Our original flash website (that everyone hated) stated that Uber was being built, and that by December 2008 we’d have the full gamut of Tier-1 providers and several Tier-2. Well, we were a little delayed in turning things up by our self-inflicted deadline, but we’re moving right along. The list of providers we’re working on bringing into the mix is getting longer every day, but the bottom line is that right now, the network is already quite Uber. 

For the record, here is the complete list of current providers:

Level(3)
Savvis
Global Crossing
Qwest
Sprint
AT&T
Telia
BTN/PCCW
Time Warner
WBS
Charter
Nuvox
ERC Broadband

We have open negotiations with a dozen other providers and once we reach acceptable agreements, we’ll be adding them in to the mix as well. Our plans for the future are big. We’re running fiber to other carrier hotels around the United States and by the end of 2009 we hope to directly peer with over 100 networks. As we make significant developments, we’ll surely blog about it so you can get excited along with us!