On Mitigating Risk When IT Vendors Fail to Deliver
A recent DSL outage that lasted almost two weeks taught us a few worthwhile lessons.
One of our clients uses a Business DSL connection sold and serviced by Covad.
Covad in turn relies on Verizon to manage portions of the infrastructure associated with the DSL connection, including the cabling in the street that leads to the client's facility. Here in New York City, much of the cabling in the streets and in older buildings is subpar, an accident waiting to happen. When trouble occurs, it can take a week or more to restore service.
This time, the client's Internet outage took two weeks for Verizon to diagnose and repair. That's much too long. So below are some of our thoughts on why the situation exists and how a business can mitigate the adverse effects of an Internet outage.
We've been installing and servicing Internet connections since ~1993.
During that time there has really only been one Internet Service Provider (ISP) we've ever found (Cogent Communications) who performed at the level of service that we believe is reasonable to expect, but unfortunately Cogent is unable to service every client facility, and the cost is ~$800 monthly. All other Internet service providers provide service that is unacceptable at some point.
What I like to say is that any of the vendors can be good or
bad, and the service generally works until it doesn't. Then when the
service doesn't work, ~50% of the time the service is restored in a
reasonable time period, say 1-2 days. Then once every few years, there
is an outage that lasts a week or more, as our client recently experienced.
To mitigate against the poor service, we came up with a dual-ISP scheme a few years ago when dual-ISP routers became available at a reasonable cost. That takes the pressure off when the primary ISP is unable to fix the problem promptly.
Another way to look at it is that the ISPs are undercharging us for their true support costs. The vendor whose service is acceptable charges ~$800 a month. So if you double a business' monthly Internet cost to ~$250, you're still well below the number that our experience has taught us is the lowest price we can pay and get acceptable service.
In the future we believe most businesses intend to do more with the Internet, not less. So, the need for absolutely reliable Internet at most businesses will increase, not diminish. Plan and act accordingly.