New Report Debunks The 'Bandwidth Hog' Myth

from the and-data-caps-are-improving-infrastructure-how? dept

ISPs and cell phone companies have long been known for raising the spectre of the “bandwidth hog” in order to justify data caps or bandwidth throttling. Considering most ISPs are also television providers, throttling and capping data usage helps push some customers back towards their TV sets. (Depending on how many other eggs are in the company’s basket, this also helps push them back into theaters and tilts their heads back to regular old radio, as these caps make streaming services very unattractive.)

Karl Bode at DSL Reports has finally obtained some data which confirms what many of us had suspected all along: the Bandwidth Hog is a convenient bogeyman rather than an actual being. Analysts Benoit Felton and Herman Wagter managed to talk an “anonymous mid size DSL company from North America” into sharing its data on customer usage. Not surprisingly, despite their open invitation to the ISPs to contest their “disruptive user” argument, no other broadband/DSL service offered to provide any data, suggesting they’re already well aware of what usage numbers actually show.

In a blog post, Felten notes that the pair took real user data for all customers connected to a single aggregation link and analyzed the network statistics on data consumption — in five minute time increments — over a whole day. What they found is that capping ISPs often don’t really understand customer usage patterns, and are confusing data consumption (how much data was downloaded over a whole period) and bandwidth usage (how much bandwidth capacity was used at any given point in time).

What they discovered is data that runs in stark contrast to a lot of the claims put out there by some familiar, larger ISPs when justifying caps and overages. Among the pair’s findings is that the top 1% of data consumers (which they call “very heavy consumers,” instead of the already adversarial “hog”) account for 20% of the overall consumption.

Looking deeper into the data, they also found that about 61% of very heavy data consumers download 95% of the time or more, but only 5% of those who download at least 95% of the time are very heavy data consumers. While 83% of very heavy data consumers are amongst the top 1% of bandwidth users during at least one five minute time window at peak hours, they only represent 14.3% of said Top 1% of users at those times.

That’s a lot of percentages and percentages of percentages. Fortunately, a commenter at DSL Reports was able to use the “dreaded” highway analogy to simplify things:

1% of vehicle drivers on the road travel a disproportionate amount of miles compared to the average driver. But they are on the road all the time. Most of the time they are on the road there is no rush hour congestion.The heavy drivers are likely to be involved in rush hour traffic jams, but only represent a small, not terribly relevant, fraction of total drivers in the traffic jam.Limiting the amount of miles a driver can drive, does nothing to widen the roads and little to keep people off the roads during traffic jams, thus does not help with congestion.

In other words, internet usage tends to be heaviest at certain points of the day, and installing caps or throttling supposedly heavy users does nothing to relieve that congestion. Instead, it punishes users across the board by hitting some of them with additional fees and offering very little in the way of improving connection or speed for the rest of the users online during these “traffic jams.”

Felten concludes that ISPs themselves need to better understand the difference between data usage and bandwidth consumption, or face driving their customers to more reasonable competitors. That’s assuming consumers have a choice, given caps exist in many markets largely due to no competition.

That’s the real problem. For many people, there are few options. And most of the ISPs are more than happy to install caps and overage fees, especially if someone in the market is already doing just that. Bode also notes the adversarial relationship with their customers that the ISPs are creating through the usage of terms like “bandwidth hog” or “disruptive user.” Rather than look into improving infrastructure, they’d much rather vilify certain paying customers in order to deflect attention away from their service limitations.

Karl Bode expands on this:

It would also be naive to assume many of the larger ISPs — stocked with number crunchers and network analysts — don’t already know everything Felten stated. However, there’s a reason that ISPs don’t like bandying real, raw data about — and it’s because there’s a few large carriers that like to use bogus science to justify anti-consumer behavior, most recently with AT&T’s announcement of caps and overages for DSL and U-Verse users. When asked to prove that these caps and overages were necessary, AT&T couldn’t — something ignored by general tech press coverage of the move.

As we’ve noted repeatedly, most carriers impose caps and overages claiming it’s due to either network congestion or financial necessity. In realty, caps and overages are implemented by carriers that simply want to jack up the cost of bandwidth so they can protect TV revenues from Internet video by making Internet video more costly and less appealing. The financial “necessity” of moving away from the flat-rate pricing model is proven false quarterly by earnings reports.

While it’s nice to finally have some data on hand to debunk the “bandwidth hog” myth, most ISPs will be able to dismiss it as not being representative of their customers’ usage patterns. After all, they’re still refusing to provide any data to back up their claims. In areas without competition, this myth will still be used as a scapegoat for everything from hard caps to lousy connections.

Filed Under: , ,

Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “New Report Debunks The 'Bandwidth Hog' Myth”

Subscribe: RSS Leave a comment
Skeptical Cynic (profile) says:

Clear is a perfect example of Boneheadedness

I had Clear WiMax from the day it was available in Atlanta. I left them over a year ago because they “Managed my Bandwidth” too often because of “over usage”.

I dropped to less than 56k speeds for more than a day anytime I tried to watch a Netflix video at the same time as I downloaded a large file after I moved to a new house in a more affluent area. Even though my monthly usage never changed it averaged the same for ever month for the last year.

Well you say they did it because I was sucking up too much bandwidth during peak times. I say false. The bandwidth I used (85% of it, by their charts) was used during the slow period during the business day. (I do IT support so I would be doing support and watch Netflix during the business day.)

I wasn’t a heavy user by any measure. 3.6GB a month.

That “throttling” pushed me away from Clear and also stopped me from recommending them. Before I saw that I was making around $1250 a month recommending them.

In the year when my service was good I had over 600 people and companies sign up for Clear.

Scooters (profile) says:

It's a problem with higher priced "services".

For the past year, Brighthouse Networks has been pushing its customers to pay more for “faster broadband”. Their words, not mine. Reluctantly, I gave into the ad campaign (at the time, it was a mere $5/mo. extra which somehow turned into $15) and tried it out.

What I discovered was a blatant rip off of consumers gullible to fall for “faster” speeds. Everything at peak time dropped down to speeds my system was already seeing before the upgrade.

Here’s the kicker: When I asked if they were throttling during peak time, they said no. So I asked, “Basically what you’re saying is I’m paying for the privilege of driving in the fast lane but because your networks allow for slower cars to be in my lane, I have to be slowed by them?”

I dropped the extra speed and the associated charge and went right back to the speeds I can never surpass, all for the wonderful price of $Suck it, customer (tax not included, as well as “fees”).

I contacted my local government regarding a data hub and bandwidth service as a means to offset their inability to allow competition to enter the marketplace, restricting us only to Brighthouse.

Hear that laughter? Yeah, it’s been over two years. I don’t think they’ll ever stop until someone’s bold enough to help finance most of the cost to get it done.

Sometimes, I truly despise Corporate America.

Anonymous Coward says:

Re: It's a problem with higher priced "services".

“I don’t think they’ll ever stop until someone’s bold enough to help finance most of the cost to get it done.”

Cities have already tried that, then corporate lobbyists come in and ban the whole projects.

What we need is someone willing to fund political campaigns that will allow competition.

anonymous says:

just another way to make more money, by increasing the number of customers, without having to do anything to improve the service by upgrading networks or putting more pipes into operation. typical company ploy. profits far more important than service given or customer satisfaction. sounds very much like another industry to me!

pixelpusher220 (profile) says:

Re: Re:

Don’t need studies, basic logic says they’re blowing smoke.

ISP claim: 1% of users are using too much bandwidth

ISP Solution: Set limits that affect 80% of users

Real world cure: Set limit just below that which the 1% are using that is causing the problem. You only affect the problem people.

Or, you know, actually market and sell what you can reasonably supply rather than unlimited data for 25Mb+ down for everybody that WILL overload your networks when people actually use that much.

PlagueSD says:

As a test, I turned my wifi off on my phone for a month just to see how much “data” I used. I didn’t change my habits, and intentionally streamed movies that month to push my data usage up. At the end of the month, I was JUST over 3gb.

I’m sure if I did nothing but stream movies, I could get pretty high, but that’s a little unreal. About 90% of the people will NEVER go above 1gb a month.

Skeptical Cynic (profile) says:

Re: Re:

I think that is true but the issue is that they buy and pass the cost on to us for the total bandwidth whether or not it is used. In IT it is simple, Bandwidth is the size of the pipe total download amount is what is used. The average company pays for Bandwidth but uses much less than that “pipe” can do in any month. But they charge us as if we did.

Some Other AC (profile) says:

Re: Re: Re:2 Re:

As a lucky user of FIOS from Verizon, my current connection is listed as 35Mb up and 35Mb down. Thru multiple test sites and times, i was able to confirm 37 to 40 down consistently and 30 to 33 up consistently. As part of a bundled package with TV and phone, retail cost is less than $150/mo before taxes/fee and includes the top tier base package and STB rental cost.

Anonymous Coward says:

Re: Re:

As a test, I turned my wifi off on my phone for a month just to see how much “data” I used. I didn’t change my habits, and intentionally streamed movies that month to push my data usage up. At the end of the month, I was JUST over 3gb.

I’m sure if I did nothing but stream movies, I could get pretty high, but that’s a little unreal. About 90% of the people will NEVER go above 1gb a month.

Yeah, I’d find it difficult to use more the 3GB on my Evo, but I can very, very easily exceed 3GB in under a day on my home DSL connection by just buying a couple of games on Steam.

This post, the article it points to, and the research discussed there is about broadband service, not mobile service, which is a whole other kettle of fish. Conflating the two is at best confusing and at worse intellectually dishonest, muddies the discussion, and ultimately works to the advantage of the entrenched telecoms that trying to extract monopoly rents from their customers in exchange for actually lowering the quality of service.

rubberpants says:

Brown Sugar

Government-granted monopolies are the heroin of the business world.

1. As soon as a business get’s a taste, they’re hooked. They can rarely transition back to a competitive environment.

2. They’ll spend millions lobbying and influencing policy to make sure they can keep getting high.

3. When people try to give them an intervention, they lash out with anger, hostility, and complete denial.

4. They’ll systematically destroy their customer relationships, waste their future opportunities, and eventually wind up bankrupt in the gutter.

Anonymous Coward says:

Re: Re: Brown Sugar

I just got a $10 credit for calling Verizon’s FiOS bullsh!t onto the carpet.

10 dollars. 10 DOLLARS! Wow, you’re rolling in it. That’s just over .5% of the average annual cost of service. I bet I could get my bill permanently reduced by 10 dollars a month, all you have to do is bitch at a bunch of customer reps.

The point is that people want a high quality of service when paying a high price; no one is really concerned with saving 10 dollars, if it means crappy service.

TtfnJohn (profile) says:

In some respects I’m not at all surprised by this revelation. Network designers in telcos and cablecos have known this for years at both analog and digital levels of transmission. It’s why the networks are designed the way they are.

It’s designed to withstand short bursts of increased (i.e. hogging) traffic and survive it quite nicely, thanks. It’s also designed to be unnoticeable to the customer. (Poorly designed or rural and suburban cable subscribers can and do still notice that bandwidth decreases between 4 and 8pm when people come home and flip in the TV and computer. The bandwidth goes back to normal after 8pm. A great deal of that is the “shared” aspect of coaxial cabling vs twisted pair.)

It’s largely been a false argument from the start. Where there is fibre to the house it’s a completely false argument as, theoretically, the bandwidth/bitstream in fibre is infinite. And before any screaming starts that there is still copper out there, I know that. The closer your bit of copper is to a cross connect that itself is fed by fibre the less the copper restricts bandwidth/bitstream transmission. (though if you wire your house with cat 3 [cheap] tel wire you deserve what you get.)

It’s a money grab, pure and simple. There’s little or no technical validity to it at all.

PS: And as AT&T well knows 99% of all data and voice switching is not done externally and internally by light not copper or relays or any of that junk in the central offices.

Paddy Duke (profile) says:

In the UK, along with the data caps, the big ISPs (with the possible exception of Virgin, the cable/fibre network operator) also throttle internet speeds at peak times.

Ostensibly this throttling is to ensure heavy users don?t hog bandwidth while there are a lot of users online. I see two huge problems with this:

First, the providers have oversold their network. They have sold more bandwidth than they are capable of providing at any one time. This is blatantly dishonest. We pay for an 3.5MB BT connection (the fastest available in my postcode) but rarely see speeds of even a third of that. A recent OFCOM study found that most UK internet users get speeds well below what they signed up to.

Second, the result of this throttling is that at the time when most people are online, the experience is at it?s worst. In the evenings on my connection it is often not possible to stream a 1 minute YouTube video in its entirety. If other users are similarly affected, the ISPs will have a lot of peeved off customers.

The rate at which the big ISP networks in the UK are being updated is also abysmal. BT have only recently started offering 20MB (though in reality this is closer to 8MB) fibre connections in heavily built up areas.

Meanwhile the smaller Virgin, whose customers actually do get the advertised speeds, and sometimes even higher ones, are rolling out 100MB connections in cities. Four years ago, when I lived in the centre of Belfast, we had a 20MB Virgin connection. It will be another two years before I can get a 20MB BT connection the large town I live in now.

Even here where we have greater competition, there is very little to differentiate between any of the providers. The prices are much the same, the speeds are much the same and the throttling and caps are all much the same. I?m not sure how this situation came about, but I find it problematic.

The only exception is Virgin. Their service and support seem to be pretty great but they are not widely available outside urban residential centres.

MichaelG says:

two ways to avoid the problem

Here are two ways the ISPs could improve their lives if they actually needed to or wanted to.

1. At night (1am to 7am), I assume their network is almost completely idle. So introduce a modified BitTorrent client that only downloads during that period (or price bandwidth based on day/night use). If you want a movie or TV program, get it at night for viewing the next day. They wouldn’t see any strain from bandwidth hogs at all.

2. Clone the NetFlix streaming library inside their walls. At 500 meg an hour for video (350 meg is more typical), a $50 terabyte drive holds 2000 hours. If NetFlix has 100,000 hours of streaming video, that’s 50 drives ($2500). Even hardened and replicated, it would cost nothing to put that much data inside the cable system servers. Then all the bandwidth currently going outside the system to NetFlix disappears with no difference to the customer. Kick some of the money saved to NetFlix. Problem solved.

Anonymous Coward says:

Re: Re: two ways to avoid the problem

Akamai pisses me off. I block it every chance I get. I like the idea, but they do strange crap like upload log files randomly to their servers. Lots of network chatter. Makes it really hard to read log files and packets.

Forget you Autodesk and Adobe. I’ll just grab the install media later.

deadzone (profile) says:

Re: two ways to avoid the problem

Or maybe all of the major isp’s could address it in a more meaningful way through investment by upgrading their aging networks to all Fiber.

The differences in areas with fiber buildouts are telling. You see 2 major things that you won’t see in your typical broadband monopoly/duopoly market: Competition and No Caps or Throttling.

Jeremy7600 (profile) says:

Re: two ways to avoid the problem

Except that hard drives prices are through the roof because of the floods in Thailand.

I don’t think you can find a 1TB for under $100 these days. (there is one one pricewatch, all others start at $105)

so your 2500 figure goes up to $5000 but not bad still.

Either way, Content Delivery Networks already do this sort of thing.

Anonymous Coward says:

It’s a little bit misleading, because they don’t seem to want to consider what these “constant” users do to available bandwidth pools.

Most users take data in short and peaky ways, usually loading a webpage with images and / or a video clip in short order, and then sitting reading it / watching the video with no additional bandwidth usage for a period of time. A great example is writing a comment here, downloading the page was a short burst, followed by the time it takes the write a comment as “dead” time.

Constant users “raise the floor” and leave less space for the burst users to get their data. If you have a 10 gig connection, but 5 or 6 gig of it is constantly in use by downloaders, you effectively only have 4 or 5 gig to support all your other burst users. It becomes more critical as you get down to the ends of the network (last mile), where you might have only a 100 meg network connection providing service to a group of end users. You get a few people downloading constantly, and much of that bandwidth disappears, slowing the burst users access.

It’s why things like Netflix are such a problem for many ISPs, because they represent a data flow that is not in keeping with their network designs or bandwidth plans.

Anonymous Coward says:

Re: Re: Re:

Network architectural isn’t something you just go “ding, here new setup”, especially when you look at “last mile” companies that might have tens of thousands of nodes connecting to their network (and dozens of customers on every node). They cannot just wander out and buy a whole bunch of new networking equipment or sling miles of fiber optic cable on a whim.

When you consider that network hardware is typically purchase on anywhere between a 3 year and 10 year depreciation cycle, you will understand that changes just don’t happen overnight.

It should also be said that if they were working their networks to be able to handle the P2P onslaught (which most seem to agree is slowly subsiding), they may find themselves once again with a network that isn’t optimized for the current trends – and by the time they change it again to be there, the target will have moved again.

Just as importantly, the ISPs continue to face downward pricing pressures in their businesses, and yet the consumers are expecting them to spend more money, to raise connectivity speeds, and to provide an endless fountain of bandwidth – all at that lower price.

You have to remember it is a real world business, and that business doesn’t move at the speed of the internet.

Trails (profile) says:

My experiences

My household is huge media consumer. I work from home, and part of my work involves imaging and dealing with 100s of meg asset files. We have netflix on computers and Wii, we also have this asian (legal) tv over ip we pay ~$40/mo for to get overseas content.

I recently switched from Rogers (the incumbent cable provider in my neck of Canuckistan) to Teksavvy a recent startup.

I couldn’t be happier. No caps, no throttling (which was hitting among other things my encrypted uploads of said 100s of megs asset files). We cancelled all cable tv, and I pay substantially less for a better connection.

The idea of caps and throttling is lame, I was paying ~$65/mo +tax for a cap of 100 gigs/mo, which, in this day and age is ludicrous. So I was paying on avg another $20-$40/mo in overage charges.

That plus throttling affecting my work, and Rogers contant billing errors (all in their favour), left me cranky as hell.

I am thrilled with the new service and will never go back to capped and throttled. Anyone who is able to switch from Bell/Rogers to Teksavvy, I HIGHLY recommend.

Don says:

My solution to being capped

If it should happen, I will cancel my cable service completely and go back to reading books. I love to read anyways and all you get from TV is what not to do in real life – which i pretty much know already. At least with books you can either be entertained or educated. Either is mental stimulation that TV lacks. I will be sad to lose the internet but i lived before it and will probably live much better without it. All those chores i avoided will finally get done.

Anonymous Coward says:

“As we’ve noted repeatedly, most carriers impose caps and overages claiming it’s due to either network congestion or financial necessity. In realty, caps and overages are implemented by carriers that simply want to jack up the cost of bandwidth so they can protect TV revenues from Internet video by making Internet video more costly and less appealing.”

This is complete bullshit! What about the ISPs who are not involved in marketing television, the local telcos, or local wireless ISPs. They do it in an attempt to limit the usage.

Highways will only support so much traffic, and it isn’t just the number of cars on the highway that causes issues, but the size of the cars (the number of packets). If you imagine the bandwidth as the highway system, you need to refine the analogy to get an acurate picture. Each service running on each actively communicating device in a home transfers some data, some more than others. Instant messaging clients typically use very little data, email without attachments is also very little (those can be SMART cars), Web sites and emails with photo attachments we will call sub compacts, music streaming services would be buses, and video streaming services would be caravans of 18 wheelers. Video files are huge, enormously huge compared to other network traffic.

Lets say you can get 1000 sedans driving down the highway at 65mph, if you throw a bunch of 18 wheelers onto that same highway the congestion will slow the entire road down substantially. Drivers complain about the slow traffic and no one wants to pay for a larger highway so you limit traffic by putting up throttling red lights at highway entrace ramps. You limit the number of people that can get on the highway to improve the efficiency of they highway system.

This used to happen back in the days of dial up when you would attempt to connect and get a busy signal. Now that everyone has always on connections its just a flood of traffic.

Anonymous Coward says:

Re: Re:

The reason semi’s slow down a highway is because they physically cannot get up to speed as fast as smaller car.

When trying to make the case of saying video files are like semi’s, these would have to be semi’s that maintain the same speed as the SMART car at all times, it’s just the end of the SMART car would cross the finish line before the end of the semi would.

Anonymous Coward says:

I think this must be the only issue I don’t agree with hMike on.

First, some caveats:
– I love unlimited data caps (I have one, and I’m a “data hog”)
– Competition kills data caps eventually (I’m in Australia: 5 years ago my cap was 10GB, the highest cap was around 50GB. Now there are caps of 350, 500, 1000GB, and even unlimited.

But there is simply no denying that caps DO reduce congestion (even if, as this data shows, it’s only really relevant at peak times). 10-15% at peak times is nothing to be sneezed at.
I have a degree in computer science, if that makes the slightest difference to the credibility of my opinion.

Bengie says:

Re: Re:

“caps DO reduce congestion”

They have shown time and time again, that one average, most people don’t even know there are caps, yet alone change their data usage because of caps.

Most people can’t link what they do with data consumption. As far as they care, reading an email takes as much bandwidth as P2P or Netflix.

Most people rank their data consumption in time spent on the internet, not on types of web services they use.

Ninja (profile) says:

Re: Download 95% of the time? Huh?

Last week I spent 72 hours downloading non-stop at max speed. I admit my habits were worse for the isp a while back when I’d spend 100% of the time for weeks using over 80% of my downstream/upstream (I had to use traffic shaping to properly navigate through the web at the time). ISP never ever throttled me. But fast forward 3 years of intensive marketing (and tons of new users) with zero investment in new infra structure and they couldn’t handle the demand at peak times (to the point I kept getting kicked out by the antenna I used to connect to). And I jumped boat. My current ISP throttles the hell out of the connection so I have to use encrypted connection and download in several chunks to avoid it. But it’s going down the same path and the throttling is getting worse and worse. Solution? Sit and cry, there’s no other competition.

Add Your Comment

Your email address will not be published. Required fields are marked *

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop »

Follow Techdirt

Techdirt Daily Newsletter

Techdirt Deals
Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...