In the race to make sure you’re connected to the Web, able to download game demos, and stream high-def video to your desktop, there’s always a mix of fantasy and reality. (How much fantasy depends on exactly which games you play and which videos you watch.) Broadband providers routinely quote specs about download speeds, always careful to include an asterisk about your location and equipment.
Well, we decided to take the asterisk and blow it to kingdom come, testing four broadband technologies in several areas and in several test scenarios. Beyond the marketing claims and theoretical speed tests, how do they really perform with everyday tasks?
We streamed movies and downloaded demos, watched YouTube in high-def, and ran several speed tests using independent benchmarking service TestMy.net. We timed YouTube buffering speed, and downloaded the same 25MB file on the same FTP server using every broadband service. To get a good mix of technologies, we tested a business cable service from Charter, Verizon LTE in a major city, a standard DSL connection, and Sprint 4G.
(By the way, some of the speed benchmarking sites are actually owned and operated by the providers themselves. We avoided those altogether. Those tests are often rigged to tap you into an internal ISP server for the best results. That’s why, when you run a line-test using the “approved” benchmark site, you might see results that are double the actual speed you’ll get in your everyday Web visits.)
To make sure we were testing in the conditions that most people will use these broadband services, we tested the two mobile technologies (LTE and Sprint 4G) on recent smartphones, and tested the two fixed broadband technologies (cable and DSL) at a commercial office space. Yes, your results may vary, but here’s an unvarnished look at how four competing broadband technologies perform in real life.
4G LTE (Verizon)
We tested Verizon LTE in Sioux Falls, South Dakota where the service just recently went live. We used an HTC Thunderbolt smartphone and tested in a parking lot, inside a mall, in a moving car, and at a park where there were not any competing wireless signals anywhere around us.
In the TestMy.net benchmark, the results were extremely good for a smartphone. Our average speed clocked in at 12.5Mbps in all the different scenarios we tested. In most cases, the speed was almost always just over 12Mbps and sometimes went as high as 15Mbps or 16Mbps. Our core test, a download of the same 25MB file using a private FTP for each provider, took just 55 seconds.
In all of our YouTube HD tests, including several Call of Duty: Modern Warfare 3 gameplay videos, there was absolutely no buffering time at all. For every video we tested, the stream started playing immediately. During each stream, there was never any re-buffering or pausing. This bodes well for LTE – it means the line is fast and reduces latency. The other possibility is that few people in the area are using LTE, so we might have been one of the few using the 4G service in that area.
Using the BlockBuster video rental service, we downloaded the movie Hanna, which clocked in at 777MB. The total download time was just 3 minutes and 45 seconds, which was much faster than the total time it took to download a 477MB episode of The Killing on the Samsung S II using Sprint 4G. We downloaded an episode of House, at 610MB, that we had uploaded to a private FTP site – the download took just 3 minutes and 23 seconds. Our overall experience with LTE was that the broadband speed was much faster than Sprint 4G, and any latency was almost unnoticeable during our tests.
4G WiMax (Sprint)
To test Sprint 4G, we used a Samsung Galaxy S II, one of the fastest and best smartphones of the year. This model has a brilliantly crisp 4-inch screen that uses AMOLED technology. We tested the phone in the car, at a shopping mall, and even at the Minnesota Twins stadium in Minneapolis. The phone uses a fast Exynos processor running at 1GHz that’s enhanced for the best Android OS speeds.
Our test results were not that favorable, however. The Galaxy tested out at 3.7Mbps on average in a variety of test conditions including the mall and at the stadium. In fact, we never edged over 4Mbps in any of the tests we performed. There could be many reasons for this low benchmark: maybe there are thousands of people using 4G in Minneapolis and were crippling the service. We re-tested at different times of the day and even found a spot where there were few competing Wi-Fi signals in an open parking lot and still had roughly the same test results – just under 4Mbps.
The core test, downloading a 25MB file using a private FTP site, took 1 minute 33 seconds. That’s over 30 seconds slower than the same download over the Verizon LTE service.
Video download time tests were also disappointing. We rented a 477MB episode for the TV show The Killing using the Samsung MediaHub app and the download took 7 minutes 55 seconds. That’s a long wait time, considering we downloaded a similar 501MB video for an episode of House using Verizon LTE and the total time from click to view was only 3 minutes 23 seconds.
Testing several YouTube HD videos, we noticed just a slight one or two second buffer time before several Call of Duty MW3 gameplay videos started playing on the Samsung phone. In some cases, these videos started playing right away, as they did on the HTC Thunderbolt using Verizon LTE.
DSL (Park Region Mutual Telecom Co.)
In the line test using TestMy.net, the DSL line benchmarked at 5.9Mbps on average. We noted a high of about 7Mbps and a low of 3.5Mbps during our testing period. In our core download test of a 25MB file, the download time was exactly 30 seconds. It’s interesting to note that this download time was 25 seconds faster than Verizon LTE, even though that TestMy.net test rated 12.5Mbps for LTE. The 5.9Mbps average was actually the speed average that TestMy.net reported for our area. Of course, your distance from the nearest DSL exchange will dramatically affect your download speeds: the further, the slower.
Latency is the killer with DSL, though. In several tests downloading episodes of The Killing using iTunes (a fairly typical use of broadband), the download times should have been under an hour for the 1.4GB episodes. Instead, the actual time was 79 minutes for a 1.42GB episode. The variance is due to the fact that a DSL connection has to deal with network traffic before the line even arrives at your home or office. Meaning, there may be a direct copper wire to your DSL modem, but the provider is dividing up access for many users on the same server before transmitting the data to your connection. At times, the DSL connection would hang altogether and there would be a pause in the video download.
We also tested latency by playing several 1080p movie trailers in YouTube HD. In the Darkest Hour trailer, the clip paused for a split second right after pressing play – most videos had at least one short pause, but would then play normally to the end. When a site is heavily optimized to speed up downloads (e.g., the FilePlanet.com demo for Crysis 2), transfer speeds over DSL also improved – it took just 35 minutes to download the 1.6GB demo for the game without any pauses.
Playing Medal of Honor during several multiplayer matches, we noticed only slight latency when other players would sometimes pop off the screen – a common problem in first-person shooters. With faster connections, these blips just occur faster and less frequently – if at all.
One of the most revealing tests: We played the 720p stream of the TV show Homeland on YouTube. In HD, there was not buffering at all, but the video played choppy and dropped several frames. It was as though the stream was laced with syrup – not jerky and still watchable, but not smooth either.
Business-class cable (Charter)
You can order up cable in any number of rate-capped flavors, but to see what the technology can really do, we went to the top end of the spectrum. Charter Business service rated for a top speed of 100Mbps (which costs $330 per month) tested out at a blazing 82Mbps on average. We had to skip TestMy.net because it doesn’t provide enough bandwidth to test 100-meg access, and instead used speedtest.charter.com. (We should note that this is one of the speed test sites owned by a provider, therefore providing higher results, but Speedtest.net test clocked in at 80Mbps.) We also made sure we tested quite a few other scenarios with this fast broadband access, including video and downloads.
One of the most interesting findings: In our most basic test, downloading a 25MB file from a private FTP, which we did with each service, the speed was instantaneous. The file did not take any time to download. We found that many Web pages also loaded almost instantly.
In downloading an episode of The Killing, which was 610MB, the total time was just 1 minute 33 seconds. The same episode of House we had stored on an FTP site, at 610MB, took 1 minute 30 seconds. Our finding was essentially that the Charter connection ran easily three times, if not four times, as fast as the Verizon LTE service. Latency was the killer for the DSL service – Charter ran about 5 times as fast.
As expected, YouTube HD streams for Call of Duty MW3 gameplay videos all started without any hesitation, played all the way through, and never had any stuttering problems or stalls. In fact, every video stream we started – an episode of the show Homefront, streams on ABC.com, etc. all flowed smoothly, although the framerate for the Homeland episode was still not perfect.
We decided to really put Charter to the test. We downloaded the Windows 8 Developer Preview, all 4.7GB, in just ten minutes. However, we also tested a Steam download of the game Deus Ex: Human Revolution, which is 8.5GB, and the download took 1 hour 25 minutes. That’s a good lesson in high-bandwidth: Massive files will still take time. Steam shows some added detail about download progress: the rate even on broadband running at 85Mbps was only 16Mbps. (That’s likely do to some throttling by Steam itself.) So there was still some wait time – but still remarkably fast.
Conclusion
If you live in an area where all of the above are available – DSL, cable, LTE, and Sprint 4G – and you’re looking for the fastest speed possible, there’s no question that cable is the fastest, if you can afford the upper tiers. In some areas, FiOS is also available and some companies offer a direct fiber connection, but cable is much more widespread in most towns including rural areas. For the best deal, we ended up preferring Verizon LTE for a steady 12-15Mbps connection speed and smooth video. The Samsung SII let us share our LTE connection as a mobile hotspot, and speeds were roughly the same on a laptop.
