Slow Bandwidth

Not Answered This question is not answered

We've been monitoring bandwidth over the past few weeks between a server in your environment with various CDNs in order to evaluate your service. The bandwidth we're seeing is about 8-16x slower than servers in other public clouds. Here is a summary of what we are doing and the results:

Test files:

Limelight:

http://cdn.cloudfiles.mosso.com/c53212/sample.gif (66KB) - 310 msec avg

http://cdn.cloudfiles.mosso.com/c53212/sample.zip (1MB) - 871 msec avg

CloudFront:

http://dp71yw0bh3bah.cloudfront.net/sample.gif (66KB) - 373 msec avg

http://dp71yw0bh3bah.cloudfront.net/sample.zip (1MB) - 1 sec avg

Internap:

http://http.cdnlayer.com/cloudharmony/sample.gif (66KB) - 409 msec avg

http://http.cdnlayer.com/cloudharmony/sample.zip (1MB) - 1.66 sec avg

For the same files with servers in the ec2, gogrid, rackspace, softlayer, and newserver public clouds, on average the 66KB file is downloading in 15-25 msec and the 1MB file in 100-200 msec.

Is there a cap on your bandwidth that might be causing this? Based on these tests, it appears that we are only getting between 5 and 8 mbps downlink from these 3 CDNs compared with 40-80 mbps in the other 5 public clouds, but the latency is about the same. 

 

  • Post Points: 3
All Replies
  • David,

    I'm acknowledging that I see your post.  I personally do not have a response to our comparisons with other companies.  I will bring it up with the team tomorrow.

    Jeremy

    vCloud Support

  • The past few weeks we've used mechanical turk to get a few thousand users around the world on various high speed internet connections (mostly residential) to run our speedtest (cloudharmony.com/speedtest). The speedtest downloads 1mb and uploads 512kB from/to various cloud services including terremark. The test order is always random. This should provide a much larger and better sample for comparing Terremark's throughput with other public clouds. Using geoip, we were also able to tag each speedtest with country, state/province and ISP name and will be able to break out the stats accordingly. We're in the process of compiling the data now and will post the results here hopefully later this week.

  • Here is a sample from the data that we have already processed (about 300 speedtest executions). Since these speedtests are mostly based on residentical broadband connections, the average throughput is much less than the pingdom tests which were run from of Internet data centers. Once we have finished processing all of the speedtests, I'll update these results:

    This first table shows the average download throughput (megabits/second) for ALL speedtests US and non-US. In this test, Terremark was 20% slower than newservers (based out of the same data center), and 47% slower than the fastest overall cloud Voxel based out of NY. It is also the 3rd slowest public cloud out of the 21 included in our tests.

    This table shows the average download throughput for only speedtests from non-US connections. In this test, Terremark was 15% slower than newservers (based out of the same data center), and 59% slower than the fastest US cloud Voxel based out of NY.

    This last table shows the average download throughput for only speedtests from US connections, Terremark was 22% slower than newservers (based out of the same data center), and 47% slower than the fastest overall cloud EC2 US East.

    These tests don't show nearly the same amount of disparity between terremark and other public clouds as the pingdom tests did. However, they do continue to show a notable difference between downlink throughput from Terremark and other public clouds including ~20% less than newservers based out of the exact same Miami data center, and ~50% less than EC2 US/East and Voxel. The global tests also continue to show Terremark near the bottom in terms of downlink throughput relative to other public clouds.

  • Hi-

    I've compiled a few report runs from your URL.. this is a fairly typical report. Our Server is a vCloud Express 2V/8R.. unloaded.. just sampling things.. while overall we've been happy with the service, we've not really done alot of heavy loading (video streaming for Tv stations) because of reports like these..

    As noted.. happy with the vCloud Express product for development purposes.. wondering outload .. if the vCloud Express platform is a better marketing tool for the non-shared products  :-)

    lets keep the thread fresh.. thanks for the input simgar.. Cheers!

    Speedtest Results

    Order Service Location Test Type Data Transferred Time (secs) Transfer Rate (Mb/s)

    1 Rackspace CDN by LimeLight  download 1.00 MB 2.10 3.81

    2 Amazon EC2 EU West IE download 1.00 MB 4.22 1.89

    3 Amazon EC2 EU West IE upload 512.00 KB 3.50 1.14

    4 VISI Cloud Services MN, US download 1.00 MB 1.62 4.95

    5 VISI Cloud Services MN, US upload 512.00 KB 2.30 1.74

    6 Amazon EC2 US East North VA, US download 1.00 MB 1.51 5.31

    7 Amazon EC2 US East North VA, US upload 512.00 KB 3.74 1.07

    8 IBM Developer Cloud  download 1.00 MB 1.22 6.55

    9 IBM Developer Cloud  upload 512.00 KB 2.59 1.54

    10 Windows Azure Platform  download 1.00 MB 1.10 7.26

    11 Windows Azure Platform  upload 512.00 KB 2.52 1.58

    12 Linode VPS Hosting - London London, UK download 1.00 MB 3.72 2.15

    13 Linode VPS Hosting - London London, UK upload 512.00 KB 5.11 0.78

    14 Google AppEngine  download 1.00 MB 1.19 6.70

    15 Google AppEngine  upload 512.00 KB 1.77 2.27

    16 Linode VPS Hosting - Fremont Fremont CA, US download 1.00 MB 2.70 2.96

    17 Linode VPS Hosting - Fremont Fremont CA, US upload 512.00 KB 2.97 1.35

    18 Linode VPS Hosting - Dallas Dallas TX, US download 1.00 MB 1.46 5.48

    19 Linode VPS Hosting - Dallas Dallas TX, US upload 512.00 KB 3.52 1.14

    20 Nirvanix SDN  download 1.00 MB 5.06 1.58

    21 SoftLayer CDN by Internap  download 1.00 MB 1.19 6.72

    22 CacheFly CDN  download 1.00 MB 0.36 22.04

    23 Technology Insight Seattle WA, US download 1.00 MB 2.42 3.31

    24 Technology Insight Seattle WA, US upload 512.00 KB 2.26 1.77

    25 Flexiscale Cloud Computing London, UK download 1.00 MB 3.30 2.42

    26 Flexiscale Cloud Computing London, UK upload 512.00 KB 3.85 1.04

    27 Linode VPS Hosting - Newark Newark NJ, US download 1.00 MB 4.26 1.88

    28 Linode VPS Hosting - Newark Newark NJ, US upload 512.00 KB 2.58 1.55

    29 VoxCAST CDN  download 1.00 MB 1.73 4.63

    30 Highwinds CDN  download 1.00 MB 1.52 5.28

    31 NewServers: Bare Metal Cloud Miami FL, US download 1.00 MB 6.76 1.18

    32 NewServers: Bare Metal Cloud Miami FL, US upload 512.00 KB 4.46 0.90

    33 GoGrid CDN by EdgeCast  download 1.00 MB 0.73 10.94

    34 Amazon S3 US West Northern CA, US download 1.00 MB 3.42 2.34

    35 VoxCLOUD Servers New York NY, US download 1.00 MB 1.69 4.74

    36 VoxCLOUD Servers New York NY, US upload 512.00 KB 2.83 1.41

    37 SimpleCDN  download 1.00 MB 1.61 4.96

    38 Linode VPS Hosting - Atlanta Atlanta GA, US download 1.00 MB 0.96 8.38

    39 Linode VPS Hosting - Atlanta Atlanta GA, US upload 512.00 KB 1.86 2.16

    40 Amazon CloudFront  download 1.00 MB 0.60 13.39

    41 CloudHarmony.com  download 1.00 MB 4.32 1.85

    42 CloudHarmony.com  upload 512.00 KB 3.39 1.18

    43 BlueLock vCloud Express Indianapolis IN, US download 1.00 MB 2.26 3.55

    44 BlueLock vCloud Express Indianapolis IN, US upload 512.00 KB 3.30 1.21

    45 prgmr Xen Servers Fremont CA, US download 1.00 MB 3.46 2.31

    46 prgmr Xen Servers Fremont CA, US upload 512.00 KB 2.94 1.36

    47 Rackspace Cloud Servers Dallas TX, US download 1.00 MB 1.54 5.20

    48 Rackspace Cloud Servers Dallas TX, US upload 512.00 KB 1.96 2.05

    49 Joyent Smart Platform CA, US download 1.00 MB 2.28 3.51

    50 Joyent Smart Platform CA, US upload 512.00 KB 3.20 1.25

    51 Amazon S3 EU West IE download 1.00 MB 2.55 3.13

    52 Amazon S3 US East North VA, US download 1.00 MB 1.75 4.56

    53 Terremark vCloud Express Miami FL, US download 1.00 MB 0.50 16.00

    54 Terremark vCloud Express Miami FL, US upload 512.00 KB 1.74 2.31

    55 Amazon EC2 US West Northern CA, US download 1.00 MB 2.73 2.94

    56 Amazon EC2 US West Northern CA, US upload 512.00 KB 3.52 1.14

    Average Download  download 34.00 MB 77.83 3.49

    Average Upload  download 11.00 MB 65.87 1.34

  • Hi!

    Disclousure.. we're an ISP coding an Application with the Hosting Controller API and vCloud API for automatic provisioning IaaS.

    We're looking to extend the Application to all Cloud IaaS Providers, but we're focused on the Iaas Providers that have good QoS metrics and and good product.

    TM has a good product, the QoS metrics have yet to be dumped out despite a post in mid-December that 'they be posted after the holidays', but, given the data being posted, there is alot of upside to be achieve in the infrastructure.

    Opinion.. The Terramark product the most polished front end, marketing, UI, etc, the nuts and bolts of the beta operation leave alot of upside potential and our wondering when when the that point in time will be realized. CIR/CBR is something we need to know upfront, not figuring out AS_Paths with CIR estimates from automated software tools the Routing Arbiters.

    Looking for feedback from the helpdesk..

    Thanks

  • Hello! I referred to this post just now in a prior post and just wanted to bump this one up the ladder.

    Based on the Post "Posted by vcs-PVieau on 12-22-2009 7:25 PM" back to my query of QoS Metrics.. "We are working on the rate metrics info.  We will have that post sometime after the Holidays."

    Whith all due respect to my brothers on the support desk, I asusme you were talking about Xmas/NewYears.. but perhaps I am mistaken.

    Can you freshen this post with data we can consume?

    Thanks

  • Hi h02332,

    Thank you for refreshing your request of QOS Metrics. I will forward your request to the right personnel and we will update you back on this thread.

    Again, thank you for your patience.

    Regards,

    Derrick Nguyen

    Terremark Cloud Support

  • All:

    I have forwarded the update request to the appropriate teams.  Our apologies for the delays.  This information is not something support has access to.  

    We appreciate the feedback and continued updates to this thread.

    Kind Regards,

    Sonia Davis

    Cloud Support Team

  • All:

    I have discussed this matter with our CTO.  He is personally working with various teams to get the data, as well as reproducing the issue and attempting to correct it.

    We will keep you posted.

    Kind Regards,

    Sonia Davis

    Cloud Support Team

  • After finishing compiling results from about 1300 unique speedtests from 115 countries, we came up with the following tables. In the end, Terremark's downlink throughput actually came out fairly decent.

    US Tests

    For US-based tests Terremark was 8th of the 18 tested.

    Non-US Tests

    For US-based tests Terremark was 11th of the 18 tested.

     

    All Tests

    For all tests (US and non-US) Terremark was 8th of the 18 tested.

     

    98% of these tests were performed from residential high-speed internet connections. The 700 US tests covered most states. Our blog provides a more detailed description of these results: http://blog.cloudharmony.com/2010/02/cloud-speed-test-results.html

    The speedtest is also updated to allow you to selectively download and upload to up to 26 different cloud services including terremark and download sizes up to 5MB: http://cloudharmony.com/speedtest

    I hope this is helpful

  • Thanks for the update simgar. I have forwarded this information to the folks who are currently working on the slow bandwidth problem.

  • simgar: Thanks for the information.

    Also, I'd like to clear up any confusion concerning QoS, rate shaping, etc. There are no caps or throttles of any kind on bandwidth for vCloud Express users. If you are experiencing performance issues, it is likely something to do with routing or latency. I have been gathering performance data and working with the network teams to determine if there is a problem and what the solution would be if a problem exists. We will provide updates as soon as we have something definitive.

    Thank you for your patience.

    Jason Lochhead
    CTO, Cloud Computing
    Terremark Worldwide

  • Hi!

    Jason, May I ask you to reply to my question, thanks!

    I need to develop a performance profile for a VM on your Platform. I've got one for the OS level, now I need network level.

    Question - What can we expect as customers for CIR/CBR to a VM at the network level? Is that virtual 10Mb Ethernet, or Virtual GigE? ;-)

    For example, I config a full up VM and want to point OC-3 (or more) Capacity on east Coast requiring a CIR/CBR in SLA for my client.. who may only need the capacity for a few hours when news breaks or something stupid like a car chase happens.. which brings in alot of traffic, quickly.. a perfect application for cloud resources..  What can we expect for network performance per VM with respect to CIR/CBR and typical network QoS metrics?

    Perhaps I should ask the question this way..   I'm willing to pay for performance QoS.. can you offer an SLA on QoS if I'm willing to pay for it? ping me offline if that is appropriate..

    The direct question is.. short of our the business to Enterprise, can you do anything for this type of application I have with the vcloud offering?

    Cheers!

  • Hi,

    I have forwarded the request to the appropriate team. This information is not something support has access to.  Thank you for your patience and for you post.

    Regards,

    Derrick Nguyen

    Terremark Cloud Support

  • Since you are not rate limiting, could it be possible that there are some routing issues? We've been doing some intracloud testing to/from terremark vcloud to about 35 other data centers. Of those 35, one of the faster connections we get is between terremark and rackspace cloud and terremar and liquidweb. However, there is a very significant different between inbound and outbound throughput. If I pull a 100 MB file off the rackspace server, I'm able to achieve 75-90 Mb/s (inbound to terremark). However, if I pull the 100 MB file from terremark to the rackspace server (outbound from terremark), I'm only able to achieve 14 Mb/s. In running a traceroute, I see that terremark routes through limelight (llnw.net) to get to rackspace, but rackspace routes through level3 to get to terremark. I get the same results for a server with liquidweb which shows about a the same difference in outbound vs inbound throughput. terremark to liquidweb goes out through alter.net and sprint, while liquidweb pulling from terremark goes through level3. The highest outbound throughput we were able to achieve from terremark to all of the 35 other data centers was consistently never higher than about 14 Mb/s. However, if I push the file to those same servers, I'm able to see much higher throughput. Thus, could the problem be related to routing through level3 which seems to be capped at about 15 Mb/s when there are obviously more optimal routes that could be taken?

    Here are a couple sample traceroutes:

    Terremark to Liquid Web:

    1  10.112.4.1 (10.112.4.1)  2.034 ms  1.616 ms  1.603 ms

    2  204.51.96.3 (204.51.96.3)  1.873 ms  1.797 ms  1.686 ms

    3   (72.46.239.97)  1.944 ms  1.446 ms  1.966 ms

    4  66.165.170.11 (66.165.170.11)  87.343 ms  87.292 ms  87.228 ms

    5  t0-0-0-5.br1.mia.terremark.net (66.165.161.89)  1.938 ms  1.933 ms  1.625 ms

    6  GigabitEthernet2-0-0.GW9.MIA4.ALTER.NET (63.65.188.29)  37.404 ms  37.026 ms  37.309 ms

    7  0.ge-1-3-0.XL3.MIA4.ALTER.NET (152.63.81.166)  43.842 ms  43.999 ms  43.704 ms

    8  0.so-6-0-0.XT1.ATL4.ALTER.NET (152.63.80.37)  44.373 ms  44.202 ms  44.051 ms

    9  0.so-6-0-0.BR1.ATL4.ALTER.NET (152.63.86.169)  38.200 ms  38.378 ms  38.027 ms

    10  sl-st21-atl-6-3.sprintlink.net (144.232.18.77)  38.071 ms  38.266 ms *

    11  sl-crs2-atl-0-4-0-0.sprintlink.net (144.232.18.134)  19.369 ms  19.350 ms  19.150 ms

    12  sl-crs2-atl-0-0-0-0.sprintlink.net (144.232.20.52)  35.147 ms  35.088 ms  35.063 ms

    13  sl-st20-chi-4-0-0.sprintlink.net (144.232.18.153)  33.735 ms sl-st20-chi-0-0-0.sprintlink.net (144.232.19.144)  33.862 ms sl-st20-chi-13-0-0.sprintlink.net (144.232.20.3)  33.732 ms

    14  63.167.95.70 (63.167.95.70)  50.961 ms  50.921 ms  50.866 ms

    15  209.59.157.44 (209.59.157.44)  40.023 ms  40.036 ms  40.017 ms

    16  lw-dc3-core5-ge1-13.rtr.liquidweb.com (209.59.157.105)  40.812 ms  40.941 ms  40.692 ms

    17  lw-dc3-dist8-po5.rtr.liquidweb.com (69.167.128.133)  41.065 ms  41.259 ms  40.949 ms

    Liquid Web to Terremark:

    1  69.167.132.2 (69.167.132.2)  0.909 ms  0.954 ms  0.922 ms

    2  lw-dc3-core5-po7.rtr.liquidweb.com (69.167.128.128)  0.903 ms  0.883 ms  0.875 ms

    3  lw-dc2-core3-ge2-15.rtr.liquidweb.com (209.59.157.104)  0.468 ms  0.544 ms  0.573 ms

    4  lw-eqx-border5-te1-2.rtr.liquidweb.com (209.59.157.225)  9.814 ms  9.850 ms  9.838 ms

    5  64.209.88.185 (64.209.88.185)  6.581 ms  6.632 ms  6.666 ms

    6  67.17.109.117 (67.17.109.117)  6.654 ms  6.598 ms  6.639 ms

    7  te-3-3.car3.chicago1.level3.net (4.68.110.193)  6.580 ms  6.572 ms  6.607 ms

    8  ae-32-56.ebr2.Chicago1.Level3.net (4.68.101.190)  14.869 ms  14.833 ms  14.818 ms

    9  ae-3-3.ebr2.Atlanta2.Level3.net (4.69.132.74)  42.850 ms  42.825 ms  42.797 ms

    10  ae-2-2.ebr2.Miami1.Level3.net (4.69.140.141)  53.128 ms  53.101 ms  52.874 ms

    11  ae-24-52.car4.Miami1.Level3.net (4.69.138.101)  41.508 ms  42.276 ms  42.383 ms

    12  DATA-RETURN.car4.Miami1.Level3.net (4.71.212.66)  39.256 ms  39.169 ms  39.221 ms

    13  t9-1.gw1.mia.terremark.net (66.165.161.94)  38.918 ms  39.381 ms  39.435 ms

    14  66.165.170.13 (66.165.170.13)  38.970 ms  39.670 ms  39.342 ms

    15   (72.46.239.98)  39.458 ms  39.577 ms  39.155 ms

    16  204.51.96.122 (204.51.96.122)  38.903 ms  39.251 ms  39.147 ms

Page 3 of 5 (74 items) 12345