So easy to deploy that the hardest part is opening the packaging

Warning:  Customer feedback seems to indicate that the hardest part of configuring these Steelheads could be in opening the packaging material!

I recently had an opportunity to host two Riverbed Users Group Meetings (RVUG) on the East Coast.  During these meetings, Riverbed customers have an opportunity to hear about the newest features and capabilities available in Riverbed products.  But for me personally, the most rewarding part of the RVUG is the round-table discussion, which is an opportunity for each customer to share about their experiences in using Riverbed in their network, including likes, dislikes, and ideas for new features that they would like Riverbed to incorporate.

It was during one of these round-table discussions that one customer gave me a serious and angry look, and complained, "It took 15 minutes for us to find a knife to open the box, but only 5 minutes to configure and install the Steelhead...the Riverbed packaging was too difficult to open!"  I stood in stunned silence for a few seconds before I realized that the customer was making a wisecrack, and then the rest of the room erupted in laughter.

Ease of deployment is something that Riverbed's customers consistently rave about.  It's the first and most obvious thing you notice about the Steelhead product when you use it for the first time.  The Steelhead configuration process basically consists of two steps:  1)  give the Steelhead an IP address, and 2) plug it into the network.  Most customers are able to deploy Steelheads--even in relatively complex network configurations--without looking at any Riverbed documentation.  The configuration process for configuring and deploying each individual Steelhead does not become more complex as the size of the network increases--configuring each Steelhead for use in a small network with five sites is essentially the same process as for a larger network with 100 or 1000 sites.  In most cases Riverbed customers are able to deploy their Steelheads without attending formal training classes, or paying professional services costs.  Riverbed's ease-of-use is particularly surprising for new Riverbed customers who have struggled in the past with Cisco WAAS and other competitive solutions.

Riverbed's competitors go to incredible--even eccentric--lengths in trying convince customers that their products are just as easy to use as Riverbed.  Cisco even created a video specifically attempting to dispel entrenched industry-wide perceptions that WAAS is difficult to use.  The video talks about the WAAS quick-start configuration wizard, and implies that the quick-start tool resolves all of the complexities and configuration difficulties that Cisco's customers have ever had with WAAS.  Similarly, Blue Coat recently issued a press release declaring that their new SGOS 5.4 software now has an "intelligent configuration wizard," and they followed Cisco's example by creating their own ease-of-use video.

How realistic are these ease-of-use claims by Riverbed's competitors?  Certainly, any device can be easy to configure and use in an isolated lab environment.  But complexity with Cisco and Blue Coat's WAN optimization products will inevitably surface when these products are exposed to typical issues found in a real-life network environment.  Though these vendors would have you believe otherwise, the reality is that their easy-to-use configuration wizards will provide very little help in addressing asymmetric routing, scalability, reporting, and proper handling of each of the many different types of applications found in a live production enterprise network.

For example, in a larger network environment, Cisco will urge that you use WCCP for traffic re-direction, because their WAAS boxes can't handle high traffic volumes in an in-path/in-line configuration.  A WCCP-based configuration in a large network environment adds significant amounts of complexity, and a huge amount of configuration work due to the number of ACL entries that will be needed.  In the case of Blue Coat, their new "intelligent configuration wizard" does nothing to help the administrator configure the numerous different knobs and settings for each of the numerous Object Caches in the ProxySG product.  Rather, the Blue Coat customer is going to have to wade through the voluminous 3000-page (three thousand pages!) SGOS 5.4 configuration manual.  For comparison purposes, the Riverbed Installation and Configuration Guide for RiOS 5.5 is only 104 pages.  Interestingly, 3000 pages is about 25% longer than the previous 2400 pages for the earlier SGOS 5.3 configuration manual, which hints at new levels of complexity in Blue Coat's new recently-released SGOS 5.4 software.

Comments

[anon]

It's funny that Ms McKoin's first answer is like a politician's:

Paraphrasing:
Q: Are these perceptions [that WAAS was hard to configure] true?
A: Applications designed for the LAN run slowly over the WAN. The new WAAS code accelerates those applications across the WAN, making them feel LAN-like.

[Humbug]

Where's the proof that Cisco's inline deployment model can not handle high traffic volumes?

[Josh Tseng]

You will see the proof when you expose the WAAS device to a moderate amount of pass-through (non-optimized) traffic while it's deployed in-line. Use an IXIA tester, or just do a high-speed transfer using BBFTP through the in-line WAAS device. At somewhere between 100 to 150Mbps of traffic, the CPU goes to 100%. At about 250Mbps of pass-through traffic, the WAAS device loses its IP address, and doesn't even recover after reboot. Note I'm talking about pass-through traffic (either TCP or UDP) that is not intercepted and optimized, because when WAAS is intercepting the TCP connection, it can just close the TCP window to throttle down the traffic if the workload gets a bit uncomfortable. But when traffic is passed-through, the WAAS box can't throttle the pass-through traffic and is forced to process and forward every packet immediately. That's where WAAS gets into trouble.

Have you found it strange that Cisco is so insistent on a WCCP-based deployment for WAAS, especially on the data center side? They never tell you why, do they? Well, try doing something like the above, and you'll understand why.

[skip]

how about talking about how riverbed breaks QOS policies in the network because Riverbed uses a tunnelling architecture unlike Cisco. oh wait, I know b'cos this is a riverbed blog. can't expect an unbiased review can we :)

[Josh Tseng]

Dude, I respect your right to post here. So please respect my right to respond to your post.

Riverbed has over 6000 customer deployments and none of them have had problems with "QoS policies." For the few customers who enforce QoS policies based on IP address, we do have an optional address transparency mode that works the same way as Cisco's WAAS product does. In other words, Cisco's "transparency" feature is not unique--Riverbed offers it too.

Josh

[Mark]

I've recently done speed testing of WAAS with inline, WCCP, and PBR. Josh, what model were you testing, and when did you test it? Your experience and mine could not be more different. Given what you describe, I'm wondering whether you did the testing on an undersized box for the traffic load you threw at it.

Cisco has been more than happy to tell my customers why WCCP - it's appropriate for load balancing, high availability, and long term maintenance/manageability. Devices can easily be brought in and out of production without moving cables or even putting an inline card into bypass (guaranteed to reset some connections), something critical in some high uptime production environments. Inline is nice for many things, but WCCP was created for a reason, and its use for WAAS is quite often the best practice.

[Josh Tseng]

Hi Mark,

What I said in my blog is that WCCP is difficult to configure in a large network. I don't doubt that it works fine if configured properly, whether you're using it with Cisco WAAS or Riverbed Steelhead. The challenge is getting it configured properly (with the correct Cisco IOS version) in a large network environment.

If you have a large network with thousands of host IP addresses and different subnets, WCCP is going to be a nightmare to configure because those ACL's have to properly sort out what traffic gets re-directed to WAAS/Steelhead, and what traffic goes directly to the WAN.

Neither WAAS nor Steelhead can optimize some types of traffic such as VoIP, SSH, encrypted SMTP, etc. In addition, (unlike Riverbed) Cisco's WAAS doesn't provide much benefit for encrypted MAPI, Oracle Forms, ICA, or SMB signed CIFS traffic. There's no use in re-directing this traffic to the WAAS devices, so you have to configure the ACL's to not re-direct this traffic to them. Use white-lists or black-lists, either way the ACL's are going to be a nightmare to configure if you have hundreds or thousands of different subnets.

What you end up with in a large network is a huge static list of ACL's that is thousands of entries long. I've seen it myself, and it's seriously impossible to manage.

This is one of several reasons why many of Cisco's customers find WAAS difficult to deploy or scale out in their large networks. If you wanted to, you could struggle with the same WCCP config issues with Riverbed, but most Riverbed customers instead choose to deploy in-path/in-line, or they use the Interceptor, which is far easier to configure and manage than WCCP.

Best of luck,
Josh

[Mark]

And about the model you tested and when? Only reason I posted in the first place was the pretty big disparity between what you experienced and what I recently confirmed through careful testing. Look, Riverbed gets good numbers. But Cisco's numbers using inline, PBR, and WCCP are far better than you describe above.

About the ACL complexity you have seen yourself, again, not consistent with what I've seen and deployed. Certainly someone could design such a monstrosity, but it's hardly necessary. That's not to say you haven't seen it. Rather, just that I've seen clean WCCP deployments (that would work equally well for Riverbed, btw) that provide for best practices device management (non-disruptive, as described above) in very large networks.

I'm a fan of inline deployments, where it meets the needs of the clients. And both vendors have solid inline solutions. Unfortunately, your statement that Cisco can't handle speeds over 150Mbps using inline isn't correct (at least absent a model and version number), and so cannot be a reason for Cisco to recommend WCCP.

[Josh Tseng]

Mark,

I was confused at your first comment because I thought you were refering to what I said in the blog. But I see now that you are refering to some comments I made in response to a different poster.

The in-path failure at 100-150Mbps throughput occurs with WAAS release 4.1.1. That was the most recent WAAS release at the time I made the comment. This specific failure has been resolved in the more recent WAAS releases.

But Cisco still generally avoids in-path deployments and recommends use of WCCP because it cannot optimize asymmetrically routed traffic while deployed in-path. And as you know, asymmetrically-routed traffic is quite common in large networks.

Take the case where the incoming SYN goes through one port-pair of an in-path deployed WAAS device, and the SYN-ACK response goes through the other port-pair on that same WAAS device. That TCP connection will not be optimized because it was asymmetrically routed, even though the same WAAS device saw both flow directions of the one TCP connection. This problem forces Cisco to use WCCP for almost all of their large WAAS deployments. Riverbed doesn't have this problem, and that's why many of our customers can use up to 10 in-path interfaces to address asymmetric routing in their large networks while Steelheads are deployed in-path.

When it comes to using WCCP, the number and complexity of ACL's does not depend entirely on your WCCP skills, but on the size and complexity of the pre-existing network and where WCCP intercept will occur. Sure it can be "clean" in a small network or a branch office location where the topology is simple. But more often than not in a real-life network things aren't so clean. If WCCP intercept has to occur on an interface where you have local LAN traffic that doesn't go over the WAN, as well as a mix of traffic that can't be optimized by WAAS, well then you have a nightmare on your hands configuring those ACLs. In a large network with 10,000 or more IP's, more likely than not you're going to have that problem on your hands.

Josh