The Riverbed Blog

Cisco's WAAS users visit the Riverbed Interop booth

This past week's Interop conference was an encouraging experience -- the Riverbed booth drew strong interest from attendees wanting to view our presentation and the Steelhead Mobile demonstration.  But there was another observation this week that was even more noticeable than in past Interop conferences from previous years--a startling number of Cisco WAAS customers were actively visiting the Riverbed booth to investigate the Steelhead solution.  I personally talked to more than a few WAAS users myself, and my colleagues who were also at the booth similarly reported being approached by inordinate numbers of Cisco WAAS users, including some individuals representing very large enterprises.

A common question they brought up was about how Riverbed addresses patches and CIFS protocol changes implemented by Microsoft.  The apparent context of this question was in reference to a recent Windows operating system patch -- KB980232 -- distributed by Microsoft several weeks ago to address some security vulnerabilities in the Windows CIFS client.  This patch causes subtle changes in the protocol behavior by Windows CIFS clients, resulting in problems during CIFS file save operations that are being optimized by Cisco's WAAS product.  The problem has also been reported by Network World.  But for these Cisco WAAS users, this recent difficulty was only the latest in a series of historic WAAS stability problems that they have faced. For this particular problem, Cisco has offered a fix, although that fix requires an upgrade to the newest and unproven WAAS 4.1.5f image (but no fix exists yet for users of WAAS 4.0).

The WAAS users I spoke with were intrigued to hear that Riverbed Steelhead customers have not been impacted by Microsoft's Windows patch.  The reasons for the stability of Steelhead deployments go far beyond Riverbed's strong business and technical relationship with Microsoft (indeed, Cisco claims a similar relationship with Microsoft).  Rather, the key difference in Riverbed's situation is the Steelhead's transparent TCP proxy architecture compared to the WAAS product's layer-7 caching/proxy approach used to optimize CIFS traffic.  In other words, WAAS is a CIFS proxy, while Riverbed is not (Steelheads are a TCP proxy, but not a layer-7 proxy). 

As a CIFS proxy, WAAS is responsible for acting as a proxy CIFS server when optimizing CIFS traffic from Windows workstations.  In such a role, WAAS must fully and adequately respond to each and every CIFS message from Windows clients receiving optimization services from WAAS.  If a Windows client initiates a message that WAAS doesn't expect or understand (such as due to the Microsoft patch), then its optimized CIFS session can potentially be impacted.  In other words, in order to properly deliver WAN optimization services for CIFS traffic, WAAS must essentially be as statefully aware of the layer-7 CIFS session as the original Microsoft CIFS client and server.  This stringent requirement to properly mimic the behavior of a CIFS server leaves WAAS vulnerable to behavioral changes in Windows clients that may occur as a result of Microsoft patches, such as KB980232.

In contrast, Riverbed Steelheads are not CIFS proxies, and they do not need to be as statefully aware of the CIFS session as a CIFS proxy would.  This is an important distinction that allows Riverbed to focus on optimizing only those messages and operations that exhibit chatty protocol behavior.  If the CIFS client exhibits unexpected behavior that the Steelhead does not understand or anticipate, the Steelhead can simply ignore those messages because it is not a CIFS proxy.  Rather, since the Steelhead is transparent to the CIFS session, it can let the CIFS client interact directly with the original Windows server in the data center for those unrecognized CIFS messages.  In this situation the Steelhead continues to offer data deduplication services in order to address bandwidth constraints in the WAN, but the Riverbed solution is not compelled to take any action on the unrecognized message or unexpected behavior that might jeopardize the stability of the ongoing CIFS session.

The fact that the Steelhead is not a layer-7 proxy for CIFS and other applications is one of the key reasons why Riverbed has consistently been able to scale Steelhead deployments for the largest and most complex network environments in the world.  While Riverbed products are not perfect, and they do have their share of bugs and other issues, nevertheless the overall track record and stability of the Riverbed solution when deployed in such demanding enterprise environments has been very remarkable.

Riverbed in HP Procurve

HP Product Manager demonstrates Riverbed inside an HP Procurve.

Interop in less than 2 minutes

So you didn't get a chance to make it to Interop this year?  Don't fret!  I had an opportunity to walk the show floor and here is the video.

Fired Up, Ready to Go - Live Demos at Interop Las Vegas

Seeing is believing, we are doing live demonstrations of Steelhead Mobile throughout the day at Interop Las Vegas at booth 2011, see for yourself how Riverbed can help you break the performance barrier!

Steelhead mobile accelerates mobile warriors

Naveen Prabhu demonstrates Steelhead Mobile, which is Riverbed's solution for accelerating mobile worker environments.

Steelhead appliance

Kenny Quan demonstrates Riverbed's flagship WAN optimization solution, the Steelhead appliance.

Branch Office Box

Faisal Memon demonstrates the Riverbed Services Platform, which provides the ability to run branch office services on a Steelhead appliance.

Network and Application Visibility with Cascade

Dinko Zidarich demonstrates Cascade, Riverbed's solution for providing visibility of both network and application performance

Riverbed Customer Rayonier Presenting to a Crowd

Mitch Walters, VP of IT @ Rayonier, giving a great presentation on the Interop show floor.

Video - Verizon Discusses Managed Service Offering Powered by Riverbed

Verizon Senior Product Manager Steve Capozzi discusses the recent announcement about how Verizon is offering a managed service based on Riverbed

Video - Riverbed discusses Verizon's managed service offering powered by Riverbed

Apurva Dave, VP of Product Marketing at Riverbed, discusses the details of the Verizon - Riverbed announcement.

Why does Riverbed's Universal Data Store architecture matter?

In a typical WAN optimization deployment, employees located at different remote branch offices need fast accelerated access to centrally-stored data on servers located the main data center.  When deploying WAN optimization into such an environment, Riverbed's universal data store architecture is essential for scalability and storage efficiency, especially for large deployments involving many branch offices. 

Riverbed's Universal Data Store

A universal data store maintains byte-level data in a single shared dictionary that is used by all peer WAN optimization devices.  A common file, web object, or any other data that is fetched to multiple remote sites is only represented a single time in the central WAN optimization device.  This property ensures optimal usage of the available storage capacity in the central Riverbed Steelhead device.

However, most non-Riverbed vendors use a per-peer data store architecture.  The properties of a per-peer data store are illustrated below:

Per-Peer Data Store (Cisco WAAS, Blue Coat, Juniper, most others)

A per-peer data store uses a separate dictionary to store and map byte-level data for each remote peer device.  Common files, emails, and other data that are sent to multiple remote peer devices must be stored multiple times in the central WAN optimization device (once for each peer).  This approach results in storage inefficiency in the central device in the main data center.  Consider the example of where a single email with a 100MB attachment is distributed to employees located at 30 different remote sites.  Such an email consumes at about 3GB of storage capacity in a WAN optimization device that uses a per-peer data store architecture; only 100MB of storage is consumed when using Riverbed's Steelhead solution.

Why this all matters:

When used to support networks with more than a few remote sites, a per-peer data store results in fragmentation of the data store capacity of the central WAN optimization device.  Poor performance results when the effective available data store capacity is inadequate.  A phenomenon known as "data store wrapping" occurs, where stored byte-level data is overwritten and erased prematurely in order to make available scarce storage capacity for new data actively being sent over the WAN.  The result is disk churn, where the WAN optimization device is constantly writing data to disk, and yet much of that newly written data is erased before it can be used even once.  While the WAN optimization device is constantly busy processing and writing data, nevertheless it delivers poor performance because this data store wrapping issue prematurely erases the disk-resident data needed to optimize newly-arriving data.

In small networks with simple topologies, the performance issues described above may not become apparent.  This includes in lab tests and limited-scale POC's, which many customers use to evaluate the efficacy of a given WAN optimization solution.  Unfortunately, some customers assume that the performance they observe in a limited-scale POC will be comparable to what they will obtain after full-scale deployment.  Tragically, these customers discover this problem only after months of frustration in an ultimately-failed deployment effort, when their chosen per-peer data store product did not deliver hoped-for results.

Optimization Strategies for Disaster Recovery

Steve Dixon, Managing Director of Riverbed Technology, discusses Optimisation Strategies for Disaster Recovery

Five key reasons to choose Riverbed over Cisco WAAS

When talking to prospective customers, I often get the question "why Riverbed?"  After all, over the past five years, Cisco has been offering a WAN optimization product that competes against Riverbed.  Surely, after five years you would think that they have addressed any problems or issues that might have affected early adopters, right?

Beyond getting into a detailed discussion about often obscure differences between Cisco WAAS and Riverbed Steelhead, what I find helpful in these situations is to point out some high-level verifiable observations about Cisco's WAN optimization efforts, such as:

1.  An extraordinary number of former Cisco WAAS customers have already switched to Riverbed -- it's interesting to note that Riverbed customers are extremely loyal—they gave Riverbed an average overall satisfaction rating of 8.41 (out of 10) in Riverbed’s most recent semi-annual customer satisfaction survey conducted in October of 2009.  However, it’s unlikely that Cisco will be conducting satisfaction surveys of their WAAS customers, because an unusually-large number of them have removed their WAAS devices, and are now Riverbed customers.   Many of these former WAAS customers have recounted to us how they had struggled with WAAS for years before finally switching to Riverbed; others are relatively recent WAAS customers who gave up after a few months when they were not able to stabilize their WAAS deployments.

2.  WAAS product development seems to be slowing or stagnating – Only a short while ago, Cisco would aggressively make bold claims about how the next upcoming WAAS software release will “catch up to Riverbed.”  But now, they don’t seem to be making those claims anymore; rather, Cisco’s development of their WAAS product seems to have stagnated. In fact, it has now been one year since Cisco last introduced a major new feature or capability for their WAAS product (SSL optimization, which was released several months late in 4.1.3).  Strangely, the most recent WAAS 4.1.5 software release introduced no new major features or capabilities except for a number of bug fixes; 4.1.5 appears to be primarily aimed at correcting and fixing bugs that afflicted earlier WAAS releases.  In that same period of time, Riverbed has released two major software versions in the past year (RiOS 5.5 and 6.0), each of which offer a number of significant enhancements and capabilities to the Steelhead solution.

3.  Cisco WAAS product capabilities continue to lag far behind Riverbed – WAAS is missing a number of key features available in the Riverbed product, including optimization of encrypted Exchange (MAPI), SMB-signed CIFS traffic, Macintosh CIFS, Citrix ICA, Lotus Notes, Microsoft SQL, Oracle Forms, and layer-7 HTTP acceleration.  WAAS also lacks Riverbed’s capabilities to prepopulate Exchange data, synchronize data stores, and address asymmetric routed traffic while deployed in-path.  Finally, Cisco lacks a high-end WAAS appliance with solid-state disks (SSD’s) comparable to Riverbed’s Steelhead 7050, and a WAAS-aware load balancer equivalent to Riverbed’s Interceptor for scaling deployments in very large networks.

4.  WAAS Mobile is not really Cisco product – Rather, WAAS Mobile is just a rebranded software product from ViaSat that is re-sold by Cisco.  This makes WAAS Mobile a very unusual product within Cisco’s product line, since Cisco does not have a strong history of reselling products from other companies (rather, Cisco is known for purchasing companies and products).  Because WAAS Mobile is not a Cisco product, the future strategic direction of WAAS Mobile is unclear.  This is an important issue for customers with mobile software requirements in WAN optimization, because ViaSat has been shifting development and marketing resources away from WAAS Mobile (including employee layoffs).

5.  WAAS data store fragmentation remains a serious scalability and performance issue – A single file accessed by 10 different branch offices must be stored 10 separate times in the central WAAS device.  In a comparable deployment using Riverbed, the central Steelhead only stores that file only once.  In other words, WAAS will consume its DRE storage capacity 10 times faster than Riverbed; with 100 branch offices, WAAS consumes storage 100 times faster than Riverbed.  We are aware of WAAS deployments where the data store “wraps” every few hours, meaning that much of the byte-level data that is written to the disk drives of the WAAS device is purged before it can be useful.  This leads to wasteful “churn” of disk drive activity from data that is repeatedly written and erased before it is used even once by WAAS.  Cisco has attempted to explain away this data store fragmentation issue as not significant, or even that their approach is superior to Riverbed’s.  That suggests Cisco has no plans to fix this problem.  In contrast, in a typical Riverbed deployment the Steelhead’s data store wraps every few months; a situation where the data store that wraps more often than once per week is reason for concern by Riverbed engineers, and a reason to upgrade the Steelhead appliance.

It’s (still) not just about bandwidth

It’s interesting to see how much has changed in the WAN (Wide Area Network) optimisation space over the last half a decade or more, yet some things haven’t changed much. Five years ago I recall a white paper had been written on the subject titled, “It’s not just about bandwidth”. At the time bandwidth reduction was a great story and the management reports within Steelhead were great at sharing how much bandwidth had been saved, but although some organisations were able to justify the business case on bandwidth alone many struggled. Saving bandwidth which gives rise to potential cost savings or as in most cases, defers bandwidth upgrades, is only part of the story and for most, a small part of the story. The real deal only happens as we speed up data across the WAN through either optimising the transport layer or better still accelerating the applications.

Now for some readers, this may not be anything new, it’s the bread and butter or simply put the basics of good solid WAN optimisation. But each week I am faced with questions from prospects & some journalists; with the advent of fibre everywhere, surely WAN optimisation struggles to add value right? Wrong!

Admittedly, if we run dark fibre from point A to point B over a relatively short distance and we connect our IT equipment at each end, with no service provider equipment in between then yes, WAN optimisation does not necessarily have a part. Bandwidth may be in plentiful supply but, more importantly, the biggest barrier to success across the WAN is latency and with dark fibre over a relative short distance the latency will be extremely low.

Today a number of service providers are upgrading their network to fibre only, offering much higher bandwidth options, in some cases up to 100Mbps and doing so cost effectively. This is great news for consumers and businesses, there’s no question about that but even with the almighty increase in bandwidth the latencies don’t go away over night. So if the issue is the speed of light, will increasing the bandwidth make the speed of light go faster? In short, no, as it still faces the same problems it did before. We can carry more traffic, hold more video conversations make more voice over IP calls and transfer larger files, but nothing goes any faster until we resolve the latency issue.

I recall a quote from Steve McCann, CTO of Riverbed Technology that was something like “The only way to overcome the speed of light is to not make the trip in the first place”. And it stands true today, more bandwidth is good, but once any bottlenecks have been overcome through either more bandwidth or a more efficient use of bandwidth the only way to make things go faster across your WAN is to optimise traffic leveraging technology such as the Riverbed optimisation system (RIOS).

So the idea that new fibre networks from service providers will provide more bandwidth is great, and I am sure technology development will improve for many, many years to come and continuingly increase in the phenomenon of more available bandwidth. But there is as much today as there was at the beginning of Riverbed and will be for many, many years to come a place for WAN optimisation technology as part of any IT infrastructure build to accelerate the data transferred across the WAN.

What do you think?