The Riverbed Blog

Is Cisco still trying to eat their own dogfood? I thought they had given up...

Cisco first began selling WAN optimization in 2004.  Over the 5+ years that they have been aggressively promoting and selling the WAFS/WAAS product to their loyal customers, Cisco has been unwilling or unable to use WAAS themselves, in their own network.

The discrepancy between Cisco's words and actions must have been uncomfortable, because in 2008 Cisco finally made formal plans to deploy WAAS in their own network infrastructure.  These plans were documented in a WAAS progress report posted on Cisco's website in 2008.  An excerpt from page 6 of this document states the following:

Cisco has approximately 300 global offices that have less than 45 Mbps bandwidth.  Cisco partner IBM will deploy Cisco WAAS in these offices during the remainder of fiscal year 2008 and part of fiscal year 2009.

2008 and 2009 passed, and there was still no sign of an operational WAAS deployment in Cisco's own network.  I wrote my first dogfood blog in June 2009, wondering how they could sell WAAS without using it themselves.  Larry Chaffin similarly asked in his Network World blog ("Could the Rumors be true about Cisco?  They don't use WAAS in their own network?").  No response from Cisco, except for a few muddled references to "pilot testing" in some of their branch office sites.  Cisco even refused to comment on the progress of their WAAS deployment when confronted at their Cisco Live! conference.  The internal WAAS deployment effort apparently was not going according to plan.

At this point, I thought Cisco had given up on their WAAS deployment, just as many of Cisco's former WAAS customers have.  But unlike former WAAS customers, Cisco's IT department is probably not allowed to turn to Riverbed.  Then I saw a new blog, dated 25 January 2010, on Cisco's website:

http://blogs.cisco.com/ciscoit/comments/cisco_internal_waas_implementation/

The internal Cisco WAAS implementation continues to  expand broadly . So far IT has deployed WAAS to 200 Cisco remote offices worldwide.  When fully deployed over the next six months, we will have implemented WAAS in approximately 300 Cisco offices and eight data centers worldwide.

Here we go again.  Years after Cisco's previous aborted attempts to deploy WAAS, it seems Cisco is going to make another concerted effort to deploy it.  But Cisco's blog also raises even more unanswered questions, such as:

1)  The blog states that 200 WAAS devices have been deployed in Cisco's branch offices, but none seem to be deployed in Cisco's data centers.  What good is WAAS if they aren't optimizing network traffic to the data centers, where the majority of their application servers are located?  (and of course, the data centers are where the WAAS per-peer data store experiences severe scaling problems...) Are these "deployed" WAAS devices doing little other than optimizing a small amount of regional traffic?

2)  If Cisco's branch offices have less than 45Mbps of WAN bandwidth as documented in Cisco's 2008 WAAS progress report, then why are they primarily deploying the WAE-674 to these offices?  The WAE-674 is rated by Cisco for 90Mbps of WAN bandwidth, so why isn't Cisco using more of the smaller and more cost-effective WAVE models?  Does Cisco lack confidence in the performance specs published in their own WAAS data sheets?  Is Cisco internally aware of a WAAS performance or scaling issue that isn't reflected in their product documentation?

3)  Cisco's blog boasts about their hopes of saving $850,000 over 3 years from their 300-site WAAS deployment, as a result of consolidating servers and storage.  $850K over 3 years???  That's hardly impressive when you consider how Riverbed customer Lantmannen expects to save $60 million over five years after deploying the Steelhead solution to 300 sites.  Furthermore, note that a single WAE-674 appliance with the WoW virtual blade enabled costs $28,500 at list price.  300 of these deployed to Cisco's branch offices amount to $8.55 million.  Even at a 50% discount, Cisco's branch deployment would amount to spending $4.275 million dollars in order to save $850,000 over three years.  After adding a few additional million dollars to the bill for the data center WAAS devices, you end up with a hugely negative ROI.

Clearly, Cisco is not attempting to deploy WAAS for the ROI, which is hugely negative (even after Cisco's 85% discount for internal transfer pricing), but to convince their concerned customers that they can eat their own dogfood.  We wish Cisco luck on their renewed WAAS deployment efforts; perhaps they will get farther than their previous attempts. But of course, if this upcoming deployment effort doesn't work either, they can always write another new blog about planning to deploy WAAS in 2012...

Will Cisco ever catch up to Riverbed?

It’s no secret that WAAS lacks many features and capabilities offered by the market-leading Riverbed product.  But Cisco continuously justifies the purchase of their less-capable WAAS product by promising to customers that the next software release will resolve all of WAAS's problems and bugs, and finally be the "Riverbed killer."  And this reasoning sometimes works; many customers have suffered with WAAS--often for years--because of the expectation that "Cisco will catch up to Riverbed.” But now times are changing--many are starting to wonder if Cisco can ever execute on these promises.

 

Cisco's promises to deliver a "Riverbed killer" now date back some five years; WAAS is not a new product.  Rather, WAAS has a history that dates back to March of 2000, almost ten years ago when it was originally created as a file caching device called ActaStor.  That product was purchased by Cisco in 2004, and eventually re-named WAAS.  Through the five-year timeframe that Cisco has owned and developed WAAS, the product has been subjected to a rushed and chaotic development environment.  There was intense pressure on Cisco’s developers to quickly introduce the new features needed to “catch up to Riverbed,” and this naturally spawned numerous bugs and other software problems that have plagued WAAS, and caused a number of serious difficulties for Cisco’s early WAAS customers.

 

As software professionals know, an emphasis on quick feature releases and short-term fixes eventually undercuts itself as the overall product becomes hard to understand, modify, and support. Today’s WAAS product appears to be afflicted with extremely complex underlying software, primarily resulting from WAAS’s rushed development environment.   Due to its complexity, in recent years a pattern developed—every time Cisco released new WAAS features and capabilities, there would be a deluge of newly created bugs in that software release.  This was the case with a number of WAAS 4.0 releases, as well as the more recent WAAS 4.1.1 and 4.1.3 releases.  Even Cisco’s own product documentation reveals that WAAS 4.1.3 was delivered to customers with more known bugs than any previous version of WAAS software.

 

Bug problems in Cisco WAAS have become such an issue that the most recent WAAS software release—version 4.1.5—introduces no major new capabilities beyond a number of minor features that are essentially bug fixes.  It seems that WAAS release 4.1.5 is purely devoted to fixing existing problems found in the earlier WAAS software releases.  It has now been nearly a year since Cisco last released a major new feature for WAAS--SSL optimization--and that feature was delivered several months late.  While Riverbed was adding a raft of new features and capabilities in latest RiOS 5.5 and 6.0 software releases, Cisco has had to effectively suspend new WAAS feature development for the past year to focus on bug fixes. 

 

If historic patterns from the past five years of WAAS product development continue, WAAS customers can expect two things from Cisco’s future WAAS development efforts:  1) a deluge of bugs with any new WAAS software release containing major new features, and 2) long waiting periods to resolve those bugs before the new software features can be used in a production environment.

No one can tell the future; perhaps Cisco’s claims of catching up to Riverbed may still come true.  However, given the complexities that seem to exist deeply within the WAAS product, I believe that outcome is unlikely.  While individual bugs can be fixed, the underlying conditions that cause bugs in the WAAS product are much more difficult to address.  These are fundamental, long-term problems that have evolved over years, which possibly can only be fixed by starting over from scratch, through a compete re-write of the WAAS software.  Not only is there a wide gap in existing product capabilities, but WAAS’s underlying software is now so complex that adding new features and capabilities takes significantly more time, effort and development resources.  As a result, many observers have come to the conclusion that the most likely future scenario is that Cisco’s WAAS will continue to fall further behind Riverbed in product capabilities.

WAN optimization, Chinese Food, and Virtualized WOCs

 
 
Selecting a WAN optimization vendor is a lot like choosing a Chinese restaurant to eat at.  We all have certain expectations about Chinese food; we know what it's supposed to taste like, and we all expect to enjoy the food that we order.  Every Chinese restaurant will claim to meet these expectations and deliver the best quality Chinese food.  But unfortunately, most of us have had at least a few disappointing experiences where the restaurant fails to deliver the quality and taste that we expect.  For myself, I am particularly wary of Chinese restaurants that advertise an "all-you-can-eat" lunch special for $4.99.

In a similar way when enterprises look for a WAN optimization solution, they have certain expectations around obtaining LAN-like performance over their WAN, 90% data reduction, 6-month ROI, and being able to centralize their application servers into their data centers.  Most WAN optimization vendors claim to deliver the same capabilities as the market-leading Riverbed solution, but for a cheaper price.  But the unfortunate reality is that cheaper WAN optimization products usually fail to meet these expectations, especially for larger deployments.  As with Chinese food, just about the only reliable way to make the right choice to get a reference from someone who has deployed that solution before you.  All too many of us have found out the hard way that the advertised $4.99 lunch special does not consistently lead to good or even adequate-quality Chinese food.

The San Francisco Bay Area where I live is crowded with Chinese restaurants; it's a very competitive market.  To differentiate themselves, some Chinese restaurants in my area offer delivery service.  You don't need to physically go to the restaurant to get their product--the food will be virtually delivered to you after you order it online or over the phone.  Would you order from a restaurant that offers a virtual delivery service?  Of course, that decision would depend on the quality of the product that they offer.  If I had a bad experience with a specific restaurant while dining-in, then it certainly wouldn't matter to me if they offered delivery service or not...there's no way I would order food from that restaurant again with or without the delivery service.

In a similar way, there has been some discussion about the benefits and convenience of virtualized WOC products, including in a Network World article by Jim Metzler.  Jim makes a good point about the advantages of having the vendor deliver the product to you in a software-only virtual package...you don't have to physically take delivery of WOC hardware in order to obtain the product and its benefits.  Riverbed understands the advantages of a virtual software WOC that runs on any brand of server hardware, and has already announced the future availability of a virtualized Steelhead product.

But a key point missing from Jim's article is that any virtualized WOC is only as good as the underlying product and technology.  If the original physical WOC product exhibits issues and problems when running in your network environment, then why would the outcome be any different when using the virtualized software version of that same product?  Just as I would not order using the deliver service from a Chinese restaurant where the food does not fit my pallet, enterprises should not purchase virtualized WOC products from vendors where the original physical WOC doesn't work for their environment.

The ABCD's of WAN Optimization

What is the first thing you think of when the term "WAN optimization" is discussed?  Believe it or not, the majority of IT professionals continue to associate WAN optimization technology solely to environments where bandwidth challenges exist.  While optimizing bandwidth is indeed one of the key value propositions of WAN optimization, there are a number of additional core value propositions that in many cases present an even higher value than bandwidth savings alone.

What are these areas that WAN optimization impacts?  Instead of going down the confusing path of going over the specific areas in detail, I'll try to make it as simple as ABC or in this case, ABCD.  A for Application Acceleration, B for Bandwidth Optimization, C for Consolidation, and D for Disaster Recovery.

Application Acceleration

Let's face it.  Most business applications have been developed with the local area network in mind.  Applications ranging from Exchange to Lotus Notes to web-based applications like SharePoint, SAP, or Oracle all perform well when clients access servers over a low latency, high bandwidth LAN.  The challenge is that as soon as you extend these applications to the WAN or increase the distance between the client and the server, application performance is poor and in some cases, up to 100 times slower.  Operations that took seconds now take minutes.  The ability to provide LAN-like performance for applications that branch office and mobile workers rely on is arguably the cornerstone value proposition of WAN optimization.   

Bandwidth Optimization

As I mentioned previously, overcoming bandwidth challenges continues to be one of the key reasons to deploy WAN optimization.  Contrary to popular belief, bandwidth is not free and in fact can be very expensive.  WAN optimization essentially eliminates all the data redundancy and the result is that between 65% to 95% of traffic is eliminated from WANs.  The result is that congested links become uncongested, smaller links perform as if they were bigger, and the need to upgrade bandwidth is deferred or eliminated altogether.  Very simple to understand ROI.

Consolidation / Cloud Computing

We have seen the pendulum swing from a centralized IT infrastructure during the 70s and 80's with central mainframes to a massive build out of a distributed IT infrastructure during the 90's where 6 million plus branch offices have been equipped with local servers.  Over the past decade or so there has been a big move back to the centralized, consolidated model where organizations want to simplify their IT and cut costs by consolidating expensive to manage branch office equipment to central data centers.  Relatively recently, we are also seeing an increased interest in moving from this private cloud data center centralized environment to a public cloud environment where infrastructure is hosted by 3rd party vendors like Amazon, Microsoft, etc.  While consolidation obviously makes sense, the challenge is performance.  When you move your file servers, mail servers, and app servers thousands of miles away from your users, performance takes a beating.  WAN optimization becomes vitamin or or the pain killer for consolidation projects.  As a vitamin, WAN optimization is the essential enabler for projects where consolidation has yet to take place.  Replace branch office servers with a WAN optimization device and users in most cases won't notice a difference, even though their server is now removed from their LAN to a data center that is thousands of miles away.  WAN optimization can also be the pain killer for organizations that have already consolidated, but have performance issues.  WAN optimization is essential for the success of any consolidation project.

Disaster Recovery

One of the often misunderstood values of WAN optimization is the impact on disaster recovery and business continuity.  The value is quite simple to understand.  WAN optimization dramatically reduces the time it takes to perform data backup operations either between the branch office and the data center or between primary and backup data centers.   For example, SnapMirror operations are in many cases 30 to 50 times faster.  Operations that take 30 hours to replicate terabytes of data can be reduced to just a couple of hours.  The impact here is that organizations that are having a hard time meeting their RTO (Recovery Time Objective) and RPO (Recovery Point Objective) need WAN optimization.  Shorter backup and recovery times combined with the fact that you can perform backup and replication operations more often, results in a much better RTO and RPO.

There you have it.  Understanding the true value of WAN Optimization is as easy as ABCD.  Thanks to Andy Rogerson at Riverbed for coming up with this clever methodology for remembering how WAN optimization impacts IT initiatives.  I would also like to add that not all WAN optimization products are the same.  Riverbed is widely recognized as the market leader with a best of breed WAN optimization suite that has been deployed by more than 7000 customers to address application acceleration, bandwidth optimization, consolidation, and disaster recovery initiatives.

Cloud security concerns

News of China's reported attempts to hack Google email accounts highlights the security-related risks and exposures of cloud computing.  Google's own perspectives on their experiences can be found here on Google SVP David Drummond's blog:

http://googleblog.blogspot.com/2010/01/new-approach-to-china.html

Though Google's complaints about international cyber-attacks are currently front-page news, the reality is that thousands of other similar attacks on public computing infrastructures occur on a daily basis.

Certainly, computer security has always been an issue since the advent of the transistor.  But as computing infrastructures have grown increasingly interconnected through networking technologies, best practices and enabling technologies have always emerged to contain the associated security risks.  In this sense cloud computing is only the latest stage in the ongoing internetworking and consolidation of disparate computing infrastructures.  Accordingly, the advent of a new generation of security challenges should not be a surprise.  It would be a mistake to simply dismiss cloud computing opportunities because of the potential for new security risks. 

Large numbers of Riverbed customers have been able to realize a private cloud computing infrastructure by using Steelhead WAN optimization to consolidate their IT infrastructure into a smaller number of data centers.  But the case of these Riverbed customers, most believe that their security has improved as a result of their consolidation practices, because their data is now centralized and consolidated into a smaller number of hardened, secure data center facilities, rather than be scattered throughout various regional or branch offices each with varying security standards.

A private cloud infrastructure only hosts data in private data center facilities that are reliably protected by established firewall, VPN, IDS/IPS, and other security technologies and best practices.  For those enterprises that view the public cloud as in their future, the adoption of private cloud infrastructures in the interim can smooth the transition to and adoption of the public cloud.  For those enterprises where security concerns and business practices don't make the public cloud feasible, a private cloud will deliver many of the efficiencies and benefits of a shared IT infrastructure without many of the security risks.