Is Optimization The End Game?

Dare I say it?

Ok, I’ll whisper it quietly; there is something beyond optimization and acceleration!

For a long time Riverbed have been telling the World about how to optimize, accelerate, monitor and report on network application traffic. But optimization and acceleration have a finite limit; there is only so much performance you can squeeze out of your infrastructure. At some point you are still going to reach a weight of traffic that the infrastructure is unable to cope with.

It is likely that the first things to suffer will be the web or application servers themselves, especially once you have removed the latency and bandwidth restrictions from your WAN using Steelheads. So what then? Do you just have to accept that this is the case, or deploy more resource to just sit there waiting for these high tides of traffic?

Obviously the answer is no, or I would not have started writing this blog post!

What’s required is intelligence in the data centre, to monitor application responsiveness and to take appropriate action when the performance hits thresholds set by the business. But what is the appropriate action? This is entirely down to the nature of the organisation and the applications that are critical to it.

A couple of examples:

Your organisation has an internal CRM; it is used to handle all the day to day admin required for your customers and also the payments application. On occasion at the end of a month, or more importantly at the end of a quarter, the application slows down to a nearly unusable state. The business requirement here is to ensure that the users inputting orders always get a good service, even at the cost of the users making administrative changes. Remembering that this may be the same user at different times.

Enter Stingray Traffic Manager; deployed in front of the application servers it is able in the first instance to have some impact in improving the performance. Secondly using a feature called Service Level Monitoring, it is able to recognise when the servers start responding slower than the acceptable threshold configured by the business. At this point the choices are numerous, in this particular instance it was decided that the best solution would be to start applying a Request Rate Shape to the less important traffic, slowing down the number of requests per second going into the admin side of the CRM. This frees resource on the application servers, that is then used by the payments service, increasing responsiveness.

When the high tide starts to subside, the service level monitor also picks this up and stops enforcing the rate shape on the admin users. From this point on everyone is getting a good level of service from the CRM.

The second example is of an online, Internet facing business, an eCommerce type site or similar (banking, gaming etc. they all have very familiar issues and requirements).

In this example all users are accessing the same application, and it is the type of user rather than what they are using that we wish to use as the key to our differentiation. Let’s imagine it is November and traffic to the site is starting to build up, with the peak periods causing slow-downs to the service. The business decides that they want to do two things: differentiate between people who have visited the site regularly (and spending lots of money), and those who have visited infrequently or not at all. They also want to make a decision based on what the user has in their shopping cart, which trumps whether they are a frequent visitor or not!

Here comes Stingray Traffic Manager again…

In this case we can use the same Service Level Monitor feature, to be the trigger, and also use Request Rate Shaping to apply different rates to the customers (based on a cookie placed in their browser from previous visits). But we will also have a dedicated “auto-scaling pool” of servers available for those customers who look to be spending big with us today!

Auto-Scaling is a feature that allows the Traffic Manager to communicate with a back end provisioning system and arrange for more resource to be made available. Hitting the API on vCenter, for example, and getting a second (or third, or forth etc.) web server spun up in our ESX Cluster to then spread the traffic load across.

In this case we will have two pools of servers for the Traffic Manager to select from, a normal pool with three servers in it, and an auto-scaling pool that starts with zero servers in it. When the normal pool is coping with traffic happily then all users connections are directed and load balanced to these servers. When it starts to get busy, then users with larger values in their shopping carts are directed to the “auto-scaling pool” that immediately starts to add servers as they are required (either to a limited number or until the service hits an acceptable level of responsiveness). In the other pool of servers because there is a limited number of nodes available, a rate shape is applied based on whether you are a Band A customer or a Band B, C or D. Band A being those customers with a cookie indicating they have spent over $200 this year, Band D meaning the customer has no cookie (i.e. likely to be a new user), and the B and C bands being lesser spending customers.

In both of these examples the result is that traffic/users/applications are prioritised and handled in the way the business requires them to be, based on the present circumstances and the logic put in place. This is the key to TrafficScript (the scripting language at the heart of the Stingray Traffic Manager) it is a tool to turn a business requirement, into the reality of what happens to the application traffic on the network.

So to answer the initial question, no, optimizations and acceleration are not the end game; they are extremely important steps on the way to the end game. But what is really needed is a holistic approach that uses the resource available to it’s furthest limit, but when that resource limit is reached business logic is applied to ensure the organisations objectives are reached.

---

 [NB1]What’s the proper byline we use here?