Request Rate Shaping – applying limits with fine-grained control
Setting the scene
Imagine you’re part of a team that oversees and manages the web applications in support of a website and service. At certain times of the year, the website and service experiences an overload of visitors looking for information or wishing to purchase products or services.
Unfortunately at this point a huge volume of requests start building up which have a detrimental effect on the performance of the website and service a visitor experiences. The response times for requests become unacceptably slow and visitors get frustrated.
This situation can be made worse if you have limited scalability in the back-end infrastructure, which cannot be scaled up to meet these demands. At this point you may wish to restrict the rate at which requests are received by the back-end servers and applications.
Applying limits with fine grained control
Allow me to suggest an approach that might help you to limit these requests in an elegant way.
One approach would be to measure the performance of the infrastructure and services with Riverbed Stingray Traffic Manager's Service Level Monitoring. If Service Level Monitoring indicates that the service's response time rises above a certain threshold, that's a strong indicator that the infrastructure performance level has dropped. Taking action at this point is all very well, but how do you know which type of users of the service to apply this to? So, before any action is taken, you need to classify the users of the service in some way as to differentiate between them. You will then be able to decide which users of the service you would want to rate limit. Because of the unique way that Stingray Traffic Manager inspects client requests you will be able to determine the location, content or frequency of the request and then classify the user accordingly.
So, you might wish to classify those visitors that are simply browsing the site and are not actively purchasing products or services, e.g. they don’t have anything in their ‘shopping cart’, as low priority during periods when the infrastructure is under strain.
Once you have decided on when to apply a rate-shaping class and which users it will affect, next you apply the rate shaping class. So, when the service and infrastructure starts to become overloaded you would limit the users in the class to a particular level of service. The result of this action is that ‘high priority’ users of the service – those actively engaged in the ‘shopping cart’ process will continue to receive the same high quality of service despite the infrastructure being overloaded.
This is just one example of how Riverbed's request rate shaping might be applied in a real world situation. See Tickets, a current Riverbed customer, is an example of how request rate shaping and Trafficscript has been used to ensure its website is always online and fast, even during extreme-peaks in traffic, so valued customers receive an excellent online service. Read the full case study.
If you who would like to get a more detailed insight into our request rate shaping capabilities, the following white paper will be of interest: Traffic Valuation and Prioritization with Stingray Traffic Manager.
Comments