Latency, a Performance Metric you can never ignore!

One of the least thought about Performance Metric for an application is Latency. It "is a time delay between the moment something is initiated, and the moment one of its effects begins or becomes detectable".

Bing and Google Agree: Slow Pages Lose Users. Another interesting stat is that Amazon found every 100ms of latency cost them 1% in sales. A broker could lose $4 million in revenues per millisecond if their electronic trading platform is 5 milliseconds behind the competition.

I think it's a crime to loose the sales 1% when there are many proven techniques to bring down the latency. Let us see how?

While you can write a full book on sources & strategies to reduce latency, this is my humble attempt to simplify the list
  • Hardware Level Latency - Things to consider includes OS, Processors, Memory, Local Storage I/O & Network I/O such as SAN/NFS.
  • Transmission Infrastructure Level Latency - Another hurdle to reduce latency is the internet infrastructure itself like DNS, Network links, Router Infrastructure, TCP Overheads - in other wors the existing World Wide Web itself is a bottleneck and the bulk of the latency is with the information round trip between the browser and the server. In other words, the lesser round trips the better.
  • Software Application Level Latency - latency from the UI scripts, Other Encrytpion/Decryption Stack, Serialization Stack, Image Processing, Bugs/Exceptions, etc.
  • Software Dependencies - internal messaging system latency, the file system, the database, frameworks(while they add to developer productivity, they could add performance overheads), Web Servers(Ex: IIS, Apache), Proxy/Cache(Ex: Squid, MemCache), VPN, Firewalls, SSL, External Services(Ex: Payment Gateways).
This list is in no way exhaustive and depends on the applicaiton in question.

If countering Latency were a process it will be an infinite loop; meaning latency has to be countered at each and every step and it lasts for the lifetime of that applicaiton.

A brief list to counter the latency are:
  • Keepup with Moore's law, Get a better hardware! Well this is not always feasible and is only true upto a limit, meaning good hardware will not go the full way.
  • Reduce round trips to counter the transmission latency.
  • Scale-out as early as possible. Use Active/Active approach, which allows all the servers to serve the users in parallel, rather the other way round with Active/Passive Approach. Scale-out early in the applicaiton evalution will mean that there is less costyl Re-Architectures down the road.
  • ACID is dead, Long live ACID properties! CAP Theoram rejects the pillar of old style DB design. It states that all four properties - atomicity, consistency, isolation & durability(ACID) cannot be had simultanouesly with distributed asyncronous systems that is the Web. But what we could do is to to design systems that are inherently partitioned and loosely coupled which uses eventual consistency. With eventual consistency you can make an update to one partition and return. You don't have to coordinate a transaction across multiple database servers, which makes a system have a higher and more variable latency.
  • Use Pareto principle, and cache more than 80% of the user requests. This while improving latency will reduce the overall costs! Ex: Reverse Proxy, MemCached.
  • Use Network Latency beating Stradegies like TOE, FastTCP, SSL Offloading, Optimize Firewalls, Use CDNs, etc.
To say that I have over simlified the latency problem with this blog post is an understatement. What I have done though isto provide a sign post which shows you the way for a better performance.

Did this post benefit you? Do you want us to write on anything specific?

Bookmark/Share this post with:
Bookmark and Share


Post a Comment