A plain english introduction to CAP Theorem
You'll often hear about the CAP theorem which specifies some kind of an upper limit when designing distributed systems. As with most of my other introduction tutorials, Lets try understanding CAP by comparing it with a real world situation.
Chapter 1: "Remembrance Inc" Your new ventrue:
Last night when your spouse appreciated you on remembering her birathday and bringing her a gift, a strange has idea strikes you. People are so bad in remembering things. And you're sooo good at it. So why not start a ventrue that will put your talent to use? The more you think about it, the more you like it. In fact you even come up with a news paper ad which explains you idea.
Remembrance Inc! - Never forget, even without remembering! Ever felt bad that you forget so much? Don't worry. Help is just a phone away! When you need to remember something, just call 555--55-REMEM and tell us what you need to remember. For eg., call us and let us know of your boss's phone number, and forget to remember it. when you need to know it back.. call back the same number[(555)--55-REMEM ] and we'll tell you what's your boss's phone number. Charges : only $0.1 per request
So, you typical phone conversation will look like this:
- Customer: Hey, Can you store my neighbor's birathday?
- You: When is it ?
- Customer: 2nd of jan
- You:(write it down aganist the customer's page in you paper note book)Stored. Call us any time for knowing you neighbor's birathday again!
- Customer: Thank you !
- You: No problem! We charged you credit card with $0.1
Chapter 2: you scale up:
You venture(企业) gets funded by YCombinator. You idea is so simple, needs nothing but a paper notebook and phone, yet so effective that is spreads like wild fire. You start getting hundres of call every day.
And there starts the problem. You see that more and more of your customers have to wait in the queue to speak to you. Most of them even hang up tired of the waiting tone. Besides when you were sick the other day and could not come to work you lost a whole day business. Not to mention all those dissatisfied customers who wanted information on that day. You decide it's time for you to scale up and bring in your wife to help you.
Your start with a simple plan:
- You and your wife both get an extension phone
- Customers still dial(555)-55-REMEM and need to remember only one number
- A pbx will route the a customers call to whoever is free and equally
Chapter3: You have you first "Bad Servie":
Two days after you implemented the new system, you get a call form you get a call from trusted cunstomr Jhon. This is how it goes:
- Jhon: Hey
- You: Glad you called “Remembrance Inc!”. What can I do for you?
- Jhon: Can you tell me when is my flight to New Delhi?
- You: Sure.. 1 sec sir (You look up your notebook) (wow! there is no entry for “flight date” in Jhon’s page)!!!!!
- You: Sir, I think there is a mistake. You never told us about your flight to delhi
- Jhon: What! I just called you guys yesterday!(cuts the call!)
How did that happen? Could Jhon be lying? You think about it for a second and the reason hits you ! Could Jhon's call yesterday reached you wife? You go to your wife's desk and check her notebook. Sure enough it's there. You tell this to your wife and she realizes the problem too.
What a terrible flaw in your distributed design! Your distributed system is not consistent! There could always be a chance that a customer updates someting which goes to either your or your wife and when the next call from the customer is routed to another person there will not be a consistent reply form Remembrance Inc!
Chapter4: You fix the Consistency problem:
Well, your competitors may ignore a bad service, but not you. You think all night in the bed when your wife is sleeping and come up with a beautiful plan in the morning. You wake up your wife and tell her.
” Darling this is what we are going to do from now”
- Whenever any one of us get a call for an update(when the customer wants us to remember something) before completing the call we tell the other person
- This way both of us note down any updates
- When there is call for search(When the customer wants information he has already stored) we don’t need to talk with the other person. Since both of us have the latest updated information in both of our note books we can just refer to it..
There is only one problem though, you say, and that is an "update" request has to involve both of us and we cannot work in parallel during that time. For eg. When you get an updat request and telling me to update too. I cannot take other calls. But that's okay becuse that most calls we get anyway are "search"(a customer updates once and asks many times). Besides, we cannot give wrong information at any cost.
"Neat" your wife says, "but there is one more flaw i this system that you haven't thougt of. What if one of us doesn't report to work on a paticular day? On the day, then, we won't be able to take 'any' update calls, because the other person cannot be updated! We will have Availability problem, i.e, for eg:if an update request comes to me I will never be able to complete that call because even though I have written the update in my note book, I can never update you. So I can never complete the call"
Chapter 5: You come up with the greatest solution Ever:
You being to realize a little bit on why distributed system might not be as easy as you thought at first. Is it that difficult to come up with a solution that could be both "Consistent and Available"? Could be difficult for others, but not for you!!! Then next morning you come up with a solution that your competitors cannpt think of in their dreams! You wake you wife up eagerly again..
"look", you tell her .. "This is what we can do to consistent and available". The plan is mostly similar to what I told you yesterday:
- Whenever any one of us get a call for an update(when the customer wants us to remember something) before completing the call, if the other person is available we tell the other person. This way both of us note down any updates
- But if the other person is not available(doesn’t report to work) we send the other person an email about the update.
- The next day when the other person comes to work after taking a day off, He first goes through all the emails, updates his note book accordingly.. before taking his first call.
Genius! You wife says! I can’t find any flaws in this systems. Let’s put it to use.. Remembrance Inc! is now both Consistent and available!
Chapter 6: Your wife gets angry:
Everyting goes well for a while. Your system is consistent. Your system works well even when one of you doesn't report to work. But what if Both of you report to work and one fo you doesn't update the other person? Remember all those days you've been waking your wife up early with your Greatest-idea-ever-bullshit? What if your wife decides to take calls but is too arnry with you adn decides not to update you for a day? Your idea totally breaks! Your idea so far is good for consistentcy and availablity but is not Partition Tolerang! You can decide to be partition tolerant by deciding not to take any calls until you patch up with your wife. Then your system will not "avaliable" during that time ...
So Let's look at CAP Theorem now. Its states that, when you are designing a distributted system you can get cannot achieve all three of Consistency, Availablity and Partition tolerance. You can pick only two of:
- Consistency: You customers, once they have updated information with you, will always get the most updated information when they call subsequently. No matter how quickly they call back
- Availability: Remembrance Inc will always be available for calls until any one of you(you and you wife)report to work even if there is a communication loss beteween you and your wife!
Bonus: Eventual Consistency with a run around clerk:
Here is another food for thought. You can have a run around clerk, who will update other's notebook when one of your's or your wife's note books is updated. The greatest benefit of this is that, he can work in background and one of your or your wife's "update" doesn't have to block, waiting for the other one to update. This is how many NoSql systems work, on node upates itself locally and a background process synchorinizes all other nodes accordingly... The only problem is that you will lose consistency of some time. For eg., a customer's call reaches your wife first and before the check has a chance to update your notebook. the cumstor' calls back and it reaches you. The he won't get a consistent reply... But that said, this is not at all a bad idea if such cases are limited. For eg. assuming a customer won't forget things so quickly that calls back in 5 minutes.
That's CAP and eventual consistency for you in simple english :)