Is it always necessary to evenly partition queries.

You are going to build a service that is going to handle a enormous amount of data and requests. Definitely, availability is one of the key issue you care about. Partitioning is a good way to divide data into multiple chunks and put these chunks on different nodes, so that both the read and the write load gets distributed.

But a question is, do we have the need to evenly partition all requests to different node?

The answer is not necessary. By creating a cluster could make our design more available, but there’s some price to pay. The more nodes a request goes through to complete, the more failure or waiting the request will encounter. So, if a query or request is time consuming, it is okay to distribute it into a bunch of servers to process since multiple servers could accelerate computing speed though there could be some failures. If a request takes a little time to finish, like index look up in a database system, it is not good idea to split requests. Having requests to fewer nodes if query is fast would be a better strategy.

Author: huadonghu

Leave a Reply

Your email address will not be published. Required fields are marked *