Service fabric gives you two mechanisms out of the box when resolving which partition you hit when calling a Reliable Service. We’ll ignore the singleton partitions as they won’t help us with sharding.
Int64range to decide which partition a numbered key falls in.
More information can be found here.
A named partition allows you to specify explicitly which partition you want to access at runtime. A common example is to specify A-Z named partitions and use the first letter of your data as the key. This splits your data into 26 partitions.
The advantages to this are that it is simple and you know which partition your data goes in without a lookup. Unfortunately as we will test later, you are unlikely to get a good distribution of your data across the partitions.
With a ranged partition the fabric tooling by default uses the entire
Int64 range as keys to decide which partition. It will then convert these into ranges or buckets depending on the partition count.
However to be able to lookup a partition we need a function which can reduce our data to an integer value. To use the configuration above we can convert our strings into an
Rather than use the ranges, you can fix your keys and plug in your own hash algorithm to resolve the partition.
We now have a key range limited to 0-25 rather than the entire
Int64 range. We can resolve a client connected to this partition in the same way, however this time we need to compute a key that fits in this smaller range. I’m using the jump consistent hash implementation in hydra.
To benchmark the distribution we have a list of around 17000 real email addresses. This should give us an idea of how the sharding strategies will distribute the data across 26 partitions. Another advantage of using one of the
Int64 methods is that they can be used with any amount of partitions.
We are looking for an even number of accounts allocated to each partition.
We can see from those results that sharding using the first character of an email produces wildly different partition sizes, not what we want! Both the jump hash and integer ranging methods produced very even parition sizes.
Based on these results I would use the ranged partitioning method, it produces provides good balancing and is fast to compute. An additional advantage is you do not need to know the partition count in the code, just map your data to an
Int64 and service fabric will do the rest.