In the previous article, we learnt about the importance of sharding in the matter of blockchain scalability. We also learnt that the current implementation of the sharding technology had a two main drawbacks.
Security Threat
Issue of decentralization
Only after solving these two main issues, we will be able to make a scalable blockchain platform. In light of this, we propose the Dynamic Monitored and Distributed Sharding (DMDS) which allows the network to maintain its security while retaining its decentralization.
So let’s get right into it! DMDS is a concept from database partition: large databases into smaller, faster and easier managed shards. Due to space shortage issues, we divide the single Fusus (Jura’s data structure) into Fusus shards. Each shard contains different sub-Fusus which operate in parallel to achieve higher performance. There are two main reasons to do this:
1. As the number of clients in the system grows, the amount of information also increases exponentially therefore each node in Fusus will find it hard to store all the transaction histories
2. As traffic increases, the frequency of compression and new generation will increase dramatically which will lead to high timing and maintenance costs.
There are three core layers in Jura’s sharding architecture.
Router Layer
The router layer has m-routers in it. Whenever a request comes in, the router directs which shard will entertain the request. Each router contains two similar hashmaps:
Illustration of the Router Layer
The special client sharding is the key of special client shard and its value is real shard address. When a request combines in, the validator checks whether the key in the map or not. If it’s in the map, the request is routed to the shard automatically otherwise request is passed to a general client shard map. The key of general client shard map is shard index, value is the real shard address:
shardIndex = hash(sharding key) * mod(the num of shards)
Sharding Key helps to identify which shard the transaction belongs to. For example: “transaction_id”. The only issue with this is the double spending problem but to make sure double spending checking works, the addresses themselves can also be used as keys. In this way, all the transaction from the same address hash range will fall in the same shard and the checking will be done inside the shard as per regular operation.
Validation Cache Layer & Transaction Flooding Defender
In this layer, each shard has a validator which makes sure the request contains the correct info or not.
However, in the current landscape, transaction flooding attacks happen occasionally. Therefore, Jura has added a cache module to each validator. If the address hash in a request is in the validator, the validator rejects the request otherwise directs to the shard.
Each cache is designed using the LRU (least recently used) algorithm. In this process, a queue is maintained which contains all the transactions request in time window T. Each node in the queue is a pair (address hash) and the head pointer of the queue is check every second and removes the least recently used node if its expired.
When a new request comes in, the validator will check whether the address hash has been in the queue or not. If not, a new node which only contains the pair will be created and added to the tail of the queue. If the address has already been in the queue, it will be moved to the tail, and the timestamp in the pair is updated. For example, if the time window T is 3 seconds:
LRU (Least Recently Used) Algorithm
It is possible that a lot of requests comes to the validator at the same time, but since the validator only keeps the very small address and timestamp, the queue can be stored in either cache or system memory.
This LRU cache will be able to handle transaction flooding attack-type problems. When this type of attack happens, only the first request will be accepted, and other will be rejected and wait for T seconds.
Monitor Layer
This layer contains multiple monitors, each of which is defined as a group of coordinator nodes in the same shard. Each monitor will oversee the general info of the shard. Sometimes, even though sharding transaction request is done using address as sharding key, the storage space and CPU that a single node uses are still not that affordable because those requests can be spread evenly. Some addresses will inherently be much more active than others. As a solution, monitor layer will be added to fix this issue. Regular node staking will be selected as the coordinator, who gets incentives based on the bid rate of the overall shard groups. Each monitor server has three models.
Illustration of Monitor Server
Storage space and a CPU hacking model will check currently shard storage space and CPU resources used for nodes. If it is the above threshold, compression will be triggered and a new genesis will be built and the top active client addresses will be identified. the system would then add their addresses to a special client shard map in the router layer for router updating. After updaing the router layer, those special clients would be routed to some new backup shards with less traffic and fewer account addresses. Since traffic and users in this shard are far less as compared to be a normal shard, Penny-spend attacks can be afforded in those backup shardis by doing regular compression each time to save storage space.
This all sounds very technical and theoretically it is extremely sound. The interlinked functionalities of these three layers make sure the security is upheld and the network remains decentralized but not letting the nodes exceed their limitations. Following this methodology, we can ensure that we can create a much more scalable blockchain platform. Now the readers might still have a few more questions about how different shards will communicate with each other? What about if the number of shards get imbalanced? Also, what about the issue of full-node functionality?
Don’t worry, we have that covered too!
- Cross-shard communication
We’re aware that even though the transactions from a sending address will always be directed the same shard, sometimes the destination of the receiving end might not be the same. The solution is to find the receiver account’s shard and send an update request. If the updating is successful, the sending account is then updated otherwise it returns an error and asks the user to try again.
- Full-node selection
This is fixed and controlled by the monitor layer. When the monitor layer finds out that some full nodes are not functional, it will automatically add new full nodes to the shard.
- Shard-number imbalance
The number of shard, in the beginning, will be fixed but when the traffic exceed the limit for more that half of the nodes, a new shard is created and half of the requests are directed to the new shard by dividing the hash range of original shard into two equal ranges.
DMDS seems like the only way forward as it’s flexible, secure and upholds the decentralization status of the entire network!
Thank you for reading. If you liked the article, clap and share!
