Dynamic committee size and dynamic sharding: implementation phase

Objective:

  1. Improve the robustness of the chain: avoid the chain blocked by the unavailability of nodes, network disconnected, inadequate node power.
  2. Adapts well with the variety of the number of validators
  3. Strengthen the security: trustless to shard committee

Length: 25 weeks

July 2020

  • Beacon sync, insert and store all shard blocks
    • Beacon node must sync and insert shard from beginning

August 2020

  • Implement new staking flow
    • Staking to become candidate
    • Candidate Life cycle (not included monitoring and report)
    • Unstake flow
    • Refine Stop auto re-staking flow
  • Implement new swap rule
    • Not included increasing/decrease (case 3,4) shard

September 2020

  • Implement solution of problem 3 (unavailability of some committee members)

October 2020

  • Implement solution of problem 2 (unavailability of >= 1/3 committee members)

November 2020

  • Implement beacon govern shard (dev 4 week)
    • Faulty pattern and error
    • Slash
    • Kick malicious committee

December 2020

  • Implement Increase/Decrease shards (dev 2 week)
  • Shard substitute monitor and report fault shard block to beacon

Key results:

  • Committee size is dynamically changed with the number of validators
  • Dynamic sharding
  • Beacon is able to verify the correctness of every single shard. As a result, a shard could verify cross-shard transactions through the beacon chain.

Note:

  • Linked to (to be update) Github

Ref Dynamic Committee Size design

16 Likes

This task will begin in 1st Jun:

  1. Design working flow @hy @hungngo @0xkumi @dungtran
  2. Implementation @hy @hungngo
  3. Unittest, local testing @hy @hungngo
  4. Integration testing by QC
4 Likes

Update 2020 Jun 8:

Github link: https://github.com/incognitochain/incognito-chain/tree/dev/dynamic-committee-size
Result:

  • Finish implement Beacon sync, insert, store all shard blocks

Next:

  • Testing
4 Likes

Update 2020 Jun 12:

Result:

  • In testing phase and fixing bug

Next week:

  • Testing
  • Move to next task while waiting for QC
2 Likes

On June 24th, 2020

  • Update the random shard assignment scheme to keep the balance of the shards’ validators.
  • Update the solution for problem 2,3

Detail design: https://docs.google.com/document/d/19KChrg0B2LmfT_t-K8vovCG-mXUkAYG88j5FQsKPXT4/edit#

4 Likes

Update 26 June 2020

Finish:

  • Agreement on refactoring code approach
  • Refactor Instruction related to committee (stake, assign, swap, stop-auto stake)

Next:

  • Refactor Committee State, remain the current assign and swap rule
  • Test after finish refactoring code

This week, we do not spend much time on the main task as the code based of incognito-chain is very huge now. After thoughtfully consideration, we decide it need to be refactor into smaller component, make it easier to maintain in the future. It might take some time now, but new staking flow can be well-implemented and well-tested with high quality code.

6 Likes

Update 3 July 2020

Finish:

  • QC testing on Beacon Sync as Fullnode
  • Implement Beacon Committee State

Next:

  • Begin implementing Shard Committee State
  • Begin implementing New Staking Flow
  • Finish implementing + unit-test Beacon Committee State
4 Likes

Yesterday, we talked about dynamic committee size - new staking flow, which supposes to be released on mainnet this September. This introduces new earing lifecycle of a validator, reduce time to sync shard’s data significantly, and also help user to unstake immediately.

image

Again, checkout the technical specs here

5 Likes

Update 10 July 2020

Finish:

  • Refactor:
    • Local testing for Beacon Committee State (transaction, stake, stop auto stake)
    • Instruction unit test
    • 80% implementation Shard Committee State
  • New staking flow, swap rule
    • Agreement for new swap rule
    • 60% design for new staking flow and swap rule

Next

  • Refactor
    • Finish Implementing Shard Committee State
    • Begin local test for both Shard and Beacon Committee State
  • New Staking flow and Swap rule
  • Finish first design for implementation
  • Begin implementation
1 Like