so cool @Mike_Wagner, i love it
[Shipped] Reduce Blockchain size by 50%
Awesome! I like the idea of the db icon either looking short in the icon or even crushed like an aluminum can.
thanks guy, we will bring more good news soon
Hi @hungngo,
I tried to get your code and run to sync data from the mainnet. I saw that at beacon block 90496, I couldnât sync it because of this error log:
â2020-03-17 22:46:09.652 beaconprocess.go:302 [CRT] BlockChain log: FORK BEACON DETECTED, New Beacon Block Height 90496, Hash 44d925ddeed6b8a39f186c9d93356d4900eba3a988229f5d9a0beb1000c91715, Expected Previous Hash 378f3a45f83d14ec6e929065c6cf08e45dd583106223fce57b13c1e3febfa230, BUT Current Best State Height 90495 and Hash 378f3a45f83d14ec6e929065c6cf08e45dd583106223fce57b13c1e3febfa230
2020-03-17 22:46:09.652 synker.go:976 [ERR] BlockChain log: -1048: Fetch Beacon Block Error
-1001: block 378f3a45f83d14ec6e929065c6cf08e45dd583106223fce57b13c1e3febfa230 not exist
Get Beacon Block By Hash Error
github.com/incognitochain/incognito-chain/dataaccessobject/rawdbv2.NewRawdbError
/Users/autononous/go/src/github.com/incognitochain/incognito-chain/dataaccessobject/rawdbv2/error.go:157
github.com/incognitochain/incognito-chain/dataaccessobject/rawdbv2.GetBeaconBlockByHash
/Users/autononous/go/src/github.com/incognitochain/incognito-chain/dataaccessobject/rawdbv2/accessor_beacon.go:66
github.com/incognitochain/incognito-chain/blockchain.(*BlockChain).verifyPreProcessingBeaconBlock
/Users/autononous/go/src/github.com/incognitochain/incognito-chain/blockchain/beaconprocess.go:300
github.com/incognitochain/incognito-chain/blockchain.(*BlockChain).InsertBeaconBlock
/Users/autononous/go/src/github.com/incognitochain/incognito-chain/blockchain/beaconprocess.go:102â
I needed to restart node, and it synced continuously again. I think this is because beacon block 90495 could not be inserted into DB, but we update the best state of beacon chain. So we need to review the reversion of this process. When I review this logging text, I think it doesnât make sense in this scenario.
Please have a look.
@thaibao i solved this problem. This problem happened because node didnât revert current state to previous state when it failed to insert new block
REPORT 14 Mar - 20 Mar 2020
Current Step
Finish:
- Fixed bug and passed all QC testcase in dev/net, local network
- Finish code review by @thaibao (letâs give him a big clap)
In progress:
- We are moving to testnet, we deployed source code with database version 2 to shard 5 in testnet
- Add time log and monitoring node operation with database version 2
To Do Next:
- If things are smooth, we will move on mass testnet committes deployment and mainnet fullnode deployment
- Merge code and support other team with our new database infrastructure
I have all the confident that our blockchain size will be 10 time smaller soon
Wonderful news over the weekend!
- We have deployed a mainnet fullnode with database v2. The blockchain size in this fullnode is only 12GB compared to the current 150GB.
Log rotation in two days consumed storage more than 5-months blockchain size. lol
Yeah, so great. Weâre trying to move some old mainnet node to this DB version and run a while before moving all nodes to this great version.
This week
All testnet node will be moved for this version
Few mainnet nodes will be moved later
@aaron @ning letâs create a long topic explaining this. this is a big improvement that is super helpful to our validators! from 150gb to 12gb, that is indeed a wonderful news @hungngo
REPORT 21 Mar - 29 Mar 2020
Current Step
Finish:
- Pass all QC testcase on both dev/net and test/net, no bug was found this week.
- Testnet: All node in testnet now using the new database version
- Mainnet: One node in each shard and beacon run with database version 2
In progress:
- Testnet: monitoring node behavior
- Mainnet: monitoring node behavior and continuously update node by node to database version 2
This weeks we received NO BAD NEWS. As scheduled, that we will deliver this feature to all node in mainnet within next 2-3 weeks.
Unfortunately, we having a trouble with instruction hash of beacon validator that use db v2. Maybe issue is in reproducing hash from instructions of block reward that mean beacon producer create a block with instruction hash which is different from beacon validatorâs hash who using db v2 . @hungngo and @hyng are going to try hard to fix this issue earlly and we can retry on beacon validator again.
All great things take more trial and error than most people will ever know.
I am sure you will fix it.
Team fixed this problem and in middle of testing and re-deployment. This problem didnât effect database v2. So every node will be deployed as schedule
REPORT 30 Mar - 4 Apr 2020
Current Step
Finish:
- We successfully deployed dbv2 to half of shards (all shard) beacon committees
- Basically everything is ready for mass deployment and official release
In progress:
- Still in deployment phase. This phase will be closed when all node in shards, beacon committees run with new database version
This proposal can be closed this week, no problem arise last week. Deployment phase is on a good track.
This is great progress @hungngo ! Will our nodes automatically update or do we have to reinstall? The current blockchain size has pushed Digital Ocean vnodes under breakeven, but those canât be downgraded, so new droplets will need to be setup. Any idea what day this might be occurring this week? Thanks!
@JG20 For manual update, which works by now, we can just follow these simple task:
- Delete our current database (all of them) and start again from the beginning
- Run with our new docker tag
@thaibao Please help us answer these question, tks a lot
- Could you tell us when will be able to make an official release?
- Will node got automatically update?
I will publish it today or tomorow. But if you guys want to run it now, please view docker tag dbv2_20200331_1
I used it for many nodes and fullnodes