[Shipped] Quality control strategy to minimize oversight

What is the problem you’re solving?

Incognito’s growing fast, new features/functions are constantly being added.

While we rapidly grow, the weight put on quality assurance also grows, which raises a big question: “How do we keep both the product growing and quality moving forward together as Incognito grows bigger every day?”

What is the solution?

  • Define Testcases, perform manual tests for every new function, find and report defects, and verify quality control actions taken.

  • Implement CD/CI processes into testing. We need to spend more time on automation testing by building a fully automatic testing system that will help us perform the regression tests on the current functions. Meanwhile, we can focus on assuring quality for the new functions.

image

Who are you?

We are @khanhj @hardy and @Rocky, a group of experienced quality control engineers, who enjoy breaking things but want to do it in smarter ways.

Why do you care?

As quality control engineers who are also community members and users of the product, we want to make sure Incognito is the best it can be. More importantly, we want to ensure the quality can keep up with the pace of the growing project.

What’s your plan? What’s your schedule?

Test plan for Feb 01, 2020 to Mar 31, 2020

Ship Date Deliverable
Delivered Perform full functional test for every new feature.
Delivered Perform regression tests.
Delivered Set up a separate chain network for testing.
Delivered Implement setup and tear down test script.
Delivered Testing Database v2

Test plan for April 2020:

Ship Date Deliverable
Delivered Setup a Jenkins server.
Delivered Testing Highway Security
Doing Integration Test pKyber Integration with Database v2
Doing Unit Test Portal v2
Doing Unit Test Consensus v2
Delivered Regression test hotfixes: getbridgewithstatus, RPC query time
Delivered Setup Jenkins pipeline so that everything will run automatically from setup, running test, teardown, and report every time there’s new feature release.

What’s your budget?

The project will be undertaken by 3 quality control engineers.
From Feb 2020 to Mar 2020:

Resource Cost Quantity Monthly Cost
Quality control engineer 1,000 PRV 3 3,000 PRV
Subtotal 3,000 PRV
TOTAL (x 2 months) 6,000 PRV

Budget plan for April 2020:

Resource Cost Quantity Monthly Cost
Quality control engineer 1,000 PRV 2 2,000 PRV
Quality control platform engineer 2,000 PRV 1 2,000 PRV
TOTAL 4,000 PRV

Is there an existing conversation around this idea?

Not yet in this forum

Is there anything else you would like the community to know?

We’re in this together. Let us know what you think! You can find more at the link below.

Reference

16 Likes

I have a suggestion, can put in the speed as one of the benchmarks. The transfer of tokens, deposits etc seem to be taking up a lot of time and CPU, in fact I specially upgraded to Apple 11 Pro for Incognito wallet, but the performance is slightly less than desirable. Thanks :smile::pray:

1 Like

Hey @anon8296499,

Thanks for your feedback. I’ll forward your request to Wallet App team.
@Mo_Incognito @tien @binh

Hey Khanh, can you please update the timeline of the project as well as provide an update for Feb?

Hi all,

Here is a quick update about status of this project in Feb 2020:

  • Complete testing new feature Batch Verification: Released 20200226_1
  • Complete Regression test on Devnet including feature refactor database, reduce data size.
  • Developing test script for new feature Uniswap Integration - In Progress

Thank you. I’ve just moved it to “work in progress” category. Disbursement for Feb has been made. We will continue disburse every subsequent week dependent on progress, so keep us updated.

A quick update on testing status of last week:

  • Continue developing test script for new feature Uniswap Integration - 15%
  • Testing Highway v2 on Testnet - 70%
  • Bug raised/Fixed: 1
1 Like

A quick update on testing status of this week: (Mar 13)

  • Continue developing test script for new feature Uniswap Integration - 50%. Including:
    • 0x aggregator
    • Kyber network aggregator
  • Testing Highway v2 on Testnet - Complete and Released on mainnet
  • Testing Database optimization feature - 90%
  • Bug raised: 1
3 Likes

Quick update on testing status of this week (Mar 20):

  • Testing Uniswap Integration on local: done
  • The new feature Uniswap Integration seem working quite well on our TESTNET:
    • Trading with 0x aggregator: done
    • Multitrade with 0x aggregator: skip
    • Trading with Kyber-network aggregator: done
    • Multitrade with Kyber-network aggregator: done
    • Test Smart Contract migration: done
    • Developing new test script: 70%
  • Testing Database optimization feature on DEVNET: done
  • Testing Database optimization feature on TESTNET: 90% (transaction, performance, reward, pDex, bridge centralize, bridge decentralize)
  • Bug raised: Issue #814
3 Likes

Shard 5 is now pretty all DB v2 nodes and I’m still working to move other shards. I think it’s so good if we recheck pUniswap feature all shard(include shard 5) to make sure it can work everywhere. Eventually, staking feature need to be tested on local for case use 1/2 old code nodes and 1/2 new code nodes to make sure DB ver2 can work correctly

Shard 5 is now pretty all DB v2 nodes and I’m still working to move other shards. I think it’s so good if we recheck pUniswap feature all shard(include shard 5) to make sure it can work everywhere. Eventually, staking feature need to be tested on local for case use 1/2 old code nodes and 1/2 new code nodes to make sure DB ver2 can work correctly

2 Likes

Hi everyone,

This is my update on testing status of last week (Mar 29):

  • Uniswap Integration: Done (including 0x aggregator single trading, Kyber-network aggregator trading, Smart Contract Migration)
  • Testing Database optimization feature on TESTNET: Done (Staking, Fault-tolerant)
  • Testing Highway v2 Security: 20%
  • Setup Jenkins Server: 80% (build Inc Highway, Inc Chain, Executing test suite Transaction)
3 Likes

We’ll update 1st thread every month based on features release plan of other proposals.

Update for April 4:

  • Preview specification of Portal v2
  • Refactor code of test script for Uniswap Integration
  • Testing Highway v2 Security: 80%
  • Setup Jenkins Server: 90% (fixing bugs)
3 Likes

Update April 11th, CD/CI integration:

  • Jenkins test jobs completed: transaction, staking, batch verification, bridge. When triggered, each will be run on remote server and reports will be collected and stored on Jenkins server
  • Jenkins job for deploying local testing chain is completed: start, stop, restart, rebuild, clear data with one click
6 Likes

A general update from QC team on April 12:

  • Continue Integration Test pUniswap Integration with Database v2
  • Finish testing Highway Security (validation)
  • Finish Regression test hotfixes: getbridgewithstatus , RPC query time
  • Finish Setup Jenkins Integration (as @Rocky updated above)
  • Prepare test script to testing feature: Consensus v2, Portal v2
5 Likes

My update on what team QC doing this week:

  • Smart Contract for Portal v2 is going to launch, i spent some time to define/review Unit test case with Devs (expect to complete this task next weekend)

  • Init new Incognito Chain on local environment with the Consensus v2, testing features: Transaction (PRV, Token), Centralize Bridge, Stake, Performance, Fault-tolerance, and pDEX. Bugs found: #861 #853 #855 #856

  • We need more time to properly implement and test Decentralize Bridge on our new Consensus mechanism.

  • Other major updates:

    • Prepare test environment for Portal v2 (BTC, BNB) - done 90%
    • Implement test case for Portal v2 - done 70%
    • Update manual test case for feature Uniswap - done 100%
    • Refactor/Update automation test script of feature Uniswap - done 50%
    • Refactor pDEX automation test script - done 25%

That’s it! Any question welcome!

6 Likes

Hi @team

I have some notes for team when make testing on portal v2, please notice about minted token in some case :
1/ Reward for custodian(how much to get from DAO, how much to withdraw)
2/ When user burn pBCT to get BTC, but custodian not send BTC to user. How to get locked PRV of custodian send(mint) to user

everything need to make sure supply of PRV is not change

Thank

2 Likes

Thank @thaibao
Yesterday I had a discussion with the dev team about that and created a detail testcase to make sure everything was right.

1 Like

My update on testing progress this week:

  • Portal v2:
    • Continue review Unit test case of Smart Contract for Portal v2 - 50% done
    • Using external tool to analysis SC Solidity code: MythX, Smartdec - done
    • Prepare test env and test case - done
    • Test case execution: trustless bridge - 40% done
  • Consensus v2: running decentralize bridge test and fault-tolerance. We expect all major issue is found within this month so that we can launch this new version on testnet in May. Bug found: #874 #864 #877
  • Refactor pDEX automation test script - done 50%
1 Like

My update on testing progress this week:

  • Consensus v2: we’re done with the local test and closed 6 bugs. We will focus on testing decentralized bridge on testnet since it’s hard to deal with Ethereum Smart Contract while running Incognito in local environment.

  • Portal v2: @Rocky is currently supporting @hardy to speed up the testing progress, we’re facing the problem that BTC-produced blocks are quite slow recently, and it takes them 2 or 3 hours to complete 1 test. Besides, some issues were found: Porting request error, picking wrong custodian, etc… Anyway, we have done 70% of the test on BTC and will start testing BNB in parallel. If you’re curious what Portal is then please check this: portal.md

  • About the new Ethereum Smart Contract, @duybao and @thachtb had done some research about SC Security Best Practices and applied a few more unittest cases. I tried to analyze solidity code with these tools: Slither, Manticore. I’m so glad that Thach has fixed some issues and upgraded compiler version to 0.6.6

That’s it. Let me know if you have any questions.

4 Likes