The State of Mainsail

Current list of all GitHub activity

Jun 26, 2023

In late April of this year the ARK Team announced the cancelation of their proposed migration to the Polygon network. This included a promise to continue working on and upgrading the ARK Core codebase. It was also announced that ARK Core would be renamed to Mainsail and all previous works and versions would be fully opensourced.

In this post we will compile a list of the first 6 weeks of Mainsail GitHub achievements in a centralized spot for anyone that may have missed it.

Mainsail Commits

From May 12th to May 19th it was reported that Mainsail had 36 commits to the repo.

From May 19th to May 26th it was reported that Mainsail had 23 commits to the repo.

From May 26th to June 2nd it was reported that Mainsail had 16 commits to the repo.

While these commits were not detailed in the Dev Report Blog, they can still be found in the GitHub.

Ramping up Development

During the first few weeks of ramping up development of Mainsail and going over all previous core work, it became more clear that the team had found a decent stride. This became evident from discord discussions with Matt as well as more detailed report blogs.

From June 2nd to June 6th, development really picked up as it was reported that Mainsail had 57 commits to the repo.

Part of the work from this week included performance boosts and consensus improvements.

The JavaScript version of the BLS crypto package was swapped out for the @chainsafe/blst-ts , resulting in a substantial performance increase and allowing for 800-1000 verifications and aggregations per second.
The BlockTime milestone was restructured to be expressed in milliseconds, with the default value now set to 8000ms.
Deprecated packages, including forger, forger-tracker, and crypto-time, have been removed from the codebase.
Consensus enhancements are in progress to improve the process of storing messages. With Mainsail, we will only store one message per stage from each validator to streamline the process. If multiple messages are received from the same validator within a stage, this is considered malicious evidence.
The P2P package was overhauled and refactored. Handlers and broadcasters have been implemented for propose, prevote, and pre-commit stages, with more work required to transfer messages over the P2P layer.
Improvements have also been made to the Crypto module, including allocating small buffers based on data size and the addition of optional metadata for blockId fields.

From June 9th to June 16th it was reported that Mainsail had 32 commits to the repo, which included new consensus mechanisms and a general clean-up of the codebase.

State cleanup: DPoS (Validators) related logic was removed. We will implement a new package similar to the validator-set-static that will implement the vote-weighted validator selection. Round-related logic was also removed from the state package. It contained methods like detectMissedBlocks and detectMissedRound, which are no longer part of the new consensus.
Commit block: We added initial support for the commit block. This is a new structure that consists of block and aggregated signatures. Aggregate’s signatures are proof that more than 2/3 of validators signed the pre-commit message for the particular block. Once the node gathers enough pre-commits it aggregates them and forms a committed block. A committed block is stored in the database and is considered final and irreversible.
Validator indices: Instead of a validator public key we use validator indices to represent a validator from the validator set. This results in much smaller messages and faster serialization.
Serialization: Serialization and deserialization fixes were applied and we can now successfully transfer propose, prevote and pre-commit over the P2P network. A delayed start was implemented which allows nodes to start consensus at the same time. This is a temporary feature, which is going to be replaced with more effective bootstrapping and message downloading. With the applied fixes we can successfully keep two or more nodes in the synced state while they are producing blocks.
Validator Registration: The registration transaction was updated. A Validator now registers their username and consensus public key. The consensus public key is different from than wallet public key. It uses BLS instead of Schnorr signature. The consensus public key is used for pre-vote, pre-commit and pre-vote message verification. BLS also allows message aggregation.
Reboot: The Genesis block was regenerated with the validator transaction updates. The static validator set now consists of the validator wallets that are registered in the genesis block.

From June 16th to June 23rd it was reported that Mainsail had 36 commits to the repo, which focused on addressing consensus-related tasks and enhancing the P2P layer.

Added verification schemas for Proposal, Prevote, and Precommit to ensure data format validation after deserialization.
Introduced support for LockedBlock, which is a block that received +2/3 Prevotes but didn’t receive +2/3 Precommits. Serialization, deserialization, and proofs were implemented, with the proof represented as an aggregated signature comprising +2/3 Prevotes.
Expanded the AJV validator to include additional formats for ipv4 and ipv6 address checks.
Redesigned and repurposed the P2P header, enabling the transmission of consensus state data such as round, height, and marks for received Prevotes and Precommits. New fields facilitate the detection of missing data and its retrieval from connected peers.
Implemented the getProposal and getMessages endpoints on the P2P module to retrieve missing data.
Utilized a bitmap representation for the validator set, reducing the size of serialized content and P2P messages by storing signed messages efficiently.

What’s Next

The most recent dev report said this upcoming week would focus on implementing persistent message storage for uncommitted consensus state, specifically Proposal, Prevotes, and Precommits. Additionally, efforts will be dedicated to consensus bootstrap and restore functionalities as well as enhancing P2P state synchronization and making improvements in that area.

Closing

We can see dev work increasing across all ARK products, not just Mainsail. We can also see more transparency week after week in the development reports. While these reports may go over the heads of everyday users, these are very important in an opensourced project. Detailed reports can attract developers and allow them to follow along in real-time and even attract some to contribute themselves.

We are excited to see the future of Mainsail and await the next report.

If you would like to follow the teams work directly you can follow the GitHub, watch the live GitHub channel in the Community Discord or read the reports from their blog.

This article was put together using official ARK SCIC posted blogs, anything outside of quoted text from official sources is strictly the opinion of The Strake Foundation.