The 8 worst issues and best features of the EOS blockchain
After having thoroughly studied and developed on the EOS blockchain, I thought it would be a good time to share what I thought were the biggest issues and characteristics of the technology. The objective of this article is to educate developers and other people using or thinking of using the EOS blockchain.
EOSIO — Software that can run scalable decentralized networks that support smart contracts
EOS — The first (of many) deployed network running EOSIO software.
BP — Block producers are the operators of the EOS network, BPs (usually a company) run a powerful computer server with the EOSIO software to run the EOS blockchain network.
This article will also highlight when aspects of the EOS blockchain that apply to all blockchains that use EOSIO software, such as the Telos blockchain.
All opinions are my own and as I currently view them. All numbers come from sources I consider reputable. This is not investment advice.
Items are ordered by my opinion of importance.
TLDR;
The worst
1. Block producers are not contributing to governance
2. Resource rate limits throttle the network and are unfair
3. Incorrect RAM price
4. Unused inflation model
5. 30ms transaction limit
6. Protocol upgrade centralization
7. Smart contract upgrade limitations
8. 3 min block finality
The best
1. Programable accounts and money
2. 0.5s blocks
3. Upgradeable smart contracts
4. Protocol upgrades are transparent
5. Scales to 4,000 tps
6. Free transactions
7. C++ contracts
8. Smart contract data management
Summary
The 8 worst issues
1. The election of block producers (BPs) is no longer encourages sound governance participation
The initial design for the EOS blockchain was to elect block producers that not only produce the blocks to a high-quality but also contributed to the ecosystem and governance of the blockchain. The idea was that the token holders should be able to elect produces based on their perceived reputation at performing these activities.
Unfortunately, the power of everyday token holders that use the system to influence the elected block producers no longer exists. This is due to a combination of cartel-like in-voting where BPs vote for each other and vote-buying. I report this because:
- Several instances where exchanges have attempted to buy votes have been reported here and here.
- The governance and ecosystem contributions, quality of blocks and other activities of the current block produces are much lower than other better BP candidates.
- The activities of the current block producers is difficult to find.
- The distribution of producers has centralized in Asia, specifically China, where reports of collusion have existed in the past, and where most of the token holder wealth exists on crypto exchanges involved in these affairs.
Why this is bad
With greedy and corrupt block producers
- Upgrades of the protocol are only approved if financially favorable for the producers and governance activities are deprioritized.
- Actively managing and governing the blockchain is deprioritized for cost-cutting.
- Changing the rules to favor reputable producers is now very difficult.
- With the concentration of production localized to a specific group that is in close collaboration, malicious collusion to attack the network rules is more likely.
The good news
- The blockchain can still be trusted as producers are still financially motivated to produce blocks on time and according to the rules. Dapp producers and users can still trust that the blockchain will run as expected, but perhaps a bit less efficiently.
- Other blockchains using EOSIO software are experimenting with the election processes to reduce the available opportunity for corrupt what produces. The Telos blockchain is leading the way for EOSIO chain governance and will be an important one to watch in the space. Until the Telos blockchain reaches the same block rewards as on EOS, it is not possible to tell for certain whether it’s governance will prevent this issue.
Update 15 Oct 2019
Several proposals now exist to address this issue:
Please comment on the article or Tweet with any other proposals and I will add them here.
Update 29 Nov 2019
There now exists an irrefutable amount of evidence that sock-puppet BPs exist on the network. Multiple parties are likely colluding, including BPs in the top 21.
Vote-buying poxies have been set up on the network by “good” block producers to compete with the corrupt one.
Block One has stated publically itself to start using it’s tokens to vote for BPs. With neary 100 million EOS tokens, they the largest voting weight of all accounts, but still will need to vote with the community to outvote the BPs which have around 330 million votes to make any changes to the network.
2. Resource rate limits throttle the network and are unfair to the poor
Applies to all EOSIO chains.
If the EOS network had 2 users:
- Bob with 10 EOS and,
- Alice with 20 EOS
When the network starts to reach its capacity for processing transactions, the network must limit transactions. What will happen is that Bob, who has less staked EOS, will be the first account to be rate-limited and will no longer be able to send any transactions to the network.
This is different to other networks where anyone can always send transactions based on the transaction fee that they pay (if you pay the most you will always be able to transact). With free transactions on EOSIO chains, this type of resource management is not possible.
The other approach would be to rate-limit Bob so that he can only send half of the transactions that Alice can send when the network is at capacity. However, with the architecture of EOSIO nodes, this adds an increased burden to block production nodes to manage.
Rate-limiting has occurred on the EOS network a few times and the unfortunate consequence has been that those with the least EOS have been unable to use the network completely. This is also affected the Dapp providers who were rate-limited as they did not have balances in the same league as large exchanges and whales and was part of a recent attack strategy on the network against EOS Bet.
Rate-limiting transactions also has the unfortunate consequence of canceling deferred transactions and means that dapp providers cannot set up on-chain cron jobs. Note: instead they can use centralized and cryptographically accountable cron services through the LiquidApps network.
Update 15 Oct 2019
When the EOS blockchain gets close to its physical limitations it activates something called “congestion mode”. A recent EOS Network Congestion Analysis by Dexaran has made it clear that there are a few additions and very cornering problems with this. For this reason this issue has been changed to priority #2.
Summary of the problem:
- By renting resources through REX, the resource exchange on EOS, it is very cheap to get the network into congestion mode.
- Once congestion mode has been activated, it takes the network too long to return to normal operating conditions (20–30 minutes).
- During congestion mode, most normal users with low balances are unable to use the blockchain as explained in this issue.
3. RAM price changes without altering the supply or demand of RAM
Applies to all EOSIO chains.
The price of RAM is set by a Bancor-like algorithm that has several inputs including the current supply of RAM and demand for RAM. It has no knowledge and does not consider the $ price of EOS.
This means that as the price of EOS increases or decreases, the cost of RAM increases by the same factor despite the fact that neither supply or demand of RAM has changed. This means that dapp providers and users need to pay more or less for RAM, despite no change in the network’s usage or capacity. In effect, the RAM price does not accurately reflect the cost of RAM on the network’s nodes.
Other systems using the “gas” accounting model such as Ethereum allows the network operators to influence the resource prices. This means it able to quickly adapt the cost of data storage to reflect the real cost to the network, which is better for their users.
Update 15 Oct 2019
Dan Larimer has Tweeted that this issue will be addressed via increasing the amount of storage available to dapps.
4. The high-inflation capital model fund is not being used properly
Ths blockchain has an annual inflation rate of 5%. 1% of this is used to remunerate block produces and allows for free transactions. The other 4% was meant to be allocated to a smart contract that would fund projects and initiatives in the EOS ecosystem. This system, called the Worker Proposal System (WPS), has never been implemented and the 4% inflation rate remains unused.
The good news
- The Telos blockchain has successfully implemented a working WPS that could be implemented on EOS
5. 30ms transaction time limit
Applies to all EOSIO chains.
All transactions must complete execution within 30ms, despite each block being able to process 200ms of transactions.
This rule means that dapps need to be careful when designing smart contracts so that they don’t go over this limit. This is an important consideration during the migration of smart contract data during smart contract upgrades as well as for cron job services that might attempt to process all records in their database set.
6. The protocol upgrades only need 15 block producers to approve
There are two types of upgrades for the EOS blockchain and both types of upgrade require only 15 of the current 21 elected BPs to approve the change.
A governance upgrade (soft fork)
When 15 BPs approve an upgrade, all nodes across the entire network will immediately apply those changes automatically. This includes changes to the EOS token, how contracts and accounts are managed, voting and election of BPs and more.
A protocol upgrade (hard fork)
All nodes in the network must upgrade the software for this upgrade to happen, however when the 15 BPs approve an upgrade, nodes that have not upgraded their software will stop syncing.
This is different from the production of blocks in which 15 of the 21 love produces need to approve a block, but the entire network of the chain is able to validate and reject them if they break the rules. This makes the production of blocks in EOS is a very decentralized processed, however, the upgrades remain the most politically centralized point. This is a security risk as a collusion of 15 nodes is able to attack the network if they choose to.
The good news
- For an external group of people that does not already operate the top 15 BPs, the attack cost to buy enough votes to upgrade the network and attack it would cost approximately $800,000,000.
- The current top 15 block produces could conceivably make this attack today, but it seems likely that if this were to happen an off-chain consensus would be made to remove the tokens of the producers and accounts that voted for them which comes to the same cost of $800,000,000, so is also a very inconceivable scenario.
- Upgrades of the protocol are done in a transparent way so detection of a malicious attack can be detected immediately
7. Smart contract upgradability is limited to a custodian model
Applies to all EOSIO chains.
The action to upgrade a contract’s code uses the “active” permission of the account. In this way, the upgrade governance of all EOSIO smart contracts is limited to the account permission structure that exists on all EOSIO blockchains. Account permissions must predefine the keys and accounts that are able to control an account. As an example check out dacautority account permissions that govern the EOS Dac contract.
The only exception to this is the eosio.prods account which is authorized to executed certain privileged actions on the system contracts, which is dynamically updated every 1 minute as part of the protocol rules.
This is unlike other blockchains where a quorum of token holders can be set as the upgradability governance mechanism, and this can be determined dynamically at the time of the upgrade.
While not a major bummer, this does mean that upgrading of smart contracts has less flexibility than other systems. It’s important to also note that upgradeability is a sorely missed feature on almost every blockchain (Ethereum, Hyperledger, Tezos…), so it is in my top features below.
8. Block finality takes nearly 3 minutes
Applies to all EOSIO chains
While it takes produces an incredibly fast 0.5 seconds to create a new block, it currently takes around 330 blocks (~2.8 minutes) to pass till that is considered finalized.
The Last Irreversible Block (LIB) defines the last block which is considered final as follows:
- A new block is created which proposes the last block that has achieved 15/21 consensus
- When the new block has achieved 15/21 consensus itself, then the proposed LIB from that block is considered the LIB.
The double LIB confirmation was implemented due to feedback from Ethereum’s Vitalik Buterin on the DPOS 3.0 proposal.
This is a bit of an issue because to consider a transaction irreversible and final requires waiting around twiddling your thumbs for 3 minutes. While not a big issue for many transactions, high-value transactions should be confirmed as finalized by participants. This check should be applied by exchanges and DEXs, and compromises the 0.5s UX that is promised by EOSIO chains.
The good news
- The BOS Core blockchain has been experimenting with this and managed to lower this to 3s. I am not sure how they have achieved this or if important compromises were made.
Update 15 Oct 2019
The BOS Core network added a 2nd layer of message confirmations for faster block finality. This is a good approach but adds network traffic and reduces verticle and horizontal scalability by 10–20% by their estimates.
Dan Larimer has Tweeted that faster finality is under development.
The 8 best features
1. Accounts and money are all ground-level features you can program with
Applies to all EOSIO chains
Accounts on EOS are a primitive feature of the network. Each account is managed by one or more private keys and users can access different dapps with the same account. The ability of users to manage multiple apps with one account, in a world of ever-growing app use, is a huge usability plus.
Building applications on the EOS network means that you have access to use and can program EOS and all other tokens on the network. Dapps can manage money via escrow and other financial primitives easily and transparently.
Combined with the flexible permissions and key management that accounts offer, users are able to easily use dapps and tokens with high levels of security and usability. For developers and entrepreneurs outside of the blockchain industry, this can be a major advantage compared to working with the traditional financial sector and challenge-request password systems.
2. Low latency 0.5s block time
Applies to all EOSIO chains
A new block is created every 0.5s. This is a set schedule that producers must adhere to. It is possible for blocks to be missed, however, the reliability of blocks is around 99.7%.
It is also possible for producers to miss an entire round of 12 blocks however this happens extremely rarely and BPs have an average reliability of around 99.97% for completing a block rounds as seen here.
The 0.5s block time an extremely powerful feature of all EOSIO chains. EOS makes the user experience similar to that of using a payment card or a web app. Personally, after using Dapps on EOSIO chains I find it frustrating going back to Ethereum, let alone using Bitcoin.
Despite recent advancements in layer 2 scaling for generation 1.0 and 2.0 blockchains, people (including Vitalik here) are starting to realize that core blockchains must scale and EOS the finest example of this.
3. Transparent standardized upgradeable smart contacts
Applies to all EOSIO chains
All smart contracts, including the system contracts that set the rules for the network, can be upgraded using a standard transaction through a smart contract. The governance of a contract is defined by the permissions of the account and therefore upgrades can be governed by:
- one private key,
- a multi signature of M of N keys or accounts,
- time delays,
- or even can be set as none upgradeable by removing the permissions of an account
For block producers this makes upgrading the rules a seamless process. An upgrade to the system contracts requires 15 out of 21 BPs to approve (see Issue 6 above) and will synchronize the change across the entire network immediately. This is unlike generation 1.0 and 2.0 blockchains where a soft fork like this would leave part of the network not upgraded during an update to the rules.
For dapp providers this makes running a company or project a more manageable endeavor. Programming bugs can be fixed as they are found in a way that software developers can rely on. This does not remove the need for proper security practices, but compared to previous blockchains, developers are now in a position to write software that they can and update reliably and transparently. With upgradeability as a norm, contracts can be made considerably more complex and advanced than was previously possible, increasing what is possible with dapps.
For users, all upgrades go through a smart contract transaction this can be viewed and audited easily and quickly. With this feature, people can understand what was upgraded and when and also see the new contract changes.
While it is possible to upgrade smart contracts on Ethereum, it is an extremely messy process with no standards and no way for people to transparently audit upgrades. Standardized upgradeability is a sorely missed feature all popular blockchains that I know of including Bitcoin, Ethereum/TRON, NEO, Tezos, Hyperledger Fabric, and even Corda.
While fundamentalists may argue that contract should never be upgradeable, all real projects that I am aware of are requesting an upgrade strategy for their smart contracts. Consensys, the biggest commercial Ethereum company, lists upgradability as one of its standard best practices for Ethereum contracts. EOSIO is the best and only example of a usable smart contract upgradeability feature.
4. Protocol upgrades are transparent and easy to coordinate
EOS is a governed blockchain. The Delegated Proof of Stake mechanism allows and expects that the block producers do more than just run the blockchain. They are chosen as delegates to ensure it’s sustainability. That means implementing upgrades to the governance rules to ensure the system is as fair and sustainable as possible. The rules are defined in the system contracts such as:
- Account creation
- Contract upgrades
- Resource management and accounting
- The EOS token
- Premium names
- Managing global inflation to pay for producers and other activities
With generation 1.0 and 2.0 blockchains, it is still extremely time and energy-consuming to coordinate an upgrade of the rules.
What is technically exciting is that such upgrades are seamless and easy to deploy as explained in Issue 3. Fixing Issues 1, 2, 4and 6 above is technically as simple as sending a transachtion to the network that 15 of the BPs approve. The other mentioned issues may require a more difficult hard-fork requiring nodes to manually upgrade their software. As previously explained coordinating a hard fork upgrade is still easier than ever, and the release of eosio v1.8 makes this even easier.
Unfortunately, improving the protocol still requires the consensus of the top block producers which, due to Issue 1, is a challenging endeavor from a political perspective.
5. It scales to at least 4,000 tps in practice, probably more
EOS scales to at least 4,000 transactions per second. This can be checked in blocks 11,302,923 and 11,302,924 on all block explorers. This busts a previous report that benchmark EOS to a maximum of 250 tps.
While 4,000 transactions per second the current practical limit reached on the EOS blockchain, the maximum capacity is quite difficult to calculate and is dependant on how the blockchain is used. The EOS capacity calculation used by Blocktivity is 1,500, which is also used by the EOS Authority explorer in their statistics.
Live benchmarking on the jungle testnet has been able to achieve up to 9,200 tps while tweaking some parameters which resulted in a higher but pretty acceptable uncle block rate and 6,500 using current EOS network conditions.
The throughput capacity of EOS is orders of magnitude higher than previous public blockchains and even higher than private blockchains like Hyperledger Fabric and Corda can achieve between two counterparties (based on discussions I have had with developers).
While the limit may not yet be known, the current usage also shows capacity of the network. On average the network records about 60–70 transactions per second, 10–100 times that on other networks. As reported by Dappradar, 32 of the top 50 most used apps in the past 7 days are on EOS and these dapps account for 62% of transaction usage amongst these top 50.
Why is EOS more scalable? Stay tuned to my Twitter or Medium account and I will in time explain how EOSIO achieves this, and what impact those decisions imply.
6. Free transactions
Applies to all EOSIO chains
Transactions on EOS are sort-of free. When you send a transaction your EOS token balance does not decrease and that is a huge user experience improvement compared to other blockchains.
To send transactions you need to have EOS tokens staked on your account. When EOS is staked (put into a special locked part of your account) this allocates you a portion of the resources of the network allowing you to send transactions. The more EOS that is staked the more resources you can access.
You can unstake EOS at any time and unstaking takes 3 days. This is both a hindrance and an account security feature at the same time.
While this model has its downsides (see Issue 3), it is a huge improvement compared to other blockchain systems where users pay per transaction and permit better consumer dapp usage.
7. The power of C++ in contracts
Applies to all EOSIO chains
EOS contracts are written in C++ which are then compiled to Webassembly. This is the code that the nodes in the network know how to execute.
Webassembly is a virtual machine environment that is highly suited for blockchain applications to run as it is efficient and portable. It’s used by other blockchain projects Parity and Hyperledger and is the planned final environment for the Ethereum network.
While multiple high-level languages can compile down to Webassembly code, C++ is one of the few languages that has a Webassembly compiler. It is 30 years old and one of the most mature, optimized and efficient languages in the world.
Being able to write contracts in a familiar and well-used language is a huge benefit for Dapp providers as training costs significantly reduced and complex applications can be quickly be developed based on mature principles and processes.
Solidity, the programming language for Ethereum, was built from the ground up for the Ethereum network. Due to this, it lacks basic support for several programming fundamentals such as the following, that are easily found in EOS programming:
- String manipulation
- Vectors, arrays and other set types
- Floating points (numbers with decimal places)
- Operator overloading and templating
- The huge amount of libraries (including cryptographic) that are available on C++
8. Smart contract data management with multiple indexes
Applies to all EOSIO chains
Smart contracts store data such as balances or escrow accounts in the RAM of the computers running the network. In generation 1.0 but this was not even possible and in generation 2.0 this is possible with very limited data structures called a key-value pair. EOSIO smart contracts store data using the popular C++ Boost library. It uses a data structure called a multi index table.
As an example, it is possible to build a database set with the user’s name and their EOS balance. It is then possible to query the database based on their name or if their balance is above a threshold. The difference here compared to previous blockchains is that multiple indexes can be used to query the database efficiently (both the name or their balance can be used, while in a key-value pair only one can be used).
This is a major advantage as it allows developers to build advanced smart contracts that could not be done before.
With the ability to upgrade smart contracts it’s also important to understand that data sometimes needs to be upgraded as well. Migration of smart contract data should follow best practices and can be complex in the face of some of the EOS network resource limitations (see Issue 5).
Summary
The scalability and the low latency are unique EOSIO characteristics in the blockchain ecosystem. This, combined with free transactions and a few other usability features make it the most scalable, user-friendly blockchain that exists.
Developing advanced dapps on the EOS network is easier and safer than ever due to its advanced smart contract storage models, C++ programming and available libraries.
Upgradeability of smart contracts is one of the most underrated features, in my opinion, and impacts the entire architecture of the blockchain allowing better experiences for developers, users and operators (block producers).
There are many issues with the use and development using the EOS network (and other EOSIO chains), some more problematic than others. I am most concerned with the RAM price and inflation as issues that need technical upgrades. Most of the issues are able to be technically solved in a relatively straightforward way by upgrading the governance and rules. This is one of the most exciting things about the EOS network — its ability to adapt and fix itself in an agile way if the elected BPs approved such changes.
But the bottom line is that with the current selection of block producers and the election process, such changes are too difficult politically to implement. The fate of the health of the EOS network relies on having genuinely committed and active delegates. Issue 1 seems to be the most contentious debate in the ecosystem. Block One has left a few hints (here and here) that it may attempt to intervene but has thus far stayed out of the discussion. Either way, the longer it is left, the worse the situation becomes.
If miraculously, this were to be fixed and a more suitable set of BPs was elected, I would hope and expect that they start to take a more active role in managing the blockchain.
Update 29 Nov 2019
While the issues on EOS may seem concerning, it is important to realize that the EOSIO ecosystem is much bigger than just EOS. There are now at least 8 public networks run using EOSIO, each trying to define it’s own value proposition and with different rules defined in their upgradeable system contracts. You can learn about different EOSIO chains at ineos.io, each supporting the same smart contracts as EOS.
If you like this article please give it a clap, post it somewhere (maybe your grandmother would like it?) or add your opinion on the Tweet. Pls tag @theblockstalk if you share…
Other related info
Dexaran shares his experience developing an EOS dapp and EOS Network Congestion Analysis
Learn how to write an EOS dapp with Michael’s Learn EOS book, I have read it and it’s pretty damn good. Use code jackandtheblockstalk for a 25% discount.
