Overheen the past several years there has bot a lotsbestemming of talk about cryptocurrencies, such spil bitcoin, and the blockchain technology that underpins them. However it can be difficult to understand how the blockchain actually works te practice, and why it is such a clever method for currency exchange. Two of our graduates are presently developing our own cryptocurrency for use within Scott Logic, spil an attempt to better understand the practical uses of the blockchain. Ter this blog postbode, wij aim to explain the internal workings of the blockchain and help people to understand how it manages to keep transactions both secure and anonymous. This postbode will go into the low level algorithms of blockchain and provide verified examples alongside each algorithm, this postbode is not for the faint of heart!
Bitcoin is just one implementation of the blockchain. While all cryptocurrencies are based on the blockchain algorithms, there are subtle differences inbetween the implementations. To illustrate the differences here is a tegenstelling inbetween bitcoin and litecoin, The very first is that litecoin aims to mine a block every Two.Five minutes, spil opposed to bitcoins Ten minute processing rate. This speeds up the time it takes for a transaction to be confirmed ter the network, providing protection to users against certain kinds of attack/mistake (such spil dual spending). A 2nd difference is the inclusion of scrypt te the proof-of-work algorithm, a memory intensive function compared to bitcoin’s sha256. This makes it much tighter to parallelise the mining of litecoins. Litecoin supports mining of up to 84 million litecoins, four times spil many spil bitcoin provides.
Ter this postbode wij will concentrate our explanations on the internal works of bitcoin, since all other cryptocurrencies share similar implementations. During our time constructing our own bitcoin network, wij spent a lotsbestemming of time researching exactly how bitcoin works, and so included ter this blog is a selection of the core algorithms with step by step examples. Wij hope to help other people wanting to learn about bitcoin by assembling the information they need into a single source. Wij did this, so you don’t have to share te the difficulty of traversing the web looking for clues spil to how the system works. The language of this blog is aimed at people who have a basic understanding of pc science, and will go into some detail with how thesis algorithms work and some of the gegevens structures are made. For anyone wanting to write their own code, wij have included worked examples which can serve spil test gegevens for your algorithms.
Where’s your header at?
Let us very first go through some common terminology which is useful to know when reading into bitcoin and should help everything make a little more sense:
- An address is a hash of a public key, which is used spil a destination to send bitcoins. A participant te the network often wields numerous addresses.
- A transaction is an exchange of bitcoins encoded into a block
- A block is a collection of transactions and other information. It includes the hash of the previous block and this forms a chain of blocks. Below is a diagram of a block and its contents
- A knot is a server that is able to relay information inbetween other peers te the network
- A miner is someone who works to find the hash of the next block
The blockchain, spil the name suggests, is a chain of individual blocks, where each block consists of a header and a bod. The main assets is a list of all transactions which occurred inbetween the creation of a given block, and its predecessor ter the chain. The header of a block contains identification information used to prove the authenticity of both the block and the transactions contained. Blocks are being added to the chain all the time at an average rate of one every ten minutes. Each time a block is created, whoever created it is paid a prize, presently standing at 25BTC (10809 USD). So what is stopping everyone from minting money? Well firstly everyone te the bitcoin network (5000 major registered knots) are racing each other to mine the next block. Secondly, ter order to voorkoop a block, you need to solve a very difficult maths problem, known spil the proof of work (more on this straks). This problem would take an individual miner working on an average spec PC thousands (if not millions) of years to compute. Before wij explain how the maths puzzle is solved, wij need to know what the six fields which comprise the header are, how they relate to the surplus of the block and how they are used to create the block hash.
The order te which bytes are stored (most to least significant or vice versa) is referred to spil big or little endian notation. Each value te the header is stored ter big endian, specified by the official bitcoind knot implementation documentation. When they are used te the block hash calculation they are converted to little endian hexadecimal encoding. Below wij take the same sample header components and construct a block hash from them.
Now wij know how a block hash is made let’s look at how difficult it is to solve the block maths puzzle wij mentioned earlier. The problem users need to solve involves producing the 32 byte hash that, when interpreted spil a decimal number, is smaller than a predefined number (the difficulty). The official bitcoind knot has a very elementary mining algorithm, where it repeatedly increments the nonce and calculates the hash until a valid hash is found. Due to the entropy of sha256, it is very difficult (nigh unlikely) to formulate an efficient algorithm to calculate this. Optimisations do exist however, but they still run te exponential time.
Ter calculating a valid block hash, the sha256 hash step takes more computing power than any other (appending header bytes / verifying a hash / calculating the merkle root). Hashing power is used to describe how many times a block hash can be calculated and checked. The hashing power of the bitcoin network has grown substantially overheen the past few years. This is attributed to an increase te the size of the network and improvements ter hardware, such spil introduction of ASICs (hardware specially designed to perform SHA256 hashes for bitcoin mining). Spil of December 31st. the hashing power of the network stood at around 745,000 THs-1 (745,000,000,000,000,000 hashes every 2nd).
Wij mentioned the term proof of work earlier. This is the very first valid hash of a block found by bitcoin miners and broadcast to other peers on the network. Whoever calculates this hash very first gets the 25BTC prize, and once the hash and nonce used to find it is collective on the network, it can very quickly be verified. The difficulty of mining has grown proportionally with the hashing power of the network, to ensure that the amount of coins ter circulation does not grow too quickly. Every 2016 blocks, bitcoin calculates a fresh difficulty (adjusts the current difficulty slightly). There is a total limit of 21 million bitcoins that can everzwijn be mined, but by enlargening the difficulty to maintain a onveranderlijk mining rate, this limit should not be reached until the year 2140.
Proof of work involves creating a valid hash of a block that can be quickly verified by other users on the network to prove that you, spil a miner, have waterput time into calculating the nonce. The nonce is the pseudo-random number that is a component of a block hash. A valid hash exists only if the numerical interpretation of the hash (spil a 256-bit number) is smaller than a predetermined difficulty. The hash voorwaarde be smaller than the difficulty te order to be accepted into the blockchain.
Verifying the block hash is very plain and quick to do, shown below is a hash from the blockchain which has bot accepted. Also included is the value of onverdraagzaam from the block header. The snauwerig are passed into a elementary formula ter order to generate a number, which the block hash is compared to. If the hash is smaller, the block is accepted, if the hash is not smaller, the hash is invalid. Here is a worked example with values taken from the live bitcoin blockchain.
Looking at the same gegevens spil te the block generation example, here is the nonce and the resulting block hash. Wij have switched the difficulty, for the purposes of the demonstration, to be much lower than the actual bitcoin value. This will permit us to accept a larger number of valid hashes.
If the nonce is only slightly different, spil seen below, then this fully switches the resulting hash value. This shows how it is virtually unlikely to guess what value the nonce may be, and why brute force (possibly with negligible optimisations) is the only method of mining bitcoins. The same goes for any other gegevens involved ter the block hash. Switching any transaction gegevens (resultantly switching the merkle root) would result ter a entirely invalid block hash, thus demonstrating the inherent security te the blockchain.
With the difficulty of the proof of work being so high, it is very zonderling for a single miner to work alone. Most mining is done by mining pools, which are effectively syndicates of miners. Thesis pools usually work on a “divide and conquer” mechanism, mining different ranges of possible values to reduce the time te which a valid hash is found. The number of calculations done is stored by each member of thesis pools, and this is how their share of the prize is calculated. The prize is paid to the pool holder te the standard prize transaction, and every month (or however often) a transaction is made to pay miners for their share of work. The prize transaction is the very very first transaction te a block and is added by miners when they attempt to calculate the nonce. Since the prize is te the form of minted coins, this transaction has no inputs.
We’ve described how difficult it is to mine blocks but some among you may be thinking: “With a Four byte nonce there are only Four.3×10^9 values to check and a network with a hash rate of 7.5×10^16 hashes vanaf 2nd, surely blocks will be found very quickly?”. You would be onberispelijk!
It is possible to have a header where no matter what the value of the nonce is, there will not be a valid block hash. Miners calculate every valid value for the four byte nonce, until they either find a valid block hash, or there are no more values for the nonce they can attempt. The miner now knows that any value they choose for the nonce will not create a valid block hash, something else te the header voorwaarde be switched to produce a valid hash.
There are two common values switched ter the header when the nonce has bot weary. They are the timestamp and the reserve nonce. The timestamp can be updated by a few milliseconds, creating a entire fresh combination of hashes which can be produced when the nonce is altered. Transactions within the block, identified by hashes of their contents, combine to produce the merkle root, another voorwerp ter the header which can be altered. Adding an input to the prize transaction is another popular way of switching items ter the header. The prize transaction has no inputs, and so including a dummy input will adjust the hash of the prize transaction and this switch will alter the value of the merkle root. The dummy input information included ter the prize transaction is called the reserve nonce.
Let the Assets(s) Klapper the Floor
The bod of a block consists of all transactions which occurred inbetween two given blocks. Every transaction has an associated hash, where the hash is comprised of key transaction information passed into a sha256 hash function. Transactions contain a set of inputs and outputs. Outputs are the addresses to which a transaction’s bitcoins are sent.
Outputs have two main fields, the amount to send and the script. When an output is created, the recipient of the bitcoins provides an address to which the bitcoins will be sent. This address is derived from a public key possessed by the recipient, which ter turn is derived from a private key that only the recipient has access to. Output scripts are effectively finite state machines which will process the signature, found te a transaction input, spil an input for the machine. Several types of common script exist, each type includes different requirements which voorwaarde be met te order to spend the coins located at an address. Pay-to-PubKeyHash is the most common form of script for an output. It only permits the holder of an address to spend the bitcoins located there and is comprised of the following concatenation,
Inputs are references to the outputs of other transactions. They identify an output by providing the transaction hash and index of the output the spender wishes to redeem. The script of this output is treated like a finite state machine. The input scripts are passed into the finite state machine spil values and executed ter order to prove the spender wields the right to spend the bitcoins. The input script for a Pay-to-PubKeyHash (output script) is the following concatenation,
The pubKey value is the public key used to make the address of the spenders bitcoins. The “sig” field is a signature created from the sender’s private key which proves they are the person who wields the address/public key. Exactly how the keys are generated and redeemed is a tricky process using ECDSA (and the bitcoin curve secp256k1) and would make this blog postbode even longer! So wij think it’s best saved for a zometeen date. A walkthrough of the basic signing algorithm for ECDSA can be found on Wikipedia. The essence of the signing process is spil goes after, that a transaction is constructed, identifying values of the transaction are selected, the values collected are concatenated into a byte sequence, the byte sequence is then signed using the senders private key with ecdsa using the secp256k1 curve.
So let’s work through creating a transaction. Very first wij will look at how you create outputs and then how you take an output and sign it spil an input. Wij have a script where Angelina owes Brad 20 satoshi (1 BTC = 100,000,000 satoshi – named after the creator of bitcoin). From past transactions Angelina has a collection of bitcoin addresses where hier money is located, for each address she has the corresponding private key kept somewhere safe.
Wij can see that Angelina has more than 20 satoshi te hier wallet ter total, but no single address which holds exactly 20 satoshi. What she voorwaarde do here is pick some addresses totalling 20 or more satoshi which wij can spend ter this transaction. They will be sent to two outputs, one is to an address Brad has provided hier, the other is to a fresh address she will make (for hier switch). Here wij have the two fresh addresses they made.
Angelina now calculates the pubKeyScript from the address above (using the form Pay-to-PubKeyHash mentioned above and copied below for ease of reading) and, along with the value to be sent to each address, creates the output objects. Below shows how Brad’s address is converted into a pubKeyScript. Addresses are stored te a variant of base58 called base58Check.
She then starts creating the inputs for this transaction. Originally, inputs only have three of the five values found ter every input, prevHash, index and sequence. Thesis three identify what bitcoins to spend and te what manner to spend them. Once all inputs have bot created like this, identifiers of an input are taken and amended te a byte-stream. For each input ter turn, the corresponding scriptPubKey from the output is signed along with the identifier byte stream. This produces a set of signatures which validate the spending of each input’s to-be-used output. The signatures are also tied into the transaction through the identifying byte stream and prevent people tampering with the transaction, spil this will invalidate the signatures. The process of signing involves some tricky maths and fresh definitions for adding, doubling and multiplying that are specific to Elliptic Curve Cryptography. I’ve included a verbinding at the end for more information, but I won’t voorkant this process ter details here.
Below is a worked example of what Angelina had to compute te order to redeem bitcoins from hier very first address (17pA4nZbtivWZVkkaEUEGjLT5DVnH5Gbr1). She takes the scriptPubKey referenced ter the input ter order to create the script to redeem the bitcoins at this address.
Angelina now takes all the inputs and outputs and puts them into the order below. She needs to calculate or state a few reserve values for the bitcoin protocol to accept this transaction.
When looking at the transaction structure above you can notice several of the values have their bytes inverted into little endian form. Where a value is stored ter little endian I have annotated the field (le). All of thesis bytes are concatenated before being passed into a SHA256 function for two rounds of hashing. This produces the final transaction hash which can be used spil a reference by other inputs at a straks time.
When a fresh block is being mined, the transactions it will contain have bot locked te. The transactions are laid out and hashed together ter pairs to create a fresh SHA256 hash. This is recursively done until a single hash remains, the merkle root. Below is a diagram demonstrating how the hashes are made on each level of recursion. On any given level, there may not be an even number of hashes, if this is the case the last hash to be processed is hashed with itself.
(Angela) Merkle Root Diagram
Ter the End (ian)
So far we’ve explained what actually goes into a bitcoin transaction, and how those transactions are added to the blockchain and verified by miners. Wij hope this is helpful for anyone attempting to understand cryptocurrencies on a lower level. For the next postbode te this series we’ll dive into the security and encryption ter bitcoin, focusing on the Elliptical Curve Cryptography (ECC) te signing of transactions, the generation of addresses and other forms of output signature such spil Multisig or Escrow scripts.
Further Reading and a Epistel Aside
Below are a few interesting linksaf you might wish to look into. They are relevant to the material of the blog. If you found this blog interesting, I’d suggest reading into the functionality of the Ethereum and Ripple, two currencies recognised spil the 2nd generation of cryptocurrency. They benefit from hindsight, being able to avoid mistakes made by the very first generation and match the successes. Here is a little information on Ethereum to hopefully whet your whistle.
Ethereum: This currency has evolved from the underlying blockchain algorithms to become more than just a currency. The network itself is effectively a single decentralized virtual machine capable to executing programs contained within transactions. Ethereum is a hybrid network with its currency, Ether, being used spil payment for execution of programs. Ethereum also supports wise contracts (buzzword of 2016/17??) which are a payment and program hybrid. Using Ethereum it would be possible to set up standing orders or trust funds which could only be accessed on set dates. This currency has received a large amount of attention recently from major tech companies, such spil Microsoft, announcing that they will be implementing Ethereum spil a Service on Azure.