Bitcoin blog

The essence of Bitcoin is innovation that be related to remittance, asset holding rights, liberation, and freedom.

bitcoinjの概要

【目次】


bitcoinjとは?

bitcoinjはBitcoinプロトコルを扱うライブラリです。 BitcoinCoreのローカルコピーなしにウォレットを維持したり、トランザクションを送受信したり、その他の高度な機能を備えています。 Javaで実装されていますが、任意のJVM互換言語が使用できます。その中にはPythonJavaScriptが含まれています。完全なドキュメントがあり、よく知られているBitcoinの多くの大規模なアプリケーションとサービスがbitcoinjで構築されています。

 

特徴

  1. 高度に最適化された軽量SPVモードは、ブロックチェーンのごく一部をダウンロードするだけで、安価な仮想プライベートサーバやスマートフォンのような制約された環境で最適に使用することが出来ます。
  2. ウォレットクラスの暗号化、手数料の計算、マルチシグ、決定論的鍵の導出、接続可能なコインの選択と操作、拡張機能のサポート、イベントリスナーを備えたウォレットクラスにより、残高の変化を最新の状態に保ちます。
  3. クライアントとサーバーの間でのマルチシグコントラクトを設定、交換チャンネル、マイナーの手数料を回避する様な少額決済手段に対してサポートします。
  4. ネットワーク処理の非同期とスレッドごとの接続の両方を提供する事で、Tor*1のように拡張性と遮断性を選ぶ事が出来ます。
  5. Bitcoinコントラスト機能を使用するアプリを簡単に実装することが出来ます。
  6. アプリを開発する時の基礎として使うことが出来るシンプルなGUIウォレットアプリです。GUIウォレットアプリのカスタマイズとJAVAを必要としないインストラーの作り方はチュートリアルを見て下さい。
  7. ネットワーク、支払いのプロトコル、チェーンファイル、ウォレットの操作する為のコマンドラインツール。
  8. 強力Bitcoinスタンダード(Btcoin上に構築された新しい拡張機能プロトコルを標準化)をサポートしています。


bitcoinj入門

以下からダウンロードすることが出来ます。

  1. JAR形式のライブラリ
  2. Moven/Gradle*2
  3. githubのコード

 

必要に応じて読んで下さい。

 

 

※重要告知:もしbitcoinjをアプリケーションに使う場合はリストにサインアップして下さい。そうする事でクリティカルなバグが見つかった場合や新しいバージョンが利用可能になった場合にいち早く知る事ができます。

 

ライセンス

このライブラリはApacheのライセンスを受けています。 ライブラリを使用することは、そのライセンスの条件に同意したとみなされます。 特に、第7章と第8章に注意してください。このライブラリはが安全に使用でき、バグがないことを保証するものではありません。実際にバグの可能性があり、このコードの使用に起因した損失は起こりえます。ですので、ライブラリで配布されているライセンスに従い、開発者は訴えることはできません(ライブラリ全体を自分で書いたようなものです)。 それに従えない場合は、このライブラリを使用しないでください。

 

ドキュメンテーション

0.14.5 API documentation

さまざまなトピックを網羅した更に深い記事がいくつかあります。順番に読むことでとても良く理解する事が出来きます。

 

基礎

 

CoreAPI

 

bitcoinjの動画

 

高度な機能

 

※0.3から開始する各リリースのリリースノートを読むことができます。

 

コミュニティ

  • Githubにあるbitcoinj見てぜひスターをつけて下さい
  • メーリングリストにある改善点について話し合い質問に答える
  • データベース上の問題の調査や報告(助けが必要な場合はデータベースを使わずにメーリングリストを使って下さい)
  • google+のbitcoinjに従って下さい
  • IRC(インターネット・リレイ・チャット)チャンネルに参加して下さい

 

*1:P2P技術を利用したSOCKSプロキシ

*2:Javaベースのオープンソースプロジェクト管理ツール

Transaction Malleability

Overview
The transaction can hold multiple transaction hashes while keeping the validity of the signature. So, double payment becomes possible by changing the expression without changing the meaning of the input script. This characteristic is called "Malleability".


Leading example of Mariability
For implementations that only validate with transaction hashes, It is interpret the transaction as floating in the air without being taken into the block chain. So, It is able to resend the transaction many times.
Let's see some examples of changing the TXID while actually maintaining the validity of the signature.


Example 1:Change the TXID by adding extra OP_CODE to ScriptSig
As mentioned above, when signing transaction data, the input script is excluded from the subject of signature(When signing, use the ScriptpubKey instead of ScriptSig). But, when generate TXID, contents are required ScriptSig.

Double payment becomes possible by changing add "OP_PUSH_DATA2(0x4d)" to origin script,the follow is one example.
f:id:adrenaline2017:20170620162819j:plain
0x48:push data of 72byte
0x41:push data of 65byte
0x4d:Push following 2-byte data
0x00:push 0 to stack

Although the TXID changes , the behavior of the script is exactly the same. there is no problem in verifying the signature.So it is possible to pretend that the transaction has not been completed and it use double payment from the original remittance.

※You can confirm that the signature is valid even if you add Unnecessary OP_CODE to the script from this site.


Example 2:Changed OP_CODE (Push_DATA) of ScripSig
The PUSH_DATA have multiple types as follow. Choose according to the number of bytes to be pushed.However, it is possible to use PUSH_DATA corresponding to a large number for a small number of bytes.
f:id:adrenaline2017:20170729074148j:plain

Same as Exsample 1. the TXID changes , the behavior of the script is exactly the same, and there is no problem in verifying the signature.


Example 3:The signature value S of ECDSA

The ECDSA take two signature values of r and s.The signature is valid even if plus and negative invert of signature value s.It can reverse plus and minus by subtracting s from n.

s = k -1 (z + dr) mod n

k:random number
z:massage hash
d:private key
r:digital signature value
n:Number of points existing on plane of the prime number


Example 4:Specify the range in which the signature is included by Sighash

SIGHASH ALL is used In generally. But when Using SIGHASH_NONE which does not include any signature at the output.Since it is possible to change the content of output any number of times,and the TXID will change.

・SIGHASH_ALL :Sign the reference destination and output of all inputs.
・SIGHASH_NONE:Sign all inputs but not sign to output.


Actual damage from Malleability
A large problem arises in Wallet due to Transaction Malleability too. A problem arises particularly with wallets that use only transaction ID to identify transactions. If a tampered transaction is captured and confirmed in the block before the correct transaction, the balance in Wallet will be mismatched. Then the correct transaction is considered double payment from the node and it will be processed as an invalid transaction.

Mining

Overview
The Mining has two meanings.

・Currency supply mecanism
・Decentralized emergent consensus


①Currency supply mechanism
At a point of jan 2009, Remuneration of mining for 1 block was 50BTC. In other word, mining means that supply new bitcoin to network.Mining reward is designed to reduce by half in 210,000 blocks (about 4 years).In 2140, it will reaches upper limit of 20,999,999,98bitcoin (13.44M block). If the reward is less than 1 satoshi, the reward for mining is lost.

f:id:adrenaline2017:20170614134249j:plain


The total number of Mining reward
Code to calculate how much the upper limit issue quantity.

#primary miners reward
start_block_reward = 50
#reward is halved every 210,000 blocks
reward_interval = 210000

#upper limite number
def max_money():
    current_reward = start_block_reward * 10 ** 8
    total = 0
    
    while current_reward > 0:
        total += reward_interval * current_reward
        current_reward /= 2
    return total

print("Total BTC to ever be created:", max_money(), "Satoshis")


②Decentralized emergent consensus
Participating node in the Bitcoin network reach a global consensus based on public ledger (blocking chain) without Centralized trust model. This is Bitcoin revolutionary invention that is dispersive mechanism to decentralized emergent consensus.The consent obtained by asynchronous interaction is established by node who following below rules.

・Verification of independent transaction
・Accumulation of independent transaction
・Verification of independent new block and blockchain transmission
・Select of independent blockchain


1.Verification of independent transaction
It is verified whether valid transaction.

・Is the transaction grammar and structure correct?
・Input and output is not empty?
・transaction data size does not over define upper limit of block size?
・Output value and Total value in the range of 0-21,000,000BTC?
・Input is not coinbase tranzaction(hash=0, N=-1)?
・”nLockTime” is lower than 31bit?
・transaction data size ≧100?
・Do not over upper limit of signature operation times?
・scriptSig can push the number to stack?
・scriptpub in isStandard format?
・Same transaction exist in the transaction pool or main branch blockchain block?
・Input is not referring to the output referred to from other transactions in the transaction pool?
・In the case of input refer to output that is coinbase transaction, number of conformation is not over than 100?
・Output referred to by input does not already used?
・Sum of each input is in the range of 0-21,000,000bitcoin?
・Sum of inputs smaller than the sum of output?
・ScripSig can unlock to corresponded scriptPub?


2.Accumulation of independent transaction
Mining nodes do not only add to memory pool and transaction pool after transaction verification, this transaction is collected to candidate block.

Priority of transaction processing
Elements related to priority of transaction
・Transaction age
・Transaction fee
・Input value of transaction size
※transaction age;The number of blocks that have elapsed since the UTXO was recorded on the blockchain

Calculation of priority is as follow

Priority = Sum (Value of input * input age ) / Transaction size

High Priority is necessary larger than 57,600,000 of priority

High Priority > 100,000,000 satoshis * 144 block / 250 byts = 57,600,000

Priority

Priority(Even if no fee) > Transaction fee/Transaction data size  > No fee

Validity period of transactions
Valid transactions have no deadline. But, If the transaction sent to the network is not transmitted to the whole, it needs to be sent again. Because, Memory pool of mining node is temporary storage.

Generation transaction
It is said that “coinbase transaction”. It is the reward you get when the mining is successful.For get remuneration,the miner generate "generation transaction" as payment of own wallet. This transaction is special and does not have UTXO in the input.Instead, the transaction has an input called coinbase.The output is a script that sends remuneration (mining remuneration and fee) to miner's address.

Constraction
Transaction hash does not have refering previous hash, Instead it fill with 0 of 32 bytes.All Output index is set to 0xFF (255), and scriptSig is a free field (coindase data) which minor can use arbitrarily.
f:id:adrenaline2017:20170614144925j:plain

Coinbase data
As defined in BIP0034 when version is "2", block hight must include in Coinbsae data.

For example,when scriptSig is...

03443b0403858402062f503253482f

03:
push 3byte to stack
443b04:
It means block hight 277,316 (little endian )
0385840206:
extra nonce
2f503253482f:
defined in BIP0016 support P2SH


Proof of work algorithm
Change the value of nonce, hash the block header with SHA256 and continue to calculate until it falls below the target value.

Characteristic of hash value with SHA256
・If input changes even for one character, the value output is completely different.
・If input is same, the same output will be returned no matter how many times calculated (anyone can check it).
・Input value can not calculate from output value.

The way to find a value smaller than the target only is only by brute force.So this work is called trust work.Verification of work is very easy. Just compare the hash value of the block header containing the answer of nonce.

Simple code to explain proof-of-work algorithm

import binascii
import hashlib
import time

max_nonce = 2 ** 32 

def proof_of_work(header, difficulty_bits):
    #When "difficulty_bits" value increase,"target" value decrease and up to difficulty
    target = 2 ** (256-difficulty_bits) 
    
    #nonce is 0 to 4294967296
    for nonce in range(max_nonce):
        #header + nonce = 'test block with transactions0, 1, 2...'
        hash_result = hashlib.sha256(header.encode('utf-8') + str(nonce).encode('utf-8')).hexdigest()
        
        # check if this is a valid result, below the target
        if int(hash_result, 16) < target:
            print ("Success with nonce %d" % nonce)
            print ("Hash is %s" % hash_result)
            return (hash_result,nonce)
            
    print ("Failed after %d (max_nonce) tries" % nonce)
    return nonce

    
if __name__ == '__main__':
    
    nonce = 0
    hash_result = ''
     
    # difficulty from 0 to 31 bits
    # Increase "difficult_bits" one by one   
    # change "nonce"value until you enter traget range.
    for difficulty_bits in range(32):
        difficulty = 2 ** difficulty_bits
        print ("Difficulty: %ld (%d bits)" % (difficulty, difficulty_bits))
    
        print ("Starting search...")
        
        # checkpoint the current time
        start_time = time.time()
        
        # make a new block which includes the hash from the previous block
        # we fake a block of transactions - just a string
        new_block = 'test block with transactions' + hash_result 
        
        # find a valid nonce for the new block
        (hash_result, nonce) = proof_of_work(new_block, difficulty_bits) 
        
        # checkpoint how long it took to find a result
        end_time = time.time()
        
        #Time to proof of work
        elapsed_time = end_time - start_time
        print ("Elapsed Time: %.4f seconds" % elapsed_time)
        
        
        if elapsed_time > 0:
            # estimate the hashes per second
            hash_power = float(int(nonce)/elapsed_time)
            print ("Hashing Power: %ld hashes per second" % hash_power)

Calcurate of target
whether nonce is right answer, blockhash will attain an objective depend on whether.Specifically, when block hash containing the value of certain nonce falls below the target value.The equation is as follows.

target value =(low of bit 3byte)* 2 ^ (8*( upper of bit 1byte - 3))

Let's calculate the target with block number 285566 as an example!

①Bits of block number “285566” is 419537774
②hexadecimal of 419537774 is ”1901a36e”
③lower 3 byte of bit is 0x01a36e
④Upper 1 byte of bit is 0x19(in the case of decimal 25)
⑤Substitute ① to ④ for above formula
⑥Check to calculated value is lower than the hash of block285566
※The hash of block number 285566 is "000000000000000149c7840096f76475d018ed0c05f1a688e608b8254a39d796"

#target calculate function
def target(bits):
    hexadecimal_bits = '%x' % int(bits)
    upperbits = hexadecimal_bits[:2]
    lowerbits = hexadecimal_bits[2:]
    upperbits = int(upperbits, 16)
    lowerbits = int(lowerbits, 16)
    target_value = lowerbits * 2 ** (8 * (upperbits - 3))
    number =64 - len('%x' % int(target_value))
    target = (number * "0") + '%x' % int(target_value)
    return(target)

#Bits of block number “285566” is...
bits = "419537774"
target_value = target(bits)

#Hash of block number "285566" is...
hash = "000000000000000149c7840096f76475d018ed0c05f1a688e608b8254a39d796"

#It can see that the hash is smaller than target_value
target_value > hash

Calculate of difficulty
Difficulty is indicator of the difficulty of finding the target block hash.

difficulty=upper limit value of target / current target

Let's calculate about difficulty of block number43420(Ask for difficulty 3.78)!
①Calculate current target
※bits of block number43420 is 474199013(hexadecimal 1c43b3e5)
②Next, calculate upper limit value of target
※The upper limit value of the target is defined as constant 0x1d00ffff
③Finally,by difficulty=upper limit value of target / current target

#target calculate function
def target(bits):
    hexadecimal_bits = '%x' % int(bits)
    upperbits = hexadecimal_bits[:2]
    lowerbits = hexadecimal_bits[2:]
    upperbits = int(upperbits, 16)
    lowerbits = int(lowerbits, 16)
    target = lowerbits * 2 ** (8 * (upperbits - 3))
    return(target)

#calculate about target of block number43420
bits = "474199013"
current_target = target(bits)

#The upper limit value of the bits is defined as constant 0x1d00ffff
upper_bits = int("1d00ffff", 16)
upper_target = target(upper_bits)

#Formula to calculate difficulty
difficulty= upper_target / current_target

print(difficulty)


3.Verification of independent new block and blockchain transmission
The node Independently verification whether effective block ,before new generated block sent to other node.
Whether valid chack it with below lists.

・Is the block's data structure and syntax valid?
・Block header hash value is smaller than target?
・Time Stamp of block is earlier than 2 hours later of the node's time?
・Block size is with in range?
・First transaction is "coinbase generation transaction"?
・The transaction included in the block satisfy the transaction rule?
※Invalid blocks are repelled by other nodes based on this rule.


4.Select of independent blockchain
The node rebuilds the new block by connecting it to the existing block chain.The node has three kinds of block set.

main blockchain
This is the chain with the most ”difficulty” accumulation. In other words, it can be said to be a chain with the most blocks.
Sometimes it has a branch of sibling block. when the branch exceeds "difficulty" of main blockchain in the future,This keeps it to be replaced by the main.

secondary blockchain
It is said that Fork.The fork occur when different nodes mined same block.Some times,there are times when new block extended except with main chain.

orphan
If a parent is not found in an existing chain even if it receives a valid block from another node, it becomes an orphan block and is retained in the orphan block pool.
The oprhan block occurs when the order of blocks mined almost simultaneously is received in reverse.

extra noce
Extra nonce is included in the coinbase transaction which is a transaction paying a minor reward.
In the early days difficulty target was low, so it was possible to find a hash value lower than target with 400 million nonces.Updated the Time Stamp and discovered a hash value lower than the target by obtaining a hash value.But, as difficulty got better, 400 million nonces were used up within 1 second, and more space was needed to find effective blocks.so,using 2 - 100 byte data space of coinbase transaction it became possible to calculate in a larger range.

Remove all ads