Centos 6 python-crypto version and Abe Blockchain Explorer

Trying to make Abe blockchain explorer work on Centos 6 for Litecoin and I kept running into Merkle hash errors:

MerkleRootMismatch: Block header Merkle root does not match its
transactions. block hash=5c64e0d3548c8dd2b850798d77c18070a3032505ce601
fa3685d329ff0bdebf4

So I put some debug prints into Abe/Datastore.py and Abe/util.py and re-run:

block_id:tx_id 8994:9075
import_block() block containing 2 transactions
   tx # 0  raw: 010000000100000000000000000000000000000000000000000000
00000000000000000000ffffffff08041e34974e028603ffffffff0100f2052a010000
00434104680cd836ad21eee163fd6dce9bcf51a11ab963301a2062b8ade9088ba356cc
f9a23f88f8c0a7890a752b0c9192d484c49dfa7db96d3d462e7326c6e8b01aeecfac00
000000
   tx # 0  len: 135
   tx # 0 hash: 9bf3e18a8563029fa8999587603721ea9c91bdd09adef6a11c1731
cbe6ee171e
   tx # 1  raw: 01000000043350859b760f33ab33224cda9c4fe1fea43684bd220b
ef7bbc282bfa98483b0b000000004a493046022100e9113332c92fb1552df4b4cdabdb
c92781d7ebb267db3381a9a016b1ab9b3173022100ff4cf10b4a4379b8a7a4a73a4e04
731b69ca50a75fcc5937ce851088ff41456401ffffffff61d9c59e262ca13e7f71789c
51ad00ad0f4d4de469475ef18d71f5ef9f565956000000004a493046022100899767ea
d9de77504a0b6526fb0f22917196a1bb3d10bec4cb341d62dc4b61f802210094f53f3a
5aa1b33effccb967784f52196fdb7af5a4a532d8f44597f013bb5c4001ffffffffd37d
c556e44a1ba47c514be37c9b0d52f4bf995caf365840b4c073e846f76bdb000000004a
493046022100c2a7e420363f72e228b3627f43b3eafa5bc6835fef0a6cc757c94c88a2
10b912022100e095e351b24846f0e043c22ecf60a07f411f3e77c22c4b1c918ef4c51f
a1496601ffffffff4a4bde7c33e83ef725cc4859469c06e5f786fb7cdc0b0e05e19cc1
ea8cf132280000000049483045022100ae6eae6e8c9be3a0b6d629f247a22ae36bc6a8
df8df383a2883dd8dae189d4cd0220275b17413a5f523564180a080c5daa9a6a415d0a
6c72e604f3ace0063c0431b201ffffffff0100c817a8040000001976a9140c397b7bfd
2b26a16f553658c831e5eb5042fc3e88ac00000000
   tx # 1  len: 503
   tx # 1 hash: 32b0a4550306abab79f4e3646bdbe98715bf5c4f75eed6acd1fd99
d9b36e11ef
   merkle() Number of tx hashes in tree 2
      MerkleRoot calcluated: 5a6a74a7ceeef7ec6bb67b53e1c1ce2c5decf8382
986b257ec4f0498b041218c

Checking block-explorer.com for block 8994 I could compare the individual transaction hashes to see the second transaction had the wrong hash calculated which then throws off the Merkle root hash of the (simple, for this block)tree.

Long story short, Centos's python-crypto-2.0.1 rpm is based on pycrypto-2.0.1 which contains a bug in the SHA256 hash routine. The pycrypto-2.1.0 changelog states:

- Fixed padding bug in SHA256; this resulted in bad digests whenever
  (the number of bytes hashed) mod 64 == 55.

So, if you notice the debug prints above show the length of the transaction with the bad hash calculation is 503 bytes...

503 % 64 = 55

There's your problem. Building pycrypto-2.1.0 and forcing it's use with PYTHONPATH=/path/to/where/you/setup/pycrypto-2.1.0 made everything work and import the previously problematic block.

On another note, while troubleshooting this, I noticed that the ancient 2.0.1 verison of pycrypto used in Centos6 provides RIPEMD160 by the name RIPEMD. Abe/util.py has a fallback to use hashlib so no exception is raised. I'm not sure of the pro's and con's on the two implementations, but you can use the pycrypto one by editing the import statement in Abe/util.py like this:

change: import Crypto.Hash.RIPEMD160 as RIPEMD160
    to: import Crypto.Hash.RIPEMD as RIPEMD160

My preferred fix for the SHA256 bug is to rebuild the rpm to either patch that specific bug, or simply update to v2.1.0 assuming it's not going to break anything else that depends on it. I'll try and put that in another post.