bitcoind follows the JSON-RPC-over-HTTP spec for reporting errors, but it sounds like Sergio's library follows a different spec. You're not the first person to complain that bitcoind returns HTTP error codes; here's a one-line patch to change that behavior with a "-rpcerrorstatus=200" argument/option:

Let me know if that solves the problem and/or if it causes any other issues; if it doesn't, I'll submit it as a PULL request.

Fixed, thanks for finding it.

To be absolutely sure, you need to extract your private keys and then run a tool to look for them in your wallet.dat file (or other files on your disk).

If you are able to compile a custom version of bitcoin and run python, here's how:
  https://gist.github.com/1361001

If you can't compile a custom version of bitcoin or run python code, then you'll have to trust other people to thoroughly test.

I've uploaded release candidate binaries for Bitcoin versions 0.4.1 and 0.5.0 to SourceForge:
  https://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.4.1/test/
  https://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.5.0/test/

The 0.4.1 release is the old wxWidgets-based Bitcoin with a critical fix to the wallet encryption feature.
The 0.5 release is the new Qt-based Bitcoin (which also includes the wallet encryption bugfix).

Report bugs using the issue tracker at github:
  https://github.com/bitcoin/bitcoin/issues

No, I've been busy. Although I have in the past said that I'm worried about long-term incentives for relaying transactions (and am not worried at all about the block reward dropping and being replaced by fees).

I skimmed the paper, and I'm really pleased to see very smart people thinking hard about the incentive structures built-in to Bitcoin. I think most people see all the digital signatures and peer-to-peer networking technology but miss that much of the brilliance of Bitcoin is how the incentives are designed.

Overall, I think the paper is most valuable as a demonstration of how to prove that a particular solution to the problem could work-- whether or not the particular solution the authors propose (rewarding the relaying nodes) is the "best" solution I'm not sure -- there are lots of dimensions of 'goodness' here -- lowest cost, fastest transaction confirmations, easiest to implement, most compatible with the network we have now, etc.

A while ago I proposed another potential solution:  have relaying clients drop their connection to 'greedy' nodes. If you have a node that sends you new blocks but isn't relaying you new transactions, maybe drop your connection to it and ban it's IP address from re-connecting to you for a while. Or maybe don't relay (or relay after delaying a couple of minutes) new-block messages that you first see from that node.  The idea is that a mild dis-incentive should be sufficient to encourage nodes to do the right thing and relay all valid transactions and blocks.

Figuring out "isn't relaying you new transactions" in a spoof-proof way would be a little tricky-- you want to see real transactions, not miner-generated "I'm pretending to be a good network citizen" transactions. A scheme where you relay transactions to half of your peers and then monitor the other half to see if you receive the transaction back from them should work to detect nodes that are relaying many-fewer-than-normal transactions or blocks.

---

I've also said repeatedly in the past I'd like to see more diversity in the networks used to transmit bitcoin transactions/blocks (with bridges between them so we're all working on the same block-chain). There doesn't have to be just one right answer to this problem, and I would love to see people simulate or experiment with variations on the existing network protocol or radically different protocols.

You mean the wiki page that describes an overly-complicated scheme with your pet feature that nobody else likes (hexadecimal amounts)?

And that we're all ignoring because we don't feel like getting into wiki editing wars with you (see the history from 9 May)?

Huh what?  Version 0.5 supports drag-and-drop of bitcoin: URIs. And there's a pull request pending for click-to-pay support.


One fear is bitcoin-address-rewriting malware, like the URL-rewriting phishing malware we have today. Actually, combining the two would be very effective (direct the user to a phishing site where all the bitcoin: URIs pay or donate to the scammers). We need better ways users can be certain they are paying who they think they are paying.

Code review and testing for the proposed fix is welcome:
  https://github.com/gavinandresen/bitcoin-git/tree/encryptionbug

Here's how I tested:

I dumped private keys from an unencrypted wallet (using bitcointools).

I wrote a little tool that took a list of private keys and a filename and reported whether or not the 32-bytes of any of the private keys appears anywhere in the file.

I also hacked a copy of bitcoin to dump any newly-generated-private keys into debug.log before they are encrypted and written to disk.

I verified that new keys for an encrypted wallet are never written to any of the Berkeley DB database files by encrypting the wallet, invalidating all of the existing keypool keys, generating new, encrypted keypool keys, sending bitcoins to a new, encrypted key, and checking all of the files for any unencrypted copies of the new private key.

So the problem became how to deal with old, previously-unencrypted keys that were ending up in the wallet.dat file.

I hoped that telling Berkeley DB to 'compact' the file would remove the 'slack' space that contained the old data (the root of the problem is BDB doesn't actually completely delete/overwrite data when you delete a key/value pair; if I missed a setting that makes it do that, please let me know). Doing that reduced the number of old private keys found by my tool, but didn't eliminate the problem.

Pieter worked on a different solution in parallel-- completely rewriting the database to a new file when encryption is turned on, then, assuming the rewrite succeeds (we trust BDB to do what a database is designed to do-- reliably write data to the disk), replace the old wallet.dat with the new wallet.dat.rewrite. During that process, all keypool keys are marked as 'used' (which actually means just not writing a 'pool' entry for them in wallet.dat.rewrite).

That works-- no old private keys made it into the new wallet.dat.

But during testing I ran into two issues:

1) The wallet.dat file isn't the only place the old private keys were found; the database transaction log (database/log.000..N) and some of the __db.00N files also contained them.  Adding a call to remove() the database environment at shutdown fixed the __db.00N issue; I had to write code to remove the database/log.000..N file on clean shutdown.

2) If more wallet database operations were performed after the rewrite (I had written code to top-up the keypool with new, secure keys before locking the wallet), old private keys could end up in the new wallet.dat.

To fix both of those issues, we modified the code so that a shutdown happens after wallet re-encryption.

... which reminds me of something else I need to test:  if you startup with a wallet encrypted by bitcoin versions 0.4.0 or 0.5.0, the wallet is re-encrypted and all of your old keypool keys are considered used.  (I believe it does not shutdown after doing that, though, but it should)

---

We've never made any guarantees that unencrypted private keys will not end up on your hard disk; if you have a virus on your system that can directly read blocks off your hard disk then it can almost certainly also read system memory, and could steal your wallet passphrase the first time you sent bitcoins.

Filling up free space on your disk (as suggested in this thread) might work, but it is a much better idea to send all of your bitcoins to new, 'born-encrypted' keys. Once fixed binaries are available, that will be easy-- just upgrade and then send all of your bitcoins to yourself, using newly-generated addresses.

---

Suggestions for more radical solutions, like storing private keys in a separate file, are out of scope. We got what we got; I personally think that a really good "deterministic private keys" solution, where the private keys for your wallet are derived from a passphrase that is only in your head (and maybe written down and stored in your safe deposit box) would be a better use of developer time rather than reworking how we store private keys on disk.

Even better would be further work on multisignature/multidevice solutions so even if your private keys are compromised an attacker can't spend your coins.

Luke-Jr is planning on supporting 0.4-based releases (finding somebody to fix wxWidgets-related bugs is an issue, though).

The issue I have with calling any pre-1.0-release 'stable' is it implies a level of maturity that I don't think we're at yet. I can see 1-year develompment->unstable->stable release cycles once we're at a solid Bitcoin 1.0 release that I can actually feel comfortable recommending to my non-geek relatives.

My fear is that developers would happily code away and use the development branch, bugs would pile up against the unstable branch (and would get ignored because developers were happily coding away on dev, and nobody really wants to do bug fixing or QA testing), and unstable would never become stable enough to tag 'stable.' But I've never led an open source software project before, so I might very well be wrong (best way to convince me is to point to other small open source projects that we can emulate-- I don't think emulating big projects like Ubuntu will work).

I agree that when a fix has been tested and is available an alert to the affected versions is a good idea.

Unfortunately, TruCoin ran into a funding crunch because an investor got cold feet and had to stop paying for anything besides directly-related-to-TruCoin work.

That's not quite right-- you need to exhaust all of the keys in your 'key pool' to be safe, so you'd have to ask for 101 new keys.

Part of the fix is marking all of the keys in the keypool as used.

A serious bug was been found in the "encrypt wallet" function of bitcoin versions 0.4 and 0.5: private keys may be left unencrypted in the wallet.dat file after encryption.

If your encrypted 0.4 wallet file is stolen, an attacker may be able to recover some or all of your private keys and steal some or all of your bitcoins.

The development team has been working on fixes for bitcoin versions 0.4 and 0.5, but it will take at least a few days to test them thoroughly. Until they are available, you should assume that your 'encrypted' wallets are as vulnerable as an unencrypted wallet, and follow all the best practices for keeping them safe (see here for a list).

It is embarrassing and astonishing that this critical a bug was not caught before the 0.4 release; constructive suggestions on how to improve the testing and release processes that do not assume access to hundreds of thousands of dollars of funds to hire security consultants or QA teams are welcome. Getting sufficient testing of code BEFORE it is released has been a chronic problem for this project.

Discussing how to fix this in IRC now.

This IS a show-stopper issue for the 0.5 release.

https://github.com/gavinandresen/Bitcoin-protocol-test-harness
Start with dumpblocks.py

No issues with export wallets/private keys. I share gmaxwell's concerns about making it easy to shoot yourself in the foot, but most of us are grown-ups and if you're talking using the RPC interface there are already plenty of ways to shoot your feet.

Remove private key I had issues with, because if you're using the 'accounts' feature then removing keypairs from a wallet (and their associated transactions) does unpredictable things to account balances. At the very least, I think it should tell you what effects it had.  Maybe a JSON result that tells you how account balances changed, e.g. { "" : 1, "John's Wallet" : 6.2, etc.}. That way, if it had an unexpected effects you would know to restore the wallet from backup.

And it seems like 'sweep private key' and 'merge wallets' is really the functionality most people want, not import private key/wallet keys. The only issue I have with them is they are slow because of the rescanning of the block chain, and they may not work or may not be secure if you don't happen to have the whole block chain downloaded.

This would be a good bitcoin.stackexchange.com question.

Do what deepceleron says... although you might want to copy JUST the two blk*.dat files (you don't need addr.dat or the database/ subdirectory).

Have you tried running rc3 on W2k?  I don't have a W2k machine to test on (I did sanity test rc2 on XP SP3), but I don't know of any reason it would fail.

I purchased this laptop:
  http://www.ebay.com/itm/290608305269

... to do Bitcoin 'gitian' builds.  It's a perfectly good laptop, runs nicely, but I'm re-selling it because its Core Duo CPU can't do 64-bit builds.

Specs are:
  1.83 Mhz Core Duo processor
  120GB memory
  2G disk
  Wifi

Running Ubuntu 11.04, includes laptop, battery and power supply.

Asking 50 BTC, I'll pay shipping anywhere in the US; if I don't get any takers here, I'll re-auction it on Ebay on Friday, Nov 11.

If you can, please help sanity test Bitcoin 0.5 release candidate 3 binaries:
 https://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.5.0/test/

Files and checksums are:

Linux and Windows binaries are created using the gitian-trusted/reproducible-build tool; the OSX binary was built by me.

I'll be writing up release notes this afternoon; assuming no 'show-stopper' bugs are found, these binaries will be the 0.5 release.

Please report bugs using the issue tracker at github:
  https://github.com/bitcoin/bitcoin/issues

 

Let me think... a problem that affects a small number of people... means you end up with 8 connections instead of more than 8... and has an easy, obvious workaround... and if fixed will be difficult to test thoroughly...

Not high on my priority list.

Things that ARE high on my priority list:

+ Getting the 0.5 release out the door
+ Improving initial startup and block download times
+ Solutions for wallet security and backup

I'm proposing one new RPC command:  'addmultisigaddress'  ... that combines several public keys into one BIP 13 -style newfangled bitcoin address.

You get the public keys from the 'validateaddress' RPC command, which I've extended to give the full public key if you give it one of your bitcoin addresses.

I extended the 'send*' RPC commands so they know how to send to the newfangled bitcoin addresses, and can send coins you received as multisignature transactions if you hold all the corresponding private keys (listtransactions will also show you them, getbalance counts them in your balance, etc).  So yes, anybody can generate new transactions to test...

Alan Reiner is proposing BIP 10 as the 'real' way to get multisignature transactions signed and spent:  https://gist.github.com/1321518
(no implementation yet, as far as I know).