Agent admitted failure to sign using the key. Permission denied (publickey). (TRIED EVERYTHING)

I've configured launchpad/bazaar before, but now it's really pissing me off:
I've added my id_rsa.pub to my launchpad

when I don't do a bzr launchpad-login I can run bzr branch lp:touei
as soon as I do: bzr launchpad-login james2432 it gives me the error in the title

log:

$ bzr branch lp:touei
You have not informed bzr of your Launchpad ID, and you must do this to
write to Launchpad or access private data. See "bzr help launchpad-login".
Branched 24 revision(s).

$ bzr launchpad-login james2432
$ cd ..
$ rm -Rf bzrtest/
$ mkdir bzrtest
$ cd bzrtest/
$ bzr branch lp:touei
The authenticity of host 'bazaar.launchpad.net (91.189.90.11)' can't be established.
RSA key fingerprint is 9d:38:3a:63:b1:d5:6f:c4:44:67:53:49:2e:ee:fc:89.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'bazaar.launchpad.net,91.189.90.11' (RSA) to the list of known hosts.
Agent admitted failure to sign using the key.
Permission denied (publickey).
bzr: ERROR: Connection closed: Unexpected end of message. Please check connectivity and permissions, and report a bug if problems persist.