Need advice about a trade

I recently traded my shaman for a priest account. Everything was going good until the guy who I traded messaged me that he wanted his account back. I said no, and he was ok with it. He has messaged me 3 or 4 times since and every time I tell him no he is ok, until this last time. Now he is threatening to tell blizzard that I stole his account if he doesn't get it back.

Also he has already traded my account to another guy, and gotten it back some how.

What should I do? What can I do to keep him from taking it?

I have already changed the email and the contact info, but other then that am I screwed cause you can't change secret question?

Thanks.