You should put all the keys in the same folder. The most important thing is to use the same certSigningRequest.csr whenever you create a new key.
You can always delete existing keys and create new ones.
This is how my keys folder look:
You can see that I have in this folder: development certificate and provision file (default.cer and default.mobileprovision), distribution certificate, store provision, and push related files. All of them are derived from the same certSigningRequest.csr file.
The IDE uses the default files unless you explicitly set a different value.