System.exit() does not kill your app if you have more than one activity on the stack. What actually happens is that the process is killed and immediately restarted with one fewer activity on the stack. This is also what happens when your app is killed by the Force Close dialog, or even when you try to kill the process from DDMS. This is a fact that is entirely undocumented, to my knowledge.
The short answer is, if you want to exit your application, you've got to keep track of all activities in your stack and finish() ALL of them when the user wants to exit (and no, there is no way to iterate through the Activity stack, so you have to manage all of this yourself). Even this does not actually kill the process or any dangling references you may have. It simply finishes the activities.