In my opinion, the proper way of handling activities is to close them using activity.finish every time you exit one of them. This way, you will never have to worry about having left an open activity.
If for some reasons you need to keep activities open, all the way long, you could use a boolean flag in order to tell each one of them to finish when you choose so. Still, I wouldn't advice doing so from my experience with such coding.