I stored a .psd file in <project>\files folder to make a .png file, which is stored as asset in the apk-file. Even this file is not added/selected in the projects file list, it is included in the apk-file. The apk-file was round 40MB with the psd file in <project>\files. If a removed this file, the apk-file was round 2MB as before!
The files are supposed to be managed from the IDE. All the files are packed as a result of how the Android packager works. This is why there is a warning for that.
There are some features that will only work with the files added with the IDE.
Shorter answer: it could have been designed differently, but the current design is also reasonable