Just an update for all who helped me on this.
I actually split up the big layout file in several smaller ones: one file for each panel.
Before showing a panel I check if the layout for it is already loaded, calling LoadLayout otherwise.
This sped up the initial startup of the app, spreading loading times over the whole lifecycle of the app.
Not all panels are always needed, and those who get loaded on demand, have very little delay, with no user impact.