Do the image views cover the whole panel? If so, their click events will be consuming the touch for the panel. You will need to leave an empty space on the panel, perhaps with a label and a move Icon to make it obvious to the user where they need to touch to move it.
If not, can you post a small test project showing the problem. So we don't have to build one.