Which is more intuitive depends on your mental model. If you think of the metadata panel as something that opens and closes, pressing up to bring it up and pressing down to bring it down makes a lot of sense. If you think of it as part of the page, that is currently out of view, it's the opposite. The current layout of MediaViewer reinforces the first model, IMO, with the image not scrolling when the metadata panel moves.
Either way, it's important to support both models. We currently do that by flashing the panel controls if you press the wrong direction - I think that's a good approach but the implementation is basically unnoticeable and should be made much more prominent.