Hi Björn,
basically ZIM files can hold whatever content, so having videos in them is theoretically no problem at all.
1st question: Does Kiwix (the most popular ZIM reader) support video? As Kiwix is using XUL, based on Firefox, chances are that Ogg/Theora videos may work out of the box.
2nd question: Do the ZIM exporter (either Extension:Collection or Kiwix' perl tools) export embedded videos as well? When using the perl tools it should be no problem to amend if needed.
/Manuel