Hi Micru,
Basically the comments are associated to each paragraph, and a little number appears next to it. When clicking it, it displays the comments.
There's already a similar implementation for MediaWiki. It's basically a modification of Extension:Comments and it allows to associate comments with sections. You can see it live at WebPlatform.org [1].
Best, Markus