I've had a similar problem for quite a while. A page called "foobar." and one called "foobar" are treated identically if you try to delete but differently if you create them. The result is the that "foobar." stays around and is not deletable.
Works for me on a version downloaded from SVN not too long ago.