The limit exists because deleting a page requires copying all of its revisions from the 'revision' table to the 'archive' table. I imagine it was a simple practical solution in 2003.
There's some discussion of the problem and links to other relevant bug reports at https://phabricator.wikimedia.org/T20493 .