If the new image is exactly the same as the one we had, it makes sense to
delete one, yes. And usually you would remove the last one to be uploaded.
For instance some people may be using of the images with the old name. We
could delete them but place a redirect to the other copy, that seems a good
compromise. Or even merge both copies at the old name, so the uploader
appears in the history.