If the new image is exactly the same as the one we had, it makes sense to delete one, yes. And usually you would remove the last one to be uploaded. For instance some people may be using of the images with the old name. We could delete them but place a redirect to the other copy, that seems a good compromise. Or even merge both copies at the old name, so the uploader appears in the history.

Kind regards