> I think the serializer should just drop empty references

This seems like the wrong place do to that. I suggest to do this in the model itself, very similar to what we are doing with empty alias groups in lists of alias groups: https://github.com/wmde/WikibaseDataModel/blob/5c78e35ee1a382d3b132ddfd760772b94d600daa/src/Term/AliasGroupList.php#L114-L127

Cheers

--
Jeroen De Dauw - http://www.bn2vs.com