Were they enabling it by putting directly into User:.../common.js or through Special:Preferences?  The former might be subject to weird caching but I always assumed it was properly invalidated on changes.  If using Special:Preferences, that sounds like a more serious bug and it would be good to know.