On search, click "Change columns",[1] click the arrow(s) to bring "Priority" on top, then save: works, doesn't it? Alternatively, save a common search with all the config you want. http://www.bugzilla.org/docs/4.4/en/html/userpreferences.html#savedsearches
Nemo