On search, click "Change columns",[1] click the arrow(s) to bring
"Priority" on top, then save: works, doesn't it?
Alternatively, save a common search with all the config you want.
http://www.bugzilla.org/docs/4.4/en/html/userpreferences.html#savedsearches
Nemo