A few thoughts:
1) If part of the issue is the more intense use of Checkuser and Oversight, a solution would be to expand checkusers and overseers outside of Arbcom.
2) A larger Arbcom would not necessarily be a bad thing if it thinned out the workload, thus forestalling burnout and enhancing case completion time.
3) Allowing admins to "rule" on user cases is fraught with danger, in my opinion, in that favoritism/cabal accusations will be rife. Various user essays notwithstanding (unabashed plug), I fear that most people do _not_ view adminship as a referendum on editor trust, and may feel rather differently had they known that these people are now authorized to judge these cases. Yes, this is somewhat oxymoronic as admins already have the power to block and propose bans, but I think there is a wiki-wide perception that Arbcom has been selected with the idea of being a court, where individual admins are more janitor/policemen. Now if the culture could be changed, that would be a different story.
4) Part of the issue, and I confess to ignorance in the greater realm of jurisprudence, being neither lawyer, nor judge, nor paralegal, is that the current process may be somewhat over-bureaucratic. As mentioned earlier, if a quorum of three Arbitrators is sufficient, and if much of the current formal drafting and voting can be replaced with a more streamlined posting of the final opinion, perhaps cases can be completed more rapidly.
--Avi