No, it makes more sense for semi-admins to be able to act against admins. Admins are better equiped to handle "hostile" acts, and should have enough experience to handle semi-admins who do something wrong.[0] We cannot expect that from new users, and it is they who must be protected against less well thought out actions of inexperienced admins.
But admins shouldn't need actions taken against them except it extreme circumstances. In extreme circumstances we would want full admins handling it, because they are more experienced. It is the routine stuff (and actions against regular users are generally routine) that can be done by semi-admins.