groups: Consolidate the setgroups permission checks