More asserts around physreg uses