Make sure the op does not change the sign of the operand