Don't do unnecessary work in polarity computation