Clean up some signedness oddities in this code noticed by clang.