Fix some more moving-from-moved-from objects issues in SmallVector