Support vector casts in more places, fixing a variety of assertion