Fix potentially N^2 behavior handling arrays with many of the