Use a SmallVector and linear lookup instead of a DenseSet - SourceMap values