Pass a MemoryBufferRef when we can avoid taking ownership.