Add a relocation visitor to lib object. This works via caching relocated