Track reference count independently from clear().