Simplify the known retain count tracking; use a boolean state instead