Fix very poor compile-time in PR19499 due to excessive tree walks in