rbtree: move some implementation details from rbtree.h to rbtree.c