ifb: fix oops when loading the ifb failed