spi: pl022: Fix race in giveback() leading to driver lock-up