MPU9250 works on I2C but not SPI

I have a strange problem with the MPU9250. I have configured it to work with the I2C interface and it works perfectly (including magnetometer). When I configure it to work with SPI, I can read the WHO_AM_I register and all the regiters that I configure during startup, successfully. I, however, do not get valid readings from the sensor registers. I have tried disabling the I2C mode, it didn't help, and I did reset the sensor paths as indicated in the MPU6500 datasheet. All register reads with SPI are valid with the exception of the sensor registers. After 16-bit two's complement (the same method used in the I2C case) the sensor values are either 0 or -255/-256 for each sensor.

Is there something that one has to configure differently if using SPI as oppose to I2C?