Getting bad data at 200 hz

I can extract clean stable quaternions from the MPU-6050 fifo at 100hz, however, when I increase the fifo rate to 200hz, the quaternion output becomes *very* noisy. Any ideas what could be causing this?

Digging a little deeper:
If I instead read accel and gyro data from the fifo at 200hz, the accel data looks good, but the gyro data comes out very noisy. Reading the gyro data directly from the sensor register is however clean at 200hz.