ICM-20600 issues when controlled via N-Channel MOSFET

Hello,

We are using ICM-20600. We are turning it off via an N-channel enhancement type MOSFET when not used, and turning it on right before using the ICM for energy saving purposes.

Initially we had following topology:

In software, we were able to use ICM instantly after turning it on with transistor. By use I mean communicate via I2C.
We decided to change this topology because of high voltage drop on transistor. Since our control signal on gate and Vdd are both 3.3V, and load is on source side, Vgs was low.

Then we used following topology:

We saw very low voltage drop on transistor and now Vgs is high enough to work properly. But now, when we try to use ICM after turning on the transistor, we get error in I2C communication. If we add a delay of ~200ms after turning on the transistor, then ICM works ok.

Why does new circuit cause such behavior and what should we do in order to be able to use ICM almost instantly after turning on the transistor.

We suspect that the reason may be behind ground connections on ICM pins themselves. In the old topology, since ICM was on GND side, it was ok that all required pins were connected to gnd (decoupling capacitors, address pin and etc.). But now, they can be connected either to drain side of transistor, so that they all become gnd when transistor is on, or they can be connected to real gnd. ICM pins with ground connections were initially connected to real gnd on PCB(except 6th pin, the power supply ground, it was connected to drain side of MOSFET), and it didn't work. Afterwards, we tried connecting all these pins to drain side, but it didn't work as well. Perhaps, some of the pins should be connected to drain, while some should be connected to real GND. Can you please help?