Controller collider not working after some time in playmode

Hi,

I'm using the Oculus Rift and set up my scene in the Newton VR example scene (deleted all unwanted objects) and put my objects in the scene. After a while of grabbing and throwing my objects around, the right controller is not able to become a collider (for pushing objects) but it still able to grab objects. The left controller has no issues (I'm right handed).

Is there a way to solve this issue?