3D text as child of controller unreadable when using Automatically Set Controller Transparency

I have added a 3D text object to the Vive controllers to be able to tell them apart in VR (just an R for the right controller and an L for the left controller). But when I tick Automatically Set Controller Transparency to true the text becomes a semi-transparent white plane. The 3D text gameobject is a child of the controller, not of the Model.

What I can see is that the material on the 3D text changes from the Font Material to Font Material (instance). If I don't tick Automatically Set Controller Transparency the text is readable and the material is the Font Material that belong to the selected font.

How can I correct this, so that the controllers become transparent, but the text is still readable? Can I set the controller transparency from a script? I noticed that both CurrentVisibility and SetVisibility() in NVRHand are private, so I can not set or call that from my own script.

Thanks!