Overview of NewtonVR
Our system allows players to pick up, drop, throw, and use held objects. Items don't pass through other items (rigidbodies), or the environment (non-rigidbodies). Rather, held items interact with other rigidbodies naturally -- taking mass into account. For example, if you have two boxes of the same mass they can push each other equally, but a balloon, with considerably less mass, can't push a box.
For more information on this style of mass-based interaction, see this article by Nick Abel, one of the creators of NewtonVR.
Items can be configured to be picked up at any point, or when grabbed, can rotate and position themselves to match a predefined orientation. This lets you pick up a box from its corner as well as pick up a gun and have it orient to the grip.
We've also created a few physical UI elements to help with basic configuration and "menu type scenarios. We also give you the option to dynamically let the controllers turn into physical objects on a button press.
Official Trailer
Grip Buttons
Grip buttons on the Oculus Touch controllers are obvious. Your middle finger rests comfortably on a trigger that we use for pick up.
The Vive controllers however, are another story. The grip buttons on the sides of the controller can be hard for people to use in combination with the trigger, or the touchpad. People's hand size varies greatly, and making a controller for everyone is difficult to be sure. But we've decided that the benefit gained by using the grip buttons on the sides of the controller outweighs the trouble users can have with them.
One of the benefits of releasing the code to this system is that if you disagree, you're welcome to change the button mappings! But the with the system's defaults, pressing grip button(s) will let you pick something up and releasing it will drop (or throw) the item.
Using the grip buttons to hold an item frees up other buttons on the controllers for items that are designed to be used while held (for example, holding a gun and then pressing the trigger button to fire). If your controller is not hovering over an interactable object, and you hit the grip button, your controller can become a physical object that you can use to interact with the world (for example, pressing a button on a control panel, or knocking over a tower of boxes).
A more philosophical reason driving our decision to go with grip buttons by default: there is intention behind the hardware design of the grip buttons, suggestive of the future of tracked controllers in VR. One of the most common actions that human hands perform is gripping objects in a variety of ways, so supporting those interactions via the controller inputs is very important. It's our expectation that tracked controllers in the near future will evolve to support more sensible or "intuitive" grip inputs, and that the discoverability, and usability problems of grip buttons in general will fade with time.
Implementation
NewtonVR handles a few big hurdles to VR development. We let you use one camera rig to do development for both the Oculus SDK as well as SteamVR. There's an item interaction system to capitalize on PhysX so held items don't go through walls. And we have a collision sound system to give your player auditory feedback when they drop something or hit something against a wall.
Basic Integration
To integrate NewtonVR into a project you must use the included NVRPlayer prefab. It handles adding the associated SteamVR or Oculus SDK components to the proper gameobjects automatically. If you want the player to have a visual head you can place a model on the Head gameobject under NVRPlayer. If you want to override hands you can do it per SDK or for all SDKs in the inspector of the NVRPlayer script on the NVRPlayer gameobject.
When you've got an item you'd like to pick up, simply drop an NVRInteractableItem component on it. You'll need to give it a Rigidbody (and ideally set the mass) if you haven't already. If the item has a specific point that you'd like to pick it up at you can create a new GameObject, parent it to your item, and position it in the location and at the rotation that you'd like the controller to be. Then set NVRInteractableItem.InteractionPoint to that new gameobject.
Closing
We hope that this system can help you make an awesome VR experience! If you use NewtonVR in one of your public projects, be sure to let us know -- we'll give you a shoutout on Twitter and add you to the list of projects using NewtonVR on NewtonVR.com.
Anybody is free to use NewtonVR for basically any purpose: game jams, commercial games, educational apps, your own framework, whatever! Check the MIT license for more info. We are actively using this system in our own projects and plan to update it as development continues.
If you have questions or comments about this system, you can contact us via Twitter at @TTLabsVR, or by creating issues on the Github project.
Development: Keith Bradner, Nick Abel
Interaction Design: Adrienne Hunter
Modeling: Carli Wood, Wesley Eldridge
Audio: Joshua Du Chene
Updated less than a minute ago