As with typical VR environments, users are encouraged to walk around and interact with their environment - and in this case, the user will be able to walk around a real operating room, without having to leave the comfort of their home.
This past spring, Nick Jushchyshyn, Program Director of Drexel University’s Animation, Visual Effects & Immersive Media program, and I visited Hahnemann University hospital to film various operating rooms in 360 stereo. This footage could then be brought into Unity, so that when wearing the VR headset, the user would feel as if they are physically present in the OR. What’s great about this method is that various operating rooms/area across discplines, and even countries could be filmed and brought in to the system - introducing users to a plethora of situations.
Of course, certain objects were and are going to be 3D modeled for the sake of enabling interactivity. First and foremost is the scalpel itself.
And of course, coupled with the scalpel, is the patient itself. Providing the patient as a 3D object to the scene allows for it’s skin tone and weight/size to be changed on demand, allowing for many possible training situations. Other small features can be added as well, such as wrinkles to signify that the patient is elderly, birthmarks, or scars for perhaps women who have had a C-section. The possibilities are truly endless.
Something I am particularly proud of are the drapes I designed to cover the patient in the environment. (Details on how it was constructed to reveal patient shape using gravity model)
Another specific feature that I am particularly proud of, is the clock I modeled that can relay real time to those in the VR space. When one is wearing a VR headset, especially when engaged in a task, it can be a hassle to remove the gear and exit the VR space simply to check the time. Now, the user onyl has to look up at the clock in the environment, and they can find out without any issue.
Leap Motion technologies are also being utilized to track the user’s hand movement and confirm that tools are being held in the right position. This issue in particular became the motivation for development of this project in the first place. We had conducted a study with surgical residents at Drexel College of Medicine, and found that PGY1 and PGY2 residents generally did not even know how to hold a scalpel the correct way, which contributes to many injuries to the patient and staff alike.
Next Step:
- Designing OR lights that can be moved by the user in the VR space to illuminate objects of interest.