The American Psychiatric Association (APA) has updated its Privacy Policy and Terms of Use, including with new information specifically addressed to individuals in the European Economic Area. As described in the Privacy Policy and Terms of Use, this website utilizes cookies, including for the purpose of offering an optimal online experience and services tailored to your preferences.

Please read the entire Privacy Policy and Terms of Use. By closing this message, browsing this website, continuing the navigation, or otherwise continuing to use the APA's websites, you confirm that you understand and accept the terms of the Privacy Policy and Terms of Use, including the utilization of cookies.

×

Abstract

Virtual reality (VR) technology offers new opportunities for the development of innovative clinical research, assessment, and intervention tools. VR-based testing, training, teaching, and treatment approaches that would be difficult, if not impossible, to deliver with traditional methods are now being developed that take advantage of the assets that are available with VR technology. As research evidence continues to indicate clinical efficacy, VR applications are being increasingly regarded as providing innovative options for targeting the cognitive, psychological, motor, and functional impairments that result from various clinical health conditions. VR allows for the precise presentation and control of stimuli in dynamic, multisensory, 3D computer-generated simulations as well as providing advanced methods for capturing and quantifying behavioral responses. These characteristics support the rationale for the use of VR applications in clinical assessment, intervention, and training. This article begins with a brief review of the history of and rationale for the use of VR with clinical populations. It then details one use case for the clinical application of VR—the exposure-therapy treatment of anxiety disorders and posttraumatic stress disorder. Although significant work is cited in other areas of clinical VR (e.g., pain management, cognitive and physical assessment and rehabilitation, eating disorders, social skills, and clinical training), a full overview of such a broad literature is beyond the scope of this article. Thus, the authors have opted to provide more in-depth analysis of one specific clinical area that clearly illustrates how VR has been successfully applied and is supported by an encouraging and evolving scientific literature.

Virtual reality (VR) technology has undergone a transition in the last 20 years that has taken it from the realm of “expensive toy” into that of functional technology. These advances stand to offer new opportunities for clinical research, assessment, and intervention in the field of psychiatry. Since the mid-1990s, clinicians and researchers have developed VR-based testing, training, teaching, and treatment approaches that would be difficult, if not impossible, to deliver with traditional methods.

During this time, a large (but still maturing) scientific literature has evolved regarding the outcomes and effects from the use of what we now refer to as clinical VR. Such VR simulation systems have targeted the assessment and treatment of cognitive, psychological, motor, and functional impairments across a wide range of clinical health conditions. Moreover, continuing advances in the underlying enabling technologies for creating and delivering VR applications have resulted in its widespread availability as a consumer product, sometimes at a very low cost.

This article provides an introductory definition of the technology, charts its historical development generally and as a clinical tool, provides a detailed use case for VR in the treatment of anxiety disorders and posttraumatic stress disorder (PTSD), and discusses the issues for its use in psychiatry. We do not review the growing literature across all the areas of clinical implementation but instead cite the major papers that review that work. Our intention is to provide a road map for those interested in learning more about this emerging and potentially transformative technology.

What Is VR?

The concept and definition of VR have been subject to debate by scientists and clinicians over the years. VR has been very generally defined as a way for humans to visualize, manipulate, and interact with computers and extremely complex data (1). From this baseline perspective, VR can be seen as an advanced form of human-computer interaction (2) that allows a user to more naturally interact with computers beyond what is typically afforded by standard mouse-and-keyboard interface devices. Moreover, some VR formats enable users to become immersed in synthetic computer-generated virtual environments.

However, VR is not defined or limited by any one technological approach or hardware set-up. An engaged VR user experience can be created through combinations of a wide variety of interaction devices, sensory display systems, and content presented in the virtual environment. Thus, there are three common variations for how VR can be created and used.

Nonimmersive VR is the most basic format and is similar to the experience of playing a modern computer or console video game. Content is delivered on a standard flat-screen computer monitor or TV, with no occlusion of the outside world. Users interact with 3D computer graphics variously using a game pad, joystick, basic mouse and keyboard, or specialized interface devices (e.g., treadmills, data gloves, and even handheld devices—e.g., the Nintendo Wii remote). Modern computer games that support user interaction and navigation within such 3D worlds, even though presented on a flat-screen display, can be technically referred to as VR environments.

Immersive VR can be produced by the integration of computers, head-mounted displays (HMDs), body-tracking sensors, specialized interface devices, and 3D graphics. These set-ups allow users to operate within a computer-generated simulated world that changes in a natural or intuitive way on the basis of a user’s motion and interaction. An HMD occludes the user’s view of the outside world while using head- and body-tracking technology to sense the user’s position and movement. It simultaneously sends that information to a computing system, which then uses those data to update the sensory stimuli presented to the user. The contingent tracking of user activity and nearly real-time updating of the 3D content is said to create an immersive virtual experience. This serves to create the illusion of being immersed in a virtual space, in which users can interact. When users are immersed in computer-generated visual imagery and sounds of a simulated virtual scene, their interaction produces an experience that corresponds to what they would see and hear if the scene were real.

Another, less-common method for producing immersive VR experiences uses stereoscopic projection screens arrayed around a user in various configurations. In one approach, six-walled projection rooms known as cave automatic virtual environments (3, 4) allow for interaction in a less-encumbered, wide-field-of-view simulated environment for multiple concurrent users. However, such systems are more costly and complex and are typically beyond the practical resources of most clinical service providers and basic researchers.

Regardless of the technical approach, the key aim of these immersive systems is to perceptually replace the outside world with the virtual world to psychologically engage users with simulated digital content designed to create a specific user experience. Immersive VR (most commonly delivered in an HMD) is typically the choice for applications for which a controlled stimulus environment is desirable for constraining a user’s perceptual experience within a specific synthetic world. This format has been often used in clinical VR applications for anxiety disorder exposure therapy, as analgesic distraction for patients undergoing acutely painful medical procedures, and in the cognitive assessment of users to measure performance under a range of systematically delivered challenges and distractions.

A Brief History of Clinical VR

VR has recently captured the public’s imagination as the next big thing in media. However, the technology for creating VR experiences and VR’s clinical use has existed for at least two decades. During the 1990s, the growing availability and rapid evolution of personal computing drove the global adoption of innovative digital technologies for the purposes of productivity enhancement, communication, and social interaction. At the same time, the advances in modern computing power required to automate processes and to store and analyze vast quantities of data did not go unnoticed by clinical researchers and providers, who imagined and prototyped novel behavioral health-care applications. Primordial efforts from this period can be seen in developments that aimed to use personal computers to enhance productivity in patient documentation and record keeping, automate the administration and scoring of psychometric tests, and allow for computer delivery of cognitive training and rehabilitation activities (5). As well, with the rapid improvements in Internet connectivity seen during the 1990s, researchers and clinicians explored the idea of enhancing access to care through Internet-based teletherapy (69) and self-help cognitive-behavioral programs (10, 11).

Since that time, the impact of computer and information technology on society has grown dramatically. This can be seen in the current adoption of and growing demand for mobile devices, high-speed network access, smart televisions, social media sites, photorealistic digital games, wearable behavioral-sensing devices, and now the “second coming of VR.” Such consumer-driven technologies, although thought of as visionary just ten years ago, have now become increasingly common and essential fixtures in the digital landscape of modern society.

The idea of using VR for clinical purposes was first recognized in the early- to mid-1990s with initial efforts to design VR simulations to deliver exposure therapy for specific phobias (e.g., fear of heights, flying, spiders, and public speaking) (12, 13) and for cognitive assessment and rehabilitation (1417). The compelling feature that drove this innovation was the idea that VR could leverage computing beyond its cardinal purpose—the automation of processes—to instead use computers to produce and deliver sensory stimuli for the creation of embodied, interactive, and immersive user experiences. This was recognized early in the visionary article “The Experience Society” by VR pioneer Myron Krueger (18), in his prophetic statement that “Virtual Reality arrives at a moment when computer technology in general is moving from automating the paradigms of the past, to creating new ones for the future” (p. 163). Viewed from this perspective, VR affords the opportunity to create highly realistic, interactive, and systematically controllable stimulus environments that users can be immersed in and interact with for human performance measurement and training as well as for clinical assessment and intervention. Clinicians and scientists who were drawn to the idea of VR during this time were often guided by the belief that its core features and assets could support the development of innovative clinical approaches that were not possible with existing traditional methodologies.

The added value for such VR systems can be seen in the technology’s capacity to create systematic human testing, training, teaching, and treatment environments that allow for the precise control of complex, multisensory, dynamic 3D stimulus presentations. Within such simulations, sophisticated behavioral interaction is possible, and researchers can precisely track, record, and analyze such physical activity to study human performance and behavior. Much like an aircraft simulator serves to test and train piloting ability under a wide variety of controlled conditions, VR can be used to create relevant simulated environments in which the assessment and treatment of cognitive, emotional, and sensorimotor processes can take place under stimulus conditions that are not easily deliverable and controllable in the physical world.

When combining VR’s stimulus control features with the ability to immerse users in functional and ecologically relevant virtual environments, early clinical VR scientists envisioned a fundamental advancement in how human assessment and intervention could be addressed. One could conjecture that this “ultimate Skinner box” perspective was what human experimental researchers and clinicians had always strived for, but they were limited by the constraints imposed by the laws of physics that govern physical reality. This “vision” drove the enthusiasm and innovative efforts seen in the fledgling area of clinical VR in the 1990s.

Unfortunately, many technical challenges needed to be overcome before this vision of clinical VR could be achieved. When discussion of the potential use of VR for human research and clinical intervention first emerged in the 1990s, the technology needed to deliver on this vision was not sufficiently mature. Consequently, during these early years, VR suffered from a somewhat imbalanced “expectation-to-delivery” ratio, as most who explored VR systems during that time will attest. Computers were too slow; 3D graphics were primitive; and user-interface devices were awkward, requiring more effort than users were willing to expend to learn how to operate them effectively. Moreover, VR HMDs were costly and bulky, and they had limited tracking speed, resolution, and field of view. As a consequence, VR commenced its “nuclear winter” period in 1995, as the public became disenchanted with the quality of a typical VR experience and generally viewed it as a failed technology. Thus, VR languished for many years in what the Gartner Group has termed “the trough of disillusionment,” the stage in technology adoption that often follows the “peak of inflated expectations” period, as described in their regularly updated “Hype Cycle for Emerging Technologies” (19).

Regardless of these technical challenges, the core vision of clinical VR was sound, and VR “enthusiasts” continued to pursue the research and development needed to advance the technology and document its added clinical value. Over the last 23 years, the technology for creating VR systems gradually caught up with the vision of creating compelling, usable, and effective clinical VR applications. This has been possible in large part because of the gradual but continuous advances in the underlying VR-enabling technologies and methods (e.g., computational speed, computer graphics, panoramic and spherical video, audio-visual-haptic displays, 3D-user interfaces, tracking sensors, speech and language processing, artificial intelligence, virtual human agents, authoring software).

Such advances have resulted in the technical capability needed to support the creation of low-cost yet sophisticated immersive and interactive VR systems capable of running on commodity-level computing devices. In part driven by the digital gaming and entertainment sectors, as well as by a nearly insatiable global demand for mobile and interactive networked consumer products, these advances in technological “prowess” and accessibility have provided the hardware and software platforms needed to produce more adaptable and high-fidelity clinical VR scenarios. This has created a state of affairs in which clinical VR applications can now usefully leverage the interactive and immersive assets that VR affords as the technology continues to get faster, better, and cheaper moving forward into the 21st century.

Moreover, since the 1990s, a significant scientific literature has evolved, almost under the radar, reporting many positive outcomes across a range of clinical applications that have leveraged the assets provided by VR (2032). A short list of the areas in which clinical VR has been usefully applied includes fear reduction among persons with specific phobias (26, 3335); treatment for PTSD, depression, and paranoid delusions (20, 27, 30, 3646); addiction treatment (47); discomfort reduction among cancer patients undergoing chemotherapy (48, 49); acute pain reduction during wound care and physical therapy with burn patients (23) and in other painful procedures (5053); improvement of body image disturbances among patients with eating disorders (54); navigation and spatial training for children and adults with motor impairments (5557); functional skill training and motor rehabilitation for patients with central nervous system dysfunction (e.g., stroke, traumatic brain injury, spinal cord injury, cerebral palsy, multiple sclerosis) (24, 5863); and the assessment and rehabilitation of attention, memory, spatial skills, and other cognitive functions in both clinical and unimpaired populations (14, 16, 17, 31, 6471).

For these varied purposes, clinical VR scientists have constructed virtual airplanes; skyscrapers; spiders; battlefields; social settings; beaches; fantasy worlds; and the mundane (but highly relevant) functional environments of the schoolroom, office, home, street, and supermarket. In essence, clinicians can apply VR environments mimicking real or imagined worlds to engage users in simulations that support the aims and mechanics of a specific clinical assessment or therapeutic approach. As a result, there is a growing consensus that VR has now emerged as a promising tool in many domains of research (72, 73) and clinical care (29, 41, 61, 74, 75).

What makes clinical VR so distinctively innovative is that it represents more than a simple linear extension of existing computer technology for human use. Because of VR’s capacity to immerse a user in an interactive computer-generated simulation, new possibilities exist that can go beyond the simple automation of previous clinical assessment and intervention approaches. Nevertheless, in deciding whether VR is ready for clinical use, one needs to consider what features VR offers that may make it especially suited for addressing the requirements of clinical and research usage (cf. 29).

On a very general level, VR can be seen to foster core processes that are relevant across a variety of clinical domains. These processes can be briefly summarized as expose (e.g., exposure therapy for anxiety disorders, PTSD, or addiction treatment), distract (e.g., distract attention away from painful medical procedures to reduce pain perception or promote discomfort reduction), motivate (e.g., motivating patients in cognitive or physical rehabilitation to perform repetitive and sometimes boring tasks by embedding those tasks in gamelike contexts), measure (e.g., measuring performance on physical and cognitive assessment activities), and engage (e.g., generally seen as captivating attention or action, which is useful for engaging participation with clinical applications).

Most clinical VR applications leverage two or more of these core processes. To illustrate this in the context of VR applications relevant in psychiatry, we detail a use case directly relevant to psychiatry—exposure therapy for anxiety disorders and PTSD. For readers interested in other areas of clinical VR usage, the literature cited in the introduction should provide a current road map for exploring a wider range of clinical applications in more detail than possible in this more focused article.

Example Use Case: VR Exposure Therapy for Anxiety Disorders and PTSD

VR Exposure Therapy for Anxiety Disorders

The use of VR to address psychiatric conditions began in the mid-1990s with its use as a tool to deliver prolonged exposure (PE) therapy targeting anxiety disorders, primarily for specific phobias (e.g., heights, flying, spiders, enclosed spaces). PE is a form of individual psychotherapy based on Foa and Kozak’s (76) emotional processing theory, which posits that phobic disorders and PTSD involve pathological fear structures that are activated when information represented in the structures is encountered. Emotional processing theory purports that fear memories include information about stimuli, responses, and meaning (76, 77). These fear structures are composed of harmless stimuli that have been associated with danger and are reflected in the belief that the world is a dangerous place. This belief then manifests itself in cognitive and behavioral avoidance strategies that limit exposure to potentially corrective information that could be incorporated into and alter the fear structure. Because escape from and avoidance of feared situations are intrinsically rewarding (albeit temporarily), phobic disorders can perpetuate without treatment.

Consequently, several theorists have proposed that conditioning processes are involved in the etiology and maintenance of anxiety disorders. These theorists invoke Mowrer’s (78) two-factor theory, which posits that both Pavlovian and instrumental conditioning are involved in the acquisition of fear and avoidance behavior. Successful treatment requires emotional processing of the fear structures to modify their pathological elements so that the stimuli no longer invoke fear, and any method capable of activating the fear structure and modifying it is predicted to improve symptoms of anxiety.

Imaginal PE entails engaging mentally with the fear structure through repeatedly revisiting the feared or traumatic event in a safe environment. The proposed mechanisms for symptom reduction involve activation and emotional processing, extinction or habituation of the anxiety, cognitive reprocessing of pathogenic meanings, the learning of new responses to previously feared stimuli, and ultimately an integration of corrective nonpathological information into the fear structure (79, 80). Thus, VR was seen early on to be a potential tool for the treatment of anxiety disorders; if an individual can become immersed in a feared virtual environment, activation and modification of the fear structure are possible. From this, the use of VR to deliver PE was the first psychological treatment area to gain traction clinically. This was perhaps in part due to the intuitive match between what the technology could deliver and the theoretical requirement of PE to systematically expose users to and engage them in progressively more challenging stimuli needed to activate the fear structure.

Moreover, even during the early days of VR, this goal was not so technically challenging to achieve. VR scenarios could be created that required little complex user interaction. Treatment involved simple navigation within a simulation that presented users with scenarios that represented key elements of the targeted fear structure, which could be made progressively more provocative (views from tall buildings, aircraft interiors, spiders in kitchens, etc.). Even with the limited graphic realism available at the time, patients with phobias were observed to be primed to suspend disbelief and react emotionally to virtual content that represented what they feared. In general, the phenomenon that patients could become psychologically immersed in VR provided a potentially powerful tool for activating relevant fears in the PE treatment of specific phobias in the service of therapeutic exposure.

Research Findings Using VR Exposure to Treat Anxiety Disorders

From this starting point, a body of literature evolved that suggested that the use of VR exposure therapy (VRET) was effective. Case studies in the 1990s initially documented the successful use of VR in the treatment of fear of flying (81, 82), claustrophobia (83), acrophobia (13), and spider phobia (84). For example, in an early wait-list controlled study, VRET was used to treat the fear of heights; patients were exposed to virtual footbridges, virtual balconies, and a virtual elevator (13). Patients were encouraged to spend as much time in each situation as needed for their anxiety to decrease and were allowed to progress at their own pace. The therapist was able to view on a computer monitor what the participant was being exposed to in the virtual environment and therefore was able to comment appropriately.

Results showed that anxiety, avoidance, and distress decreased significantly from pre- to posttreatment for the VRET group but not for the wait-list control group. Examination of attitude ratings on a semantic differential scale revealed positive attitudes toward heights for the VRET group and negative attitudes toward heights for the wait-list group. The average anxiety ratings decreased steadily across sessions, which indicates habituation for those participants in treatment. Furthermore, seven of the ten VRET treatment completers exposed themselves to height situations in real life during treatment, although they were not specifically instructed to do. These exposures seemed to be meaningful, including riding 72 floors in a glass elevator and intentionally parking at the edge of the top floor of a parking deck.

Rothbaum et. al. (85) then compared VRET with both an in vivo PE therapy condition and a wait-list control in the treatment of the fear of flying. Treatment consisted of eight individual therapy sessions conducted over six weeks, with four sessions of anxiety-management training followed either by exposure to a virtual airplane (VRET) or exposure to an actual airplane at the airport (PE). For participants in the VRE group, exposure in the virtual airplane included sitting in the virtual airplane, taxiing, taking off, landing, and flying in both calm and turbulent weather, according to a treatment manual (82). For PE sessions, in vivo exposure was conducted at the airport during sessions 5–8.

Immediately after the treatment or wait-list period, all patients were asked to participate in a behavioral avoidance test consisting of a commercial round-trip flight. The results indicated that each active treatment was superior to the wait-list control and that there were no differences between VRET and in vivo PE. For wait-list participants, there were no differences between pre- and posttreatment self-report measures of anxiety and avoidance, and only one of the 15 wait-list participants completed the graduation flight. In contrast, participants receiving VRET or in vivo PE showed substantial improvement, as measured by self-report questionnaires, willingness to participate in the graduation flight, self-report levels of anxiety on the flight, and self-ratings of improvement. There were no differences between the two treatments on any measures of improvement. Comparison of posttreatment with the six-month follow-up data for the primary outcome measures for the two treatment groups indicated no significant differences, which indicates that treated participants maintained their treatment gains. By the six-month follow-up, 93% of treated participants had flown since completing treatment.

Since that time, an evolved body of literature of controlled studies has emerged, and the efficacy of VRET has been documented in multiple independent meta-analyses and reviews of the literature (26, 3335). These reviews concur with the finding that VR is an efficacious approach for delivering PE, that it outperforms imaginal PE, and that it is as effective as in vivo exposure (which is not always a feasible approach for treating anxiety disorders).

VRET for the Treatment of PTSD

In the late 1990s, researchers began to test the use of VRET for the treatment of PTSD by immersing users in simulations of trauma-relevant environments in which the emotional intensity of the scenes could be precisely controlled by the clinician in collaboration with the patients’ wishes. Traditional PE typically involves the graded and repeated imaginal reliving of the traumatic event in the safety of the therapeutic setting. However, although the efficacy of imaginal exposure has been established in multiple studies with diverse trauma populations (45, 86, 87), many patients are unwilling or unable to effectively visualize the traumatic event. This is a crucial concern, because avoidance of cues and reminders of the trauma is one of the cardinal symptoms of the DSM-5 (88) diagnosis of PTSD. In fact, research on this aspect of PTSD treatment suggests that the inability to emotionally engage (in imagination) is a predictor for negative treatment outcomes (89).

Thus, VRET offers a way to circumvent the natural avoidance tendency by directly delivering multisensory and context-relevant cues that aid in the confrontation and processing of traumatic memories without demanding that the patient actively try to access his or her experience through effortful memory retrieval. Similar to PE for specific phobias, this approach is believed to provide a low-threat context in which the patient can begin to therapeutically process the emotions that are relevant to the traumatic event as well as decondition the learning cycle of the disorder through an extinction learning process. The rationale for this approach was bolstered after the start of the wars in Iraq and Afghanistan with series of Institute of Medicine committee reports (9092) indicating that cognitive-behavior therapy with trauma-focused exposure has the highest level of research evidence in support of its therapeutic efficacy.

The first effort to apply VRET for PTSD began in 1997, when researchers at Georgia Tech and Emory University began testing the Virtual Vietnam VR scenario with Vietnam veterans (44). This occurred more than 20 years after the end of the Vietnam War. During those intervening years, in spite of valiant efforts to develop and apply traditional psychotherapeutic and pharmacological treatment approaches to PTSD, the progression of the disorder for some veterans significantly affected their psychological well-being, functional abilities, and quality of life, as well as that of their families and friends. This initial effort yielded encouraging results in a case study of a 50-year-old male Vietnam veteran who met DSM-IV-R criteria for PTSD (93). Results indicated posttreatment improvement on all measures of PTSD and maintenance of these gains at a six-month follow-up, with a 34% decrease in clinician-rated symptoms of PTSD and a 45% decrease in self-reported symptoms of PTSD.

This case study was followed by an open clinical trial with Vietnam veterans (44). In this study, 16 male veterans with PTSD were exposed to two HMD-delivered virtual environments (a virtual clearing surrounded by jungle scenery, and a virtual Huey helicopter), in which the therapist controlled various visual and auditory effects (e.g., rockets, explosions, whether it was day or night, and shouting). After an average of 13 exposure-therapy sessions over five to seven weeks, there was a significant reduction in PTSD and related symptoms. (For more information, see the nine-minute Virtual Vietnam documentary video at http://www.youtube.com/watch?v=C_2ZkvAMih8.)

Similar positive results were reported by Difede and Hoffman (37) for PTSD that resulted from the attack on the World Trade Center, in a case study using VRET with a patient who had failed to improve with traditional imaginal exposure therapy. This group later reported positive results from a wait-list controlled study using the same World Trade Center VR application (38). The VR group demonstrated statistically and clinically significant decreases on the “gold standard” Clinician Administered PTSD Scale (CAPS) relative both to pretreatment and to the wait-list control group, with a between-groups posttreatment effect size of 1.54. Seven of ten people in the VR group no longer carried the diagnosis of PTSD, whereas all of the wait-list controls retained the diagnosis after the waiting period. Treatment gains were maintained at six-month follow-up. Also noteworthy was the finding that five of the ten VR patients had previously participated in imaginal exposure treatment with no clinical benefit. Such initial results are encouraging and suggest that VR may be a useful component of a comprehensive treatment approach for persons with combat- or terrorist-attack-related PTSD. (For more information, see the virtual World Trade Center video at http://www.youtube.com/watch?v=XAR9QDwBILc.)

BRAVEMIND Virtual Iraq-Afghanistan Exposure-Therapy System for PTSD

With this history in mind, the University of Southern California’s Institute for Creative Technologies (ICT) created an immersive VRET system for combat-related PTSD focused on veterans of Operation Enduring Freedom (OEF) and Operation Iraqi Freedom (OIF). The treatment environment was initially based on recycling virtual assets that were built for the commercially successful X-Box game and the tactical-training simulation scenario Full Spectrum Warrior. Over the years, other existing and newly created assets developed at the ICT have been integrated into this continually evolving application. The BRAVEMIND virtual Iraq-Afghanistan application now consists of a series of 14 virtual scenarios designed to resemble the general contexts that most service members experience during an OEF-OIF deployment, including Middle-Eastern-themed city, village, and roadway environments (see Figure 1).

FIGURE 1.

FIGURE 1. Scene from BRAVEMIND Roadway and City Scenarios

For example, the Iraq and Afghanistan city settings have a variety of elements, including a marketplace, desolate streets, checkpoints, ramshackle buildings, warehouses, mosques, shops, and dirt lots strewn with junk. Access to building interiors and rooftops is available, and the backdrop surrounding the navigable exposure zones creates the illusion of being embedded in a section of a sprawling, densely populated mountainous or desert city. The user can also be positioned inside of a Humvee or mine-resistant ambush-protected vehicle, which supports the perception of travel in a convoy or as a lone vehicle. These scenarios have selectable positions as a driver, as a passenger, or in the more exposed turret position above the roof of the vehicle. The number of soldiers in the cab can also be varied, as well as their capacity to become wounded during certain attack scenarios (e.g., improvised explosive devices, rooftop or bridge attacks).

In addition to the visual stimuli presented in the VR HMD, the clinician can deliver directional 3D audio, vibrotactile, and olfactory stimuli (e.g., burning rubber, cordite, garbage, body odor, smoke, diesel fuel, Iraqi food spices, and gunpowder) into the BRAVEMIND scenarios in real time. The presentation of all ambient and additive combat-relevant stimuli into the VR scenarios (e.g., helicopter flyovers, bridge attacks, exploding vehicles, and detonation of improvised explosive devices) can be controlled in real time through a separate “Wizard of Oz” clinician’s interface (see Figure 2), while the clinician is in full audio contact with the patient. The clinician’s interface is a key feature that provides a clinician with the capacity to customize the therapy experience to the individual needs of the patient. This interface allows a clinician to place the patient in VR scenario locations that resemble the setting in which the trauma-relevant events occurred. Ambient light and sound conditions can be modified to match the patients’ description of their experience. The clinician can then gradually introduce and control real-time trigger stimuli (visual, auditory, olfactory, and tactile), through the clinician’s interface, to foster the anxiety modulation required to promote extinction learning and emotional processing in a customized fashion based on the patient’s past experience and treatment progress.

FIGURE 2.

FIGURE 2. Clinician Interface (Left) and User Interacting in Driving (Center) and Foot-Patrol (Right) Modes

This package of controllable multisensory stimulus options was included in the design of the BRAVEMIND system to allow a clinician the flexibility to engage users across a wide range of unique and highly customizable levels of exposure intensity. As well, these same features have broadened the system’s applicability as a research tool for studies that require systematic control of stimulus presentation in combat-relevant environments (94). A direct link to a YouTube channel with videos that illustrate features of this system and include former patients discussing their experience with the VRET approach can be found at http://www.youtube.com/user/AlbertSkipRizzo.

Research Findings Using the BRAVEMIND System for Combat-Related PTSD

Early clinical tests of the BRAVEMIND virtual Iraq-Afghanistan system produced promising results. Initially, three published case studies reported positive results using this system (9597), and in the first open clinical trial, analyses of 20 active-duty treatment completers (19 male, one female; mean age=28, age range=21–51) produced positive clinical outcomes (42). For this sample, mean pre- to posttreatment PTSD Checklist—Military Version (PCL-M) (98) scores decreased in a statistical and clinically meaningful fashion: 54.4 (SD=9.7) to 35.6 (SD=17.4). Paired pre- to posttreatment t test analysis showed these differences to be significant (t=5.99, df=19, p<0.001). Correcting for the PCL-M no-symptom baseline of 17 indicated a greater than 50% decrease in symptoms, and 16 of the 20 completers no longer met PCL-M criteria for PTSD at posttreatment. Mean Beck Anxiety Inventory (99) scores significantly decreased 33%, from 18.6 (SD=9.5) to 11.9 (SD=13.6; t=3.37, df=19, p<0.003), and mean Patient Health Questionnaire (100) depression scores decreased 49%, from 13.3 (SD=5.4) to 7.1 (SD=6.7; t=3.68, df=19, p<0.002). The average number of sessions for this sample was just under 11.

Overall, 80% of the treatment completers in this sample showed both statistically and clinically meaningful reductions in PTSD, anxiety, and depression symptoms, and anecdotal evidence from patient reports suggests that they saw improvements in their everyday life. These improvements were also maintained at three-month posttreatment follow-up. In another open clinical trial (101) with active-duty Army service members (N=24), the results indicated significant pre- to posttreatment reductions in PCL-M scores and a large treatment effect size (Cohen’s d=1.17). After an average of seven sessions, 45% of those treated no longer screened positive for PTSD, and 62% had reliably improved.

A series of randomized controlled trials (RCTs) were then conducted. In an early trial (102), active-duty service members with PTSD (N=19) were randomized to VRET (N=9) or imaginal exposure (N=10). At posttreatment, VRET reduced CAPS (103) scores (p<0.05), whereas the imaginal PE showed no significant changes. Both groups showed significant change (p<0.05) on the PCL-M, however. In a small, preliminary, quasi-RCT (104) that used a VRET simulation of Iraq comparable to the ICT version described above, seven of ten participants with PTSD showed a 30% or greater improvement with VR, whereas only one of nine participants in a treatment-as-usual group showed similar improvement.

The results of these two RCTs are variously limited by small sample sizes, lack of blinding, and use of a single therapist. In the case of McLay et al. (104), moreover, the VRET comparison was with a set of relatively uncontrolled usual-care conditions. These findings, however, have added to the incremental evidence in support of the use of VRET for combat-related PTSD.

More equivocal findings were reported by Reger et al. (105) in an RCT comparing VRET, PE, and a wait-list control with active-duty OIF-OEF soldiers with PTSD (N=162). Although both VRET and PE demonstrated significantly more improvement of PTSD and depressive symptoms relative to the wait-list control condition, no significant differences were observed between VRET and PE at posttreatment. Moreover, greater improvement in PTSD symptoms at the three- and six-month follow-ups was found with PE.

One possible explanation for these follow-up results, which are in sharp contrast to previous findings indicating strong durability of VRET treatment gains (39, 42, 46), is that the study employed an early version of the system that contained only four VR scenarios. It therefore might have provided less relevant content to this specific group of active-duty service-member clients. Previous feedback from clinicians using this system indicated that when the client’s trauma experience was not well matched to the available content in this initial system, clinicians shifted to imaginal PE. Such feedback informed the design of the BRAVEMIND system, with its expansion to 14 diverse scenarios. Clinical trials with that version of the system are underway.

There are also reports from two other large-scale VRET trials, which examined the augmentation of the traditional exposure component with additional psychosocial treatment (36) and with a pharmacological supplement (46). Beidel et al. (36) combined VRET with trauma management therapy (TMT) in an intensive daily outpatient program conducted over three weeks. VRET was delivered each morning, and TMT (106) was conducted each afternoon as a highly structured group intervention that focused on social reintegration, anger management and problem-solving training, and brief behavioral activation for depression. With an analyzed sample size of 102 and a 2% dropout rate, the authors reported a 2.06 effect size, with 65.9% of participants no longer meeting diagnostic criteria for PTSD. Similar positive effects were reported in other clinical domains, and treatment gains were maintained at six-month follow-up. Although it is impossible to determine the differential effects of VRET versus the psychosocial TMT components, the results from the combination of these approaches in an intensive format are promising, especially in light of recent criticism of PE approaches (107).

Finally, Rothbaum et al. (46) compared the effects of five VRET sessions augmented by d-cycloserine (DCS), which has been found to facilitate extinction in other fear-based disorders (108); alprazolam; and placebo in a study with 156 OEF and OIF veterans with PTSD. Although there were no differences in treatment outcome across medication conditions, with the exception of posttreatment and three-month follow-up CAPS scores indicating that the alprazolam group showed a higher rate of PTSD than the placebo group, PTSD symptoms significantly improved across all conditions at posttreatment and at the three-, six-, and 12-month follow-ups. Moreover, VRET resulted in improvement in psychobiological measures of startle and cortisol reactivity to a trauma-relevant scene (109), which provides further support for the effectiveness of this form of exposure-based therapy using VR (25).

An ongoing RCT using the BRAVEMIND system is nearing completion at the time of this writing; it investigates the additive value of DCS with VRET and PE (110). Recent evidence of both VRET and DCS effectiveness was reported by Difede et al. (39) in a clinical trial with World Trade Center PTSD clients. In a double-blinded controlled comparison between VRET+DCS and VRET+placebo, both groups had clinically meaningful and statistically significant positive outcomes, with the DCS group achieving equivalent gains with fewer sessions. This finding is in contrast with two reports that found no additive value when DCS was added to imaginal PE for PTSD treatment in civilian (111) and military (112) groups. The current ongoing RCT (110) is important for determining whether DCS differentially improves PTSD treatment outcomes across PE and VRET conditions, in view of previously reported mixed findings in this literature.

In conclusion, the overall trend of the published findings is encouraging for the view that VRET can be safely applied clinically and may be an effective approach for delivering an evidence-based treatment (PE) for PTSD. At the least, with the exception of the follow-up data from Reger et al.’s (105) trial, these studies suggest that VRET is as efficacious as traditional PE. However, more research is needed in the form of high-quality RCTs using the current BRAVEMIND system before this can be fully ascertained.

Discussion and Conclusions

General simulation technology has a long history of adding value in aviation training; military planning; automotive, equipment, and architectural design; and robotic surgery practice (113). When researchers leverage these same simulation technology assets, but in a form factor that can deliver VR experiences in a clinicians’ office, hospital, or research laboratory, a powerful and engaging set of virtual tools for psychiatric applications becomes available. The initial step for creating useful VR simulations for clinical purposes is to first look at known processes operating in physical reality that are based on existing theory and are thought to underlie evidence-based approaches to assessment and intervention. With that as a starting point, one can thoughtfully specify the simulation assets required to create VR applications that can amplify treatment effects, provide more reliable and valid diagnostic assessments, break down barriers to care, or simply reduce costs by automating processes.

For example, we know that the use of imaginal exposure therapy for anxiety disorders and PTSD is evidence based in the physical world. From that, one can see a direct case for using the virtual world to deliver ecologically relevant simulations, within which stimuli can be precisely controlled. This allows for the titration of progressively more provocative stimuli designed to pace exposure for the end goal of promoting extinction learning and subsequent fear and anxiety reduction.

Similarly, we know that the sheer amount of physical rehabilitation activity that a stroke survivor engages in is related to improved functional outcomes (all other factors being equal). From that, it is logical to hypothesize that if compelling game-based VR rehabilitation tasks are developed, it might be possible to motivate users to do more repetitions, which could lead to improved outcomes. These thumbnail examples illustrate how a combination of the VR assets cited earlier (expose, distract, motivate, measure, and engage) can inform the rationale for clinical use cases that add value over existing traditional methods.

The research support for the use of clinical VR applications is also promising, albeit not fully mature. There seems to be consensus in the literature that VR can produce equivalent or better outcomes for exposure-based approaches for anxiety disorders and PTSD treatment (2527, 114). As well, in other areas of clinical VR application, consistent findings have been produced in support of VR as an effective distraction tool for reducing the perception of pain among patients undergoing acutely painful medical procedures (23, 53). A growing body of research indicates that VR can increase participation in physical rehabilitation, with patients reporting more motivation to engage in rehabilitation tasks in a game-based VR context, as compared with traditional rehabilitation approaches (115). Cognitive assessment methods using VR have enhanced performance measurement, producing promising results in construct validation studies and for distinguishing between clinical groups and healthy controls (70, 116118). Finally, virtual humans with varying levels of artificial intelligence have been shown to engage users in credible interactions in support of clinical training (virtual patients), as always-available and tireless health-care guides, and recently in the role of clinical assessors (22, 119126).

As we look to the future, we see clinical VR as one of the larger domains of general VR usage. In the recent Goldman Sachs (74) market analysis looking at the future of VR in 2025, the gaming and entertainment sector of course garnered the largest market share. Although this is to be expected, given the public’s chronic demand for new and better ways to consume media, the little-noticed item in that market analysis is that health care came in second place for the VR market share. This is not a surprise to researchers and clinicians who have worked in this area over the years, especially as we see health-care costs becoming one of the largest line items in the U.S. government’s budget, after defense.

Interest in clinical VR by actual therapists also seems to be substantial and growing. Norcross et al. (75) surveyed 70 psychotherapy experts regarding interventions they predicted to increase in the next decade. VR was ranked fourth out of 45 options, with other computer-supported methods (teletherapy, mobile apps, online cognitive-behavioral therapy self-help) occupying the other three of the top five positions.

Professional interest in the clinical uses of VR technology has clearly accelerated and will likely continue to be fueled by a societal zeitgeist in which this form of immersive and interactive technology inspires the public’s attention and imagination. Although it was previously hamstrung by costs, complexity, and clinician unfamiliarity with VR equipment, the technology has evolved dramatically in the consumer marketplace, with new low-cost, easily accessible, high-fidelity product offerings that are poised to drive wider scale adoption. This will result in a probable future scenario whereby VR headsets will become like toasters—although people might not use the headset every day, every household will have one. When such market penetration occurs, the general public will have more access to a range of VR experiences. This may serve to accelerate the uptake of clinical VR as users, more familiar with the technology, begin to imagine its value beyond the world of digital games and entertainment and come to expect it as a part of their health care.

The momentum generated by the growing public awareness of VR, coupled with advances in the science and technology, has created a unique opportunity for psychiatry. Our analysis of the history, theoretical basis, and research findings to date leads us to predict that the application of clinical VR will have a significant impact on future research and practice. Certainly there is still much work to be done to advance the science in this area, particularly with the need to dismantle studies to better specify active ingredients for promoting clinical improvements. Research also needs to better specify which patients will best derive value from a VR approach versus more traditional methods.

However, when one considers that psychology as a science has been around for about 125 years, with a focus on studying human behavior and interaction in the physical world, it only makes sense that we may need a few more years to evolve the science for how humans behave and interact in the virtual world. In view of the current enthusiasm for VR generally across society and specifically in the clinical community, coupled with emerging scientific support, we believe that it is a fairly safe bet that clinical VR applications will soon become standard tools in the toolbox of psychiatry researchers and practitioners and will only grow in relevance and popularity in the future. For access to a large library of online videos demonstrating many of the VR applications discussed here, go to https://www.youtube.com/user/albertskiprizzo.

Drs. Rizzo and. Talbot are with the University of Southern California Institute for Creative Technologies, Los Angeles. Dr. Koenig is with Katana Simulations Pty Ltd., Adelaide, Australia.
Send correspondence to Dr. Rizzo (e-mail: ).

The efforts described here were variously sponsored by the Army Research Lab, U.S. Army, U.S. Air Force, Defense Advanced Research Projects Agency, Defense Center of Excellence, Infinite Hero Foundation, Office of Naval Research, and Telemedicine and Advanced Technology Research Center.

These views reflect the opinions of the authors and not necessarily the position or policy of the U.S. government or foundations.

The authors report no financial relationships with commercial interests.

References

1 Aukstakalnis S, Blatner D, Roth S: Silicon Mirage: The Art and Science of Virtual Reality. Berkeley, CA, Peachpit Press, 1992Google Scholar

2 Rizzo AA, Buckwalter JG, Neumann U: Virtual reality and cognitive rehabilitation: a brief review of the future. J Head Trauma Rehabil 1997; 12:1–15CrossrefGoogle Scholar

3 Cruz-Neira C, Sandin DJ, DeFanti TA: Surround-screen projection-based virtual reality: the design and implementation of the CAVE; in Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques. Edited by Whitton MC. New York, ACM, 1993Google Scholar

4 DeFanti TA, Acevedo D, Ainsworth RA, et al.: The future of the CAVE. Cent. Eur. J. Eng. 2011; 1:16–37. Available at doi: 10.2478/s13531-010-0002-5CrossrefGoogle Scholar

5 Robertson I: Does computerized cognitive rehabilitation work? A review. Aphasiology 1990; 4:381–405CrossrefGoogle Scholar

6 Cuijpers P, Van Straten A, Andersson G: Internet-administered cognitive behavior therapy for health problems: a systematic review. J Behav Med 2008; 31:169–177CrossrefGoogle Scholar

7 Putrino D: Telerehabilitation and emerging virtual reality approaches to stroke rehabilitation. Curr Opin Neurol 2014; 27:631–636CrossrefGoogle Scholar

8 Rizzo AA, Strickland D, Bouchard S: The challenge of using virtual reality in telerehabilitation. Telemed J E Health 2004; 10:184–195CrossrefGoogle Scholar

9 Stamm BH: Clinical applications of telehealth in mental health care. Prof Psychol Res Pr 1998; 29:536–542CrossrefGoogle Scholar

10 Carlbring P, Westling BE, Ljungstrand P, et al.: Treatment of panic disorder via the Internet: a randomized trial of a self-help program. Behav Ther 2001; 32:751–764CrossrefGoogle Scholar

11 Spek V, Cuijpers P, Nyklícek I, et al.: Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med 2007; 37:319–328CrossrefGoogle Scholar

12 Lamson RJ: Virtual therapy of anxiety disorders. Cyberedge Journal 1994; 4:6–8Google Scholar

13 Rothbaum BO, Hodges LF, Kooper R, et al.: Virtual reality graded exposure in the treatment of acrophobia: a case report. Behav Ther 1995; 26:547–554CrossrefGoogle Scholar

14 Brown DJ, Kerr SJ, Bayon V: The development of the Virtual City: A user centred approach; in Proceedings of the 2nd European Conference on Disability, Virtual Reality and Associated Technologies (ECDVRAT). Edited by Sharkey P, Rose D, Lindstrom J. Reading, UK, University of Reading, 1998.Google Scholar

15 Cromby J, Standen P, Newman J, et al.: Successful transfer to the real world of skills practiced in a virtual environment by student with severe learning disabilities; in Proceedings of the 1st European Conference on Disability, Virtual Reality and Associated Technologies. Edited by Sharkey PM. Reading, UK, University of Reading, 1996.Google Scholar

16 Pugnetti L, Mendozzi L, Motta A, et al.: Evaluation and retraining of adults’ cognitive impairment: which role for virtual reality technology? Comput Biol Med 1995; 25:213–227CrossrefGoogle Scholar

17 Rizzo AA: Virtual reality applications for the cognitive rehabilitation of persons with traumatic head injuries; in Proceedings of the 2nd International Conference on Virtual Reality and Persons With Disabilities. Edited by Murphy HJ. Northridge, CSUN, 1994. http://www.csun.edu/~hfdss006/conf/1994/proceedings/Thi~1.htmGoogle Scholar

18 Krueger MW: The experience society. Presence (Camb Mass) 1993; 2:162–168Google Scholar

19 Gartner’s 2016 Hype Cycle for Emerging Technologies Identifies Three Key Trends That Organizations Must Track to Gain Competitive Advantage. Stamford, CT, Gartner Inc 2016. http://www.gartner.com/newsroom/id/3412017.Google Scholar

20 Botella C, Serrano B, Baños RM, et al.: Virtual reality exposure-based therapy for the treatment of post-traumatic stress disorder: a review of its efficacy, the adequacy of the treatment protocol, and its acceptability. Neuropsychiatr Dis Treat 2015; 11:2533–2545CrossrefGoogle Scholar

21 Dascal J, Reid M, IsHak WW, et al.: Virtual reality and medical inpatients: a systematic review of randomized, controlled trials. Innov Clin Neurosci 2017; 14:14–21Google Scholar

22 Freeman D, Reeve S, Robinson A, et al.: Virtual reality in the assessment, understanding, and treatment of mental health disorders. Psychol Med 2017; 47:2393–2400. https://www.cambridge.org/core/services/aop-cambridge-core/content/view/A786FC699B11F6A4BB02B6F99DC20237/S003329171700040Xa.pdf/virtual_reality_in_the_assessment_understanding_and_treatment_of_mental_health_disorders.pdfCrossrefGoogle Scholar

23 Hoffman HG, Chambers GT, Meyer WJ 3rd, et al.: Virtual reality as an adjunctive non-pharmacologic analgesic for acute burn pain during medical procedures. Ann Behav Med 2011; 41:183–191CrossrefGoogle Scholar

24 Howard MC: A meta-analysis and systematic literature review of virtual reality rehabilitation programs. Comput Human Behav 2017; 70:317–327CrossrefGoogle Scholar

25 Maples-Keller JL, Bunnell BE, Kim SJ, et al.: The use of virtual reality technology in the treatment of anxiety and other psychiatric disorders. Harv Rev Psychiatry 2017; 25:103–113CrossrefGoogle Scholar

26 Morina N, Ijntema H, Meyerbröker K, et al.: Can virtual reality exposure therapy gains be generalized to real-life? A meta-analysis of studies applying behavioral assessments. Behav Res Ther 2015; 74:18–24CrossrefGoogle Scholar

27 Rizzo AA, Cukor J, Gerardi M, et al.: Virtual reality exposure therapy for PTSD due to military combat and terrorist attacks. J Contemp Psychother 2015; 45:255–264CrossrefGoogle Scholar

28 Rizzo AA, Shilling R, Forbell E, et al.: Autonomous virtual human agents for healthcare information support and clinical interviewing; in Artificial Intelligence in Mental Healthcare Practice. Edited by Luxton DD. Oxford, UK, Academic Press, 2015.Google Scholar

29 Rizzo AS, Koenig ST: Is clinical virtual reality ready for primetime? Neuropsychology 2017; 31:877–899CrossrefGoogle Scholar

30 Rizzo AA, Shilling R: Clinical virtual reality tools to advance the prevention, assessment, and treatment of PTSD. Eur J Psychotraumatol 2017; 8(sup5):1414560CrossrefGoogle Scholar

31 Rose FD, Brooks BM, Rizzo AA: Virtual reality in brain damage rehabilitation: review. Cyberpsychol Behav 2005; 8:241–271CrossrefGoogle Scholar

32 Slater M, Sanchez-Vives MV: Enhancing our lives with immersive virtual reality. Frontiers in Robotics and Artificial Intelligence 2016; 3:1–47. https://pdfs.semanticscholar.org/7409/3f46bd53222fd7ae1dc7a92030991e76d4d9.pdfGoogle Scholar

33 Opriş D, Pintea S, García-Palacios A, et al.: Virtual reality exposure therapy in anxiety disorders: a quantitative meta-analysis. Depress Anxiety 2012; 29:85–93CrossrefGoogle Scholar

34 Parsons TD, Rizzo AA: Affective outcomes of virtual reality exposure therapy for anxiety and specific phobias: a meta-analysis. J Behav Ther Exp Psychiatry 2008; 39:250–261CrossrefGoogle Scholar

35 Powers MB, Emmelkamp PMG: Virtual reality exposure therapy for anxiety disorders: A meta-analysis. J Anxiety Disord 2008; 22:561–569CrossrefGoogle Scholar

36 Beidel DC, Frueh BC, Neer SM, et al.: The efficacy of trauma management therapy: a controlled pilot investigation of a three-week intensive outpatient program for combat-related PTSD. J Anxiety Disord 2017; 50:23–32CrossrefGoogle Scholar

37 Difede J, Hoffman HG: Virtual reality exposure therapy for World Trade Center post-traumatic stress disorder: a case report. Cyberpsychol Behav 2002; 5:529–535CrossrefGoogle Scholar

38 Difede J, Cukor J, Jayasinghe N, et al.: Virtual reality exposure therapy for the treatment of posttraumatic stress disorder following September 11, 2001. J Clin Psychiatry 2007; 68:1639–1647CrossrefGoogle Scholar

39 Difede J, Cukor J, Wyka K, et al.: D-cycloserine augmentation of exposure therapy for post-traumatic stress disorder: a pilot randomized clinical trial. Neuropsychopharmacology 2014; 39:1052–1058CrossrefGoogle Scholar

40 Falconer CJ, Rovira A, King JA, et al.: Embodying self-compassion within virtual reality and its effects on patients with depression. BJPsych Open 2016; 2:74–80CrossrefGoogle Scholar

41 Freeman D, Bradley J, Antley A, et al.: Virtual reality in the treatment of persecutory delusions: randomised controlled experimental study testing how to reduce delusional conviction. Br J Psychiatry 2016; 209:62–67. doi: 10.1192/bjp.bp.115.176438CrossrefGoogle Scholar

42 Rizzo A, Difede J, Rothbaum BO, et al.: Development and early evaluation of the Virtual Iraq/Afghanistan exposure therapy system for combat-related PTSD. Ann N Y Acad Sci 2010; 1208:114–125CrossrefGoogle Scholar

43 Rizzo AA, Buckwalter JG, Forbell E, et al.: Virtual reality applications to address the wounds of war. Psychiatr Ann 2013; 43:123–138CrossrefGoogle Scholar

44 Rothbaum BO, Hodges LF, Ready D, et al.: Virtual reality exposure therapy for Vietnam veterans with posttraumatic stress disorder. J Clin Psychiatry 2001; 62:617–622CrossrefGoogle Scholar

45 Rothbaum BO, Schwartz AC: Exposure therapy for posttraumatic stress disorder. Am J Psychother 2002; 56:59–75CrossrefGoogle Scholar

46 Rothbaum BO, Price M, Jovanovic T, et al.: A randomized, double-blind evaluation of D-cycloserine or alprazolam combined with virtual reality exposure therapy for posttraumatic stress disorder in Iraq and Afghanistan War veterans. Am J Psychiatry 2014; 171:640–648CrossrefGoogle Scholar

47 Bordnick PS, Yoon JH, Kaganoff E, et al.: Virtual reality cue reactivity assessment: a comparison of treatment- vs. nontreatment-seeking smokers. Res Soc Work Pract 2013; 23:419–425CrossrefGoogle Scholar

48 Chirico A, Lucidi F, De Laurentiis M, et al.: Virtual reality in health system: beyond entertainment. A mini-review on the efficacy of VR during cancer treatment. J Cell Physiol 2016; 231:275–287. doi: 10.1002/jcp.25117CrossrefGoogle Scholar

49 Schneider SM, Kisby CK, Flint EP: Effect of virtual reality on time perception in patients receiving chemotherapy. Support Care Cancer 2011; 19:555–564. doi: 10.1007/s00520-010-0852-7CrossrefGoogle Scholar

50 Gold JI, Kim SH, Kant AJ, et al.: Effectiveness of virtual reality for pediatric pain distraction during i.v. placement. Cyberpsychol Behav 2006; 9:207–212CrossrefGoogle Scholar

51 Mosadeghi S, Reid MW, Martinez B, et al.: Feasibility of an immersive virtual reality intervention for hospitalized patients: an observational cohort study. JMIR Ment Health 2016; 3:e28. doi: 10.2196/mental.5801CrossrefGoogle Scholar

52 Tashjian VC, Mosadeghi S, Howard AR, et al.: Virtual reality for management of pain in hospitalized patients: results of a controlled trial. JMIR Ment Health 2017; 4:e9. doi: 10.2196/mental.7387CrossrefGoogle Scholar

53 Trost Z, Zielke M, Guck A, et al.: The promise and challenge of virtual gaming technologies for chronic pain: the case of graded exposure for low back pain. Pain Manag 2015; 5:197–206. doi: 10.2217/pmt.15.6CrossrefGoogle Scholar

54 Riva G: The key to unlocking the virtual body: virtual reality in the treatment of obesity and eating disorders. J Diabetes Sci Technol 2011; 5:283–292CrossrefGoogle Scholar

55 John NW, Pop SR, Day TW, et al.: The implementation and validation of a virtual environment for training powered wheelchair manoeuvres. IEEE Trans Vis Comput Graph 2018; 25:1867–1878. doi: 10.1109/TVCG.2017.2700273CrossrefGoogle Scholar

56 Rizzo AA, Schultheis MT, Kerns K, et al.: Analysis of assets for virtual reality applications in neuropsychology. Neuropsychol Rehabil 2004; 14:207–239CrossrefGoogle Scholar

57 Stanton D, Foreman N, Wilson P: Uses of virtual reality in clinical training: developing the spatial skills of children with mobility impairments; in Virtual Reality in Clinical Psychology and Neuroscience. Edited by Riva G, Wiederhold B, Molinari E. Amsterdam, IOS Press, 1998Google Scholar

58 Deutsch JE, Westcott McCoy S: Virtual reality and serious games in neurorehabilitation of children and adults: prevention, plasticity, and participation. Pediatr Phys Ther 2017; 29(Suppl 3):S23–S36CrossrefGoogle Scholar

59 Holden MK: Virtual environments for motor rehabilitation: review. Cyberpsychol Behav 2005; 8:187–219CrossrefGoogle Scholar

60 Klamroth-Marganska V, Blanco J, Campen K, et al.: Three-dimensional, task-specific robot therapy of the arm after stroke: a multicentre, parallel-group randomised trial. Lancet Neurol 2014; 13:159–166CrossrefGoogle Scholar

61 Lange B, Koenig S, Chang C-Y, et al.: Designing informed game-based rehabilitation tasks leveraging advances in virtual reality. Disabil Rehabil 2012; 34:1863–1870CrossrefGoogle Scholar

62 Merians AS, Jack D, Boian R, et al.: Virtual reality-augmented rehabilitation for patients following stroke. Phys Ther 2002; 82:898–915Google Scholar

63 Merians AS, Fluet GG, Qiu Q, et al.: Integrated arm and hand training using adaptive robotics and virtual reality simulations; in Proceedings of the 2010 International Conference on Disability, Virtual Reality and Associated Technology. Edited by Sharkey P, Sánchez J. Reading, UK, University of Reading, 2010Google Scholar

64 Bogdanova Y, Yee MK, Ho VT, et al.: Computerized cognitive rehabilitation of attention and executive function in acquired brain injury: a systematic review. J Head Trauma Rehabil 2016; 31:419–433CrossrefGoogle Scholar

65 Brooks BM, McNeil JE, Rose FD, et al.: Route learning in a case of amnesia: a preliminary investigation into the efficacy of training in a virtual environment. Neuropsychol Rehabil 1999; 9:63–76CrossrefGoogle Scholar

66 Matheis RJ, Schultheis MT, Tiersky LA, et al.: Is learning and memory different in a virtual environment? Clin Neuropsychol 2007; 21:146–161CrossrefGoogle Scholar

67 Ogourtsova T, Souza Silva W, Archambault PS, et al.: Virtual reality treatment and assessments for post-stroke unilateral spatial neglect: a systematic literature review. Neuropsychol Rehabil 2017; 27:409–454CrossrefGoogle Scholar

68 Parsons TD, Rizzo AA, Rogers S, et al.: Virtual reality in paediatric rehabilitation: a review. Dev Neurorehabil 2009; 12:224–238CrossrefGoogle Scholar

69 Passig D, Tzuriel D, Eshel-Kedmi G: Improving children’s cognitive modifiability by dynamic assessment in 3D immersive virtual reality environments. Comput Educ 2016; 95:296–308CrossrefGoogle Scholar

70 Rizzo AA, Bowerly T, Buckwalter JG, et al.: A virtual reality scenario for all seasons: the virtual classroom. CNS Spectr 2006; 11:35–44CrossrefGoogle Scholar

71 Valladares-Rodríguez S, Pérez-Rodríguez R, Anido-Rifón L, et al.: Trends on the application of serious games to neuropsychological evaluation: a scoping review. J Biomed Inform 2016; 64:296–319CrossrefGoogle Scholar

72 Bohil CJ, Alicea B, Biocca FA: Virtual reality in neuroscience research and therapy. Nat Rev Neurosci 2011; 12:752–762. doi: 10.1016/j.jbi.2016.10.019CrossrefGoogle Scholar

73 Larson EB, Feigon M, Gagliardo P, et al.: Virtual reality and cognitive rehabilitation: a review of current outcome research. NeuroRehabilitation 2014; 34:759–772. doi: 10.3233/NRE-141078CrossrefGoogle Scholar

74 Virtual and Augmented Reality: Understanding the Race for the Next Computing Platform. New York, Goldman-Sachs, 2016. http://www.goldmansachs.com/our-thinking/pages/technology-driving-innovation-folder/virtual-and-augmented-reality/report.pdfGoogle Scholar

75 Norcross JC, Pfund RA, Prochaska JO: Psychotherapy in 2022. A Delphi poll on its future. Prof Psychol Res Pr 2013; 44:363–370CrossrefGoogle Scholar

76 Foa EB, Kozak MJ: Emotional processing of fear: exposure to corrective information. Psychol Bull 1986; 99:20–35CrossrefGoogle Scholar

77 Foa EB, Steketee G, Rothbaum BO: Behavioral/cognitive conceptualizations of post-traumatic stress disorder. Behav Ther 1989; 20:155–176CrossrefGoogle Scholar

78 Mowrer OA: Learning and behavior. New York, Wiley, 1960CrossrefGoogle Scholar

79 Bryant RA, Moulds ML, Guthrie RM, et al.: Imaginal exposure alone and imaginal exposure with cognitive restructuring in treatment of posttraumatic stress disorder. J Consult Clin Psychol 2003; 71:706–712CrossrefGoogle Scholar

80 Foa EB, Hearst-Ikeda D: Emotional dissociation in response to trauma: an information-processing approach; in Handbook of Dissociation: Theoretical and Clinical Perspectives. Edited by Michelson LK, Ray WJ. New York, Plenum Press, 1996CrossrefGoogle Scholar

81 Rothbaum BO, Hodges L, Watson BA, et al.: Virtual reality exposure therapy in the treatment of fear of flying: a case report. Behav Res Ther 1996; 34:477–481CrossrefGoogle Scholar

82 Rothbaum BO, Hodges LF, Smith S: Virtual reality exposure therapy abbreviated treatment manual: fear of flying application. Cognit Behav Pract 1999; 6:234–244CrossrefGoogle Scholar

83 Botella C, Baños RM, Perpiñá C, et al.: Virtual reality treatment of claustrophobia: a case report. Behav Res Ther 1998; 36:239–246CrossrefGoogle Scholar

84 Carlin AS, Hoffman HG, Weghorst S: Virtual reality and tactile augmentation in the treatment of spider phobia: a case report. Behav Res Ther 1997; 35:153–158CrossrefGoogle Scholar

85 Rothbaum BO, Hodges L, Smith S, et al.: A controlled study of virtual reality exposure therapy for the fear of flying. J Consult Clin Psychol 2000; 68:1020–1026CrossrefGoogle Scholar

86 Bryant RA: Psychosocial approaches of acute stress reactions. CNS Spectr 2005; 10:116–122CrossrefGoogle Scholar

87 Van Etten ML, Taylor S: Comparative efficacy of treatments of posttraumatic stress disorder: an empirical review. JAMA 1998; 268:633–638Google Scholar

88 Diagnostic and Statistical Manual of Mental Disorders, 5th ed. Arlington, VA, American Psychiatric Publishing, 2013CrossrefGoogle Scholar

89 Jaycox LH, Foa EB, Morral AR: Influence of emotional engagement and habituation on exposure therapy for PTSD. J Consult Clin Psychol 1998; 66:185–192CrossrefGoogle Scholar

90 Institute of Medicine: Treatment of Posttraumatic Stress Disorder: An Assessment of the Evidence. Washington, DC, National Academies Press, 2008.Google Scholar

91 Institute of Medicine: Treatment for Posttraumatic Stress Disorder in Military and Veteran Populations: Initial Assessment. Washington, DC, National Academies Press, 2012Google Scholar

92 Institute of Medicine: Treatment for Posttraumatic Stress Disorder in Military and Veteran Populations: Final Assessment. Washington, DC, National Academies Press, 2014Google Scholar

93 Rothbaum BO, Hodges LF: The use of virtual reality exposure in the treatment of anxiety disorders. Behav Modif 1999; 23:507–525CrossrefGoogle Scholar

94 Rizzo AA, Buckwalter JG, John B, et al.: STRIVE: Stress Resilience in Virtual Environments: a pre-deployment VR system for training emotional coping skills and assessing chronic and acute stress responses. Stud Health Technol Inform 2012; 173:379–385Google Scholar

95 Gerardi M, Rothbaum BO, Ressler K, et al.: Virtual reality exposure therapy using a virtual Iraq: case report. J Trauma Stress 2008; 21:209–213CrossrefGoogle Scholar

96 Reger GM, Gahm GA: Virtual reality exposure therapy for active duty soldiers. J Clin Psychol 2008; 64:940–946CrossrefGoogle Scholar

97 Rizzo AA, Graap K, Mclay RN, et al: Virtual Iraq: Initial Case Reports From a VR Exposure Therapy Application for Combat-Related Post Traumatic Stress Disorder. IEEE XPlore Virtual Rehabilitation International Conference, Sept 27–29, 2007, Venice, Italy. doi: 10.1109/ICVR.2007.4362152Google Scholar

98 Blanchard EB, Jones-Alexander J, Buckley TC, et al.: Psychometric properties of the PTSD Checklist (PCL). Behav Res Ther 1996; 34:669–673CrossrefGoogle Scholar

99 Beck AT, Epstein N, Brown G, et al.: An inventory for measuring clinical anxiety: psychometric properties. J Consult Clin Psychol 1988; 56:893–897CrossrefGoogle Scholar

100 Kroenke K, Spitzer RL: The PHQ-9: a new depression diagnostic and severity measure. Psychiatr Ann 2002; 32:509–515CrossrefGoogle Scholar

101 Reger GM, Holloway KM, Candy C, et al.: Effectiveness of virtual reality exposure therapy for active duty soldiers in a military mental health clinic. J Trauma Stress 2011; 24:93–96CrossrefGoogle Scholar

102 Roy MJ, Costanzo ME, Blair JR, et al.: Compelling evidence that exposure therapy for PTSD normalizes brain function. Stud Health Technol Inform 2014; 199:61–65Google Scholar

103 Blake DD, Weathers FW, Nagy LM, et al.: The development of a clinician-administered PTSD scale. J Trauma Stress 1995; 8:75–90CrossrefGoogle Scholar

104 McLay RN, Wood DP, Webb-Murphy JA, et al.: A randomized, controlled trial of virtual reality-graded exposure therapy for post-traumatic stress disorder in active duty service members with combat-related post-traumatic stress disorder. Cyberpsychol Behav Soc Netw 2011; 14:223–229CrossrefGoogle Scholar

105 Reger GM, Koenen-Woods P, Zetocha K, et al.: Randomized controlled trial of prolonged exposure using imaginal exposure vs. virtual reality exposure in active duty soldiers with deployment-related posttraumatic stress disorder (PTSD). J Consult Clin Psychol 2016; 84:946–959CrossrefGoogle Scholar

106 Turner SM, Beidel DC, Frueh BC: Multicomponent behavioral treatment for chronic combat-related posttraumatic stress disorder: trauma management therapy. Behav Modif 2005; 29:39–69CrossrefGoogle Scholar

107 Steenkamp MM: True evidence-based care for posttraumatic stress disorder in military personnel and veterans. JAMA Psychiatry 2016; 73:431–432CrossrefGoogle Scholar

108 Ressler KJ, Rothbaum BO, Tannenbaum L, et al.: Cognitive enhancers as adjuncts to psychotherapy: use of D-cycloserine in phobic individuals to facilitate extinction of fear. Arch Gen Psychiatry 2004; 61:1136–1144CrossrefGoogle Scholar

109 Norrholm SD, Jovanovic T, Gerardi M, et al.: Baseline psychophysiological and cortisol reactivity as a predictor of PTSD treatment outcome in virtual reality exposure therapy. Behav Res Ther 2016; 82:28–37CrossrefGoogle Scholar

110 Difede J, Rothbaum BO, Rizzo A: Enhancing Exposure Therapy for PTSD: Virtual Reality and Imaginal Exposure with a Cognitive Enhancer. Randomized Controlled Trial. Bethesda, MD, U.S. National Library of Medicine, 2010. http://clinicaltrials.gov/ct2/show/NCT01352637Google Scholar

111 De Kleine RA, Hendriks GJ, Kusters WJ, et al.: A randomized placebo-controlled trial of D-cycloserine to enhance exposure therapy for posttraumatic stress disorder. Biol Psychiatry 2012; 71:962–968CrossrefGoogle Scholar

112 Litz BT, Salters-Pedneault K, Steenkamp MM, et al.: A randomized placebo-controlled trial of D-cycloserine and exposure therapy for posttraumatic stress disorder. J Psychiatr Res 2012; 46:1184–1190CrossrefGoogle Scholar

113 Applications of Virtual Reality. Virtual Reality Society, 2017. https://www.vrs.org.uk/virtual-reality-applications/Google Scholar

114 Bouchard S, Dumoulin S, Robillard G, et al.: Virtual reality compared with in vivo exposure in the treatment of social anxiety disorder: a three-arm randomised controlled trial. Br J Psychiatry 2017; 210:276–283. doi: 10.1192/bjp.bp.116.184234CrossrefGoogle Scholar

115 Granic I, Lobel A, Engels RC: The benefits of playing video games. Am Psychol 2014; 69:66–78CrossrefGoogle Scholar

116 Man DWK, Ganesan B, Yip CCK, et al.: Validation of the virtual-reality prospective memory test (Hong Kong Chinese version) for individuals with first-episode schizophrenia. Neuropsychol Rehabil 2016; 1–14. doi: 10.1080/09602011.2016.1251949CrossrefGoogle Scholar

117 Nir-Hadad SY, Weiss PL, Waizman A, et al.: A virtual shopping task for the assessment of executive functions: validity for people with stroke. Neuropsychol Rehabil 2017; 27:808–833CrossrefGoogle Scholar

118 Parsons TD, Rizzo AA: Initial validation of a virtual environment for assessment of memory functioning: virtual reality cognitive performance assessment test. Cyberpsychol Behav 2008; 11:17–25CrossrefGoogle Scholar

119 Bickmore TW, Utami D, Matsuyama R, et al.: Improving access to online health information with conversational agents: a randomized controlled experiment. J Med Internet Res 2016; 18:e1CrossrefGoogle Scholar

120 Rizzo AA, Kenny P, Parsons T: Intelligent virtual humans for clinical training. International Journal of Virtual Reality and Broadcasting 2011; 8. https://www.jvrb.org/past-issues/8.2011/2902/820113.pdfGoogle Scholar

121 Rizzo AA, Talbot T: Virtual reality standardized patients for clinical training; in The Digital Patient: Advancing Medical Research, Education, and Practice. Edited by Combs CD, Sokolowski JA, Banks CM. New York, Wiley, 2016CrossrefGoogle Scholar

122 Rizzo AA, Lucas G, Gratch J, et al.: Clinical interviewing by a virtual human agent with automatic behavior analysis; in 2016 Proceedings of the International Conference on Disability, Virtual Reality and Associated Technologies. Edited by Sharkey P, Rizzo AA. Reading, UK, University of Reading, 2016. http://www.icdvrat.org/2014/papers/ICDVRAT2014_S03N3_Rizzo_etal.pdfGoogle Scholar

123 Lucas GM, Gratch J, King A, et al.: It’s only a computer: virtual humans increase willingness to disclose. Comput Human Behav 2014; 37:94–100CrossrefGoogle Scholar

124 Lucas G, Rizzo A, Gratch J, et al.: Reporting mental health symptoms: breaking down barriers to care with virtual human interviewers. Frontiers in Robotics and AI, 2017CrossrefGoogle Scholar

125 Talbot TB, Sagae K, John B, et al.: Sorting out the virtual patient: how to exploit artificial intelligence, game technology and sound educational practices to create engaging role-playing simulations. Int J Gaming Comput-Mediated Simulations 2012; 4:1–19CrossrefGoogle Scholar

126 Talbot T, Rizzo AA: Virtual human standardized patients for clinical training; in Virtual Reality Technologies for Health and Clinical Applications: Psychological and Neurocognitive Interventions, vol. 2. Edited by Rizzo AA, Bouchard S. New York, Springer, in pressGoogle Scholar