Abstract
The goal of this research and creative activity is to create an interactive dramatic experience in immersive virtual reality. The human participant is the protagonist of the drama, which is designed to evoke her emotional response. The research has two interconnected aspects; the adaptation of dramatic tools from existing narrative media to interactive, immersive VR; and the appropriation of artificial intelligence techniques for the creation of responsive, believable, computer-controlled characters. I consider that intelligent agents bring life into virtual environments and are a sine qua non for the creation of interactive drama. I am currently collaborating with Dave Pape, developer of the VR Authoring framework Ygdrasil, and Stuart C. Shapiro developer of the SNePs AI system in order to marry sophisticated VR and AI tools for this work.

The important questions here are: can, and how can, a virtual, interactive situation provoke an emotional response in the user? VR is recognized as an effective place of treatment for phobias, because it can evoke fearful reactions. [10] Bernard Perron has studied how suspense and shock play out in both cinema and interactive media, using very similar dramatic tropes. [11] It has been argued that one role of modern story-tellers (novelists, film-makers, game designers) is to represent the unconscious of the reader/audience [12], and that a practical, common sense understanding of psychology allows them to create powerful emotions in an audience. [13] Our own experience building and exhibiting The Thing Growing, convinces us that participants will respond psychologically and emotionally to virtual dramatic situations. We similarly rely on practical psychology to devise situations they will respond to.

Agents are a core element for evoking these psychological responses. We use them to simulate emotion in order to stimulate emotion in the participant. For example the Thing agent in The Thing Growing keeps the user off-balance by simulating love and fawning on her, then simulating anger and spurning her. A pattern of blowing hot and then cold is common in intimate relationships. We observed that users were moved, responding with body language and utterances that indicated typical human counter-responses; satisfaction when they were praised; confusion, self-doubt or annoyance when they were criticized; aggrievement when they were spurned. [14]

Dramatic twists are used in conjunction with the emotions that the agents' simulate in order to heighten the effect on the user. During the vigil scene in The Trial The Trail Patofil's role is to simulate cheerful playfulness and stimulate the user into a sense of happy security laced with a touch of mischief- - then the bad guys attack. They are casually aggressive. They alternate between insults, sweet-talking, and making kissing, howling and barking noises. Common sense psychology tells us that many people will be unnerved by a gang of virtual bullies surrounding and taunting them. And, moreover, that we can increase this impact if it contrasts with a previously induced happy-go-lucky mood.

We believe that the fact that agents in a VR system are peers in terms of size with the human user, and can have a physical presence, adds to their effectiveness as emotional stimulants. VR agents can get too close to the user, can penetrate her personal space. One user described the Thing agent thus "It's so real, dancing there in my face ..." [15] The bad guys in The Trial The Trail are big. We are currently experimenting with ways to make them push and jostle the user as well as simply crowding her -- we are interested in discovering whether we can make this feel effective enough, or whether the lack of haptic feedback will be a problem.

Testing the participant's state of mind
In order for our dramas to be effective, we must not only stimulate emotion in the user, but provide some mechanism for detecting these emotions, so that the agents can respond appropriately and that subsequent scenes can take them into account. In our specific case, we need to parlay the information from the tracking system and wand of our VR system into interpretations of the user's state of mind. Tracking sensors on the user's head and two hands give feedback about their position and orientation. Information from the wand controller consists of the state of the joystick and the three buttons available. The snare situation needs to contain an activity that can be detected by these devices. The activity exists in a narrative context, so what the user does can be interpreted in the light of that context. The agents are a crucial part of establishing feedback loops that can reveal something about the user. For example, the Thing tries to teach the user a dance, here is a summary of the possible actions we can detect, and the interpretations we make of those actions in order to trigger a response in the Thing agent.

• Joystick detects the user drives away: Interpretation = the user is disobedient: Thing's response = follow and remonstrate with her
• Tracking system detects the user dances badly: Interpretation = the user doesn't care about the task or is shy: Thing's response = encourage or complain
• Tracking system detects the user dances OK: Interpretation = the user is obedient: Thing's response = praise her

This process of detecting the user's state of mind is reductive -- we are confronting the user with some set of simple options disguised by the agents' personalities and cloaked in narrative logic. We have chosen what elements of the user's behaviour we want to pay attention to, and when. The times that we check on the user are discrete and have a certain duration. For example, we check if the user is dancing correctly only when she is told to dance, not all the time, and the duration of each dance period is only a few seconds. Every check is tied to the few moments of narrative time around the check, so that the response in not a simple reaction but tied to the narrative flow. For example, the Thing's responses change over narrative time. If the user drives away, the Thing will first whine and beg to persuade her to dance; then get angry; and, if the behaviour persists, finally storm off. If the user dances badly, first the Thing cajoles, then tells the user off.

The agents' personalities also help us in the process of interpreting the user -- and this interpretation can go beyond detecting emotion to detecting attitudes relevant to the psychological terrain that the virtual drama explores. For example, in The Trial The Trail Patofil and Filopat take up different positions relevant to the quest and its challenges. Patofil is reckless and insouciant, believes the journey is more important that the arrival, and is dubious whether the heart's desire exists. Filopat follows rules, adheres to duty and fervently believes in the quest. Patofil stimulates the user to disobey and to be cavalier, even cynical. Filopat provokes defiance to authority, yet also tends to urge humanity and caring. The user is encouraged to side with one, then the other. These alliances implicitly include an adherence to the particularly philosophical position of that agent. An example of the detection of such an alliance comes when we detect whether the user follows Filopat's injunctions to stay still and meditate during the vigil scene -- or not.

Re-composable snares
In building VR drama we are concerned to minimize waste. We don't want to build a lot of scenes that the user never visits. At the same time we don't want the user to feel forced through our scenes on a glaringly narrow linear path. Dividing the story into modular snare structures makes it possible to recombine the snares into a different order as a response to different users: to a degree. Stories are dependent on dramatic ordering for meaning to emerge [15]. Typically it is a specific combination of snares that creates the psychological movement we are interested in. For example early user tests of the vigil scene we are building for The Trial The Trail suffer because the users don't have an established relationship with Patofil and Filopat -- which would make them tend to like (and believe, and act with) Patofil rather than Filopat.

Like several other research groups we choose to divide our virtual dramas into acts that will always follow in a linear fashion. [16] At a lower level of granularity we can adjust the ordering of snare elements within the act to accommodate and respond to different users but still maintain the overall psychological purpose of the act. For example, the second act of The Thing Growing was designed to ensnare the user into some typical patterns of dysfunctional relationships and to leave her feeling ambiguous about the Thing. The whole act is centered around a dancing trope and the default version had these parts (or snares) in the following order.

• ·The Thing teaches the user dance steps
• ·A rift occurs between user and Thing, and the Thing leaves (precipitated by one or the other)
• ·The Thing rescues the user from difficulty
• ·The Thing copies as the user dances

(In the last part, tracking data from the user, was fed to the Thing's body with a slight delay added - people very much liked the feeling of agency of seeing this virtual creature mirror their motions.)
Some users would not dance with the Thing, because they didn't want to, or because they didn't understand, in this case we changed the ordering thus:

• ·The Thing teaches the user dance steps (FAILS)
• ·A rift occurs between user and Thing, and the Thing leaves (precipitated by Thing)
• ·The Thing rescues the user from difficulty
• ·The Thing copies as the user dances
• ·The Thing teaches the user dance steps

In this version users did not have to dance at an early point in their virtual experience, and were either softened up, more relaxed, or excited by the Thing copying them, and therefore amenable to learning dance steps from it. In both cases, it was vital to maintain the section where the Thing taught the user to dance, because this was the part where the Thing most revealed its dysfunctional nature and solicited dysfunctional responses from the user; the part that would leave the user most conflicted about this creature which was the ultimate goal of the act.

Agents are a vital part of making the re-ordering of snares work smoothly. Phoebe Sengers critiques some behavioral-based agents for not giving the user appropriate narrative cues that explain why their behavior changes. [17] In this case we want to abruptly change the order of high level behaviours (scenes) of the story system as a whole, without losing a sense of narrative logic. The elements of each part remain the same, most of the agent's behaviour remains the same, but the agent has specific actions (speech, animation, navigation) to cover the different transitional moments gracefully. By maintaining its emotional and psychological believability throughout, the agent supports a narrative sequence vital for advancing the psychological goal of the act.

The more elaborate the story, the more possible recombinations of snares there can be, and the more work in providing the agents with appropriate transitional behaviours. In The Trial The Trail we are trying to formalize this structure in the script our agents use, so that we can handle greater complexity. But we still do not want to do unnecessary work and so we rely on observing user-tests to find places in our narrative where users absolutely balk at exploring the dramatic paths we offer.

The agents also have an important role in inhibiting the exponential growth of snare recombination. Sometimes instead of offering a whole different pathway, we simply use agents to bully or cajole the user into a certain performance. This may appear to be a pretty thin disguise for pushing the user along our psychological trajectory. But given the narrative context and agents' personalities it can be effective. For example the Thing is a control freak; its role is to try persistently to make the user do as it wants. Patofil and Filopat act out good cop/bad cop routines with the same purpose of controlling/persuading the user to follow some path. In social situations in real life we are influenced by those around us and inhibit our behavior because of politeness, liking, hate; we build our dramatic situations and agents to have the same kind of influence on users.

Believable Agents
The purpose of the agents is to work with the dramatic structure to engender emotions in the user. For this they must be believable. Their interrelation with the dramatic strategies make them believable at a high level, giving them coherent personalities with their own agendas, agency, desires. But what are the more fundamental elements that make them appear "alive?"

Some researchers argue that a high degree of photo-realistic resolution is necessary to make agents believable. [18] We disagree. The Thing, Patofil, Filopat, and the bad guys are all quite abstracted. The choice here is to make the agents read symbolically rather than literally. Scott McCloud suggests that viewers can more easily identify with simply drawn, iconic, cartoon characters. [19] In the same way we believed that simply designed characters are fleshed out by the user's imagination, and facilitate her own emotional memories seeping into the experience. This choice means we also avoid the problems of highly photo-realistic agents that jar the user because they are not perfect.

Working with networked VR, we noted that the simple tracking systems typically used in CAVE-like systems are immensely effective at imparting a sense of life to the avatar -- the virtual representation of a person -- as she appears to all the other participants in the shared virtual world. We concluded that a more abstract avatar, coupled with natural body language, often reads as a living "entity" better than a photo-realistic (but never really human) humanoid. Therefore we use motion capture techniques to animate our visually simple agents, augmented where necessary with procedural animation. For example, Patofil and Filopat have cloth-like wings that are animated with a mass-spring system. As we have already mentioned agents in VR have a physical presence that can effect a user emotionally, and strengthen her feelings of co-presence with a live entity.

Part of making the agent a believable character in a drama, is to have it speak appropriately and with an appropriate emotional tone. Since current voice-generating software does not do a good job of rendering emotional qualities of the voice, we pre-record phrases for each character creating a dialog library large and flexible enough so there is a response for every eventuality, and which includes redundant phrases so the character does not get stuck repeating the same lines.

I argued above that we want to avoid waste and so devise strategies for recombining certain snares, which represent fairly large chunks of the drama. At the lowest level of granularity, a level within these snares themselves, the agent is composed of discreet performances (lines of speech, animations, navigation strategies) lasting a few seconds, which are combined on the fly. These basic elements are easy to produce and we accept that we must have many redundancies. While many of these tiny performances will not be used during any one run-through of the drama, we want to ensure that there are enough to cover all the possible responses needed for that particular participant. For example the Thing agent has many different ways of praising a user who dances well - typically only a subset of them are used.

That Pesky Free Agent
I have said that the goal of our work is to create an immersive experience that involves the participant centrally as the protagonist of a psychological drama. Obviously, the participant's role in the process is key.

I am influenced by semiotic theory that describes the processes by which a user decodes audio-visual or textual stimuli and thereby inserts herself into the creation of meaning. [20] In our drama we want to set up situations where the user can follow the narrative clues and cues provided by the agents, involve her own emotional, reasoning and acting systems, and by so doing explore for herself the psychological terrain that our story aims to evoke. There are two obvious problems.

One: are people willing or interested in immersing themselves in an experience that is seeking to give them a controlled emotional experience? We have found that some people are not, and they do not enjoy our dramas, but others are willing to suspend belief about the control this kind of interactive drama exerts and explicitly play along with it in order to increase their sense of involvement in the drama. [21] Jane McGonigal has written of a similar tendency in participants of pervasive games who actively "perform belief" that the boundary between the game and real-life is permeable in order to intensify their game experiences. [22]

Two: given that everyone has different experiences to draw from how can we hope to control the proliferation of personal connotations that will lead people to make up different stories about what is happening and therefore respond in very diverse emotional ways to the same stimuli? How, in short can we ensure that the users actually create for themselves precisely the kind of emotional roller coaster ride that we have devised? Common sense psychology gives us a sense of how many people might respond. In interviews with participants in The Thing Growing, we learned that the kind of stories and conjectures that people were making up in their minds about the Thing changed in detail -- but what was common was that they were engaged in trying to understand it and make a comprehensible model of its motivations. [23] They would interpret lucky coincidences quite elaborately and attribute to the agent far more intelligence than it had. But by and large participants interpreted and responded to the Thing's personality and attitudes in ways we had anticipated. We anticipated correctly because we built and tested the acts, scenes, and snares of our drama iteratively. Our experience in user testing, showed quite a surprising convergence of behaviour in participants. We refined the drama to cover the fairly few major tendencies we observed and made a decision not to accommodate outlier behaviour.

Issues, Problems and Challenges
Our user tests for both The Thing Growing and The Trail The Trail have shown that it is not always easy to get the user to pay attention to agents. Part of the problem is participants' expectations. People are more used to VR environments that they fly through and explore rather than social interactions with agents. Agents in video games are enemies or fairly unimportant followers. Some spout instructions which may be useful but which often become repetitive and can be ignored. A crucial moment at the start of the interaction with an agent needs to establish that it is responsive and expecting to be dealt with. For example, the Thing tries to encourage the participant to dance. Initially some people just wandered passed it, ignoring it. We focused on this moment. The participant's navigation was slowed down. The agent stayed firmly in front of her, suggesting that they loosen up before dancing and instructing, "Raise your hands above your head." The agent watched to see that both hands were up, commenting if they weren't, establishing that it knew what the user was doing. The participant was hooked. As we build and test The Trial The Trail we have observed one participant who was so focused on trying to rescue Patofil from her set of bad guys that he ignored those that were surrounding him. The scene is designed to accommodate more aggressive users running at the bad guys, and less aggressive ones backing away. However, we now need to detect this particular kind of user and give them a very early and definitive push, to make them pay attention to their own plight and the agents confronting them.

Our psychological arc is also vulnerable if the user's frustrations with the interface fight with the kind of emotional state we are trying to encourage. The vigil scene in The Trial The Trail is meant to leave the user feeling happy. It makes Patofil and the user co-conspirators against Filopat, and the activities they do instead of the vigil are designed to be playful. One problem that can occur is that the inexperienced participant may become frustrated at her inability to master the activity -- for example hitting the whisps. The snare needs to be sensitive to the participant's responses and the state of mind that may be inferred from them, so that contingency plans for achieving the current goal can be called into play.

A very substantial challenge is balancing the user's expectations of being able to act freely and experimentally in an interactive context, with our goal of providing an authored emotional journey. The Thing Growing was very simple, but with the more elaborate story of The Trial The Trail, will our user tests show the same kind of convergent behaviour so that we will only have to make a manageable number of alternate routes to implement the psychological arc effectively?

Josephine Anstey, University at Buffalo, Dept. of Media Study, 231 CFA, Buffalo, NY 14260

Acknowledgements
I would like to thank Stuart C. Shapiro, Dave Pape, Orkan Telhan, Trupti Devdas Nayak, Mike Kandefer, Anthony Ekeh and Paul Visco for their work on The Trial The Trail. This project has been supported by an IRCAF grant from the University at Buffalo.

References
[1] D. Spoto, The Dark Side of Genius, The Life of Alfred Hitchcock (Little, Brown and Company: Boston, 1983)

[2] S. Nath, (2003) "Story, plot & character action: Narrative experience as an emotional braid" in Technologies for Interactive Digital Storytelling and Entertainment Conference, 2003, pp. 1-18

[3] J. Anstey, "Writing a story in virtual reality" in M. E. Hocks and M. Kendrick (eds.), Eloquent Images (The MIT Press: Cambridge, Massachusetts: 2003), pp. 283-304.

J. Anstey, P. Pape, D. Sandin. D. (2000) "The Thing Growing: Autonomous characters in virtual reality interactive fiction" in IEEE Virtual Reality, 2000.

[4] J. Anstey, P. Pape, S.C. Shapiro, V. Rao, (2003) "Virtual drama with intelligent agents" in Hybrid Reality: Art, Technology and the Human Factor, Proceedings of the Ninth International Conference on Virtual Systems and MultiMedia, International Society on Virtual Systems and MultiMedia, 2003.

J. Anstey, P. Pape, S.C. Shapiro, O. Telhan, T. Devdas Nayak, "Psycho-drama in VR" in The Fourth Conference on Computation Semiotics, 2004, pp. 5-13

[5] C. Bandera, S.C. Shapiro, H. Hexmoor, "Foveal machine vision for robots using agent based gaze control" in Final Technical Report #613-9160001 (Amherst Systems Inc.: Buffalo, NY, September 1994).

H. Hexmoor, J. Lammens, S.C. Shapiro, "Embodiment in GLAIR: a grounded layered architecture with integrated reasoning for autonomous agents" in D. D. Dankel II and J. Stewman (eds.), Proceedings of The Sixth Florida AI Research Symposium, FLAIRS 1993, pp. 325-329.
H. Hexmoor, S.C. Shapiro, "Integrating skill and knowledge in expert agents" in P. J. Feltovich, K. M. Ford, and R. R. Hoffman (eds.) Expertise in Context (AAAI Press/MIT Press: Cambridge, MA, 1997), pp. 383-404.
S.C. Shapiro, H.O. Ismail, "Anchoring in a grounded layered architecture with integrated reasoning" in Robotics and Autonomous Systems, 43(2-3), 2003, pp. 97-108

[6] S.C. Shapiro, W.J. Rapaport, "SNePS considered as a fully intensional propositional semantic network" in N. Cercone and G. McCalla (eds.), The Knowledge Frontier (Springer-Verlag: New York, 1987), pp 263-315.
S.C. Shapiro, W.J. Rapaport, "The SNePS family" in Computers & Mathematics with Applications, 23(2-5), 1992, pp. 243-275 [Reprinted in 16, pp. 243-275].

S.C. Shapiro and The SNePS Implementation Group, SNePS 2.6 User's Manual, Department of Computer Science and Engineering, University at Buffalo, The State University of New York, Buffalo, NY, 2002. Available at http://www.cse.buffalo.edu/sneps/Manuals/manual26.ps

[7] D. Pape, D. Sandin, "Alive on the grid" in 6th World Multiconference on Systemics, Cybernetics and Informatics, 2002

[8] D. Pape, T. Imai, J. Anstey, M. Roussou, T. DeFanti, "XP: An authoring system for immersive art exhibitions" in Fourth International Conference on Virtual Systems and Multimedia, 1998, pp. 528-533

[9] J. Anstey, P. Pape, S.C. Shapiro, O. Telhan, T. Devdas Nayak, "Psycho-drama in VR" in The Fourth Conference on Computation Semiotics, 2004, pp. 5-13

[10] M. McMahan, "Immersion, engagement and presence" in M. J. P. Wolf and B. Perron (eds.), The Video Game Theory Reader (Routledge: New York, 2003), pp 67-86

[11] P. Perron, (2004) "Sign of a threat: The effects of warning systems in survival horror games" in The Fourth Conference on Computation Semiotics, 2004, pp. 132-141

[12] M. McLuhan, "The galaxy reconfigured" in N. Wardrip Fruin and N. Montfort (eds.), The New Media Reader (The MIT Press: Cambridge, Massachusetts, MA, 2003), pp. 194-202

[13] Ibid. [11]

[14] J. Anstey, "The Thing Growing:: Video documentation," Retrieved February 25, 2005 from http://www.ccr.buffalo.edu/anstey/VDRAMA/THING/MOVIES/thing_mfashow_hires.mov

[15] J. Anstey, "Writing a story in virtual reality" in M. E. Hocks and M. Kendrick (eds.), Eloquent Images (The MIT Press: Cambridge, Massachusetts, MA, 2003), pp. 283-304

[16] A. Bobick, S. Intille, J. Davis, F. Baird, C. Pinhanez, L. Campbell, Y. Ivanov, A. Schutte, A. Wilson, "The Kidsroom: A perceptually-based interactive and immersive story" in Presence 8(4), 1999, pp. 367-391.

N. Szilas. "Idtension: a narrative engine for interactive drama" in Technologies for Interactive Digital Storytelling and Entertainment Conference, 2003, pp. 187-203

[17] P. Sengers, "Do the thing right: an architecture for action-expression" in Second International Conference on Autonomous Agents (ACM Press: New York, 1998), pp. 24-31

[18] R. Pausch, J. Snoddy, R. Taylor, S. Watson, E. Haseltine, "Disney's Aladdin: first steps toward storytelling in virtual reality" in SIGGRAPH 96, 1996, pp. 193-203

[19] S. McCloud, Understanding Comics (Tundra Publishing: Northampton, MA, 1993)

[20] J. Anstey, P. Pape, "Being there: Interactive Fiction in virtual reality" in Consciousness Reframed 3, CaiiA (University of Wales College: Newport, 2000).

R. Barthes, S/Z (Hill and Wang: New York, 1974).

J. Williamson, Decoding Advertisements: Ideology and Meaning in Advertising (Marion Boyars: London, 1978)

[21] J. Anstey, P. Pape, "The provoking thing: A VR relationship" in Emotional and Intelligent II: The Tangled Knot of Social Cognition, Papers from the 2001 AAAI Fall Symposium, Technical Report FS-01-02 (AAAI Press: Menlo Park, California, 2001), pp. 3-8

[22] J. McGonigal, "A real little game: The performance of belief in pervasive play" in Digital Games Research Association (DiGRA), "Level Up" Conference, 2003

[23] J. Anstey, "The Thing Growing: Interviews 1," Retrieved February 25, 2005 from http://www.ccr.buffalo.edu/anstey/VDRAMA/THING/MOVIES/thing_talk1.mov

J. Anstey, "The Thing Growing: Interviews 2," Retrieved February 25, 2005 from. http://www.ccr.buffalo.edu/anstey/VDRAMA/THING/MOVIES/thing_talk2.mov