The importance of audio on player experience and the causality of immersion.

Introduction:

From a two-dimensional image with a single control, to a virtual reality setup where infinite worlds can be explored and experienced in person, video games have evolved drastically since the first-ever gaming console in 1972, the Magnavox Odyssey (Computing History, n.d.). This console, originally considered an 'incredible breakthrough' at the time of launch, consisted of a plastic sheet placed on the television screen to form the game environment. However, it lacked one crucial element; audio. A few months later in November 1972, Atari released Pong (Digital Sound and Music in Computer Games, n.d.), a 2-D game where players would control a paddle to hit a ball onto their opponent’s side to score points. Due to the inclusion of a 4-bit 2-channel sound card that could produce singular monaural sound, the noticeable difference was that this console could produce something that would forever change the way we interact with video games: sound.

Due to the ever-increasing reality of video game graphics and environments, the depth of immersion has improved. Sound has also progressed, moving from single monotonous beeps to a multi-textured ambient environment offering thousands of sounds during a brief exposure of gameplay, granting the player a sense of immersion thus improving the players’ overall experience. “Previous studies have shown that immersion is an area of game development that has attracted great interest from gamers and researchers” (Gormanley, 2013). Passionate gamers are using ‘immersion’ mods, such as alterations in lighting, weather or even different sound packs, to make the virtual world more immersive and to allow for enhanced experiences for the player. Popular immersion mods can be associated with Stalker: Shadow of Chernobyl (2007), Elder Scrolls V: Skyrim (2011) and Witcher 3 (2015) (see. NexusMods, 2018 & Tarason, 2018).

As video games slowly increased in popularity, journalists and researchers also became more interested in exploring the physical and mental impact of video games on the player experience regarding the audiovisual aspect of the environment - how sound and visuals react together to create different experiences, in a field of study known as Ludomusicology. Zhang J & Fu X (2015) identified how important background music was to improve player experience and how sound can induce immersion. Immersion is the process of entering a specific state of mind and being “completely involved in something” (Cambridge Dictionary, n.d.). By registering certain factors of immersion, e.g. talking to oneself during gameplay or a sudden increase of game performance, researchers can find out if music or sound acts as an immerser, or if it is caused by the physical graphics. By exploring the relationship between game audio and computer game immersion, Huiberts (2010) developed and referred to an auditory framework known as IEZA (Interface Effect Zone Affect), a 2-dimensional structure that is used to analyse and explain auditory environments within video games, offering an insight into player immersion.

The results found in Zhang & Fu’s study showed that participants “expressed more serious time distortion than those participants without hearing background music” (Zhang & Fu, 2015) and achieved significantly higher scores when the background music is active. Time distortion is a concept of “perceptual distortion, at times, experienced in modified states of awareness, wherein time seems to pass either with extreme quickness or with great slowness” (Pam, 2013). Time passing with extreme quickness can be seen as a factor of immersion.

Gormanley (2013) explored immersion further with a focus on background music and sound effects within online games. A prototype game was created allowing for different applications of audio within gameplay. After obtaining 30 responses from computing students, he found over half of his participants thought the background music to be immersive.

Music and sound together can immerse the player and lead to higher performance within gameplay. Tafalla (2007), with an altered focus on gender, researched whether there was a correlation between the presence of sounds and the performance of the players. He discovered that male gamers scored twice as many points when playing DOOM (1993), a first-person shooter, with the audio enabled. The ambient soundtrack consists of horrifying screams, heavy weapon shots, powerful explosions and pulsing basslines, offering the male players a sense of heightened excitement granting them immersion into the game environment, whereas, typically, female players found it be a stressor. Cassidy and MacDonald (2009) researched into whether sound added to or detracted from player concentration, this time with a focus on driving games, played with four varying conditions: silence, car sounds, player-selected music and researcher-selected music. They found that people playing with pre-determined music found it “highly arousing” and drove the fastest with the most significant number of mistakes.

Yamada et al. (2001) broke the trend of ‘music improves performance’, as results from their study show that people had surprisingly better scores with the sound turned off, suggesting that there were higher levels of concentration. Moore (2013) explored the impact of audio in a text-based video game on player emotions, whether they enjoyed it more or less with audio. The results found within this study showed that sound did not affect the participant's emotional experience of the game. However, the inclusion of audio influenced the time it took participants to complete the game.

Although there has been an increased amount of research into how music and sound affects player performance within one particular genre, there is a lack of research containing comparable results spread across multiple genres. This particular research has been designed to focus on and provide useful information on similar elements that add to the findings of cross-genre studies, whilst also seeing if immersion can be granted without sound. If results could offer proof that sound makes a difference, it could help developers improve game environments.

Qualitative methods allowed for the exploration of emotions and feelings of the participants at a specific time and require the “studied use and collection of a variety of empirical materials – case study, personal experience and interviews, along with observations” (Denzin & Lincoln, 2005). As this study focused on developing “a complete, accurate, clear and articulate description” (Brown.uk, n.d.) of player experience, it took techniques taken from empirical and phenomenological research. The fragile nature of emotions meant the results were not an “empirical generalisation applied to a population” (USCLibraries, n.d.), but instead allowed for interpretations, contextualisation and gaining insight into phenomena. Through observations and interviews, I tried to address the questions of ‘How does sound and music affect player experience within multiple genres of video games’ and, with a smaller focus, ‘How does sound help in achieving a state of immersion?’.

Unlike quantitative research, this type of study has less control and requires different types of interpretations due to the subjective nature of the results. The individual movement of each player creates a slightly altered experience as it triggers different phenomena within the game environment. As a researcher, it was difficult not to influence any results by asking questions that could have been leading towards a specific answer.

To gain a greater understanding into the phenomena of how music and sound affected the player experience, participants were gathered through purposive sampling, also known as subjective sampling, a “non-probability sample that is selected based on characteristics of a population and the objective of the study” (Crossman, 2018). In this study, five people of varying gaming experience, ranging from low-high, were chosen to see if previous experience, or a preference for a particular genre, made a difference.

Before the study started, participants were made aware of the overall research aims, potentially resulting in them focusing more on the study environment than the game itself. Results may have differed had I not explained it beforehand, but it could have caused ethical issues due to horror games, so consent was gathered beforehand.

During the study, observations were carried out as people are often not willing to express their true opinions in a one-to-one format as it may make them feel embarrassed or exaggerate responses as they cannot remember quite how they felt. Due to the ethnographical nature of this study, observations allowed me to focus on how people react within their natural setting. While the participants played the game individually, I sat behind them observing any changes in body language, e.g. breathing slowing down or speeding up, and whether there was a noticeable difference or not based on sound, whilst using an observation rubric (see Appendix A). One issue with observations is specific changes in body language could have been open to subjective interpretation, such as foot movement.

By gathering and understanding information, the perception of the participants’ experience was carried out through a constructivist qualitative approach. The idea of constructivism puts forwards the idea that “learning is an active, constructive process” and that “people actively construct or create their own subjective representations of objective reality” (David, 2015). Participants crafted their perception of the world by interacting with objects and activities located within a specific environment.

After the experiment, a face-to-face semi-structured interview (see Appendix B) with each participant, consisting of open and closed questions, took place lasting between 7’06” – 14’30” (see Appendix C for range of timings). Due to its format and the non-specific answers given by the participant, multiple questions branched off into other sub-categories as each person answered differently.

The experiment for each participant lasted about two hours; the observation itself for ninety minutes prior to the interview. As well as observations, hand-written notes were taken throughout which seemed to be less intrusive than using video recordings and allowed for notating the participants’ current behaviour. The full research design and physical setup can be found in Appendix D. The linear design of the study allows it to be easily repeated and reproduced for other researchers to see if results are comparable. Content was analysed with a focus on common themes leading to categorisations of information.

Throughout the study I noticed the inclusion of sound enhanced player experience and the levels of immersion seemed to increase. Whilst referencing the observation rubric, there was a significant increase in body movement when sound was active. Although not as noticeable, there was an increase of in-game performance with music on. This study focused on whether there was a considerable difference between genres with sound on and off. There were copious amounts of data gathered from my own observations that showed how music and sound affect each participant’s experience.

Genre Difference:
- The importance of music and sound varied between genres. The sound seemed to be more important in the horror game than in the other games as horror often relies heavily on sonic aspects.
  - P1: “Horror was definitely the most immersive one”.
  - P2: “I was really aware of the sound… because there’s a lot going on in the horror one like creaking and heartbeats”
- Accuracy improved and fewer mistakes were made.
- From observations, when sound was introduced, each participant, especially Participant 4, stopped what they were doing and followed audible clues.
- Different game genres seemed to access different moods. The most stressful game (Call of Duty) seemed to be the one where music enhanced the most, yet Ori and the Blind Forest, a simple 2-D platformer, sound enhanced the least.
Music and Sound:
- If sound is of a poor quality or inactive, the game environment is also considered to be of a poor quality.
- Participants very rarely spoke about sound separately to music. They would describe audible elements as ‘music’.
Immersion:
- Gaming experience had some impact on this as players with a genre preference did not connect as much to other genres.
- From observation, female participants did not become as involved in the game environment as males when sound was introduced, although they still performed better than without sound.
- Levels of immersions varied between participants, P3 talked about how they usually struggle with concentration levels normally but music and sound helped them to become more engaged with the game environment.
- The main result coming from the study was that sound was a core factor within immersion
  - P4: “More with sound, I was more immersed”
  - P5: “Immersion. It immerses you into the game”
- Frustration:
  - Frustration appeared to be a relevant factor within the study as when participants were dying a lot on Call of Duty, it created frustration and led to a removal from the game environment.
    - P2: “In call of duty when I kept dying… it was annoying”
    - P4: “I got a bit frustrated, I thought, I’m going to give up”
  - Experience:
    - A lack of experience caused problems for some participants as they seemed to become more frustrated than others, meaning it was harder for them to access the state of immersion within complex games [e.g. Call of Duty]
  - Controls:
    - Certain participants also found the controls to be unnecessarily confusing, making them focus more on the controls than on the game sound and environment.
- Time Distortion:
  - Time distortion, the unconscious realisation of 'warped' time passing, was apparent throughout all of the studies as participants did not feel each activity was ten minutes in length. However, this varied due to sound being active or inactive, as with sound, the participants felt the study was much shorter, indicating a sense of immersion.

This particular study was important as it helped show that sound is a critical element within game creation. With reference to previous literatures, this study explored the two main questions ‘How does music and sound affect player experience within multiple genres of video games?’ and ‘How does sound help to achieve a state of immersion?’.

Although the term immersion is often debated by academics, one clear finding that emerged throughout this study was, regardless of the video game genre or type, music and sound is necessary for the player to become immersed within the game environment. It also became clear that, with sound, players’ performance improved and the state of immersion, gathered through experience of time distortion, frustration and experience, was easier to access. Without sound, players were more easily frustrated: with sound, player performance improved. The difference between game genres did not affect the overall findings: sound still created immersion and helped improve the game environment.

The results from this study coincide with Cassidy and Macdonald (2009)’s results as both studies show that music and sound helped increase player concentration and improve the player’s experience, in this particular case, within multiple games. In all cases, music did enhance the player experience which led to a slight increase in game performance, with accuracy improving in Call of Duty and less deaths occurring within Ori and the Blind Forest. It can be argued that participants’ gameplay improved because of past experience, however music certainly seemed to play a part. This was more evident in ‘Participant 3’ as I noticed concentration levels increased drastically with sound on as they were quieter and moved around less within the experiment environment. These particular findings from the study are important as they show that chaotic game environments need to be improved in a way that can appeal to players who struggle with concentration levels so as to keep their attention at a peak but not to overwhelm them.

My findings show that multi-textured sonic environments are crucial within video games as sound helps create a moment for the player, therefore acting as an immerser. Video games offer players an escape from reality so immersion is a crucial factor and, without sound, this state of mind would not be so easily accessible. Results such as this help to show game developers the areas that need to be prioritised and improved. Throughout the study, participants showed higher amounts of time distortion, with sound active. This was a focus within Zhang & Fu (2015)’s study as they looked at how crucial sound is within player experience for achieving immersion.

Although not a heavy focus of this experiment, certain results, taken from my own observations, showed that male gamers performed better with sound than female gamers which coincides with Tafalla (2007). This may be due to sound playing a bigger role on male gamers or it could just be a lower amount of experience from both female participants.

These findings demonstrate that, if the sound element of video games is not of a high quality, the players will not become immersed nor will they want to play the game. The results found within my study can be applied to professional practices within the creation of game environments and audio development, to help game developers become aware of the issues that lie within sound creation and to further increase player experience, or even to other researchers wanting to explore field of Ludomusicology.

Appendix A:

Observation Rubric:

Appendix B:

Interview Schedule/Structure

Opening

Hi, thanks for taking the time to let me interview you. My name is Jonny, I am a student at the University of Huddersfield and I am currently researching the effect of music and sound on player experience.

I would like to ask you a few questions about how you interacted with the game environment and what you thought of your experience.

I aim to use this information to help improve gaming environments and see if sound/music is critical within immersion.

This interview should take about 20 minutes. Do you have time to answer these questions?

[consent form]

Body:

Make sure they know if they’re talking about general preference or exact moment

First block – Past Experience and General Attitudes
1. What is your experience of video games?
2. What is setup preference? Controller, playback device (speaker/headphones)
3. What other games have you played?
4. Do you have other media devices on whilst playing games?
  1. It would take away from immersion – cause distractions?

Second block – Awareness (Indicator of immersion) – Specific to moment
1. What was your thought of the games played?
  1. If a player doesn’t have much experience of video games and don’t become immersed it could mean they don’t like the game rather than the environment
2. Can you remember what the music sounded like? Describe it
  1. Observe which genres they mention
3. Did the experiment environment affect how you enjoyed the game?
  1. How did it affect you?
4. Would you know how long the test was if it wasn’t mentioned beforehand?
  1. Indicator of immersion is time distortion

Third block - Focus on immersion – Specific to moment
1. Why could you/couldn’t you focus on the music?
2. How did the game environment distract you from sound?
3. What role does immersion play for game experience?
4. Did you enjoy playing the game?
  1. Were you frustrated at any point?
    1. How did the controls affect this? [if so]
    2. Yes - How did frustration affect immersion?
    3. No - How did immersion lead to the enjoyment of your experience?

End with open question
1. Is there anything I haven’t asked about that you think I should know?
  1. Explain the study then ask them if they wanted to add/correct something
2. How did music and sound affect your player experience?

Closing:

I hope you enjoyed the game environment and I am grateful you took the time for this interview and study. Is there anything else you can think of that would benefit this study?

Appendix C:

Interview Timings:

Average interview time lasting: 11’00”

Range of interviews: 07’06” – 14’30”

Appendix D:

Full Research Design:

5 participants of varying gaming experience
Each participant played three video games of different genres for twenty minutes each (ten minutes with sound, ten minutes without) in an environment of dimmed lighting.
- Call of Duty: Modern Warfare 3 (2011)
  - A first-person shooter/action game set in the modern day.
  - The specific level played was ‘Black Tuesday’ where the player must destroy the radio jammer placed on top of New York’s Stock Exchange.
- Ori and the Blind Forest (2015)
  - An atmospheric platform game.
  - The level played is the prologue where Ori, a small spirit guardian, must help seek guidance after his mother’s death.
- Among the Sleep (2014)
  - A first-person atmospheric horror game.
  - The level played is the prologue where the player assumes the role of a young child who tries to find his mother with help from his teddy bear.

Full Physical Setup:

Desktop PC
Monitor
Sony MDR7506 Pro Closed Headphones (Noise-cancelling)
- Headphones are immersive by nature as they cancel out background noises and they offer a wider stereo image (180’) than speakers.
- Binaural imaging gave participants a literal feel for the environment.
Sumvision Nemesis Wraith Gaming Keyboard
Havit 3200DPI Gaming Mouse
Zoom H4n Pro Recorder – Audio recording through experiment and interviews
In-game audio set to 100% - Sound, Music, Ambience and Voice Over (if applicable)
Computer audio set to 30%

Appendix E:

Interview Transcriptions:

Name:

Person 1

Person 2

Person 3

Person 4

Person 5

Age:

21

22

27 Gender:

Female

Male

Female

Male

Gaming Experience:

Casual

Moderate-High

Minimal

Moderate

High

Setup Preference:

Console

Computer

Favourite Genre:

Role-Playing Games (RPG)

Casual

FPS/Football Games

Fantasy

Sound Preference:

Speaker

Headset

Speaker

Name:

Person 1

Person 2

Person 3

Person 4

Person 5

Length of Interview:

12’15”

11’21”

14’30”

09’48”

07’06”

Methodology:

Results:

Discussion

Person 1: Female, 22, moderate experience of video games (casual), preferred genre RPG.

Interviewer: What is your experience of video games?

Person 1: As in what kind of video games?

I: Yeah video games in general

P1: I guess I play it quite a bit but I wouldn’t say I am a ‘gamer’ – more casual gamer.

I: What is your setup preference? Controllers? Mouse and Keyboard?

P1: Yeah, I usually play with controllers so it would always be on a console as I don’t have my own pc

I: Do you play with headphones? Like a headset?

P1: No

I: What other games do you play besides the ones we played today

P1: I play smash, league, lots of role-playing games like neon and stuff like that

I: Do you have any other media devices on whilst playing games? So, do you listen to Spotify or YouTube whilst playing?

P1: Yeah, I listen to Spotify, YouTube and iTunes

I: And would you normally mute the sound of the game?

P1: No, I don’t get it when people do that. That’s the one thing I don’t get. A lot of my friends do, they’ll play a game with Spotify running in the back ground (especially X-Box players) and I don’t understand why they do that, so I personally don’t.

I: What was your thought of the games played in general?

P1: What I thought? As in, did I enjoy them?

I: Yeah

P1: Yeah, I quite enjoyed Cod (Call of Duty: Modern Warfare 3) and Ori and the Blind Forest but not quite so much the third one (Among the Sleep)

I: Why was that?

P1: Just because I didn’t really like the controls very much, I had to turn the sensitivity down a bit

I: Was it too high or did it just feel clunky?

P1: No, it was too high and it was blurry so I had to turn the sensitivity down. But, I’m not a fan of horror video games, I like watching other people playing them but don’t like playing them myself

I: Can you remember what the music sounded like in any of them?

P1: Ori and the blind forest is just amazing; the music was amazing. Cod doesn’t really have a defining sound other than gunshots and screams and I guess the 3rd one is just generic horror – high strings etc.

I: Did the experiment environment affect how you enjoyed the game? So me sitting here observing?

P1: Hmmm, not really. I guess if I had a technical difficulty I would just look at you, but not really.

I: Would you know how long the test was if I hadn’t mentioned it beforehand? So, did time feel distorted with sound/without or not at all?

P1: No, not really, it didn’t feel like 10 minutes. It felt like I had a bit longer than 10 minutes

I: When in the game environment, why could you/couldn’t you focus on the music? In the cod you said it was just gunshots, did you pick up the ambience or the music at all?

P1: I honestly can’t remember off the top of my head, all I can remember is the gunshots, that is the core of the game and they’re not really known for their music.

I: What about Ori?

P1: Oh Ori, I definitely remember the music, the woman vocalising and stuff like that because you can tell they focus on the music compared to cod.

I: And among the sleep being high-string horror?

P1: Really generic horror.

I: How did the game environment itself distract you from sound? Say in Ori when you were fighting the creatures, did that distract you from the music?

P1: When an enemy appears you kind of switch to focusing on gameplay just because that is the forefront of what you are doing at the moment. When you are just walking, you’re enjoying the environment but then when there’s an action you need to do, your focus shifts slightly. The music is still there but not as much

I: Do you reckon the sound takes over rather than the music itself?

P1: Yeah, I guess so. You’re listening out for those sounds to see if you’re hitting your enemy

I: Speaking of sounds in an environment, did you think that localised sounds were more important? So, if you heard a sound to your right, would you turn towards it?

P1: I feel like for a game like Ori, it wouldn’t matter so much as it’s a 2-d platformer but for Cod and most definitely the horror game that would be really important obviously in horror if you hear something on the right there’s probably going to be something on your right you need to look out for. Cod probably as well but as there’s so many sounds going on you don’t really know what the hell is going on anyway.

I: Is that why horror is so successful because of how limited the sound/music is?

P1: There isn’t as much going on yeah so definitely a more important aspect when it comes to horror games. I don’t know whether I would say the music is successful anyway as it’s such a niche genre of music so you can’t go crazy on the melodies and that.

I: How important would you say sound and music is in games?

P1: Oh, really important, I feel like 50% of the game is music and sound. If you have a bad sounding game it’s going to take away from the gameplay experience. It brings us back when you asked the question “do you listen to Spotify?” with the game, I feel like you’re taking away your experience of the game if you’re doing that. If you’re playing a horror game and you’ve got Earth Wind and Fire playing in the background, it doesn’t make sense.

I: What role would you say immersion plays for game experience? Do you feel immersed with music on?

P1: Horror was definitely the most immersive one and sound did play a big part in it as it getting you into the mood of “oh crap, somethings about to happen”. I’d say Ori, it creates a mood but doesn’t immerse you, personally when I played it. But if anything, the music was so good that I didn’t really want to play the game, I just wanted to sit and listen to the music. Cod, the sounds are there just to make you feel like you’re in a war zone)

I: Were you aware of vocal noises you made? Such as laughs and remarks?

P1: Yeah, the thing is, in horror games you make those remarks to draw yourself away from the horror that’s about to unfold. I do know that I do commentary whenever I play a game. It’s just like the process of playing games like “oh no”.

I: Do you reckon that enhances the experience for yourself?

P1: Yeah because you are playing the game. So if you make remarks then it’s like you’re getting into it.

I: So could it be argued you were getting yourself immersed?

P1: Yeah, I guess you could say I was getting immersed.

I: Was it due to music or not?

P1: Again, I would say the horror yes, the other two no. Music is important but I think gameplay is still how the engine runs [meaning its more important]

I: Did you enjoy playing the games?

P1: Yeah, I would definitely play Ori and the blind forest again

I: Were you frustrated at any point?

P1: I was frustrated at Cod and the Horror game from the controls just because Ori is very simple, it’s just left right up and down jump. It’s something about aiming games, anything that uses physics-based engines as it doesn’t always work fluidly. I just hate aiming.

I: Did you feel the frustration you had took you away from the game environment?

P1: Yeah, when you’re frustrated you want to be like “I don’t want to play anymore”. Say if Cod got a bit frustrating I could be like “right, I’m not playing anymore” but I guess because I’m in an experiment situation it’s not as if I could walk away from the game, well I could’ve. But it wasn’t causing me any discomfort, just frustration.

I: Overall, just to re-clarify, how did music and sound effect player experience?

P1: Definitely better with, especially horror game as I didn’t even know what I was doing at first. You know with horror you listen to the sounds as well if something bad is about to happen there will be a sound signal and because I wasn’t hearing that I wasn’t sure if someone was coming, so I was running around like everything was perfect. Cod I could’ve played with the sound. And Ori is so beautiful that you just want to listen to the music.

Person 2: Male, 21, moderate experience of video games, preferred genre RPG, FPS & Open World.

Interviewer: What is your experience of video games

Person 2: I used to play quite a lot of video games on the Xbox whilst I was a teenage, but I’m too poor to buy games now so I don't really do it anymore

I: What genre's in particular?

P2: I like fantasy open world stuff like Skyrim, also I've got a few first-person shooters. Mainly open world stuff.

I: Do you have a setup preference? Controller?

P2: Yeah, Controller

I: Do you play through speakers normally or through headset?

P2: A bit of both really

I: Do you have other media devices on whilst playing games? Spotify/iTunes

P2: If I am properly playing a game, like doing a campaign or something then I will be focused on the game. But if I am messing around on something like GTA, I might have music on, on my phone

I: Would you mute game sounds?

P2: Possibly. I would turn off the radio in the car on GTA

I: What was your thought of specific games today?

P2: I thought they were pretty good. I was surprised about the 3rd one [Ori and the Blind Forest]. I enjoyed that more than I expected. I had a bit more previous experience with call of duty

I: Can you remember what the music sounded like?

P2: Not really on the horror one, was there music on the horror one?

I: Yeah

P2: Was it more like ambience and stuff?

I: Yeah, like high-strings

P2: I remember the heartbeat and stuff like that and a lot of weird noises, but I can’t really remember the music

I: Did the experiment environment affect you in anyway?

P2: Not really, other than I was talking to you from time to time

I: If I didn’t tell you how long the test was, would you know it was 10 minutes?

P2: I think that it definitely felt shorter on Ori and the Blind Forest with the sound, I think it felt longer in the call of duty one but I think that’s because I got stuck on that one section where I kept dying. And the horror one, I feel kind of equal on that one. It was more interesting with the sounds on because it was. The time felt quite the same.

I: How did the game environment distract you from sound?

P2: How focused I was on the sound?

I: Yeah, maybe during stressful moments

P2: In call of duty I wasn’t really focused that much on the sound obviously because the gameplay is more tense, I guess, and you’re concentrating on lots of stuff but I was really aware of the sound in the other two because there’s a lot going on in the horror one like creaking noises and heartbeats and all that. Also, the last one, that it added a very relaxing element to the game that kind of changed the whole aesthetic I guess.

I: How did your experience change with sound and without sound?

P2: In call of duty, I was able, with sound, I was more aware of what I was supposed to or where I was supposed to go because there’s people in game telling you stuff but without sound you have to

read the subtitles, there’s already a lot going on anyway, so you can’t really do that. With the horror one [Among the sleep] that wasn’t really scary at all before the sound came on because I was just a child with a teddy. And the last one wasn’t really a dramatic change, but it was, kind of, a lot more peaceful and relaxing.

I: Did you feel frustrated at any point?

P2: In general, or just with sound elements?

I: I think both really

P2: I guess in the bit in call of duty when I kept dying, I wasn’t angry or anything, was just a bit annoying

I: Did that annoyance make you feel as though you were removed from the game environment?

P2: Well I guess the nature of the death in the game, where it stops and there’s a loading screen, that obviously takes you out of the experience, but as soon as you start the game you’re just launched back into it and also, I was kind of, when I did die, I might have talked to you at some point so that took me out of the game. But I wasn’t really frustrated or anything

I: Were you aware of physical movement? So your leg was bouncing, your body was moving to the side where you aimed?

P2: Really? No I wasn’t aware, I mean, my leg bounces. I do that naturally all the time

I: Is there anything I haven’t asked about that you think I should know?

P2: I guess at the point where I was playing the second game, the horror one, when there was a sound coming from upstairs of somebody moving furniture, that took me out of the experience a little bit.

I: As it is a study including immersion, do you think sound is an immerser or not?

P2: Yes

I: Would you say it’s crucial?

P2: It depends how it’s used on how crucial it is. In the last game I wouldn’t say sound was incredibly crucial because it’s just sounds of the forest and when you’re shooting those things, but it still adds to the immersion of being in a game but in something like call of duty it’s a lot more crucial because there’s a lot more going on and you have to pay attention to the characters of the game and also in the horror game it’s crucial as it completely changes the atmosphere of the game.

Person 3: Feale, 22, limited/minimal experience of video games, preferred genre casual.

Interviewer: What is your experience of video games

Participant 3: I don’t really have much

I: Although limited experience, do you have a favourite genre?

P3: I like adventure and casual

I: What is your set up preference? Mouse and Keyboard? Xbox?

P3: Usually a computer [mouse and keyboard]

I: When you listen through sound, is it done through headphones or speaker?

P3: Headphones

I: Apart from the games today, what other games have you played?

P3: Sims and Goat Simulator

I: When you play games, do you have any other media devices on while playing? Spotify, YouTube,

iTunes?

P3: Sometimes

I: When you do that do you mute the original game sound

P3: Probably lower the volume

I: So not fully muted?

P3: No

I: What was your thought of the games played today?

P3: I enjoyed all 3 of them

I: Any favourites?

P3: I preferred the last one [Ori and the Blind Forest]

I: Why is that?

P3: Because it was more of an adventure game

I: Did sound play a part in that decision?

P3: Yeah, I really liked the music

I: Can you remember what the music sounded like in the other games? If so, could you describe them

P3: The music in call of duty was quite tense so I didn’t enjoy it that much and the music in Among the Sleep it was quite eerie

I: Do you think that helped create the environment?

P3: Yeah, it helped me to get more involved in the game compared when I didn’t have the sound on

I: How does the environment experiment affect how you enjoyed the game?

P3: When I had the sound off, I was aware that you were sat behind me, watching. But with the

sound on, I didn’t notice as much, I was more involved with the game, concentrating more.

I: Would you know how long the task was if not mentioned beforehand?

P3: Probably not, when I was playing the games I lost track of time

I: More with sound? Less with sound? Or just the same?

P3: Time seemed to go quicker when I had the sound on

I: How did the game environment itself distract you from sound?

P3: I think because there was so much sound in that game, I only noticed certain things, so I was aware of all these different sounds, but I didn’t really pick up on what was being said, more just background music. I know the things that were said were being repeated again and again but I didn’t really pick out any specific words.

I: One thing you commented on which I found quite interesting after Among the Sleep, is you said

with the sound on, it detracted away from the game’s poor quality, could you explain?

P3: When I was playing the game with the sound off, I felt quite dizzy playing it as the camera kept jumping around quite a lot so the picture was good really but it just moved around too much and my focus was on that rather than the actual game but when the sound was on, I didn’t notice that as much.

I: Do you think that sound is an important factor of gaming?

P3: Yeah, I think that without sound, people wouldn’t concentrate as much so they wouldn’t get as involved in the game, they would get bored easily and they wouldn’t play the game for anywhere near as long, they wouldn’t find it interesting and they wouldn’t really know what was going on because you need sound to go alongside it so you understand what you’re doing. Especially for me because I struggle to pay attention to things, that would be a key thing to have on a game.

I: At times, for example the horror, would you say you were more reliant on sound than on visuals [for the horror environment]?

P3: Yes, because I think for that game, it was the sound that told you what to do, I was waiting to hear the voices of the characters on what I should be doing. The music helped to build on that so if the music was faster, I would know that something was happening but probably wouldn’t have known that otherwise.

P3: Yeah, I enjoyed three of them in different ways. All three were different genres of games and each one used sound in different ways, either to build suspense or create an atmosphere.

I: Were you frustrated at any point?

P3: Overall, I felt sound was a good thing but in call of duty, I thought there was too much of it, because there was the music, people talking and the explosions, there was too much, I felt that was kind of confusing and frustrating.

I: What about with Ori and the Blind Forest?

P3: I found that one a lot better because the music was a lot more relaxed so although there was other sounds going on, it didn’t bother me as much

I: How did less controls help in Ori?

P3: It was definitely easier in that way

I: With frustration how did it affect your ability to enter the game environment?

P3: For example, with call of duty when there are all these noises, it was kind of overwhelming and it put me off what I was doing, even though I knew what I was supposed to be doing, there was so much going on I just couldn’t focus on it, I couldn’t do it all at once. I was all over the place with the controls

I: Is there anything I haven’t asked that you think I should know?

P3: I just think that when there was sound, I definitely concentrated a lot better, I was able to get

more involved with the game and understand it as well. I think even though when the sound’s off, even though there were subtitles, I still think it is very difficult, like with the horror game, it was hard to just read those and concentrate on what was going on as well so you definitely needed the sound there to play it properly.

I: One last question I forgot to ask, were you aware of any body language movements? For example, with sound on, especially the horror, you were a lot quieter than before and your breathing slowed down. You also leant forward during tense moments. Were you aware of any of these?

P3: I was aware I was quieter because I was focusing but I wasn’t aware of the other things but I think I found on Call of Duty as well, I was a lot quieter as well playing that game, even though I found the sound distracting at times, when I did have the sound, it made me feel more involved and more focused on it. One last point I want to make, when I had sound off, I noticed I was quite fidgety and looking round the room a lot and getting quite bored but every time the sound was on, I stopped doing a lot of that.

Person 4: Male, 22, low-moderate experience of video games, preferred genre fps and football (fifa).

Interviewer: What is your experience of video games?

Participant 4: Right, my experience of video games I had an Xbox growing up, played it occasionally, has a Wii at one point and then from age 11 didn’t play much myself but I played it when I went to my mate’s house. Wouldn’t call myself a gamer though

I: What is your set up preference?

P4: Prefer to play on a controller

I: Does the sound come through speaker or headphones?

P4: it would come through headphones, well, I guess when I played Xbox as a kid it came through the speakers but when I was at my mates’ house it came through headphones.

I: Apart from the ones today, what other games have you played?

P4: GTA, Call of Duty zombies, Far Cry 3 and Fifa; that’s probably the most video games I play. Do you count like apps on your phone as video games?

I: I guess so yeah

P4: Then I play 10/10 which is more of a puzzle than a game

I: Do you have any other media devices playing whilst on games? Spotify/Youtube?

P4: No I don’t really like listening to music at the same time as playing games

I: What was your thoughts of games played today?

P4: I enjoyed them a lot, 3 different games. I would probably say [Call of duty] Mode warfare was my favourite, I feel like that was the one I was into the most, 2nd [Among the sleep] was my least and I enjoyed the 3rd [Ori] was but I think that was mainly, not the gameplay itself, but just how beautiful it looked and the soundtrack as well.

I: Can you remember what the music sounded like in any of the games?

P4: Yes, the first it was pumping you up. It was more orchestral with rock music. Second one, I don’t remember the music as such, it was more sound elements. Third one “phwoar” beautiful.

I: Did the experiment environment affect how you enjoyed the game?

P4: Umm, my eyes were hurting a bit in the first as it was a bit dark. It added to the game because it meant I could speak outload which I wouldn’t have done normally

I: How does the game affect how long you thought the test lasted for?

P4: I tend to lose track of time in video games.

I: More or less sound with sound?

P4: More with sound, I was more immersed.

I: Would you say your lack of experience of games you from enjoying the game and sound? For example, the keyboard controls.

P4: Yeah, they were annoying me in the first one. If I knew what I was doing, I’d enjoy it more.

I: During mw3, I noticed from my own observations that when with sound, your accuracy improved and you were more frustrated with death. How did sound affect that?

P4: I felt I was more in there because I could hear bullets flying past me and when it hit me it was like “OOH”, when they hit me in the game, I felt more pain because it…

I: So, you had a bond with your character?

P4: Yeah, I had an empathetic bond with my character

I: Relating to among the sleep, it seemed, all the games really, it seemed when there was no sound, you compensated and made your own noises, why was that?

P4: I don’t know, just like, I couldn’t hear anything so I was imagining what was going on, imagining the sounds and in the case of that one in particular, they were very different to what I thought they would be. Why was I making those sounds? Don’t know, entertain myself. Add the other element that wasn’t there.

I: Were you aware throughout the game of change of body language?

P4: My body language?

I: Yeah, in Among the Sleep, you had heavier breathing. You also leant forward.

P4: Oh god, no I didn’t. Didn’t realise.

I: With that in mind, how important would you say music and sound is within video games?

P4: I think it’s important, I think its underrated. I guess, when you think of a video game you don’t really think “oh I really enjoyed listening to the sound”, however when you don’t have the sound…

I: You take it for granted?

P4: Yeah, you take it for granted. It puts you in the world, it’s like if you walked around the streets and you couldn’t hear anything yet walked around when you could hear anything, whole new vibes.

I: Did your frustration at any point take you away from the game environment?

P4: Yeah, in Ori, I don’t know if you noticed but it took me several attempts to kill this one creature and I got a bit frustrated, I thought, I’m going to give up. With Mw3, it was part of the fun and I got a bit annoyed during among the sleep, where I didn’t know what I was doing so I had to ask you for advice.

I: Which took you out of the environment?

P4: Yeah because if I was by myself I probably would’ve quit, but you just think, yeah this is that bit, what am I meant to do?

I: With Ori, did sound make a difference to your frustration? For example, would you have quit?

P4: Without sound, I would have quit the game.

Is there anything I haven’t asked about that you think I should know?

P4: Yeah, in among the sleep, there’s a lot going on there, a lot of vibes. In particular, when that random woman came out of nowhere. I don’t think I would’ve jumped at all if I hadn’t had the sound. I would’ve just been like, oh … there’s a figure… whereas I’m in the sound and hearing “woaooahaha… boom!”. I think it added a lot because the visuals aren’t even that amazing so I think sound was most important in that one out of the three, I think the one it was least important was ori but the one I liked the most was in Ori.

Person 5: Male, 27, high experience of video games, preferred genre fantasy.

Interviewer: What is your experience of video games

Participant 5: Well they’re pretty much my life, aren’t they? I suppose they are part of my few hobbies. I can’t really give you a clearer answer than that. They have probably knocked 20 years off my life due to stress. They make me escape from the fact that, in all seriousness, the only reason I got into them was A. I was very depressed and it was a bit of escapism. I’ve got a crap job and most of friends at home are annoying, I could just forget about them for the time

I: What is your setup preference? Controller or mouse and keyboard?

P5: Mouse and keyboard.

I: What other games have you played?

P5: I could count them on one hand. For console, loads, I couldn’t even name them we’d be here all day but computer games, WoW [World of Warcraft], Rift, League of Legends,

Runescape, Age of Empires, Starcraft, there ya go. That’s probably about it.

I: Do you have any other media devices on whilst playing games?

P5: Yeah, my other computer. I probably watch something or talk to people on discord or rage at people on discord.

I: Do you feel that can take you out of the game environment?

P5: No. It’s something to do. If I’m doing something really AFK [away-from-keyboard] like on wow or something I want something to watch, or on RuneScape. I’m not gonna sit there staring at woodcutting, am I? like ‘ooh he hit cut that one a bit differently’. I need something else to do.

I: What is your thought of the games played today?

P5: I hate cod but the other two were cool

I: Could you remember what the music sounded like?

P5: Ori was pretty fantasy, among the sleep I was too busy concentrating on not getting caught by the ghost. Oh yeah there was some interesting heartbeat thing, and when you were about to walk into a new room there was some weird like tension building and things [horror specific].

I: Did the experiment environment affect the game?

P5: No

I: Would you know how long the test was if it wasn’t mentioned beforehand?

P5: No. May as well be honest, you get lost in the game.

I: Did it feel longer with sound on?

P5: If anything, it felt longer with sound off.

I: How did the game environment distract you from the sound?

P5: A lot, especially in Among the Sleep. I knew it was a horror, had you not mentioned it before, I probably would’ve noticed it [sound] even less. In ori it was pretty clearer, music is a big part of it. If you have to concentrate on something that you don’t know what’s going to happen, I didn’t even concentrate on the music.

I: Tell me how sound affected your overall experience?

P5: Immersion. It immerses you into the game. Even when I play league on quiet, I don’t feel like I am play it. It just feels like I am looking at stuff and all I can hear is [keyboard taps] and [mouse clicks] and it’s just boring.

I: Would you say it enhances it?

P5: Yeah definitely.

I: How important would you say sound is?

P5: Very. Probably the most. Besides gameplay, it’s probably the most important isn’t it. If it’s a storyline, you need sound for that. Cut scene, you need sound. Quite important isn’t it. Imagine releasing a game with no sound, it would be terrible. Who would buy that?

I: Were you frustrated at any point throughout the game?

P5: No, it was league, then yes, all the time. No.

I: When you were playing with sound, were you aware, for example in cod, your accuracy improved?

P5: Yeah, because I could actually hear where gunshots were coming from.

I: Is there anything else that I haven’t asked about that you think I should know? In terms of relation of and between sound and music in games?

P5: No…? Maybe quality of sound. If it’s poor sound effects then it won’t bring you into the environment. And if the music is just terrible, it would just get on your nerves and you wouldn’t want to play it

References:

Bethesda Game Studios. (2011). The Elder Scrolls V: Skyrim [Video game]. Maryland: Bethesda Softworks

Brown.uk. (n.d.). Qualitative methodologies: ethnography, phenomenology, grounded theory and more. Retrieved from http://www.brown.uk.com/teaching/qualitativepostgrad/handout.pdf

Cambridge Dictionary. (n.d.). IMMERSE | meaning in the Cambridge English Dictionary. Retrieved from: https://dictionary.cambridge.org/dictionary/english/immerse

Cassidy, G. and MacDonald, R. (2009). The effects of music choice on task performance: A study of the impact of self-selected and experimenter-selected music on driving game performance and experience. Musicae Scientiae.

CD Projekt. (2015). The Witcher 3: Wild Hunt [Video game]. Warsaw: CD Projekt

Computing History. (n.d.). Magnavox Odyssey - Computing History. Retrieved from: http://www.computinghistory.org.uk/det/16909/Magnavox-Odyssey/

Crossman, A. (2018). What You Need to Understand About Purposive Sampling. Retrieved from: https://www.thoughtco.com/purposive-sampling-3026727

David, L. (2015) Constructivism - Learning Theories. Retrieved from: https://www.learning-theories.com/constructivism.html.

Denzin, N and Lincoln, Y. (2005). The Discipline and Practice of Qualitative Research. Retrieved from: https://uk.sagepub.com/sites/default/files/upm-binaries/40425_Chapter1.pdf

Digital Sound and Music in Computer Games. (n.d.). History - Digital Sound and Music in Computer Games. Retrieved from: http://3daudio.info/gamesound/history.html

Gormanley, S. (2013). Audio immersion in games — a case study using an online game with back. Retrieved from: https://link.springer.com/article/10.1007%2FBF03392344

GSC Game World. (2007). S.T.A.L.K.E.R.: Shadow of Chernobyl [Video game]. California: THQ

Huiberts, S. (2010). The Role of Audio for Immersion in Computer Games

Infinity Ward. (2011). Call of Duty: Modern Warfare 3 [Video Game]. California: Activision

Krillbite Studio. (2014). Among the Sleep [Video Game]. Norway: Krillbite Studio.

Moon Studios. (2015). Ori and the Blind Forest [Video Game]. Washington: Xbox Game Studios.

Moore, G. (2013). The Effect of Sound on the User Experience of Playing a Video Game

NexusMods. (2018). FCR3 - Immersion and Gameplay Tweaks. Retrieved from: https://www.nexusmods.com/witcher3/mods/2788

Pam, N. (2013). Time Distortion in Psychology Dictionary. Retrieved from: https://psychologydictionary.org/time-distortion/

Tafalla, R. (2007). Gender Differences in Cardiovascular Reactivity and Game Performance Related to Sensory Modality in Violent Video Game Play. Journal of Applied Social Psychology.

Tarason, D. (2018). The 3 essential STALKER mods keeping the series alive for fans. Retrieved from: https://www.pcgamesn.com/stalker-series/stalker-mods

USCLibraries. (n.d.). Research Guides: Organizing Your Social Sciences Research Paper: Qualitative Methods. Retrieved from: http://libguides.usc.edu/writingguide/qualitative

Yamada, M. Fujisawa, N., Komori, S. (2001). The effect of music on the performance and impression in a racing game.

Zhang, J., & Fu, X. (2015). The Influence of Background Music of Video Games on Immersion

All data gathered is confidential and no names relating to the person have been used or stored onto any system. All information was collected under GDPR 2018 rights to keep the identities of the individuals anonymous.