There is no "uncanny valley" for sound...

[ThunderPeel2001]

...so why isn't it given more attention in games?

How many times have you walked two feet away from a character only to have their voice drop to an inaudible volume? How many times have you heard things crystal clear through brick walls that weren't in any way muffled? Why doesn't sound echo in certain chambers. Why don't the sound of my foot steps change for each type of ground I can walk on? Concrete, grass, sand, mud... If it's raining and I stand by a window or a roof, when can't I hear the rain pounding on the window or roof?

According to weirdmeister genius David Lynch, "sound is half of the experience". Films create entire soundtracks from nothing, the technology exists. So why don't developers bridge the "uncanny valley" for sound, since it can be done (relatively) easily?

Edited October 13, 2008 by ThunderPeel2001

[toblix]

The Thief game series had a pretty heavy focus on sound. A lot of it uses (and requires) hardware acceleration like EAX, but my impression is that this stuff can now be done on the CPU as well, although I guess occlusion sound may be too demanding compared to whatever else you can get (most likely graphically) from the same cpu cycles. For some reason sound acceleration hasn't had the same development as graphics -- you still can't "expect" any form of hardware sound capability the way you can expect Direct3D (or I guess just DirectX now) or OpenGL support in a moden computer.

The Half-Lives have pretty awesome material-specific sounds.

Also, I guess maybe because it's known in the industry that 80% of allc computers have tinny shit speakers.

[Nachimir]

I don't think it's really an "uncanny valley" since we immediately recognise it as crap implementation rather than it being something creepy, but you're right about sound being tragically neglected in games, and at game development events too. GDC has an audio track, but outside of that I hardly ever see anything about it in print or schedules.

Black Cat once hacked the Unreal engine for Thievery UT (Multiplayer thief, versus guards) to use pathnodes for sound propagation, and it kind of worked. Nonetheless, it also meant players using headphones with the volume up could easily detect even faint sounds and sometimes very easily find Thieves or avoid guards.

[Nachimir]

Holy coincidence Batman.

People are working on it, and have been constrained by many limitations - good audio can take up a lot of space, and take a lot of producing.

I once expected that we should be able to make things that sound realistic just because we can sample, but there's a lot more to it than that, as I found out even by attempting to put ambient sound in a UT level. The expectation was kind of like expecting to be able to make photoreal games just because we can photograph things.

[elmuerte]

Sounds isn't limited much by technology. A lot of game devs simply do not care for that level of detail, and a shit load of players don't care either.

For example. One of the most noticeable things is the sound of footsteps, in most games the footsteps always have the same sound no matter on what surface person is walking on. Simply changing the sound of footsteps makes a bit difference. For example in Little Big Adventure you had different sounds, they even went so far that one foot made the sound of carpet, and the other of brick.

Another big thing is the usage of music. In movies everything is static, so you can perfectly time the music and produce a proper flow. This adds a lot to the mood in the game. For interactive entertainment you need to use dynamic music. But this isn't used that often, even though the technology has been around for years. Some games that come to mind with excellent dynamic music would be Unreal and Star Reach/Space Federation.

As for environment effecting the sound. I've seen a couple of games where they they did do it. But it wasn't always that obvious (and had various bugs). Don't know exactly which games had it, but I'm sure it was at least of of the Rainbow Six games. Of course there were a couple of other games around the SBLive! era when real time sound modifications came up. But these days I don't really notice much difference.

So... it does loke like the sound department doesn't get the level of detail other department get.

[Chris]

http://chrisremo.com/bloggin/2008/06/18/attentionvoicedirectorsmusttourgent-advice/

http://chrisremo.com/bloggin/2008/06/19/video-game-voice-acting-advice-ii-the/

[brkl]

Yes, for the love of all things holy, I hope this is something that will get better ASAP.

I've been playing around with the idea of doing some analysis of, say, Mass Effect, and trying to see where it fails to produce natural sounding dialogue and why. I think a lot of it stems from the fact that we are still basically having someone read aloud passages that were never written with that in mind. RPG dialogue is still pretty much what it was when Baldur's Gate or Planescape: Torment came out, and even though Torment has perhaps the greatest dialogue to be found in games, reading that aloud will not sound like conversation.

[n0wak]

See also

http://www.brainygamer.com/the_brainy_gamer/2008/09/voice-acting.html

[Zaarin]

Another big thing is the usage of music. In movies everything is static, so you can perfectly time the music and produce a proper flow. This adds a lot to the mood in the game. For interactive entertainment you need to use dynamic music. But this isn't used that often, even though the technology has been around for years. Some games that come to mind with excellent dynamic music would be Unreal and Star Reach/Space Federation.

LEC patented the shit out of this technology with imuse though. I bet they could do better rumble in joypads too if there weren't patents waiting around every corner to steal all your money.

[Jake]

LEC patented the shit out of this technology with imuse though. I bet they could do better rumble in joypads too if there weren't patents waiting around every corner to steal all your money.

iMuse is cool, and elements of it are patented, but the audio tools in XNA/XACT, and the popular middlware FMod (featured hugely in the opening splash screen to LittleBigPlanet, but used by hundreds of developers) allow for complex realtime mixing, filtering, randomization, pitch shifting (for instance to make everfootstep ever play at the same rate and tone so you never hear the exact same footstep twice), iMuse style interactive loop markers and transitions to segue music from space to space, 3D positioning including occlusion detection to make sounds' reverb and mix properties change if they suddenly find themselves behind a wall, and a whole bunch more. It takes a lot more implementation time and uses more system memory than most developers are willing to spare for "just the sound," but the technology is all there right now in various places. I wish more people paid attention to it, because I think good audio helps games more than it gets credit for. Good sound doesn't sell a game like a good screenshot (or a good trailer with great sound done by an outside mixing house) does, though, so I imagine it's a very hard fight to get time and money invested in improving audio quality.

I'm sure this difficulty is compounded by fewer and fewer studios having their own in-house audio team to fight for such improvements. At LucasArts in the '90s, they had all of their sound designers and composers in-house, and to my understanding, some of them were also programmers to varying degrees of proficiency. That is a rare combination -- having the staff in-house to be able to promote and fight for a better audio solution, and to have that department also within itself contain the resources necessary to program in the new audio functionality. Most game studios seem to have neither of those things -- most sound and music work is outsourced or done through a partner company, and most music programming and implementation is either middleware, or coded in at its barest level early in the project and then declared good enough, while that programmer goes on to something else.

[Thrik]

I think the best example I've seen yet is Battlefield 2142's sound (eg: pane of glass between two players muffles the sound, but sound coming from other directions gets through as you'd expect), and even for that you need a fairly expensive X-Fi sound card.

It does make a huge difference though, and while not even close to what's possible it does show what can be done today. The game is literally packed with quality sound engineering, largely because it was the de facto flagship game for the X-Fi cards.

[Nachimir]

That is a rare combination -- having the staff in-house to be able to promote and fight for a better audio solution, and to have that department also within itself contain the resources necessary to program in the new audio functionality. Most game studios seem to have neither of those things -- most sound and music work is outsourced or done through a partner company, and most music programming and implementation is either middleware, or coded in at its barest level early in the project and then declared good enough, while that programmer goes on to something else.

Yeah, in my experience only the biggest have their own studios on site and can keep it running full time.

Sounds isn't limited much by technology. A lot of game devs simply do not care for that level of detail, and a shit load of players don't care either.

Your second statement is true, but I don't think the first one is entirely accurate. Sound still largely relies on sampling, which isn't equivalent to the kind of sophistication we have in, say, lighting modeling and generation. I've met plenty of people working on better sound technology, but dynamic generation of sound, for instance, is still a very hard problem and most human ears can easily tell the difference between synthesis and a sample of something.

When you're including multiple samples of stuff, and having to code systems to handle them, you're putting the workload and space requirements up a fair bit - more than many studios are willing, I'm sure. Better compression technology helps on the one hand with this.

It may well be the lack of priority that has led to this shortfall in sound R&D compared to visuals or physics; perhaps we could do much better even with present tech.

[DanJW]

As for environment effecting the sound. I've seen a couple of games where they they did do it. But it wasn't always that obvious (and had various bugs). Don't know exactly which games had it, but I'm sure it was at least of of the Rainbow Six games. Of course there were a couple of other games around the SBLive! era when real time sound modifications came up. But these days I don't really notice much difference.

Actually I've notied that most games I play have an option for EAX or EAX2 in the audio tab. However 50% of the time it seems to cause problems such as sound cutting out (probably because my sound card is an ancient SBLive! Value!)

[ThunderPeel2001]

Cool, I need to find time to read all those articles. I remember that Hidden & Dangerous had phenomenal sound, especially for the time. (Stand next to a shack when it's raining and hear the rain pound the roof!) It's a shame that there were bugs that let you men fall through the scenery...

I think the next step will be some sort of sound physics middleware. (Anyone up for a challenge?*)

* Note: I cannot code.

[toblix]

I'll start.

public class SoundPhysicsMiddleware extends AbstractMiddleware{

public static void main(String[] args){

}

}

[Noyb]

I'll start.

public class SoundPhysicsMiddleware extends AbstractMiddleware{

public static void main(String[] args){

}

}

Because all the big game companies use Java.

You'd want either a constructor or a set of static methods, not an main function that automatically runs god-knows-what for an object you want others to use

[DanJW]

Because all the big game companies use Java.

(I think he was joking).

[elmuerte]

public class SoundPhysicsMiddleware extends AbstractMiddleware{

   public SoundPhysicsMiddleware(){
       super();
   }

   public static void main(String[] args){
     SoundPhysicsMiddleware spm = new SoundPhysicsMiddleware ();
     try {
       spm.run();
     }
     catch (Throwable e){
        System.err.println("I can't let you do that Dave. "+e.toString());
        e.printStackTrace();
     }
   }

}

next

[toblix]

No, I was seriously starting work on a Java based audio physics middleware, using Idle Forums as a code repository.

public interface Sound{

  public void play();
  public void loop();
  public void stop();
  public void setPhysics(Physics physics);

[n0wak]

code

next

Now do it in Malboge!

[ThunderPeel2001]

We're gonna be rich!!!

[elmuerte]

Now do it in Malboge!

B_:]]=65:{W10T43c1*q_LnIl)F!h}|#dc>av^:yr[7onF31poh@lNdi;as%7cEE~BXA{UxY<:t8NSqQ
4n!1F..-,GFEDC&Nq?!7~Zk9WygTAt2baM.Kn8*kj'EV$eA.>``<ML[JZYWFlk0R.@f>*)ha'I7c#!!l
^A@.-x<dVO7764P3!1jE.,BTF)(&%%M">!}HkFiyxTS3Qbr<L-Km7)YX~%$Tz!~a<^]]9J%u5W31S0RQ
lxdMuaIrH#\!!2^]zzTwXd:98rR5JmHkjjJIHfev(a`NML!=<Y:9iyx/4@2rNponnI7ZYhhV$T/RxwvN
^99wZX4s2D}/AmlNMvh''eH]"!D~XA?hTf;W:s87p5PO!1LXJg+G*((a&`#9K![l|zz7xv4-P1rN_K]m
*ZY'~ffezRbav*]9r[ZuGWUqj/A@l,NMu:'ed6#4`~|]?UZfe::8s6p4JO21Lii-H**E'=&`_^\~}ZYX
y7wv4uQbO/L-,mkkj'EV|B@!bw`;^y\ZYoX"EqT/{mlxj*)gfrqG"[~~}{{y-x;ucUNS5^o3N~Li.-ff
*)cPa%@?Kn}}kX8ywv4uQb0/Ln,mlkjiEVUB@!bw`;z99ZYYX"rqT/h-lxN*):'rqc"a~~^{{[-x;QcU
NS6^o3N~ki.,fA*)c'a%M^Kn6YkXVV6v4utO=<o-J\kH5'Wff#"?a}`*)y8ZYY4V!qTB{m?ej*u:f_^6
En~~|{{ygYXuVtTM6Q3I!ZFKhzBeFcbt<;:?"[<l:zzUxwuts1rMM-,mlG"!~%$0@!b,v_]\rJZ55FrT
}0RzPN*c)'98dc5"~_|A?U>=XW:UNSp4PI[l}KiCffwEbPBA^]>=Z}GFW105vuQr0/;:Klkj('h}${z!
~w+u]9xq%YX43q0o/Q,xNM)KfHG#5ECC|j\hZSeu)8TSp53nNGYihgB+*Eu'B%#9>7<Z|3EVwBetcbrq
_:9lHHiiDg|A/b>aO;^y[[i5t"2qpi.PlkjiLJseGpEaZ}^j\y>YvQ)UN7qQ]nN0}KJIUGFE(C&_:?>~
}5:9z7x/Rt21*q.^m+lZ"'EC$Adb~}`<zy[qpo5sUk1oQg-Nd*hJ`&G]\"`BX|VUy<Rv9ONr5KJIm0Fj
-Cg*@d'=<`:98\}543W10T43s+Op(L&%Ij"!Ef|Bcyx>_u;sr8Yon4lkj0Qgf,dc)a`_%]\"ZY}WVz=S
RvV8Nr5Ko2HGkEi,Bf)?>=a$:^!76Z43Wx0T.-Q+Op(Lm%Ij"Fg}|Bzyx>_u;yrqp6Wml2pi/Pf,Mc)J
`_%F\[!YX|?UTx;QPtNMqQ3Im0FEiIH*@d'=<`#98\}54Xy1Uv.-Q+*N.n&Jk#"Fg}|{A!~`=^ts9wp6
tm3Tjih.Oed*Ka'H^]#aCY}WVzZSRvPOs6Lp3Im0Fj-CBAe(>bB;:^!76Z432Vw/S3s+Op(Lm%Ij"F&f
|Bcy?}_ut:[q7Xnm3Tji/gfe+Lb(`_%cE[!BX|\>TxX:PtTS5Ko2HGkEiCBf@d'=a$:^>~[54Xy10Tu-
Qr*N.-m%Ij(!E}|B"!aw=^t:[qp6tVl2ji/Pf,dc)J`&G]#DZ~XW{UTSwW9Os6LKJnNM/EiCBf)?>b%;
:^!76Z{3Wx0Tu-Qr*No'K%$Hi!~De{z@aw=^ts9Zp6Wm3Tjih.Oed*hgI_%F\"CY}@VzZY;Qu8NMLp3I
m0FEiIBf)?>b<;_"8\<|43Wx0Tu-,+Op(Lm%I#G!~D|{z@aw=^t:xZpo5m3Tjih.lNd*hJ`&G]#DZY}]
?Uy<RQuONr5KoImGFEi,Bf@?c=<`@?!\}5Yz2V6v.Rs+O)(L&Jk#"Fg}C#"bx>vu;sr8Yo5Vlkj0Qg-N
d*Ka'H^]#DZY}@Vz=SRv9Os6LKoIHlFEi,BAe?>b<;_9]=65Yz21Uv.Rs+*N(L,l$#Gh~}Cdz@aw=^ts
9Zpon4rTj0Qg-kMc)a`&G]\"ZYX|?UySRvPOs6Lp3ImGFEi,Bf@?>b%;_"87[;{X21Uv.Rs+O)M'&%Ij
"!E}|B"bx>v<]s9wYo5sUkj0hg-Nd*hJ`_%F\[!BX|VzZ<Rv9ONMqQ3IHlFEi,Bf@?c=<`#98\<|4X81
0Tu-Qr*No'Kl$Hi!~D|{Abx>_u;\r8Yonm3Tji/Pfe+iKa`&d]#DZY}]?UTxXQuOs6LpJIm0FEiCBf)?
>bB;_?!7[|43Wx0T.-Qr*No'&J$#"Fg}|B"bxw=ut:r8po5ml2poQg-kMc)J`&d]#[Z~^]?zTSwQuOsS
R4o2HlLK-Cg*@dD&<`#98\<;{3Wx0/S3s+*N.nK%$H"!Ef|{Ayx>_u;\r8vXnm3kj0hg-Nd*hJ`&G]#D
ZY}]VUy<RvVONr5KJn1GFj-CgA@d>=aA#9]~65Y32Vw/.R,+Op('K%$H"!Ef|Bcy?`v<]s9Z7on4Ukj0
Qgf,jLb(I_%]\"CYX|VUySRv9ONMq4Jn1GFj-Cg*@?c&<;_"8\}54X21U/.R,P*)Mn&%IjGh~}Cdz@x>
_ut:[q7Xn4Ukj0hg-ed*Ka'eG]#DZY}@Vz=SRv9ONr5KJnHGkK-Cg*@?>b<;_"87[|43W10/St,+OpM-
m%Ij"F~D|{Abxw=^t:xZp6Wm3kj0Qgf,Mcb(I_^$E[Z~AW{>TSw:PtT6Lp3IHlLEiCg*@d'=aA#9]~65
Y9y10T4-Qr*No'&J*j"Fg}|Bcyx>vu;sr8pon4rTj0Qgf,dc)a`&G]#[!BXW{UTx;QPt7Mq4JIHl/EiC
Bf)?>b%;:^87[|43Wx0/S-Qr*No'&Jk#G'g}|Bzy?`v<]sr8Yon4Ukj0nPfe+Lb(`_%F\"CY}]?UyY;v
POsMq4JnHGk.Dh+Ae(>=a$:9]76Z{32Vw/.R2r*No'Kl$H"!Ef|{Ayx>|^t:xZp6tVlk1oQ.fed*Ka'e
G]#DZY}@VUySw:PtT6LpJIHl/jDCg*@dD=a$:^!7[|43W10T.-Qr*)M'&Jk#"F~}C#cy?`v<]s9Zpo5m
lk1Rhg-ed*Ka`&G]\"CY}WVz=SRvV8NrRKo2HGk.DCg*@?cC%;_"8\}543Wx0Tu-,P*)M-m%$Hi!E}|B
cy?`=^zs9Zp6Wml2ji/gfe+Lba'_^$bD!YX|VUy<RvV8NrLKo2HGk.DCgA@d>=aA#9]~65Y9y1Uv.Rs+
Op(Lm%$H"!E}|Bcy?`v<]sr8Yon4rqSi/Pf,jLb(feG$E[!_AW{UySRv9Os6LKoIm0Fj-CgG)d>=a$_9
87[|4X8x0T4t,P0p(Lm%$H"!E}|Bcy?`v<]sr8Yon4rqSi/Pf,jLb(feG$E[!_AW{UySRv9Os6LKoIm0
Fj-CgG)d>=a$_987[|4X8x0T4t,P0/oLm%$Hi!E}|{Ay?`v<zy[q7uWm3qSi/gfe+Lba'H^$E[Z~XW{>
Tx;Qu8NMqQ3nHlFEi,BAe?>bB$:^!7[|43W10Tu-,Pq)Mn&%I#"!E%e{z@xw={]sr8po5Vlk1Rhg-kMc

[elmuerte]

cancel that... made I typo in the code.. somwhere, I'm offering a free "waste your time and don't get it back"-card for the first person to find it.