#5 in a series of articles about the technology behind Bang & Olufsen loudspeakers

 

The the BeoSystem 4, the BeoVision 11, and the BeoPlay V1 all use Bang & Olufsen’s “TrueImage” algorithm – but what is TrueImage and why did we choose to use it instead of a commercial standard from someone else?

The Source: How many channels are coming in?

The first question we have to address is that of how many channels your source has. There are many standards these days for audio – particularly if we’re talking about audio that accompanies video.

1 channel – mono: If you’re watching the evening news, you’re probably getting a single audio channel which is basically a feed from the new anchor’s microphone. This might also be the case if you’re listening to a podcast – maybe…

2 channels – stereo: Technically, “stereo” doesn’t automatically imply only two channels – although most people mean “two channels” when they say “stereo”. (I am equally guilty of this behaviour.) There is a plethora of materials recorded in 2-channel stereo, going as far back as the 1940’s! Lots of television channels are broadcast in 2 channels (I remember a time when, if you wanted to hear a TV broadcast in stereo, you had to tune your FM stereo for the audio being simultaneously broadcast on the radio…). Almost all music is distributed in 2-channel stereo. Actually – almost everything except for movies and video games has two channels of audio. In case you aren’t already aware,

• there are two channels because you have two ears – one channel is for the left, and the other is for the right
• stereo was originally patented by Alan Blumlein in the 1930’s
• Blumlein originally patented stereo for sound “especially when associated with picture effects as in talking motion pictures” – it wasn’t originally for sitting and just listening to music – no snobbery!
• Just because there are two channels of audio doesn’t necessarily mean that those two channels are different. As I type this, my two kids are watching a television show that is in mono, but broadcast as a 2-channel signal. All that means is that the two audio channels are identical. Most people call this “dual mono”.

4 channels: For a while, there was a system available from Dolby called “Dolby Stereo” in cinemas, but “Dolby Surround” in home systems. These two were essentially the same thing. This worked by encoding (sort of…) 4 channels of audio (Left Front, Centre Front, Right Front, and Surround) into a 2-channel stream (like on a video tape or an optical film). When you played back the 2-channel signal, it could be “decoded” into the original four channels. This system wasn’t perfect because of the encoding system. So, if a signal was in the centre channel, it was also showing up (albeit a little quieter) in the Left and Right Front channels. The same was true of the Surround signal – it also bled into the Left and Right Front channels. However, it was better than nothing, so it saw a wide distribution. In fact, it’s still possible to buy DVD’s like this one that are encoded in the original Dolby Surround format.

5.1 channels – surround sound (aka multichannel audio): Most movies released in the past few years on DVD have been mixed and distributed in 5.1 Surround Sound – but what, exactly does “5.1” mean? Well, the “5” part means that there are five main audio channels. these are the Left Front, Centre Front, Right Front, Left Surround and Right Surround channels. Whether or not you use 5 loudspeakers to reproduce these channels is up to you. For example, if you are using a BeoVision 11’s internal loudspeakers as the centre channel, then you are sending the Centre Front signal to two internal loudspeakers – just like the “dual mono” case I talked about above. Some people have really big home theatres, so they need more than one loudspeaker for each of the two surround channels – just like in a real movie theatre.

That leaves us with the “.1” which is the short-hand way of saying that there is an extra audio channel in the signal called an “LFE” channel (it stands for “Low Frequency Effects” or “Low Frequency Enhancement” depending on who you ask…) The reason we say it is only 0.1 of a channel is because it usually contains about one tenth of the frequency range of the other channels.

It’s important to note here that 5.1 channel  systems are not like the Dolby Stereo system I talked about above. 5.1-channel systems are always “discrete” (and not necessarily “discreet”) meaning that if audio is supposed to be in one channel, it doesn’t bleed to adjacent channels. In other words, something in the Left Front channel is only in the Left Front channel and nowhere else.

It’s also important to note that the “LFE channel” and the “subwoofer channel” are not necessarily the same thing – so you should be careful not to say “LFE” if you mean “subwoofer” and vice versa.

It is also important to note that, if you are listening to a music-only 5.1 recording, you should turn off the LFE channel. Multichannel music recordings are almost always recorded, mixed, and mastered in 5.0 channels – no LFE. However, the business people that run the record labels don’t want to hear complaints from customers that are worried that their subwoofer isn’t working with their new disc, so they tell the mastering engineer to bleed a little bass to the LFE channel just to stop people from complaining. Since the only people who want something on the LFE channel are people with degrees in law, marketing, and business (and have no experience whatsoever with music or recording), there is absolutely no good reason to listen to it. This is why there is a menu item on the BeoVision 11 and BeoPlay V1 that says “LFE Input – On/Off” – it allows you to shut down the LFE channel input if you’re listening to a multichannel music recording.

It is also interesting, but less important to note here that not all audio formats are stuck with the low-frequency limit  on the range of the LFE channel – in other words, in some systems (like SACD, DVD-Audio, Dolby TrueHD, and DTS HD-Master Audio, for example) the LFE channel can be used for more than just low frequencies, which brings us to…

6.0 channels – surround sound (aka multichannel audio): There are a small number of recordings, almost all of them music recordings on SACD (like this one), that are ostensibly in 5.1, but are actually in 6.0. This is because the recording engineer knew that:

• you shouldn’t use the LFE channel for music recordings
• the LFE channel in the distribution format (in this example, an SACD) has a full frequency range

So, in these cases, the LFE channel was used for capturing a sixth full-range audio channel instead of just a low-frequency channel. Typically, that sixth channel was used for height information, so the “correct” way to hear the recording is to send that audio channel to a loudspeaker above the listening position, preferably in or near the ceiling. This is why there is a menu item on the BeoVision 11 and BeoPlay V1 that says “LFE Input To Ceiling – On/Off”. It takes the LFE audio input channel and re-labels it a Ceiling channel so that it is re-routed to the appropriate loudspeaker. Of course, if you don’t route this audio channel properly, you probably just get a little something coming out of your subwoofer, and nobody is going to complain about that, as anyone with a degree in law, marketing, or business will tell you.

6.1 channels – surround sound (aka multichannel audio): There are a small number of movies that were released in systems that sort of combined the 5.1-channel discrete system and the “encoding” (more accurately called “matrixing” but we won’t get into that) system like the one used in the Dolby Stereo system. In these cases, the Left Surround and the Right Surround are used to contain a Centre Surround channel. So, you get a 6-channel system which fixes the problem that some people notice where the two surround channels don’t hang together very well… There is at least one fully discrete 6.1 system which is basically the same, only better.

7.1 channels – surround sound (aka multichannel audio): In the past couple of years, some of the really big-budget movies have been released in 7.1-channel surround sound – this is pretty easy to do because both Blu-ray and HDMI can handle it. This should make everything better, but, sadly, it’s where things start falling apart.

In theory, when we say “7.1 surround”, I know what you mean, and vice versa – and in most (possibly all) cases, that might be true. A “standard” 7.1 loudspeaker configuration is basically the same as the 5-channel configuration with a Left Back and Right Back loudspeaker added. The result is shown in the figure below.

The problem is that this is not the only possible version of 7.1. In fact, there are at least seven different variants of 7.1. Including the one above, these are as follows:

1. Standard: Centre Front,L and R Front, L and R Surround, L and R Back, LFE
2. Front Wide: Centre Front, L and R Front, L and R Front Wide, L and R Surround, LFE
3. Front Height: Centre Front, L and R Front, L and R Surround, L and R Front Height, LFE
4. Centre Height: Centre Front, L and R Front, L and R Surround, Centre Back, Centre Front Height, LFE
5. Centre Overhead: Centre Front, L and R Front, L and R Surround, Centre Back, Ceiling, LFE
6. Side Height: Centre Front, L and R Front, L and R Surround, L and R Surround Height, LFE
7. Rear Surround: Identical to the Standard configuration but with a slightly different loudspeaker placement (with the surround and back loudspeakers placed closer together).

So, we have a problem. Just because a Blu-ray is encoded in 7.1 doesn’t necessarily mean that the channel allocations or the loudspeaker positions are the same from disc to disc.

More than 7.1 audio channels: To infinity and beyond! Believe it or not, there are some formats and systems that have more than 7.1 audio channels

• 9.1 channels:  This format can be found, for example, in Dolby ProLogic IIz-encoded materials
• 10.2 channels: This format is built on the 5.1 standard, and includes 5 more main channels (including two dipole “diffuse radiators”) and an extra LFE channel. It was originally proposed by Tomlinson Holman
• 11.1 channels: This format is seen, for example, in the DTS Neo:X system
• 22.2 channels: This format is an experimental one built, in part, by Kimio Hamasaki at NHK
• Dolby Atmos – which doesn’t really have a set number of channels (although it can handle playback systems with up to 64 loudspeakers) and is only available in theatres
• A monster called Wavefield Synthesis, but that typically needs hundreds of loudspeakers to work reasonably well.
• and others
• and more to come (to quote Madonna’s character from Dick Tracy “More is better than nuthin’ – but nuthin’s better than more!”)

So, let’s assume that anything between 1.0 and 7.1 channels of audio can arrive at the input of our television, since HDMI can currently support up to 8 channels of audio. Let’s now think about the output.

Loudspeakers: How many channels are going out?

Let’s say that you have just had your brand-new BeoVision 12 (with an equally new BeoSystem 4) installed. The BeoVision 12 is equipped with a single loudspeaker channel (although it has 4 loudspeaker drivers built-in – four 2″ midranges and one 3/4″ tweeter, each with its own dedicated ICEpower amplifier). We’ll also assume that you don’t have any external loudspeakers attached – you only have the internal single loudspeaker. This means that, no matter how many audio channels come into the BeoSystem 4 – from a 1.0 mono news program up to a 7.1 Blu-ray – you’ll expect to be able to hear everything you’re supposed to hear. This means that the BeoSystem 4 will usually have to mix more than one channel at its input down to one channel at its output (the BeoVision 12 loudspeaker) – a process called downmixing.

But what about if tomorrow, the truck from the B&O shop shows up and drops off 6 new loudspeakers and a subwoofer. You set up the loudspeakers as your Left and Right Front, Left and Right Surround, Left and Right Back and Subwoofer – all in the correct places like in the drawing above – and you sit down and turn on a CD. Now you have only two audio channels coming into the system, but you probably want to hear something coming out of all of those loudspeakers. This means that we have to do something to get signals to go to more than two loudspeakers – we have to upmix the signal from 2.0 to 7.1. In fact, since most of the things you’ll be listening to and watching will be 1.0 to 5.1 (not many movies are in 7.1 yet) – you’ll probably be upmixing most of the time.

Now let’s take a typical situation – you have a 5.1 loudspeaker system (5 main loudspeakers and a subwoofer) and you have input signals that can range from 1.0 up to 7.1. This means that, sometimes, when you have a 1.0 or a 2.0 input from a TV channel or a CD,  you want to upmix to 5.1. Sometimes, when you have a 5.1 input, from a DVD, for example, you want the channels to go straight through unaffected. Sometimes, you’ll watch a Blu-ray in 7.1 and you want to downmix to your 5.1 output. And, of course, you don’t want to have to do anything for this to happen – it should just happen automatically.

And, then, there are the people who go big – they have a loudspeaker on every audio output of the BeoSystem 4 – so they have the BeoVision 12 plus 11 other loudspeakers. So, they’ll probably be upmixing everything all the time.

So, the moral of the story is that, as a manufacturer, we don’t know how many loudspeakers you have. The BeoSystem 4 and BeoVision 11 are not like a typical AVR that you buy at your local big-box electronics store. They are far more capable in many aspects…

 

Philosophy: What should an upmixer do?

Part 1: Behaviour

If you’re like me, you’re a realistic purist. This means that I firmly believe that, if two channels come into the input of the system, you should only have two loudspeakers playing those two channels – unless it sounds better doing it differently.

In other words, if I’m at home, alone, sitting in the “sweet spot”, perfectly located between my two front Left and Right loudspeakers, I want 2.0 in and 2.0 out. No mucking about.

However, if I’m sitting on the sofa with the family, not in the sweet spot, and I have use the “purist” configuration of 2.0 in and 2.0 out, things just don’t work. All of my phantom images will collapse into the closest loudspeaker and that’s that. So, in this case, if I can have an upmixer that can correct the situation and put the centre images back in the centre, whilst maintaining some spaciousness and width, I’ll turn it on.

However, not everyone is like me – not everyone is going to be turning an upmixer on and off according to which chair they’ve put their bum on. Most likely, most people will turn on an upmixer and leave it on (because they want to hear sound coming out of all the loudspeakers they’ve paid for).

So, in a perfect world, in my not-so-humble opinion: if you’re sitting in the sweet spot, and you turn the upmixer on and off, you should not hear any different whatsoever. However, if you’re not in the sweet spot, turning on the upmixer should improve things. (At least, as I said, that’s my philosophy. There are those who say that, even when you’re sitting in the sweet spot, an upmixer should enhance things like spaciousness and envelopment – but I disagree, since I’m a part-time purist…)

Part 2: Technique

So, let’s say that you’re faced with having to make an upmixer. You can basically do one of two things:

1. Take the input, analyse the audio signal, decompose it into more audio channels, and send those to more loudspeakers.
2. Take the input signal and create new audio channels – for example, feed it into a reverberation unit to make echoes and reverb that are not in the original recording.

The purist in me very firmly believes that an upmixer should not add anything to the original recording. It should derive components of the audio signal that are in the original recording and distribute them to the various loudspeakers. But it should NEVER add something that wasn’t in the original. So, no reverberation – no concert hall sound slapped on top of the original recording. (At least, that’s my philosophy. Some commercially available upmixers are built on the idea that they should do exactly this – for example, simulating a system where your stereo recording is played in a “good” listening room, and then a multichannel version of that simulation is played through your real system. In my opinion, this is misguided at best, since I don’t want the drums in the opening of Lyle Lovett’s song “Penguins” or the anchor on the evening news to sound like they’re being played in a bathroom.)

 

Bringing it all together

When we were designing the audio signal flow for the BeoSystem 4, BeoVision 11 and BeoPlay V1, we knew that we needed an upmixer. Believe it or not, the first question was not “can we licence one?” (in other words “what can we buy?”) – it wasn’t even “what does everyone else use?” The first question was “what do we need?” which turned out to be a very interesting question indeed!

We decided that we needed an upmixer with the following features:

• it has to understand all of the audio formats from 1.0 to all of the 7.1 variants (because we don’t know what you’re going to play)
• it has to be able to upmix from any of those formats to any other of those formats (because you should be free to set up your loudspeakers however you want)
• it has to be able to downmix from any of those formats to any other of those formats (because you should be free to set up your loudspeakers however you want)
• it has to be able to neither downmix nor upmix if the input format matches the output format
• it has to be able to switch seamlessly between downmixing, upmixing and through-putting automatically with changes in the signal (because you might switch from watching the news to watching a movie, either because you switched to watching a Blu-ray disc, or because the movie was the next thing coming on that TV channel…
• it has to allow some adjustment of some parameters by the end user (everyone likes a little salt and pepper now and again…)
• if you’re sitting in the sweet spot
  • it should sound like it’s not doing anything, either spatially or timbrally (a fancy word for “tonal balance-ly”)
• if you’re not in the sweet spot
  • it should improve the centre image location
  • it should improve spaciousness
  • it shouldn’t push the left and right images wider or narrower
  • it shouldn’t mess up the timbre too much

Let’s just look at the second of those requirements, since that was a killer. What this means is that you, the customer, should be able to set up your loudspeakers using any current standard of loudspeaker configuration from 1.0 or 7.1 (all seven variants) and the upmixer should take care of everything for you. How can this be done? Well, our solution was to set about to make a master format that encompassed all of those standard formats. In other words, if I wanted to build a loudspeaker configuration in a (large) room that could be used to play back all of those formats (one at a time) – how many loudspeakers would I need and where would they have to be? The answer was 16 (although, I have to admit, a small part of me was hoping that the answer would be 42 – but I always want the answer to be 42…)

As you can see in the table above, we found that we need 16 channel allocations in order to be able to accommodate all of the standard surround sound configurations. Those of you with a BeoSystem 4, BeoVision 11, or BeoPlay V1 should recognise the master list of 16 allocations as the Speaker Role options in your menus.

As soon as we had that, we knew that we needed to build our own algorithm. So, I was given the job…

From there, it was more of an organic process than anything else. I locked myself in the listening room for a couple of months with a 16-channel system made of BeoLab 3’s (yes… I had 16 BeoLab 3’s running in a room that is only 6 m x 5 m x 2.5 m – there wasn’t much room for people to visit and chat) connected to a multichannel sound card connected to my Macintosh running Max/MSP and MATLAB. I started by inputting a 2.0 stereo signal and built an algorithm that derived signals from the stereo signal to send to the various loudspeakers – always making sure that, sitting in the sweet spot, I couldn’t hear the difference (too much…). However, sitting away from the sweet spot, adding the extra channels had to improve things. In many respects, this task was a lot like being a real tonmeister – but instead of taking a bunch of input signals from microphones in front of the orchestra or band, and mixing that down to two loudspeakers, I was taking two inputs (from a CD) and breaking it down into its constituent components for the various loudspeakers. Almost like reverse-engineering a recording.

 

 

Of course there were challenges. The system has to work for pop recordings (where every instrument is recorded in mono with a single microphone and the fake reverb is generated digitally) and orchestra recordings (where, at the opposite end, you have two microphones in a concert hall and nothing else) and everything in between. This took some tweaking – but that’s why it took a while to build. The other big challenge was ensuring that it works for more than just a 2.0 input – that it also behaves for other input formats.

In the end, we had an upmixer that had “grown” in Max/MSP. The problem then was how to implement it, since the televisions don’t run Macintosh software inside. The solution to this was reverse engineering – I took my Max Patcher and converted it to a flowchart that was given to our “real” programmers to write the DSP code that would live inside the final products. So, they wrote the code and we ran it on the early prototype of the television (which actually just looked like a raw screen and a printed circuit board on a piece of MDF, as you can see in the photo below…).

 

 

 

One of the “tricks” that I did when making the signal flow was to leave myself a bunch of parameters that I could tune after the whole algorithm was implemented. So, once the Max/MSP + Matlab version on my Macintosh was converted to software inside the final hardware, and that was verified to make sure the conversion was correct, I could start tweaking. This meant leaving the BeoLab 3’s in the listening room, but taking out the Mac and the sound cards and replacing them with an early television prototype (see the photo above… it was VERY early…). From there I could make minor adjustments in the upmixer’s behaviour, including the custom tuning of each step on each of the different sliders that are in the final menus (like “Frequency Tilt” and “Stage Width” – just to name two).

Finally, we had to make sure that the upmixing and downmixing worked with different combinations of inputs (from 1.0 to 7.1 signals) and output configurations (from one loudspeaker to 12, sometimes with subwoofers, sometimes not) and different loudspeaker types (mixing and matching different sets of BeoLab loudspeakers to make sure things held together even if you don’t have 7 BeoLab 5’s in your listening room).

 

The Wrap-up

Of course, this was the short version of the full story. The longer version would talk about how our first real version of the TrueImage upmixer  in a commercial product was made for the first Advanced Sound System that we put in the Audi A8. It would also talk about how TrueImage isn’t a fixed algorithm – it’s constantly evolving to meet the needs of the product it’s put in. So it’s more like a toolbox of possibilities that we can work with in the development process of any of our products that have to negotiate a situation where the number of audio input channels isn’t equal to the number of output channels. This also means that, as time goes on, the TrueImage algorithm will certainly change to meet the needs of the signals that we can receive and the outputs that our products can deliver. However, that would be too much information for this week.