Encoders express testing (part 1):
MS Windows Media Audio (.WMA), MPEG-1 Layer III (.MP3, Lame Encoder) and Ogg Vorbis (.OGG)

Author: (c)Alex Y. Radzishevsky
, http://radzishevsky.info
20.Sep.2000

The modern line of development of various algorithms for encoding of audio data tangles both amateurs and professionals. Codecs are appearing in the great amount and no one knows what to use. MP3 (MPEG-1 Layer III) is used everywhere but it happens machanically because people don't think that there is anything else besides MP3. But it is clear that MP3 is already not so young and therefore it may be not so qualitative comparing to other existing algorithms. The given testing will not be detailed, it only will show practical results of using of some modern codecs, namely: MS Windows Media Audio (.WMA) WMA, MPEG-1 Layer III (.MP3, used Lame Encoder) and Ogg Vorbis (.OGG). I have stopped on a choice of these codecs for one very important reason: all they are free-of-charge for the end user and, means, have very good prospects of development and distribution. Such very important branch of codecs, as MPEG-2 AAC (Advanced Audio Coding) owing to separation and incompatibility of versions of this algorithm among themselves does not participate in testing. Today there are following versions MPEG-2 AAC: Homeboy AAC, AT*T a2b AAC, Liquifier AAC, Astrid/Quartex AAC and Mayah AAC. The unique leader among all listed modifications is Liquifier AAC (.LQT), however it's not free and besides, its firm-manufacturer recently has made concessions before Microsoft, having handed over thus position WMA. For this reason Liquifier AAC in testing did not participate.

Tested encoding algorithms: MS WMA, MPEG-1 Layer III, Ogg Vorbis.

Method of testing: coding of an original .WAV-fragment by each of codecs, back decoding to .WAV and comparison resulting amplitude-frequency response of original and received .WAV-files. Because Ogg Vorbis codes only on VBR and does not allow to specify bitrate definitely, the selection of coding parameters for all codecs was carried out so that the compressed files were approximately of one size.

Used encoders: OggEnc v0.4 (Ogg Vorbis), Windows Media Audio V7 (in the complete set of Windows Media Encoder), Lame Encoder v3.86.

Fragments taken for testing: the first fragment (tst1.wav, 7056 Kb, PCM WAV 44.1 KHz, 16 bit, stereo) has duration 40 seconds, spectral structure with mainly average frequencies; the second fragment (tst2.wav, 7079 Kb, PCM WAV 44.1 KHz, 16 bit, stereo) 40 seconds sated spectrum on all frequencies, third fragment (tst3.wav, 10058 Kb, PCM WAV 44.1 KHz, 16 bit, stereo), duration 1 minute very much sated spectrum, is especial on high frequencies (Jean Michel Jarre, Oxygene VII).

Remarks. CBR means "constant bitrate", VBR means "variable bitrate". All given diagrams are for the left channel (may be in the different scales).

Test 1 

Resulting amplitude-frequency response (all diagrams are for the left channel)

Test 2 

Resulting amplitude-frequency response:

Test 3 

Resulting amplitude-frequency response:

Resulting amplitude-frequency response (in the bigger scale)

Test 4 

Resulting amplitude-frequency response

Test 5 

Resulting amplitude-frequency response

Resulting amplitude-frequency response (in the bigger scale)

Let's stop on it and make a little conclusion. But before, one very important remark. It is necessary to say (important!), that used method of testing (comparing of diagrams of amplitude-frequency response) can't make us absolutely sure in accuracy of the results and conclusions made on these results. The reality is that two signals with even absolutely coincident diagrams can sound harshly different (in our case this phenomenon can be explained by using in different codecs of different psychoacoustic models). But nevertheless this method was chosen because all other testing methods are also not precise and moreover not so demonstrative.

First of all, WMA shows the best results even on lowest tested bitrates (guys in Microsoft had worked very good), though cuts off the signal on 20 Khz. And in general, WMA, frankly speaking, has surprised me pleasantly. Ogg Vorbis is not less strong competitor, and though he also lags behind from WMA very little, but has a more flexible coding mechanism (it supports all without exception bitrates in the range from 8 up to 512 Kbps, also declared support of coding more than 2 channels), and also not cuts of the signal on 20 Khz even on lowest bitrates. MP3 in execution of Lame Encoder lives similar last days on my computer - under all tests, carried out by me, MP3 is worse than other tested codecs. Other conclusions I shall give to make you, the diagrams more than eloquent. But never forget: no diagram can convince you of quality as your own ears :-)!