Table of Contents
Advertisement
C H A P T E R
2 :
2.5 Selection of the Reference File
As a rule of thumb, the reference file should be a signal that comes as close as
possible to the kind of signal which shall be applied to the device under test in
real life. E.g., if you design a special headset for female call center agents, you
should use a test stimulus that contains mostly female speech. If the device
should be used by male and female users as well as children, you should
perform separate tests with typical stimuli for each of these cases. For the
assessment of MPEG audio codecs that are used for the transmission of high
quality music between broadcast studios, real music should be used. Especially
with wide band music codecs a variety of at least six to ten different test
samples should be selected, since the performance of audio codecs differs
widely depending on the test material.
The duration of the test sequence should be within the range of approximately
four to eight seconds. Longer tests will lead to averaging effects (short distortions
may be averaged down by a long but almost perfect transmission) and shorter
sequences may not be long enough to contain representative parts of the signal.
If for any reason very long reference files are desired, OPERAs feature of
measuring just a short sequence out of the entire input signals could be
selected. Details regarding this feature under the Trigger option menu are
explained in chapter 4.
The sample rate of the reference file is frequently already defined by the
algorithm that shall be used for the evaluation of the recorded data. PEAQ
according to ITU-R BS.1387 for example requires 48kHz sample rate, although
the implementation in OPERA will deliver reliable results at 44.1kHz, too. Most
speech quality measures are defined for 8 and 16kHz sample rate only. For
more details, refer to the description of the individual algorithms or the standard
documents that apply.
The selection of the sample format should mainly be driven by considering the
capabilities of the underlying hardware. While using the audio interfaces
provided by OPERA, it makes sense to select the 16bit linear. Since currently all
measures use 16bit linear internally, any higher resolution, although supported
by the hardware, will not result in more accurate measurements. When
performing test calls with the voice board, the sample format should be 8bit
mu-law or 8bit A-law (G.711). Otherwise the measurement will include at least
one more step of encoding, since the DSP on the voice board will convert all
input data back to G.711.
A set of typical wide band audio examples is mentioned in the ITU-R rec.
BS.1387. Speech samples are also provided by the ITU-T, in the Series P
Supplement 23.
2.6 How to Assess "Signal Enhancers"
Signal enhancers are pieces of equipment that try to make the processed signal
sound better than the original signal, like e.g. noise reduction systems etc.
When the input signal of the enhancer is taken as the reference and the output
signal as the test signal of any perceptual measure, the result will usually be the
opposite of what is expected. In general, the enhanced signal will be graded
down the more, the better your enhancer works. This is because perception
W H I C H
M E A S U R E M E N T
20
D O
I
N E E D ?
Advertisement
Table of Contents
