OPTICOM PEVQ User manual

PEVQ™Analyzer
Perceptual Video Quality-of-Experience Measurement
OPTICOM Product Line:
Video
Qu lity
Testing
Voice
Quality
Testing
Audio
Quality
Testing
Data
Quality
Testing
The new PEVQ Anal zer evaluates the quality of
multimedia (QCIF, CIF, VGA), SD and HD video for-
mats based on perceptual video quality-of-
experience measurement, reliably and objectively.
The system allows for synchronous SD and HD
play-out and recording over a range of interfaces
(HD-SDI, DVI) and comes with a high-perfor-
mance, front accessible array of hard disk drives.
Perceptual Evaluation of Video Quality (PEVQ) is
an accurate, reliable and fast video quality meas-
urement technology, developed by OPTICOM. It
provides mean opinion score (MOS) analysis of
the video quality degradation occurring through a
network, e.g. in mobile and IP-based networks.
PEVQ can be ideally applied to test video telepho-
ny, video conferencing, video streaming, and IPTV
applications.
PEVQ is a full-reference, end-to-end measurement
based on signal analysis: The degraded video sig-
nal output from a network is analyzed by compar-
ison to the undistorted original reference video
signal on a perceptual basis. Based on the
approach to model the human visual system,
PEVQ can detect anomalies in the video signal and
quantify them by a multitude of key performance
indicators ( PIs). With PEVQ one can analyze the
video quality based on a human visual model, thus
representing the true image quality as perceived
by subjects, for example paying subscribers. Its
value proposition is obvious: equipment manufac-
turers, network operators and service providers
can analyze QoE issues and optimize next genera-
tion networks and thus avoid customer's dissatis-
faction, which otherwise might lead to complains
or churn.
OPTICOM, the leading provider of signal based
perceptual measurement technology for voice,
audio and video, offers its new easy to use PEVQ
testing technology in combination with its proven
audio quality analysis techniques PESQ
(Perceptual Evaluation of Speech Quality) and
PEAQ (Perceptual Evaluation of Audio Quality).
The combination with these state-of-the-art test
algorithms allows for audio-visual quality analysis
and thus provides useful means for lip-sync arte-
fact measurement.
Users can choose among the PEXQ software suite
for Windows™ and the PEVQ Analyzer. PEVQ is
also available as an OEM solution and has been
adopted by a range of leading industry partners
(see www.pevq.org for further information on
third parties' product availability).
PEVQ is based on the earlier PVQM technology
developed by PN Research and new develop-
ments of OPTICOM. The PEVQ analyzer is pro-
duced in cooperation with DVC Germany
(www.digitalvideo.de).
PEVQ has been verified by the Video Quality
Experts Group in 2008 and is going to be proposed
for International Standardization.
Key Fe tures:
• Accurate, reliable and fast
objective analysis of per-
ceived video quality
• Full reference based end-
to-end quality analysis
• Outputs MOS score
that correlates well
with subjective MOS
• Additional PIs for
detailed analysis
• Measurement of multime-
dia (QCIF, CIF, VGA), SD
and HD video quality
• Applications: IPTV, stream-
ing video, 3G, video
telephony
• Performance verified and
recommended for stan-
dardization by the Video
Quality Experts Group
(www.vqeg.org) 2008
PEXQ™, 3SQM™, PEAQ™, PEVQ™, PEDQ™ and the OPTICOM logo are registered trademarks of OPTICOM GmbH; PESQ™ is a registered trademark of OPTICOM
GmbH and Psytechnics Ltd.; the 'Single-sided Speech Quality Measure' and 'The Perceptual Quality Experts' are trademarks of OPTICOM GmbH.
This information may be subject to change. All other brand and product names are trademarks and/or registered trademarks
of their respective owners. All rights reserved. Copyright © 2008 OPTICOM GmbH - www.opticom.de
Voice Qu lity Testing
PSQM ITU-T P.861
PESQ ITU-T P.862
3SQM ITU-T P.563
ECHO
Audio Qu lity Testing
PEAQ ITU-T BS.1387
Video Qu lity Testing
PEVQ
Network Qu lity Testing
VQmon
NEW:
VQEG approved for
standardization

The PEXQ split screen feature of the viewer
window allows for a 1:1 comparison of the
degraded (test) video with the original ref-
erence by horizontal and/or vertical split of
the overlay. This display feature is ideally
suited for codec developers and cause
analysis, as sliding through the image may
visualize even slight deviations from the
original (left half of the image). In this HD
IPTV example, bit errors were inserted dur-
ing the IP transport (visible in the right half
of the image).
PEXQ™, PEAQ™, PEVQ™, PEDQ™, 3SQM™ and the OPTICOM logo are registered trademarks of OPTICOM GmbH; PESQ™ is a registered trademark of OPTICOM GmbH and
Psytechnics Ltd.; the ‘Single-sided Speech Quality Measure’ and ‘The Perceptual Quality Experts’ are trademarks of OPTICOM GmbH. This information may be subject to
change. All other brand and product names are trademarks and/or registered trademarks of their respective owners.
All rights reserved. Copyright © 2008 OPTICOM GmbH – www.opticom.de
2PEVQ™ Analyzer
Comfortable Graphical User Interface
Split View Overlay
2
1
3
The PEVQ Analyzer desktop combines PEXQ (1), OPTICOM's unified graphical user interface
providing a Quad- PI set for voice, audio, video and data analysis, and QuickClip (2) (pow-
ered by Drastic Technologies) for comfortable play-out and record. The PEXQ view window
(3) allows for visual frame-by-frame overlay comparison of original, degraded (test) signal
and PEVQ internal error patterns.

3PEVQ™ Analyzer
Principle
Human Visual Model View
The internal PEVQ views expose an insight
to the human visual model at work. From
left to right:
1) Frame of the original reference
sequence
2) Corresponding frame of the degraded
(test) signal
3) Degraded signal after temporal and
spatial alignment
4) Difference error pattern according to
PEVQ's human visual model, this per-
ceptible error forms the basis for the
MOS calculation.
In the example shown, it can be noted that
the degraded signal contained a lower
frame rate than the original, which leads
to temporal and spatial alignment arte-
facts that may contribute to the error pat-
tern for the particular frame analyzed.
(This example was pictured intentionally
for illustration purposes).
The algorithm can be divided up into four
separate blocks. The first block – pre-pro-
cessing stage – is responsible for the spa-
tial and temporal alignment of the reference
and the impaired signal. This process
ensures that only corresponding frames are
compared to each other.
The second block calculates the perceptual
difference of the aligned signals. Perceptual
means that only those differences are taken
into account which are actually perceived
by a human viewer. Furthermore the activity
of the motion in the reference signal pro-
vides another indicator representing the
temporal information. This indicator is
important as it takes into account that in
frame series with low activity the percep-
tion of details is much higher than in frame
series with quick motion.
The third block in the figure classifies the
previously calculated indicators and detects
certain types of distortions.
Finally, in the fourth block all the appropri-
ate indicators according to the detected dis-
tortions are aggregated, forming the final
result - the mean opinion score (MOS). The
MOS value describes the video quality on a
range from 1 for very bad quality, to 5 for
excellent quality. Besides the final quality
score additional indicators are provided at
the output of the algorithm for further cause
analysis.
This approach to video quality estimation
includes the effects of both packet level
impairments (loss, jitter) and signal related
impairments such as blockiness, jerkiness,
blur and distortions caused by coding
processes.
Future releases will be further optimized in
computational performance and prediction
accuracy for high definition video.
OPTICOM’s PEVQ is one of the two top-
ranked models in the 2008 Video Quality
Experts Group (VQEG) multimedia evalua-
tion and approved for standardization.
Temporal indicators
Temporal
and spatial
alignment Distortion
classifi-
cation
Reference Signal
Impaired Signal Y
Cr
Cb
Perceptual
difference
(Y, Cr, Cb)
PEVQ Score
Indicators
for cause
analysis
}
time
time
Distortion
indicator
integration
PEVQ is a full reference, intrusive measurement algorithm for video quality. Its
basic structure is shown in the figure below.

PEVQ™ Analyzer
PEXQ™, PEAQ™, PEVQ™, PEDQ™, 3SQM™ and the OPTICOM logo are registered trademarks of OPTICOM GmbH; PESQ™ is a registered trademark of OPTICOM GmbH and
Psytechnics Ltd.; the ‘Single-sided Speech Quality Measure’ and ‘The Perceptual Quality Experts’ are trademarks of OPTICOM GmbH. This information may be subject to
change. All other brand and product names are trademarks and/or registered trademarks of their respective owners.
All rights reserved. Copyright © 2008 OPTICOM GmbH – www.opticom.de
4
The PEVQ Analyzer employs the
QuickClipXO clip-recorder toolset (powered
by Drastic Technologies) for comfortable
system configuration, play-out and record
control. A variety of common SD (625/50
or 525/60) and HD (720p, 1080i, 1080p)
format settings are supported.
Clip Recorder/Player
Detailed graphical output for PEVQ: The
'Summary' window (top) includes information on
the analyzed reference and test video files like
frame rate, duration and pixel resolution. On the
right the overall PEVQ MOS is depicted in a
coloured bar graph that turns from red (bad quali-
ty) to green (good quality). The lower window
allows for more detailed cause analysis based on
a number of state-of-the-art PIs, including
PSNR. The radar chart to the right presents a
multidimensional error indicator, including tem-
poral, luminance and chrominance information
along with indicators for blur, jerkiness and
blockiness.
Comprehensive Numerical
and Graphical KPI Set

5PEVQ™ Analyzer
PEVQ MOS
The PEVQ MOS value lies within a range
from 1 (bad) to 5 (excellent). The PEVQ
MOS is based on a multitude of percep-
tually motivated parameters.
Distortion indic tors
For a more detailed analysis the percep-
tual level of distortion in the luminance,
chrominance and temporal domain are
provided.
Del y
The min, max and mean delay of each
frame of the test signal related to the
reference signal, as well as delay vs.
time.
Brightness
The brightness of the reference and
degraded signal.
Contr st
The contrast of the distorted and the ref-
erence sequence.
PSNR
To allow for a coarse analysis of distor-
tions in different domains the PSNR is
provided for the Y, Cb and Cr components
separately.
Jerkiness
describes the smoothness of a video
playback which is often impaired by
down-sampling, coding processes and
perturbed transmissions.
Blur
is a distortion characterized by reduced
sharpness of contour edges and spatial
detail.
Blockiness
is often the result of a low bit rate cod-
ing that uses a block matching algorithm
for the motion estimation and a coarse
quantization for the image blocks.
Fr me Skips nd Freezes
are temporal artefacts occurring in video
transmissions caused by e.g. overloaded
networks.
Effective Fr me R te
Down-sampling of a video signal on a
frame by frame basis often results in loss
of information which often leads to the
degradation of the video signal. The
effective frame rate is an indicator quan-
tifying the severeness of such a process.
Tempor l Activity nd Sp ti l
Complexity
Temporal activity and spatial complexity
indicators quantify the amount of activi-
ty/movement and spatial detail in the
video content. These indicators are
derived from ITU-T recommendation
P.910.
Output
New in PEXQ: The composite result view
for audio-visual uality analysis reporting
the video MOS (PEVQ) (upper frame) and
the audio MOS (PESQ) (middle frame).
The combination of perceptual video and
audio uality measures allows for
detailed lip-sync analysis based on time-
variant delay analysis (lower frame).

PEXQ™, PEAQ™, PEVQ™, PEDQ™, 3SQM™ and the OPTICOM logo are registered trademarks of OPTICOM GmbH; PESQ™ is a registered trademark of OPTICOM GmbH and
Psytechnics Ltd.; the ‘Single-sided Speech Quality Measure’ and ‘The Perceptual Quality Experts’ are trademarks of OPTICOM GmbH. This information may be subject to
change. All other brand and product names are trademarks and/or registered trademarks of their respective owners.
All rights reserved. Copyright © 2008 OPTICOM GmbH – www.opticom.de
6PEVQ™ Analyzer
In the course of International
Standardization of video quality metrics, in
2007 and 2008 the Video Quality Experts
Group (www.vqeg.org) conducted a huge
test series of 41 validation tests run across
twelve different labs in Europe, USA and
Asia. OPTICOM's PEVQ showed superior
performance and is going to be proposed
for standardization. Most importantly,
PEVQ could also significantly outperform
PSNR, a commonly used industry metrics
for objective video quality estimation.
The accuracy of PEVQ has been trained on
a huge number of human visual test data-
bases. The photo shows OPTICOM's sub-
jective test setup which is in line with ITU-
T Rec. P.910 and further adaptations devel-
oped by VQEG for the multimedia test
series.
Sub ective Validation
Superior Accuracy

7PEVQ™ Analyzer
PEVQ MOS Results for H.264
1
1,5
2
2,5
3
3,5
4
4,5
5
64kbit 64kbit 128kbit 128kbit 320kbit 704kbit
Bitrate
PEVQ MOS
Shuttle
Calendar
Football
MOS = 3.3
MOS = 3.3
MOS = 2,5
MOS = 2,5
MOS = 1.6
MOS = 1.6
The image quality of advanced video
codecs, like H.264, does very much depend
on the content material and in particular at
lower bitrates significant performance
deviations can be observed for various
programme material. In the diagram one
and same H.264 encoding was applied to
three test sequences of differing, individ-
ual complexity. As a result, the measured
MOS scores vary between 3,3 and 1,6, a
significant content depending difference
which outlines why NGNs carrying IPTV
and multimedia services need to be ana-
lyzed by true perceptual signal analysis
like PEVQ. An example that clearly shows
why packet-loss analysis based E-model
estimates simply can't tackle this level of
performance issue.
Headend/Aggregator Core Network Metro Network
Home
Content
Provider/
Studio
Home
Mobile PDA
Cell
PEVQ
Analyzer
IP
Access Network
Stream
Stream
Stream
Decode/
Transcode/
Reformat
Decode/
Transcode/
Reformat
Decode/
Transcode/
Reformat Encrypt/
Server
Encrypt/
Server
Encrypt/
Server
STB/TV
PC
VoIP
STB/TV
PC
VoIP
Reference 1
Reference 2
Test 1 Test 2 Test 3
Regional Service Provider
VOD File
VOD File
VOD File
Legacy
Cable Network
IP (IPTV)
Core Network
IMS
Ecosystem
SD/HDTV
SD/HDTV
Mobile TV
Dish
Farm
Application to IPTV and IMS Architectures
Product Differentiator
PEVQ is a full reference video quality measure that compares the original signal from the
satellite downlink (reference1) or a stored reference from the studio (reference2) with the
test signal, degraded by transmission artefacts. Test Signals can be acquired e.g. after
transcoding at the headend (test1), at the edge of the core (test2) or the metro (test3)
network, i.e. at the subscriber broadband tools (STB or Mobile TV). The application to
network segments represents a very useful toolset for service
assurance and SLA compliance testing.

Specifications
Functionality
• Perceptual Evaluation of Video Quality-
PEVQ, with mapping to MOS scale
(1 bad, … 5 excellent quality)
• Analysis of multimedia, SD and HD video
formats
Input
• Raw YUV, AVI Files with RGB24, YUV444,
YUV422 or YUV420 data, any frame rate from
2,5 up to 60 fps
• Video test sequences of 6 to 20 seconds
duration
In Cooperation with
DVC –
Digit lvideo Computing
GmbH
Seestraße 7
82211 Herrsching, Germany
Phone: +49-81 52 - 9 30 10
Fax: +49-81 52 - 9 13 31
[email protected]
www.digitalvideo.de
PEVQ - Perceptu l Ev lu tion of Video Qu lity
PEXQ™, 3SQM™, PEAQ™, PEVQ™, PEDQ™ and the OPTICOM logo are registered trademarks of OPTICOM GmbH; PESQ™ is a registered trademark of OPTICOM
GmbH and Psytechnics Ltd.; the 'Single-sided Speech Quality Measure' and 'The Perceptual Quality Experts' are trademarks of OPTICOM GmbH.
This information may be subject to change. All other brand and product names are trademarks and/or registered trademarks
of their respective owners. All rights reserved. Copyright © 2008 OPTICOM GmbH - www.opticom.de
Sales Contacts:
OPTICOM GmbH
Nägelsbachstraße 38
91052 Erlangen, GERMANY
Phone: [phone]
Fax: [phone]
[email protected]
www.opticom.de
North Americ :
Telchemy Inc.
Phone [phone]
JDSU - Actern U.S.
Phone [phone]
Europe, L tin Americ ,
Middle E st & Afric ,
Asi P cific, CIS Countries:
JDSU - Actern
Germ ny GmbH
Phone: [phone]
Through our distributor net-
work, we are represented in
more than 80 countries. To find
your local sales office, please
contact [e-mail]
OPTICOM GmbH
Nägelsbachstraße 38
91052 Erlangen, GERMANY
Phone: [phone]
Fax: [phone]
[email protected]
www.opticom.de
PVQ-101-000-D
• PEVQ Analyzer, Desktop Version
(DVC Boxxster)
• Single channel SD/HD SDI Interface for cap-
turing video (uncompressed reference video
play-out OR test record)
• Available with internal SATA-2-Storage 480
GB, 720 GB or 1200 GB (others on request)
PVQ-102-000-R
• PEVQ Analyzer, Rack-mountable Version
(DVC Cliprecorder AVX2)
• Dual channel SD/HD SDI Interface for cap-
turing video (uncompressed reference video
play-out AND test record)
PVQ-00X-XXX-S
• PEVQ Analyzer, Software-only Version
(for upgrade of existing DVC
Boxxster/Cliprecorder systems)
Optional available on re uest:
• Rack-mountable 1U breakout box incl.
SD/HD SDI I/O and analog/digital audio
• SD/HD SDI to DVI converter
• System flight-case (wheeled)
NOTE:
The PEVQ Analyzer comes with pre-configured
system installation, system documentation,
hard disk array as specified, break-out cables
for analog video and analog/digital audio
(Stereo), USB keyboard and mouse. The system
does NOT include a monitor as shown on the
first page.
Ordering Information
About OPTICOM
OPTICOM GmbH is the leading vendor for
voice, audio and video quality measurement
technology and OEM products for mobile and
IP based network testing. With PSQM, PESQ,
PEAQ and P.563, the pioneers in perceptual
quality testing have been providing by now
four international world-class standards for
voice and audio quality measurement since
the foundation of OPTICOM as a spin-off
from Fraunhofer’s MP3 development team in
1995.
After the great success with PESQ - the
International Standard for voice quality test-
ing, the experts from Germany now also
source PEVQ, the new industry standard to
measure a perceptual video quality KPI for
streaming, conferencing and messaging
applications.
PE Q, the next level of Perceptual Quality
Measurement, is the ideal ‘all-in-one’ test
suite for developers, manufacturers and
operators, while the ‘ ’ just symbolizes the
ongoing evolution of perceptual QoE metrics:
Based on PESQ, PEAQ, PEVQ and PEDQ the
software provides the most comprehensive
standards-based MOS-KPI set to score voice,
audio/visual and data quality as experienced
by subscribers. OPTICOM’s proven OEM
technology can be found in most state-of-
the-art products of leading T&M vendors,
see also www.opticom.de/company/cus-
tomers-licensing.html.
OPTICOM GmbH is a privately held company
located in Erlangen, Germany.