2.5 ‘KPI Overview’ Worksheet
2.5.1 Session KPIs
Session (MO) | KPI | Threshold |
|---|
Session Setup Success Rate (%) | 91.3% | 98 |
Session Setup Delay (sec) | 3.75 | 4 |
Conversation Start Delay (ms) | 75.04 | 200 |
Session Completion Success Rate (%) | 95.2% | 98 |
Session Disconnect Delay (ms) | 160.3 | 200 |
SIP Handshake Time (ms) | 249.83 | 200 |
IMS Latency (ms) | 3600.05 | 4000 |
These KPIs deal with VoLTE sessions.
• Session Setup Success Ratio: Session setup success is triggered by reception of 200 OK Response to INVITE Request.
By contrast, session setup failure is triggered:
o on reception of SIP Status Code 401, 403, 404, 405, 408, 415, 420, 480, 486, 488, 500, 501, 504, 600, 603, 604, or 606;
o on SIP Request CANCEL;
o if SessionSetupFailureTimeout is exceeded. This is a TEMS Discovery-internal parameter; see chapter
3. The value used when generating the report appears from the table in section
2.5.5.
• Session Setup Delay: Time from (SIP) INVITE Request to 180 Ringing (SIP) Response. Specifically:
o For MO, time from INVITE sent until 180 Ringing received.
o For MT, time from INVITE received until 180 Ringing sent.
Setup delay is calculated only for successful session setups.
• Conversation Start Delay: Time from 200 OK Response (INVITE) to first received RTP Packet. Specifically:
o For MO, time from 200 OK received until first RTP packet received.
o For MT, time from 200 OK sent until first RTP packet received.
• Session Completion Success Ratio: Session completion success is triggered by reception of 200 OK Response to BYE Request.
By contrast, session completion failure is triggered:
o if BYE request is sent/received with reason phrase other than 101 (User Triggered), 107 (New Dialog Established, i.e. redirected to voicemail) or 105 (Moved to eHRPD);
o if no SIP Status Code 200 OK for BYE request is received before SessionHangupTimeout (TEMS Discovery-internal parameter, see chapter
3);
o if SIP Status Code 400, 403, 404, 405, 406, 408, 413, 415, 420, 480, 481, 486, 487, 488, 500, 501, 503, 504, 600, 603, 604, or 606 is received after SIP BYE Request has been sent;
o if no RTP packet is received for a period longer than SessionDropTimeout (TEMS Discovery-internal parameter, see chapter
3).
Timeout values used when generating the report appear from the table in section
2.5.5.
• Session Disconnect Delay: Time from BYE Request sent until 200 OK Response (BYE) received.
• SIP Handshake Time: Time from PRACK Request sent until 200 OK Response (PRACK) received.
• IMS Latency: Time from (SIP) INVITE Request sent from UE ‘a’ until (SIP) INVITE received by UE ‘b’ plus the time from 180 Ringing (SIP) Response sent by UE ‘b’ until 180 Ringing (SIP) received by UE ‘a’.
Signaling diagrams:
Session setup
Session release
2.5.2 Quality KPIs
Quality | KPI | Threshold |
|---|
Acceptable Call Quality (%) | 89.6% | 98 |
AQM Score Downlink (MOS) | 3.61 | 2.7 |
AQM Score Uplink (MOS) | 3.55 | 2.7 |
Speech Path Delay (ms) | 537.0 | |
These KPIs are concerned with voice quality.
• Acceptable Call Quality (%): The quality of a call is judged according to the algorithm in ETSI TR 102 506 V1.2.1, which takes into account
o the recency effect: bad samples lower the perceived quality more if they occur towards the end of call;
o the worst sample of the call: a single very bad sample has a strong negative impact on the perceived quality of the entire call.
A call is deemed acceptable if its quality score is above the relevant MOS threshold, as detailed in the ‘AQM’ rows of the table.
A further threshold is set for the KPI itself.
• AQM Score Downlink (MOS), AQM Score Uplink (MOS): The voice quality is judged by one of the following algorithms:
o POLQA (Perceptual Objective Listening Quality Assessment), standardized in ITU-T Recommendation P.863;
o PESQ (Perceptual Evaluation of Speech Quality), standardized in ITU-T Recommendation P.862.1.
• Speech Path Delay (ms): Roundtrip time as measured along the actual audio path: from and back to the external handset connector of the UE.
2.5.3 Registration KPIs
Registration | KPI | Threshold |
|---|
Registration Success Rate (%) | 100.0% | 98 |
Registration Delay (ms) | 797 | 500 |
These KPIs deal with registration towards IMS.
• Registration Success Ratio (%): Registration success is triggered by reception of 200 OK Response to REGISTER Request.
Cases:
1. UE power-up (initial) registration: First REGISTER request sent, followed by response 401 Auth. Challenge; second REGISTER request sent followed by response code 200 OK (REGISTER). See signaling diagram below.
2. Re-registration: REGISTER request sent, followed by response code 200 OK (REGISTER).
3. UE power-down de-registration: REGISTER request sent with Expires value set to zero, followed by a NOTIFY request and then by response code 200 OK (REGISTER).
By contrast, session setup failure is triggered:
o on reception of SIP Status Code 400, 403, 404, 405, 406, 408, 413, 415, 420, 480, 481, 486, 487, 488, 500, 501, 503, 504, 600, 603, 604, or 606;
o on expiration of SIPRegistrationFailureTimeout. This is a TEMS Discovery-internal parameter; see chapter
3. The value used when generating the report appears from the table in section
2.5.5.
• Registration Delay (ms): Time from (SIP) REGISTER request to 200 OK (on REGISTER) response. In case of new registration, the delay is measured from the first REGISTER request. Compare Registration Success Ratio above.
Signaling diagram:
Remark: The first response to the REGISTER is normally a failure response, indicating that authentication must be done. The UE then sends a second REGISTER request with authentication information. After correct authentication, the UE receives the 200 OK message.
2.5.4 Presence KPIs
Presence | KPI | Threshold |
|---|
Presence Subscription Success Rate (%) | - | |
Presence Subscription Delay (ms) | | |
Publish Success Rate (%) | 100.0% | |
Publish Request Delay (ms) | 286.67 | |
These KPIs concern subscription to and publishing of user presence information.
• Presence Subscription Success Ratio (%): Subscription success is triggered by reception of 200 OK Response to NOTIFY Request followed by successful SUBSCRIBE.
By contrast, session setup failure is triggered:
o if SIP Status Code 412, 423, or 489 is received;
o if PresenceSubscriptionTimeout is exceeded. This is a TEMS Discovery-internal parameter; see chapter
3.
• Presence Subscription Delay (ms): Time from (SIP) SUBSCRIBE Request to 200 OK (NOTIFY) Response.
• Publish Success Ratio (%): Publish success is triggered by reception of 200 OK Response to PUBLISH Request.
By contrast, publish failure is triggered:
o if SIP Status Code 412, 423, or 489 is received;
o if PublishTimeout is exceeded. This is a TEMS Discovery-internal parameter; see chapter
3.
• Publish Request Delay (ms): Time from (SIP) PUBLISH Request to 200 OK (PUBLISH) Response.
2.5.5 Timer Settings
Timer Settings | | Unit |
|---|
Session Setup Failure Timeout | 35000 | ms |
Session Drop Timeout | 9000 | ms |
Session Hangup Timeout | 3000 | ms |
Registration Timeout | 10000 | ms |
Subscription Timeout | 5000 | ms |
Publish Timeout | 5000 | ms |
This table shows the TEMS Discovery timer settings used in producing the report.
2.5.6 LTE Statistics
LTE Statistics | KPI | |
|---|
RRC Connection Setup Success Rate (%) | 78.7% | |
RRC Connection Setup Time (ms) | 38.49 | |
RRC Connection Completion Success Rate (%) | 92.4% | |
Handover Interruption Time Control Plane (ms) | 8.21 | |
Handover Interruption Time User Plane (ms) | 308.77 | |
• RRC Connection Setup Success Ratio (%): Setup success is triggered by RRC Connection Setup Complete message, preceded by RRC Connection Request sent from the same device in the same logfile.
By contrast, RRC connection setup failure is triggered:
o on reception of RRC Connection Reject;
o on expiry of timer T300;
o if no RRC Connection Setup Complete is sent on reception of RRC Connection Setup.
• RRC Connection Setup Time (ms): Time from RRC Connection Request to RRC Connection Setup Complete.
• RRC Connection Completion Success Ratio (%): Successful completion is triggered by RRC Connection Release message.
By contrast, RRC connection drop is triggered:
o on reception of RRC Re-establishment Request followed by a RRC Connection Re-establishment Reject;
o on expiry of timer T301;
o if no RRC Connection Release is received before new RRC Connection Setup attempt.
• Handover Interruption Time Control Plane (ms): Time from RRC Connection Reconfiguration indicating handover to RRC Connection Reconfiguration Complete.
• Handover Interruption Time User Plane (ms): Time from last RTP packet on old cell to time of reception of first RTP packet on new cell (only if no silence indication).
