WO2012136248A1 - Assessing network user quality of experience based on correlation of signaling events - Google Patents

Assessing network user quality of experience based on correlation of signaling events Download PDF

Info

Publication number
WO2012136248A1
WO2012136248A1 PCT/EP2011/055264 EP2011055264W WO2012136248A1 WO 2012136248 A1 WO2012136248 A1 WO 2012136248A1 EP 2011055264 W EP2011055264 W EP 2011055264W WO 2012136248 A1 WO2012136248 A1 WO 2012136248A1
Authority
WO
WIPO (PCT)
Prior art keywords
signaling
event information
signaling events
events
information
Prior art date
Application number
PCT/EP2011/055264
Other languages
French (fr)
Inventor
Petri Kristian SILEN
Jyrki Petteri BERG
Sampo Kristian TORIKKA
Jani-Pekka Virtanen
Original Assignee
Nokia Siemens Networks Oy
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nokia Siemens Networks Oy filed Critical Nokia Siemens Networks Oy
Priority to PCT/EP2011/055264 priority Critical patent/WO2012136248A1/en
Publication of WO2012136248A1 publication Critical patent/WO2012136248A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/50Network service management, e.g. ensuring proper service fulfilment according to agreements
    • H04L41/5061Network service management, e.g. ensuring proper service fulfilment according to agreements characterised by the interaction between service providers and their network customers, e.g. customer relationship management
    • H04L41/5067Customer-centric QoS measurements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14Network analysis or design
    • H04L41/142Network analysis or design using statistical or mathematical methods
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/12Network monitoring probes
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04QSELECTING
    • H04Q2213/00Indexing scheme relating to selecting arrangements in general and for multiplex systems
    • H04Q2213/13166Fault prevention

Definitions

  • the invention relates to apparatuses, methods, a system, computer programs, computer program products and computer- readable media.
  • an apparatus comprising: at least one processor and at least one memory including a computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to: obtain signaling event information, the signaling event information being gathered in a plurality of networks; determine signaling events from the signaling event information, and choose signaling events for combining from the determined signaling events and combine the chosen signaling events for obtaining
  • a method comprising: obtaining signaling event information, the signaling event information being gathered in a plurality of networks; determining signaling events from the signaling event information, and choosing signaling events for combining from the determined
  • an apparatus comprising: means for obtaining signaling event information, the signaling event information being gathered in a plurality of networks; means for determining signaling events from the signaling event information, and means for choosing signaling events for combining from the determined signaling events, and combine the chosen signaling events for obtaining
  • a computer program embodied on a computer-readable storage medium, the computer program comprising program code for controlling a process to execute a process, the process comprising: obtaining signaling event information, the signaling event
  • Figure 1 illustrates an example of a system
  • Figure 2 is a flow chart
  • Figure 3 illustrates an example of an apparatus.
  • Embodiments are applicable to any user device, such as a user terminal, relay node, server, node, corresponding component, and/or to any communication system or any combination of different communication systems that support required functionalities.
  • the communication system may be a wireless communication system or a communication system utilizing both fixed networks and wireless
  • NTN next generation network
  • One target of the NGN is that one network transports all kinds of information and services, such as voice, data and media by
  • All-IP is also used to describe this idea. All-IP is aimed to provide a merged data/voice/multimedia network. It should be appreciated that service-related functions are designed to be
  • QoS Quality of Service
  • QoS Quality of Service
  • a service having a required Quality of Service may be achieved by managing latency or delay, delay variation or jitter, bandwidth, and/or packet loss parameters by a network. For example, a long delay may cause a noticeable echo, delay variations may cause strange sound effects and packet loss may cause interruptions in a conversation or in the run of a video clip, etc.
  • Ad hoc networks comprise a plurality of (mobile) nodes coupled by wireless links forming arbitrary time- varying wireless network topologies. Nodes of an ad hoc wireless network may move randomly and self-organize themselves in an arbitrary fashion.
  • the exemplary communications system used herein is the Universal Mobile
  • UMTS Telecommunications System
  • UTRAN radio access network
  • WCDMA wideband code division multiple access
  • GSM global system for mobile communications
  • TD- SCDMA time division synchronous code division multiple access
  • LTE long term evolution
  • LTE-Advanced long term evolution advanced
  • WiMAX worldwide interoperability for microwave access
  • Bluetooth® personal area network
  • PCS communications services
  • ZigBee® wideband code division multiple access
  • WCDMA wideband code division multiple access
  • UWB ultra- wideband
  • sensor networks sensor networks
  • MANETs mobile ad-hoc networks
  • Internet Protocol multimedia
  • IMS subsystems
  • Figure 1 is a simplified illustration of a communications system or cooperation of a plurality of communications systems to which embodiments according to the invention are applicable.
  • Figure 1 shows parts of two cellular radio systems which comprise radio network elements 100, 118 into which functionalities of a base station and a radio network controller are integrated.
  • the communications network may include separate base stations and radio network controllers, but in this example the part of the network is depicted as its simplest.
  • the radio network controller 100 has bi-directional radio links 102 and 104 to user devices 106 and 108 and the radio network controller 118 has bi-directional radio links 120, 122 to user devices 124, 126.
  • the user devices may be fixed, vehicle-mounted or portable.
  • the radio network element includes transceivers, for instance. From the transceivers of the radio network element a connection is provided to an antenna unit that establishes bi ⁇ directional radio links to the user devices.
  • the radio network element 100 is further connected to a core network 110 (CN) and the radio network element 118 is further connected to a core network 116.
  • the counterpart on the CN side can be a media gateway (MGW) or a serving GPRS (General Packet Radio Service) support node (SGSN) , a gateway GPRS support node (GGSN) , where GPRS is an
  • MGW media gateway
  • GPRS General Packet Radio Service
  • SGSN Serving GPRS (General Packet Radio Service) support node
  • GGSN gateway GPRS support node
  • the GPRS Core Network provides mobility management, session management and transport for Internet Protocol packet services in Global System for Mobile Communications (GSM) and Universal Mobile Telecommunications System (UMTS) networks .
  • GSM Global System for Mobile Communications
  • UMTS Universal Mobile Telecommunications System
  • GTP GPRS Tunneling Protocol
  • the communication systems are also able to communicate with other networks, such as the Internet 112.
  • the network topology may differ from what is depicted in Figure 1.
  • a counterpart on the CN side in the GSM may be a mobile switching centre (MSC) that is a service delivery node for the GSM (and also for the code division multiple access systems) which is responsible for routing voice calls as well as other services.
  • MSC mobile switching centre
  • the MSC sets up and releases end-to-end connections, handles mobility and handovers, etc.
  • two communications networks are in
  • correlation server may be a stand-alone network element or it may operationally or even physically be integrated in one of the core networks serving the communications networks which in this case may be thought as some kind of sub-networks in the end-user's point of view, etc.
  • Arrow 128 depicts that a correlation server may be coupled to an external network (not shown) .
  • the external network may be any kind of communication network.
  • Figure 2 is usually related to a server or host that may operate as a correlation server.
  • An embodiment of the invention relates to correlation of end-user experiences in computer and/or communication networks.
  • One target is to enrich information on end-user experiences in a centralized manner for obtaining improved views of them.
  • the processing of information may be carried out in real-time.
  • Term "real-time" may in this application refer to a processing delay which is
  • An acceptable delay may vary according to the service at issue. Typical delay-critical services are voice calls, conventional videoconferences and telepresence. It is also possible that protocols used determine limits to
  • the embodiment starts in block 200.
  • the embodiment is especially suitable to be carried out by a correlation server or a corresponding element.
  • the signaling event information is gathered in a plurality of networks.
  • Signaling event information may be typical signaling event information (SEI) used in telecommunication networks collected by network elements (NEs) , signaling transfer points (STPs) and/or signaling link probes (SLPs) , and conveyed to a correlation server (CS) typically in real ⁇ time .
  • SEI signaling event information
  • NEs network elements
  • STPs signaling transfer points
  • SLPs signaling link probes
  • CS correlation server
  • a signaling event takes place when a network element or an STP
  • a signaling message may be according to any signaling protocol used in communication networks (signaling system 7 (SS7) that is a telephone signaling protocol or Internet Protocol (IP), for
  • SEI Signaling event information
  • SEI may be any piece of information or a full information element (IE) inside a signaling message.
  • SEI may for example be a subscriber identity, such as a mobile station international ISDN
  • MSISDN integrated services digital network number
  • IMSI international mobile subscriber identity
  • LAC location area code
  • CID cell Id
  • cause code specified by 3rd Generation
  • an end-user is served by utilizing cooperation of a plurality of networks, such as a wireless network and core network.
  • network may mean one network in the sense that it operates according to one radio or other protocol, a combination of several networks established for providing services to users, such as a network which comprises a "basic” network and one or more ad hoc networks operating under the control of the "basic” network, a combination of networks operating under the control of different operators with negotiated rules for cooperation control, etc.
  • the interpretation of the term “network” is emerging along with the progress of the information society and thus it has to be interpreted broadly both in the real-world and as an abstract concept.
  • Figure 1 One example of a possible arrangement for the cooperation of a plurality networks is shown in Figure 1.
  • correlation server is shown as an option by reference number 114.
  • a correlation server may store signaling information (in real-time) into a database and/or internal data structure for processing.
  • Signaling and thus signaling events are in relation to actions needed for providing services to users. Signaling is needed for establishing a call, establishing a tunnel for data transfer, providing security and encryption, carrying out handovers, obtaining location information, etc. Communications networks are typically quite
  • Each event concerning signaling is called a signaling event.
  • signaling event usually, several signaling events which may use different protocols, take place on different layers and between different network elements are required for providing a service, such as a call, for an end-user.
  • a service such as a call
  • For possible fault detection it is common that many signaling events involved in the faulty service must be searched for.
  • signaling event information is recognized and then unpacked for this determination .
  • a signaling event takes place when a network element or a STP sends/transmits/receives/forwards a signaling message (SM) , or when an SLP captures a signaling message in a signaling link it monitors.
  • a signaling message may be according to any signaling protocol used in
  • Signaling system 7 that is a telephone signaling protocol or Internet Protocol (IP) for example
  • IP Internet Protocol
  • a correlation server may use correlation as means for choosing suitable ones from multiple signaling events (SE) according to defined correlation rules (CRs) .
  • SE signaling events
  • CRs correlation rules
  • correlation rule may be defined as one or more pieces of signaling event information that need to match between two different signaling events.
  • the correlation server may create combined information (CI) from signaling events that have been correlated in real-time according to information combination rules (ICRs) .
  • ICR may be defined as signaling events that have been correlated.
  • CI is thus typically a subset of all SEIs from correlated SEs.
  • a call is controlled in a mobile network by a switch, such as a mobile switching server (MSS) , and the call is conveyed as a VoIP-transmission between media gateway (MGW) elements in an Internet
  • a switch such as a mobile switching server (MSS)
  • MMS mobile switching server
  • MGW media gateway
  • the switch responsible for establishing and finishing the call as well as collecting location information and the MGW of the network, where the call is established, transfers voice and/or sound into a suitable format for the IP-network and takes care of routing.
  • the quality of the call may only be evaluated by analyzing the quality of IP-traffic via the MGW elements by using one or more methods usually used for this purpose.
  • the switch (MSS) cannot provide necessary parameters, since it does not see IP-traffic.
  • an end-user report comprises these pieces of information instead of a plurality of separate reports that may even be stored in different databases in different networks.
  • Pieces of information may be correlated or chosen by using an appropriate criterion, usually a common denominator.
  • the common denominator may be subscriber identification or a global call reference number, for example.
  • the criterion is called a correlation rule. If put in a simplified manner, it may be said that first the pieces of information are correlated to find suitable ones from the information space, and then combined in a desired way to produce the kind of end-user reports which are useful in evaluating end-user experience.
  • a correlation rule or criterion may, for example, be at least one of the following: one or more parameters identifying the end-user, one or more parameters indentifying elements used for providing the service and one or more parameters identifying the service and/or service status in question.
  • the correlation server may convey combined information in real-time to one or more external systems, such as to one or more external network being responsible of network control functionalities in the networks the signaling event information is gathered from.
  • the end-user experience information may thus be enriched in a centralized manner.
  • Some examples of signaling event information to be enriched are control plane experience information of a mobile switching center (MSC) server and user plane experience information of a multimedia gateway.
  • Another example is enriching radio network control (RNC) data of a wide band code division multiple access (WCDMA) radio network with a mobile type international mobile equipment identity (IMEI) information, which is available to an MSC server but not to an RNC.
  • RNC radio network control
  • WCDMA wide band code division multiple access
  • IMEI mobile type international mobile equipment identity
  • the embodiment ends in block 208.
  • the embodiment is repeatable in many ways. One example is shown by arrow 210 in Figure 2.
  • steps/points may be performed simultaneously or in an order differing from the given one.
  • Other functions can also be executed between the steps/points or within the steps/points and other signaling messages sent between the illustrated messages.
  • Some of the steps/points or part of the steps/points can also be left out or replaced by a corresponding step/point or part of the step/point.
  • conveying, transmitting and/or receiving may herein mean preparing a data
  • An embodiment provides an apparatus which may be any node, host, server, user device or any other suitable apparatus capable to carry our processes described above in relation to Figure 2.
  • Figure 3 illustrates a simplified block diagram of an apparatus according to an embodiment especially suitable for network control.
  • an apparatus 300 such as a node device, host or server, including facilities in a control unit 304 (including one or more processors, for example) to carry out functions of embodiments, such as choose and combine signaling events for obtaining information on user experience in the plurality of networks.
  • a control unit 304 including one or more processors, for example
  • an apparatus 300 may include at least one processor 304 and at least one memory 302 including a computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to: obtain
  • Yet another example of an apparatus comprises means 304 for obtaining signalling event information, the signalling event information being gathered in a plurality of networks; means 304 for determining signaling events from the signaling event information, and means 304 for choosing signaling events for combining from the
  • an apparatus comprises an obtainer configured to obtain signaling event information, the signaling event information being gathered in a plurality of networks; a determining unit configured to determining signaling events from the signaling event information, and a chooser configured to choose signaling events for combining from the determined signaling events and combine the chosen signaling events for obtaining information on user experience in the plurality of networks.
  • An apparatus may in general include at least one
  • the memory units may include volatile and/or non-volatile memory.
  • the memory unit may store computer program code and/or
  • Each of the memory units may be a random access memory, hard drive, etc.
  • the memory units may be at least partly removable and/or detachably operationally coupled to the apparatus.
  • the memory may be of any type suitable for the current technical environment and it may be implemented using any suitable data storage technology, such as semiconductor-based technology, flash memory, magnetic and/or optical memory devices.
  • the memory may be fixed or removable.
  • the apparatus may be a software application, or a module, or a unit configured as arithmetic operation, or as a program (including an added or updated software routine) , executed by an operation processor.
  • Programs also called program products or computer programs, including software routines, applets and macros, can be stored in any
  • Computer programs may be coded by a programming language, which may be a high-level programming language, such as objective-C, C, C++, Java, etc., or a low-level programming language, such as a machine language, or an assembler.
  • a programming language which may be a high-level programming language, such as objective-C, C, C++, Java, etc., or a low-level programming language, such as a machine language, or an assembler.
  • routines which may be implemented as added or updated software routines, application circuits (ASIC) and/or programmable circuits. Further, software routines may be downloaded into an apparatus.
  • the apparatus such as a node device, or a corresponding component, may be
  • a computer or a microprocessor such as single-chip computer element, or as a chipset, including at least a memory for providing storage capacity used for arithmetic operation and an operation processor for executing the arithmetic operation.
  • Embodiments provide computer programs embodied on a distribution medium, comprising program instructions which, when loaded into electronic apparatuses, constitute the apparatuses as explained above.
  • processor to perform embodiments of the methods described above .
  • the computer program may be in source code form, object code form, or in some intermediate form, and it may be stored in some sort of carrier, distribution medium, or computer readable medium, which may be any entity or device capable of carrying the program.
  • carrier include a record medium, computer memory, read-only memory, electrical carrier signal, telecommunications signal, and software distribution package, for example.
  • the computer program may be executed in a single electronic digital computer or it may be distributed amongst a number of computers .
  • the techniques described herein may be implemented by various means. For example, these techniques may be implemented in hardware (one or more devices) , firmware (one or more devices) , software (one or more modules) , or combinations thereof.
  • the apparatus may be implemented within one or more application specific integrated circuits (ASICs) , digital signal processors (DSPs) , digital signal processing devices (DSPDs) , programmable logic devices (PLDs) , field programmable gate arrays (FPGAs) , processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described herein, or a combination thereof.
  • ASICs application specific integrated circuits
  • DSPs digital signal processors
  • DSPDs digital signal processing devices
  • PLDs programmable logic devices
  • FPGAs field programmable gate arrays
  • processors controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described herein, or a combination thereof.
  • firmware or software the firmware or software
  • implementation can be carried out through modules of at least one chip set (e.g., procedures, functions, and so on) that perform the functions described herein.
  • the software codes may be stored in a memory unit and executed by processors.
  • the memory unit may be implemented within the processor or externally to the processor. In the latter case it can be communicatively coupled to the processor via various means, as is known in the art.
  • components of systems described herein may be rearranged and/or complimented by additional components in order to facilitate achieving the various aspects, etc., described with regard thereto, and they are not limited to the precise configurations set forth in the given figures, as will be appreciated by one skilled in the art .

Landscapes

  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

The invention relates to an apparatus comprising at least one processor and at least one memory including a computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to: obtain signaling event information, the signaling event information being gathered in a plurality of networks; determine signaling events from the signaling event information, and choose signaling events for combining from the determined signaling events and combine the chosen signaling events for obtaining information on user experience in the plurality of networks.

Description

Description
Title
ASSESSING NETWORK USER QUALITY OF EXPERIENCE BASED ON CORRELATION OF SIGNALING EVENTS
Field
The invention relates to apparatuses, methods, a system, computer programs, computer program products and computer- readable media.
Background
The following descrip Lion or oacKgrouna art may include insights, discoveries , understandings or disclosures, or associations together with disclosures not known to the relevant art prior to the present invention but provided by the invention. Some such contributions of the invention may be specifically pointed out below, whereas other such contributions of the invention will be apparent from their context .
The expectations of end-users regarding the quality of service are becoming higher. The change of customer habits introduces more demands for the availability of services in communications networks .
Brief description
According to an aspect of the present invention, there is provided an apparatus comprising: at least one processor and at least one memory including a computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to: obtain signaling event information, the signaling event information being gathered in a plurality of networks; determine signaling events from the signaling event information, and choose signaling events for combining from the determined signaling events and combine the chosen signaling events for obtaining
information on user experience in the plurality of
networks .
According to yet another aspect of the present invention, there is provided a method comprising: obtaining signaling event information, the signaling event information being gathered in a plurality of networks; determining signaling events from the signaling event information, and choosing signaling events for combining from the determined
signaling events and combine the chosen signaling events for obtaining information on user experience in the plurality of networks.
According to yet another aspect of the present invention, there is provided an apparatus comprising: means for obtaining signaling event information, the signaling event information being gathered in a plurality of networks; means for determining signaling events from the signaling event information, and means for choosing signaling events for combining from the determined signaling events, and combine the chosen signaling events for obtaining
information on user experience in the plurality of
networks .
According to yet another aspect of the present invention, there is provided a computer program embodied on a computer-readable storage medium, the computer program comprising program code for controlling a process to execute a process, the process comprising: obtaining signaling event information, the signaling event
information being gathered in a plurality of networks; determining signaling events from the signaling event information, and choosing signaling events for combining from the determined signaling events and combine the chosen signaling events for obtaining information on user experience in the plurality of networks. List of drawings
Some embodiments of the present invention are described below, by way of example only, with reference to the accompanying drawings, in which
Figure 1 illustrates an example of a system;
Figure 2 is a flow chart, and
Figure 3 illustrates an example of an apparatus.
Description of embodiments
The following embodiments are only examples. Although the specification may refer to "an", "one", or "some"
embodiment ( s ) in several locations, this does not
necessarily mean that each such reference is to the same embodiment ( s ) , or that the feature only applies to a single embodiment. Single features of different
embodiments may also be combined to provide other
embodiments .
Embodiments are applicable to any user device, such as a user terminal, relay node, server, node, corresponding component, and/or to any communication system or any combination of different communication systems that support required functionalities. The communication system may be a wireless communication system or a communication system utilizing both fixed networks and wireless
networks. The protocols used, the specifications of communication systems, apparatuses, such as servers and user terminals, especially in wireless communication, develop rapidly. Such development may require extra changes to an embodiment. Therefore, all words and expressions should be interpreted broadly and they are intended to illustrate, not to restrict, embodiments.
A next generation network (NGN) is a concept of
architectural evolutions in relation to core and access networks of a communication system. One target of the NGN is that one network transports all kinds of information and services, such as voice, data and media by
encapsulating these services into packets in the similar manner as on the Internet. Term "all-IP" is also used to describe this idea. All-IP is aimed to provide a merged data/voice/multimedia network. It should be appreciated that service-related functions are designed to be
independent from underlying transport-related
technologies. One example of suitable technologies to be used in this context is Voice over Internet protocol
(VoIP) .
Quality of Service (QoS) is an important issue in
providing VoIP, multimedia and other corresponding
services. An end-user expects that their connection is not delayed or dropped due to other traffic. A service having a required Quality of Service (QoS) may be achieved by managing latency or delay, delay variation or jitter, bandwidth, and/or packet loss parameters by a network. For example, a long delay may cause a noticeable echo, delay variations may cause strange sound effects and packet loss may cause interruptions in a conversation or in the run of a video clip, etc.
The picture is even more complicated when a network topology and/or node connectivity change in the course of time. One example of such networks is so-called ad-hoc networks. Ad hoc networks comprise a plurality of (mobile) nodes coupled by wireless links forming arbitrary time- varying wireless network topologies. Nodes of an ad hoc wireless network may move randomly and self-organize themselves in an arbitrary fashion.
With reference to Figure 1, we examine an example of a communications system to which embodiments of the
invention may be applied. The exemplary communications system used herein is the Universal Mobile
Telecommunications System (UMTS) radio access network (UTRAN) . It is a radio access network which includes wideband code division multiple access (WCDMA) technology and can also offer real-time circuit and packet switched services. The embodiments are not, however, restricted to the systems given as examples but a person skilled in the art may apply the solution to other communication systems. Figure 1 depicts examples of simplified system
architectures only showing some elements and functional entities, all being logical units, whose implementation may differ from what is shown. The connections shown in Figure 1 are logical connections; the actual physical connections may be different. It is apparent to a person skilled in the art that the system typically comprises also other functions and structures than those shown in Figure 1.
The embodiments are not, however, restricted to the system given as an example but a person skilled in the art may apply the solution to other communication systems provided with the necessary properties. Some examples of other options for suitable systems are groupe special mobile or global system for mobile communications (GSM) , time division synchronous code division multiple access (TD- SCDMA) , long term evolution (LTE, the same as E-UTRA) , long term evolution advanced (LTE-Advanced) wireless local area network (WLAN or WiFi) , worldwide interoperability for microwave access (WiMAX) , Bluetooth®, personal
communications services (PCS) , ZigBee®, wideband code division multiple access (WCDMA) , systems using ultra- wideband (UWB) technology, sensor networks, mobile ad-hoc networks (MANETs) and Internet Protocol multimedia
subsystems (IMS) .
Figure 1 is a simplified illustration of a communications system or cooperation of a plurality of communications systems to which embodiments according to the invention are applicable. Figure 1 shows parts of two cellular radio systems which comprise radio network elements 100, 118 into which functionalities of a base station and a radio network controller are integrated. The communications network may include separate base stations and radio network controllers, but in this example the part of the network is depicted as its simplest.
The radio network controller 100 has bi-directional radio links 102 and 104 to user devices 106 and 108 and the radio network controller 118 has bi-directional radio links 120, 122 to user devices 124, 126. The user devices may be fixed, vehicle-mounted or portable. The radio network element includes transceivers, for instance. From the transceivers of the radio network element a connection is provided to an antenna unit that establishes bi¬ directional radio links to the user devices.
The radio network element 100 is further connected to a core network 110 (CN) and the radio network element 118 is further connected to a core network 116. The counterpart on the CN side can be a media gateway (MGW) or a serving GPRS (General Packet Radio Service) support node (SGSN) , a gateway GPRS support node (GGSN) , where GPRS is an
abbreviation of a general packet radio service, etc. The GPRS Core Network provides mobility management, session management and transport for Internet Protocol packet services in Global System for Mobile Communications (GSM) and Universal Mobile Telecommunications System (UMTS) networks .
GPRS Tunneling Protocol (GTP) is an Internet protocol of the GPRS core network. It enables a user device to
maintain Internet connections while moving. It conveys subscriber data from the subscriber's current SGSN to the GGSN which is handling the session.
The communication systems are also able to communicate with other networks, such as the Internet 112.
As to other possible systems, such as the GSM, the network topology may differ from what is depicted in Figure 1. As an example, a counterpart on the CN side in the GSM may be a mobile switching centre (MSC) that is a service delivery node for the GSM (and also for the code division multiple access systems) which is responsible for routing voice calls as well as other services. The MSC sets up and releases end-to-end connections, handles mobility and handovers, etc.
In the example, two communications networks are in
cooperation with each other and operationally linked for this reason by a correlation server (CS) 114. The
correlation server may be a stand-alone network element or it may operationally or even physically be integrated in one of the core networks serving the communications networks which in this case may be thought as some kind of sub-networks in the end-user's point of view, etc. Arrow 128 depicts that a correlation server may be coupled to an external network (not shown) . The external network may be any kind of communication network.
In the following, some embodiments are disclosed in further details in relation to Figure 2. The embodiment of Figure 2 is usually related to a server or host that may operate as a correlation server.
The expectations of end-users regarding the quality of service are becoming higher. The change of customer habits introduces more demands for the availability of services in communications networks. In order to meet these
expectations, operators need tools for evaluating or verifying the quality of service and/or user experience each end user is experiencing. One challenge is to obtain reliable real-time information about a user experience when the service used deploys multiple networks, such as a radio network and core network. In such a case, typically, information needs to be searched from multiple data bases and sources which typically results in an uncorrelated real-time view of a user experience. Unfortunately, this, in many cases, also needs manual work for matching data records together. Additionally, it is possible that some information, which is available in a core network, is not available in a radio network leading to insufficient data for end user level troubleshooting.
An embodiment of the invention relates to correlation of end-user experiences in computer and/or communication networks. One target is to enrich information on end-user experiences in a centralized manner for obtaining improved views of them. The processing of information may be carried out in real-time. Term "real-time" may in this application refer to a processing delay which is
adequately unnoticeable in an end-user's point of view. An acceptable delay may vary according to the service at issue. Typical delay-critical services are voice calls, conventional videoconferences and telepresence. It is also possible that protocols used determine limits to
acceptable end-to-end or other kinds of delays in which case these limits may be applied.
The embodiment starts in block 200. The embodiment is especially suitable to be carried out by a correlation server or a corresponding element.
In block 202, signaling event information is
obtained. The signaling event information is gathered in a plurality of networks.
Signaling event information may be typical signaling event information (SEI) used in telecommunication networks collected by network elements (NEs) , signaling transfer points (STPs) and/or signaling link probes (SLPs) , and conveyed to a correlation server (CS) typically in real¬ time .
According to one definition, a signaling event (SE) takes place when a network element or an STP
sends/transmits/receives/forwards a signaling message (SM) , or when an SLP captures a signaling message in a signaling link it monitors. A signaling message may be according to any signaling protocol used in communication networks (signaling system 7 (SS7) that is a telephone signaling protocol or Internet Protocol (IP), for
example) .
Signaling event information (SEI) is usually information with regard to a signaling event. SEI may be any piece of information or a full information element (IE) inside a signaling message. SEI may for example be a subscriber identity, such as a mobile station international ISDN
(integrated services digital network number) (MSISDN) or international mobile subscriber identity (IMSI), network entity, such as a location area code (LAC) or a cell Id (CID) , or a cause code specified by 3rd Generation
partnership project (3GPP) .
Typically, an end-user is served by utilizing cooperation of a plurality of networks, such as a wireless network and core network. In this application term "network" may mean one network in the sense that it operates according to one radio or other protocol, a combination of several networks established for providing services to users, such as a network which comprises a "basic" network and one or more ad hoc networks operating under the control of the "basic" network, a combination of networks operating under the control of different operators with negotiated rules for cooperation control, etc. The interpretation of the term "network" is emerging along with the progress of the information society and thus it has to be interpreted broadly both in the real-world and as an abstract concept. One example of a possible arrangement for the cooperation of a plurality networks is shown in Figure 1. A
correlation server is shown as an option by reference number 114.
In block 204, signaling events are determined from the signaling event information. A correlation server may store signaling information (in real-time) into a database and/or internal data structure for processing.
Signaling and thus signaling events are in relation to actions needed for providing services to users. Signaling is needed for establishing a call, establishing a tunnel for data transfer, providing security and encryption, carrying out handovers, obtaining location information, etc. Communications networks are typically quite
complicated from this viewpoint.
Each event concerning signaling is called a signaling event. Usually, several signaling events which may use different protocols, take place on different layers and between different network elements are required for providing a service, such as a call, for an end-user. For possible fault detection, it is common that many signaling events involved in the faulty service must be searched for. Thus, a need for determining signaling events from signaling event information exists. Typically, signaling event information is recognized and then unpacked for this determination .
As put forward above, according to one definition, a signaling event (SE) takes place when a network element or a STP sends/transmits/receives/forwards a signaling message (SM) , or when an SLP captures a signaling message in a signaling link it monitors. A signaling message may be according to any signaling protocol used in
communication networks (signaling system 7 (SS7) that is a telephone signaling protocol or Internet Protocol (IP) for example) .
In block 206, the signaling events are chosen for combining and the chosen signaling events are combined for obtaining information on user experience in the plurality of networks. A correlation server may use correlation as means for choosing suitable ones from multiple signaling events (SE) according to defined correlation rules (CRs) . A
correlation rule may be defined as one or more pieces of signaling event information that need to match between two different signaling events. Further, the correlation server may create combined information (CI) from signaling events that have been correlated in real-time according to information combination rules (ICRs) . ICR may be defined as signaling events that have been correlated. CI is thus typically a subset of all SEIs from correlated SEs.
In an exemplifying case, a call is controlled in a mobile network by a switch, such as a mobile switching server (MSS) , and the call is conveyed as a VoIP-transmission between media gateway (MGW) elements in an Internet
Protocol ( IP) -network . In this case, the MSS is
responsible for establishing and finishing the call as well as collecting location information and the MGW of the network, where the call is established, transfers voice and/or sound into a suitable format for the IP-network and takes care of routing. The quality of the call may only be evaluated by analyzing the quality of IP-traffic via the MGW elements by using one or more methods usually used for this purpose. The switch (MSS) cannot provide necessary parameters, since it does not see IP-traffic.
Thus by choosing and combining signaling events, it is possible to enhance the observation of the quality of service. In the exemplary case, information on
establishing the call and user' s location is combined with information on the quality of service obtained in reality. Enriched user information may be accomplished in such a manner that peaces of information (which typically are in the form of reports or records) are combined, typically in real-time, and stored in a database. Thus an end-user report comprises these pieces of information instead of a plurality of separate reports that may even be stored in different databases in different networks.
For obtaining useful and enriching combination of
information, it is relevant to choose right pieces of information for combining. Thus pieces of information may be correlated or chosen by using an appropriate criterion, usually a common denominator. In the exemplifying case, the common denominator may be subscriber identification or a global call reference number, for example. In this application, the criterion is called a correlation rule. If put in a simplified manner, it may be said that first the pieces of information are correlated to find suitable ones from the information space, and then combined in a desired way to produce the kind of end-user reports which are useful in evaluating end-user experience.
A correlation rule or criterion (or rules or criteria) may, for example, be at least one of the following: one or more parameters identifying the end-user, one or more parameters indentifying elements used for providing the service and one or more parameters identifying the service and/or service status in question.
It should be understood that information obtained may also be used for statistical analysis, if collected within a longer period of time and not only for real-time use.
Therefore is it possible for operators to identify
possible bottlenecks in their network.
The correlation server may convey combined information in real-time to one or more external systems, such as to one or more external network being responsible of network control functionalities in the networks the signaling event information is gathered from.
The end-user experience information may thus be enriched in a centralized manner. Some examples of signaling event information to be enriched are control plane experience information of a mobile switching center (MSC) server and user plane experience information of a multimedia gateway. Another example is enriching radio network control (RNC) data of a wide band code division multiple access (WCDMA) radio network with a mobile type international mobile equipment identity (IMEI) information, which is available to an MSC server but not to an RNC.
The embodiment ends in block 208. The embodiment is repeatable in many ways. One example is shown by arrow 210 in Figure 2.
The steps/points, signaling messages and related functions described above in Figure 2 are in no absolute
chronological order, and some of the steps/points may be performed simultaneously or in an order differing from the given one. Other functions can also be executed between the steps/points or within the steps/points and other signaling messages sent between the illustrated messages. Some of the steps/points or part of the steps/points can also be left out or replaced by a corresponding step/point or part of the step/point.
It should be understood that conveying, transmitting and/or receiving may herein mean preparing a data
conveyance, transmission and/or reception, preparing a message to be conveyed, transmitted and/or received, or physical transmission and/or reception itself, etc on a case by case basis.
An embodiment provides an apparatus which may be any node, host, server, user device or any other suitable apparatus capable to carry our processes described above in relation to Figure 2.
Figure 3 illustrates a simplified block diagram of an apparatus according to an embodiment especially suitable for network control.
As an example of an apparatus according to an embodiment, it is shown an apparatus 300, such as a node device, host or server, including facilities in a control unit 304 (including one or more processors, for example) to carry out functions of embodiments, such as choose and combine signaling events for obtaining information on user experience in the plurality of networks. This is depicted in Figure 3.
Another example of an apparatus 300 may include at least one processor 304 and at least one memory 302 including a computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to: obtain
signaling event information, the signaling event
information being gathered in a plurality of networks; determine signaling events from the signaling event information, and choose signaling events for combining from the determined signaling events, and combine the chosen signaling events for obtaining information on user experience in the plurality of networks.
Yet another example of an apparatus comprises means 304 for obtaining signalling event information, the signalling event information being gathered in a plurality of networks; means 304 for determining signaling events from the signaling event information, and means 304 for choosing signaling events for combining from the
determined signalling events, and combine the chosen signaling events for obtaining information on user experience in the plurality of networks.
Yet another example of an apparatus comprises an obtainer configured to obtain signaling event information, the signaling event information being gathered in a plurality of networks; a determining unit configured to determining signaling events from the signaling event information, and a chooser configured to choose signaling events for combining from the determined signaling events and combine the chosen signaling events for obtaining information on user experience in the plurality of networks. Although the apparatuses have been depicted as one entity in Figure 3, different modules and memory may be
implemented in one or more physical or logical entities. An apparatus may in general include at least one
processor, controller or a unit designed for carrying out control functions operably coupled to at least one memory unit and to various interfaces. Further, the memory units may include volatile and/or non-volatile memory. The memory unit may store computer program code and/or
operating systems, information, data, content or the like for the processor to perform operations according to embodiments. Each of the memory units may be a random access memory, hard drive, etc. The memory units may be at least partly removable and/or detachably operationally coupled to the apparatus. The memory may be of any type suitable for the current technical environment and it may be implemented using any suitable data storage technology, such as semiconductor-based technology, flash memory, magnetic and/or optical memory devices. The memory may be fixed or removable.
The apparatus may be a software application, or a module, or a unit configured as arithmetic operation, or as a program (including an added or updated software routine) , executed by an operation processor. Programs, also called program products or computer programs, including software routines, applets and macros, can be stored in any
apparatus-readable data storage medium and they include program instructions to perform particular tasks. Computer programs may be coded by a programming language, which may be a high-level programming language, such as objective-C, C, C++, Java, etc., or a low-level programming language, such as a machine language, or an assembler.
Modifications and configurations required for implementing functionality of an embodiment may be performed as
routines, which may be implemented as added or updated software routines, application circuits (ASIC) and/or programmable circuits. Further, software routines may be downloaded into an apparatus. The apparatus, such as a node device, or a corresponding component, may be
configured as a computer or a microprocessor, such as single-chip computer element, or as a chipset, including at least a memory for providing storage capacity used for arithmetic operation and an operation processor for executing the arithmetic operation.
Embodiments provide computer programs embodied on a distribution medium, comprising program instructions which, when loaded into electronic apparatuses, constitute the apparatuses as explained above.
Other embodiments provide computer programs embodied on a computer readable medium, configured to control a
processor to perform embodiments of the methods described above .
The computer program may be in source code form, object code form, or in some intermediate form, and it may be stored in some sort of carrier, distribution medium, or computer readable medium, which may be any entity or device capable of carrying the program. Such carriers include a record medium, computer memory, read-only memory, electrical carrier signal, telecommunications signal, and software distribution package, for example. Depending on the processing power needed, the computer program may be executed in a single electronic digital computer or it may be distributed amongst a number of computers .
The techniques described herein may be implemented by various means. For example, these techniques may be implemented in hardware (one or more devices) , firmware (one or more devices) , software (one or more modules) , or combinations thereof. For a hardware implementation, the apparatus may be implemented within one or more application specific integrated circuits (ASICs) , digital signal processors (DSPs) , digital signal processing devices (DSPDs) , programmable logic devices (PLDs) , field programmable gate arrays (FPGAs) , processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described herein, or a combination thereof. For firmware or software, the
implementation can be carried out through modules of at least one chip set (e.g., procedures, functions, and so on) that perform the functions described herein. The software codes may be stored in a memory unit and executed by processors. The memory unit may be implemented within the processor or externally to the processor. In the latter case it can be communicatively coupled to the processor via various means, as is known in the art.
Additionally, the components of systems described herein may be rearranged and/or complimented by additional components in order to facilitate achieving the various aspects, etc., described with regard thereto, and they are not limited to the precise configurations set forth in the given figures, as will be appreciated by one skilled in the art .
It will be obvious to a person skilled in the art that, as technology advances, the inventive concept may be
implemented in various ways. The invention and its
embodiments are not limited to the examples described above but may vary within the scope of the claims.

Claims

Claims
1. An apparatus comprising:
at least one processor and at least one memory including a computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to:
obtain signaling event information, the signaling event information being gathered in a plurality of
networks ;
determine signaling events from the signaling event information, and
choose signaling events for combining from the determined signaling events and combine the chosen
signaling events for obtaining information on user
experience in the plurality of networks.
2. The apparatus of claim 1, wherein the signaling event information is collected by network elements (NEs) , signaling transfer points (STPs) and/or signaling link probes (SLPs) , and conveyed to a correlation server (CS) .
3. The apparatus of claim 1 or 2, further configured to :
store signaling information into a database and/or internal data structure for processing.
4. The apparatus of any preceding claim, further
configured to:
use correlation for choosing suitable ones from multiple signaling events (SE) according to a predefined
correlation rule (CR) .
5. The apparatus of claim 4, wherein the predefined correlation rule is defined as one or more pieces of signaling event information that need to match between two different signaling events.
6. The apparatus of any preceding claim, wherein the predefined correlation rule is at least one of the
following: one or more parameters identifying the end- user, one or more parameters identifying elements used for providing the service and one or more parameters
identifying the service and/or service status in question.
7. The apparatus of any preceding claim, the apparatus comprising a server, host or node.
8. A computer program comprising program instructions which, when loaded into the apparatus, constitute the modules of any preceding claim 1 to 6.
9. A method comprising:
obtaining signaling event information, the signaling event information being gathered in a plurality of networks;
determining signaling events from the signaling event information, and
choosing signaling events for combining from the
determined signaling events and combine the chosen
signaling events for obtaining information on user
experience in the plurality of networks.
10. The method of claim 9, wherein the signaling event information is collected by network elements (NEs) , signaling transfer points (STPs) and/or signaling link probes (SLPs) , and conveyed to a correlation server (CS) .
11. The method of claim 9 or 10, further comprising:
storing signaling information into a database and/or internal data structure for processing.
12. The method of any preceding claim, further comprising: using correlation for choosing suitable ones from multiple signaling events (SE) according to a predefined
correlation rule (CR) .
13. The method of claim 12, wherein the predefined
correlation rule is defined as one or more pieces of signaling event information that need to match between two different signaling events.
14. The method of any preceding claim, wherein the
predefined correlation rule is at least one of the
following: one or more parameters identifying the end- user, one or more parameters identifying elements used for providing the service and one or more parameters
identifying the service and/or service status in question.
15. An apparatus comprising means for carrying out the method according to any one of claims 9 to 14.
16. A computer program embodied on a computer-readable storage medium, the computer program comprising program code for controlling a process to execute a process, the process comprising:
obtaining signaling event information, the signaling event information being gathered in a plurality of
networks ;
determining signaling events from the signaling event information, and
choosing signaling events for combining from the
determined signaling events and combine the chosen
signaling events for obtaining information on user
experience in the plurality of networks.
17. The computer program of claim 16, wherein the
signaling event information is collected by network elements (NEs) , signaling transfer points (STPs) and/or signaling link probes (SLPs) , and conveyed to a
correlation server (CS) .
18. The computer program of claim 16 or 17, the process further comprising:
storing signaling information into a database and/or internal data structure for processing.
19. The computer program of any preceding claim, the process further comprising:
using correlation for choosing suitable ones from multiple signaling events (SE) according to a predefined
correlation rule (CR) .
20. The computer program of claim 19, wherein the
predefined correlation rule is defined as one or more pieces of signaling event information that need to match between two different signaling events.
21. The computer program of any preceding claim, wherein the predefined correlation rule is at least one of the following: one or more parameters identifying the end- user, one or more parameters identifying elements used for providing the service and one or more parameters
identifying the service and/or service status in question.
PCT/EP2011/055264 2011-04-05 2011-04-05 Assessing network user quality of experience based on correlation of signaling events WO2012136248A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/EP2011/055264 WO2012136248A1 (en) 2011-04-05 2011-04-05 Assessing network user quality of experience based on correlation of signaling events

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2011/055264 WO2012136248A1 (en) 2011-04-05 2011-04-05 Assessing network user quality of experience based on correlation of signaling events

Publications (1)

Publication Number Publication Date
WO2012136248A1 true WO2012136248A1 (en) 2012-10-11

Family

ID=44625674

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2011/055264 WO2012136248A1 (en) 2011-04-05 2011-04-05 Assessing network user quality of experience based on correlation of signaling events

Country Status (1)

Country Link
WO (1) WO2012136248A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104717673A (en) * 2013-12-17 2015-06-17 中兴通讯股份有限公司 Method and device for analyzing dropping rate of terminals
WO2016080808A1 (en) * 2014-11-20 2016-05-26 Samsung Electronics Co., Ltd. Method and device for sharing enriched information associated with a call

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2000074430A2 (en) * 1999-06-02 2000-12-07 Accenture Llp System and method for customer driven qos in a hybrid communication system
WO2001045370A1 (en) * 1999-12-15 2001-06-21 Tekelec Methods and systems for analyzing multi-protocol signaling message traffic in a mobile telecommunications network
EP1215924A2 (en) * 2000-12-18 2002-06-19 Inet Technologies, Inc. System and method for verifying usage and quality of interconnection services for a communication network
US20110188397A1 (en) * 2009-10-16 2011-08-04 Mccann Thomas M Methods, systems, and computer readable media for multi-interface monitoring and correlation of diameter signaling information

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2000074430A2 (en) * 1999-06-02 2000-12-07 Accenture Llp System and method for customer driven qos in a hybrid communication system
WO2001045370A1 (en) * 1999-12-15 2001-06-21 Tekelec Methods and systems for analyzing multi-protocol signaling message traffic in a mobile telecommunications network
EP1215924A2 (en) * 2000-12-18 2002-06-19 Inet Technologies, Inc. System and method for verifying usage and quality of interconnection services for a communication network
US20110188397A1 (en) * 2009-10-16 2011-08-04 Mccann Thomas M Methods, systems, and computer readable media for multi-interface monitoring and correlation of diameter signaling information

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104717673A (en) * 2013-12-17 2015-06-17 中兴通讯股份有限公司 Method and device for analyzing dropping rate of terminals
WO2016080808A1 (en) * 2014-11-20 2016-05-26 Samsung Electronics Co., Ltd. Method and device for sharing enriched information associated with a call
US10356571B2 (en) 2014-11-20 2019-07-16 Samsung Electronics Co., Ltd. Method and device for sharing enriched information associated with a call

Similar Documents

Publication Publication Date Title
US11240702B2 (en) System and method for quality of service in a wireless network environment
US9037710B2 (en) Method and apparatus for correlating end to end measurements through control plane monitoring of wireless traffic
WO2019223498A1 (en) Data processing method, and data transmission method and apparatus
US8423014B2 (en) Method and system for quality of service (QoS) monitoring for wireless devices
EP1487160B1 (en) Service usage records for mobile data communications
US8514723B2 (en) Traffic monitoring by lowest transmission layer marking
US20050163047A1 (en) Method and system for processing quality of service (QOS) performance levels for wireless devices
EP2210370B1 (en) Method and monitoring component for network traffic monitoring
US20070086336A1 (en) Application layer metrics monitoring
US9674728B2 (en) Method and apparatus for managing a degree of parallelism of streams
US10146682B2 (en) Method and apparatus for improving non-uniform memory access
WO2021136636A1 (en) Grouping qos flow for user equipment to user equipment communications
US9270561B2 (en) Method and apparatus for applying uniform hashing to wireless traffic
CN105208605B (en) Link information sending method and device and flow control method and device
KR101533719B1 (en) Realtime data analysis apparatus based on streaming
US8144649B2 (en) Communication control apparatus, wireless communication apparatus, communication control method, and wireless communication method
US8611343B2 (en) Method and apparatus for providing a two-layer architecture for processing wireless traffic
WO2012136248A1 (en) Assessing network user quality of experience based on correlation of signaling events
US7453817B1 (en) Central traffic correlation system
JP7377364B2 (en) Systems and methods for adaptive collection of quality of experience (QoE) measurements
FI123641B (en) Real-time management of packet data service traffic in mobile communications
CN116472702A (en) Data processing node device and information transmission method executed in the same
US8767567B1 (en) Intelligent assignment of wireless communication overhead messaging resources
Panchmatia Intelligent integrated insight into wireless networks

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11712253

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11712253

Country of ref document: EP

Kind code of ref document: A1