WO2018022951A2

WO2018022951A2 - Collection of vnf (virtual network function) performance measurements related to virtualized resources

Info

Publication number: WO2018022951A2
Application number: PCT/US2017/044280
Authority: WO
Inventors: Joey Chou
Original assignee: Intel IP Corporation
Priority date: 2016-07-29
Filing date: 2017-07-28
Publication date: 2018-02-01
Also published as: EP3491783A2; WO2018022951A3

Abstract

Techniques for facilitating PM (Performance Measurement) data notification are discussed. In various aspects, techniques are discussed for: creating PM jobs at a NFVI (Network Function Virtualization Infrastructure) based on a request from a NM (Network Manager), providing an acknowledgement of PM job creation, and reporting available PM data to the NM.

Description

COLLECTION OF VNF (VIRTUAL NETWORK FUNCTION) PERFORMANCE MEASUREMENTS RELATED TO VIRTUALIZED RESOURCES

REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Application No.

62/368,824 filed July 29, 201 6, entitled "SYSTEM AND METHOD TO COLLECT THE VNF PERFORMANCE MEASUREMENTS RELATED TO VIRTUALIZED RESOURCES", the contents of which are herein incorporated by reference in their entirety.

FIELD

[0002] The present disclosure relates to core network technology of a

communication network, and more specifically to techniques associated with collection of VNF (Virtualized Network Function) performance measurements related to virtualized resources.

BACKGROUND

[0003] Network Function Virtualization (NFV) involves the replacement of physical network nodes with Virtual Network Functions (VNFs) implemented via Virtualization Resources (VRs) that perform the same function as the physical node.

BRIEF DESCRIPTION OF THE DRAWINGS

[0004] FIG. 1 is a diagram illustrating an architecture of a system of a network in accordance with some embodiments.

[0005] FIG. 2 is a diagram illustrating components of a core network in accordance with some embodiments.

[0006] FIG. 3 is a block diagram illustrating components, according to some example embodiments, of a system that can support NFV in connection with various aspects discussed herein.

[0007] FIG. 4 is a block diagram illustrating components, according to some example embodiments, able to read instructions from a machine-readable or computer-readable medium (e.g., a non-transitory machine-readable storage medium) and perform any one or more of the methodologies discussed herein. [0008] FIG. 5 is a block diagram of a system employable by a Network Manager (NM) that facilitates data notification in connection with a VNF (Virtual Network

Function) related VR (Virtualization Resource) PM (Performance Measurement), according to various aspects described herein.

[0009] FIG. 6 is a block diagram of a system employable by an Element Manager (EM) that facilitates data notification in connection with a VNF related VR PM, according to various aspects described herein.

[0010] FIG. 7 is a block diagram of a system employable by a Virtual Network

Function Manager (VNFM) that facilitates data notification in connection with a VNF related VR PM, according to various aspects described herein.

[0011] FIG. 8 is a block diagram of a system employable by a Virtualized

Infrastructure Manager (VIM) that facilitates data notification in connection with a VNF related VR PM, according to various aspects described herein.

[0012] FIG. 9 is a block diagram of a system employable by a Network Function

Virtualization Infrastructure (NFVI) that facilitates data notification in connection with a

VNF related VR PM, according to various aspects described herein.

[0013] FIG. 10 is a diagram illustrating an example PM data collection technique for virtualized networks that can be employed in various aspects discussed herein.

[0014] FIG. 11 is a flow diagram of a method that facilitates PM data notification, according to various aspects discussed herein.

[0015] FIG. 12 is a flow diagram of a method that facilitates creation of a VNF related VR PM job and associated data notification by a NM according to various aspects described herein.

[0016] FIG. 13 is a flow diagram of a method that facilitates creation of a VNF related VR PM job and associated data notification by an EM according to various aspects described herein.

[0017] FIG. 14 is a flow diagram of a method that facilitates creation of a VNF related VR PM job and associated data notification by a VNFM according to various aspects described herein.

[0018] FIG. 15 is a flow diagram of a method that facilitates creation of a VNF related VR PM job and associated data notification by a VIM according to various aspects described herein. [0019] FIG. 16 is a flow diagram of a method that facilitates creation of a VNF related VR PM job and associated data notification by a NFVI according to various aspects described herein.

DETAILED DESCRIPTION

[0020] The present disclosure will now be described with reference to the attached drawing figures, wherein like reference numerals are used to refer to like elements throughout, and wherein the illustrated structures and devices are not necessarily drawn to scale. As utilized herein, terms "component," "system," "interface," and the like are intended to refer to a computer-related entity, hardware, software (e.g., in execution), and/or firmware. For example, a component can be a processor (e.g., a microprocessor, a controller, or other processing device), a process running on a processor, a controller, an object, an executable, a program, a storage device, a computer, a tablet PC and/or a user equipment (e.g., mobile phone, etc.) with a processing device. By way of illustration, an application running on a server and the server can also be a component. One or more components can reside within a process, and a component can be localized on one computer and/or distributed between two or more computers. A set of elements or a set of other components can be described herein, in which the term "set" can be interpreted as "one or more."

[0021] Further, these components can execute from various computer readable storage media having various data structures stored thereon such as with a module, for example. The components can communicate via local and/or remote processes such as in accordance with a signal having one or more data packets (e.g., data from one component interacting with another component in a local system, distributed system, and/or across a network, such as, the Internet, a local area network, a wide area network, or similar network with other systems via the signal).

[0022] As another example, a component can be an apparatus with specific functionality provided by mechanical parts operated by electric or electronic circuitry, in which the electric or electronic circuitry can be operated by a software application or a firmware application executed by one or more processors. The one or more processors can be internal or external to the apparatus and can execute at least a part of the software or firmware application. As yet another example, a component can be an apparatus that provides specific functionality through electronic components without mechanical parts; the electronic components can include one or more processors therein to execute software and/or firmware that confer(s), at least in part, the functionality of the electronic components.

[0023] Use of the word exemplary is intended to present concepts in a concrete fashion. As used in this application, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless specified otherwise, or clear from context, "X employs A or B" is intended to mean any of the natural inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, then "X employs A or B" is satisfied under any of the foregoing instances. In addition, the articles "a" and "an" as used in this application and the appended claims should generally be construed to mean "one or more" unless specified otherwise or clear from context to be directed to a singular form. Furthermore, to the extent that the terms "including", "includes", "having", "has", "with", or variants thereof are used in either the detailed description and the claims, such terms are intended to be inclusive in a manner similar to the term

"comprising."

[0024] As used herein, the term "circuitry" may refer to, be part of, or include an Application Specific Integrated Circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or group), and/or memory (shared, dedicated, or group) that execute one or more software or firmware programs, a combinational logic circuit, and/or other suitable hardware components that provide the described functionality. In some embodiments, the circuitry may be implemented in, or functions associated with the circuitry may be implemented by, one or more software or firmware modules. In some embodiments, circuitry may include logic, at least partially operable in hardware.

[0025] Embodiments described herein may be implemented into a system using any suitably configured hardware and/or software. FIG. 1 illustrates an architecture of a system 1 00 of a network in accordance with some embodiments. The system 100 is shown to include a user equipment (UE) 101 and a UE 102. The UEs 101 and 102 are illustrated as smartphones (e.g., handheld touchscreen mobile computing devices connectable to one or more cellular networks), but may also comprise any mobile or non-mobile computing device, such as Personal Data Assistants (PDAs), pagers, laptop computers, desktop computers, wireless handsets, or any computing device including a wireless communications interface.

[0026] In some embodiments, any of the UEs 101 and 102 can comprise an Internet of Things (loT) UE, which can comprise a network access layer designed for low-power loT applications utilizing short-lived UE connections. An loT UE can utilize technologies such as machine-to-machine (M2M) or machine-type communications (MTC) for exchanging data with an MTC server or device via a public land mobile network (PLMN), Proximity-Based Service (ProSe) or device-to-device (D2D) communication, sensor networks, or loT networks. The M2M or MTC exchange of data may be a machine-initiated exchange of data. An loT network describes interconnecting loT UEs, which may include uniquely identifiable embedded computing devices (within the Internet infrastructure), with short-lived connections. The loT UEs may execute background applications (e.g., keep-alive messages, status updates, etc.) to facilitate the connections of the loT network.

[0027] The UEs 101 and 102 may be configured to connect, e.g., communicatively couple, with a radio access network (RAN) 1 10— the RAN 1 10 may be, for example, an Evolved Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access Network (E-UTRAN), a NextGen RAN (NG RAN), or some other type of RAN. The UEs 101 and 102 utilize connections 103 and 104, respectively, each of which comprises a physical communications interface or layer (discussed in further detail below); in this example, the connections 103 and 104 are illustrated as an air interface to enable communicative coupling, and can be consistent with cellular communications protocols, such as a Global System for Mobile Communications (GSM) protocol, a code-division multiple access (CDMA) network protocol, a Push-to-Talk (PTT) protocol, a PTT over Cellular (POC) protocol, a Universal Mobile Telecommunications System (UMTS) protocol, a 3GPP Long Term Evolution (LTE) protocol, a fifth generation (5G) protocol, a New Radio (NR) protocol, and the like.

[0028] In this embodiment, the UEs 101 and 1 02 may further directly exchange communication data via a ProSe interface 105. The ProSe interface 105 may alternatively be referred to as a sidelink interface comprising one or more logical channels, including but not limited to a Physical Sidelink Control Channel (PSCCH), a Physical Sidelink Shared Channel (PSSCH), a Physical Sidelink Discovery Channel (PSDCH), and a Physical Sidelink Broadcast Channel (PSBCH).

[0029] The UE 102 is shown to be configured to access an access point (AP) 106 via connection 107. The connection 107 can comprise a local wireless connection, such as a connection consistent with any IEEE 802.1 1 protocol, wherein the AP 106 would comprise a wireless fidelity (WiFi®) router. In this example, the AP 106 is shown to be connected to the Internet without connecting to the core network of the wireless system (described in further detail below). [0030] The RAN 1 1 0 can include one or more access nodes that enable the connections 1 03 and 104. These access nodes (ANs) can be referred to as base stations (BSs), NodeBs, evolved NodeBs (eNBs), next Generation NodeBs (gNB), RAN nodes, and so forth, and can comprise ground stations (e.g., terrestrial access points) or satellite stations providing coverage within a geographic area (e.g., a cell). The RAN 1 1 0 may include one or more RAN nodes for providing macrocells, e.g., macro RAN node 1 1 1 , and one or more RAN nodes for providing femtocells or picocells (e.g., cells having smaller coverage areas, smaller user capacity, or higher bandwidth compared to macrocells), e.g., low power (LP) RAN node 1 12.

[0031] Any of the RAN nodes 1 1 1 and 1 12 can terminate the air interface protocol and can be the first point of contact for the UEs 101 and 102. In some embodiments, any of the RAN nodes 1 1 1 and 1 12 can fulfill various logical functions for the RAN 1 1 0 including, but not limited to, radio network controller (RNC) functions such as radio bearer management, uplink and downlink dynamic radio resource management and data packet scheduling, and mobility management.

[0032] In accordance with some embodiments, the UEs 101 and 102 can be configured to communicate using Orthogonal Frequency-Division Multiplexing (OFDM) communication signals with each other or with any of the RAN nodes 1 1 1 and 1 1 2 over a multicarrier communication channel in accordance various communication techniques, such as, but not limited to, an Orthogonal Frequency-Division Multiple Access (OFDMA) communication technique (e.g., for downlink communications) or a Single Carrier Frequency Division Multiple Access (SC-FDMA) communication technique (e.g., for uplink and ProSe or sidelink communications), although the scope of the embodiments is not limited in this respect. The OFDM signals can comprise a plurality of orthogonal subcarriers.

[0033] In some embodiments, a downlink resource grid can be used for downlink transmissions from any of the RAN nodes 1 1 1 and 1 12 to the UEs 101 and 1 02, while uplink transmissions can utilize similar techniques. The grid can be a time-frequency grid, called a resource grid or time-frequency resource grid, which is the physical resource in the downlink in each slot. Such a time-frequency plane representation is a common practice for OFDM systems, which makes it intuitive for radio resource allocation. Each column and each row of the resource grid corresponds to one OFDM symbol and one OFDM subcarrier, respectively. The duration of the resource grid in the time domain corresponds to one slot in a radio frame. The smallest time-frequency unit in a resource grid is denoted as a resource element. Each resource grid comprises a number of resource blocks, which describe the mapping of certain physical channels to resource elements. Each resource block comprises a collection of resource elements; in the frequency domain, this may represent the smallest quantity of resources that currently can be allocated. There are several different physical downlink channels that are conveyed using such resource blocks.

[0034] The physical downlink shared channel (PDSCH) may carry user data and higher-layer signaling to the UEs 101 and 102. The physical downlink control channel (PDCCH) may carry information about the transport format and resource allocations related to the PDSCH channel, among other things. It may also inform the UEs 101 and 102 about the transport format, resource allocation, and H-ARQ (Hybrid Automatic Repeat Request) information related to the uplink shared channel. Typically, downlink scheduling (assigning control and shared channel resource blocks to the UE 102 within a cell) may be performed at any of the RAN nodes 1 1 1 and 1 12 based on channel quality information fed back from any of the UEs 101 and 102. The downlink resource assignment information may be sent on the PDCCH used for (e.g., assigned to) each of the UEs 101 and 1 02.

[0035] The PDCCH may use control channel elements (CCEs) to convey the control information. Before being mapped to resource elements, the PDCCH complex-valued symbols may first be organized into quadruplets, which may then be permuted using a sub-block interleaver for rate matching. Each PDCCH may be transmitted using one or more of these CCEs, where each CCE may correspond to nine sets of four physical resource elements known as resource element groups (REGs). Four Quadrature Phase Shift Keying (QPSK) symbols may be mapped to each REG. The PDCCH can be transmitted using one or more CCEs, depending on the size of the downlink control information (DCI) and the channel condition. There can be four or more different PDCCH formats defined in LTE with different numbers of CCEs (e.g., aggregation level, L=1 , 2, 4, or 8).

[0036] Some embodiments may use concepts for resource allocation for control channel information that are an extension of the above-described concepts. For example, some embodiments may utilize an enhanced physical downlink control channel (EPDCCH) that uses PDSCH resources for control information transmission. The EPDCCH may be transmitted using one or more enhanced the control channel elements (ECCEs). Similar to above, each ECCE may correspond to nine sets of four physical resource elements known as an enhanced resource element groups (EREGs). An ECCE may have other numbers of EREGs in some situations.

[0037] The RAN 1 1 0 is shown to be communicatively coupled to a core network (CN) 1 20— via an S1 interface 1 1 3. In embodiments, the CN 120 may be an evolved packet core (EPC) network, a NextGen Packet Core (NPC) network, or some other type of CN. In this embodiment the S1 interface 1 13 is split into two parts: the S1 -U interface 1 14, which carries traffic data between the RAN nodes 1 1 1 and 1 12 and the serving gateway (S-GW) 122, and the S1 -mobility management entity (MME) interface 1 15, which is a signaling interface between the RAN nodes 1 1 1 and 1 12 and MMEs 121 .

[0038] In this embodiment, the CN 1 20 comprises the MMEs 121 , the S-GW 122, the Packet Data Network (PDN) Gateway (P-GW) 123, and a home subscriber server (HSS) 124. The MMEs 121 may be similar in function to the control plane of legacy Serving General Packet Radio Service (GPRS) Support Nodes (SGSN). The MMEs

121 may manage mobility aspects in access such as gateway selection and tracking area list management. The HSS 1 24 may comprise a database for network users, including subscription-related information to support the network entities' handling of communication sessions. The CN 120 may comprise one or several HSSs 124, depending on the number of mobile subscribers, on the capacity of the equipment, on the organization of the network, etc. For example, the HSS 124 can provide support for routing/roaming, authentication, authorization, naming/addressing resolution, location dependencies, etc.

[0039] The S-GW 122 may terminate the S1 interface 1 13 towards the RAN 1 1 0, and routes data packets between the RAN 1 10 and the CN 120. In addition, the S-GW

122 may be a local mobility anchor point for inter-RAN node handovers and also may provide an anchor for inter-3GPP mobility. Other responsibilities may include lawful intercept, charging, and some policy enforcement.

[0040] The P-GW 123 may terminate an SGi interface toward a PDN. The P-GW

123 may route data packets between the EPC network 123 and external networks such as a network including the application server 130 (alternatively referred to as application function (AF)) via an Internet Protocol (IP) interface 125. Generally, the application server 130 may be an element offering applications that use IP bearer resources with the core network (e.g., UMTS Packet Services (PS) domain, LTE PS data services, etc.). In this embodiment, the P-GW 123 is shown to be communicatively coupled to an application server 130 via an IP communications interface 125. The application server 130 can also be configured to support one or more communication services (e.g., Voice-over-Internet Protocol (VoIP) sessions, PTT sessions, group communication sessions, social networking services, etc.) for the UEs 101 and 102 via the CN 120.

[0041] The P-GW 123 may further be a node for policy enforcement and charging data collection. Policy and Charging Enforcement Function (PCRF) 126 is the policy and charging control element of the CN 120. In a non-roaming scenario, there may be a single PCRF in the Home Public Land Mobile Network (HPLMN) associated with a UE's Internet Protocol Connectivity Access Network (IP-CAN) session. In a roaming scenario with local breakout of traffic, there may be two PCRFs associated with a UE's IP-CAN session: a Home PCRF (H-PCRF) within a HPLMN and a Visited PCRF (V- PCRF) within a Visited Public Land Mobile Network (VPLMN). The PCRF 126 may be communicatively coupled to the application server 130 via the P-GW 123. The application server 130 may signal the PCRF 126 to indicate a new service flow and select the appropriate Quality of Service (QoS) and charging parameters. The PCRF 126 may provision this rule into a Policy and Charging Enforcement Function (PCEF) (not shown) with the appropriate traffic flow template (TFT) and QoS class of identifier (QCI), which commences the QoS and charging as specified by the application server 130.

[0042] FIG. 2 illustrates components of a core network in accordance with some embodiments. The components of the CN 1 20 may be implemented in one physical node or separate physical nodes including components to read and execute instructions from a machine-readable or computer-readable medium (e.g., a non-transitory machine- readable storage medium). In some embodiments, Network Functions Virtualization (NFV) is utilized to virtualize any or all of the above described network node functions via executable instructions stored in one or more computer readable storage mediums (described in further detail below). A logical instantiation of the CN 120 may be referred to as a network slice 201 . A logical instantiation of a portion of the CN 120 may be referred to as a network sub-slice 202 (e.g., the network sub-slice 202 is shown to include the PGW 123 and the PCRF 126).

[0043] NFV architectures and infrastructures may be used to virtualize one or more network functions, alternatively performed by proprietary hardware, onto physical resources comprising a combination of industry-standard server hardware, storage hardware, or switches. In other words, NFV systems can be used to execute virtual or reconfigurable implementations of one or more EPC components/functions. [0044] FIG. 3 is a block diagram illustrating components, according to some example embodiments, of a system 300 to support NFV. The system 300 is illustrated as including a virtualized infrastructure manager (VIM) 302, a network function

virtualization infrastructure (NFVI) 304, a VNF manager (VNFM) 306, virtualized network functions (VNFs) 308, an element manager (EM) 310, an NFV Orchestrator (NFVO) 31 2, and a network manager (NM) 314.

[0045] The VIM 302 manages the resources of the NFVI 304. The NFVI 304 can include physical or virtual resources and applications (including hypervisors) used to execute the system 300. The VIM 302 may manage the life cycle of virtual resources with the NFVI 304 (e.g., creation, maintenance, and tear down of virtual machines (VMs) associated with one or more physical resources), track VM instances, track performance, fault and security of VM instances and associated physical resources, and expose VM instances and associated physical resources to other management systems.

[0046] The VNFM 306 may manage the VNFs 308. The VNFs 308 may be used to execute EPC components/functions. The VNFM 306 may manage the life cycle of the VNFs 308 and track performance, fault and security of the virtual aspects of VNFs 308. The EM 310 may track the performance, fault and security of the functional aspects of VNFs 308. The tracking data from the VNFM 306 and the EM 310 may comprise, for example, performance measurement (PM) data used by the VIM 302 or the NFVI 304. Both the VNFM 306 and the EM 310 can scale up/down the quantity of VNFs of the system 300.

[0047] The NFVO 312 may coordinate, authorize, release and engage resources of the NFVI 304 in order to provide the requested service (e.g., to execute an EPC function, component, or slice). The NM 314 may provide a package of end-user functions with the responsibility for the management of a network, which may include network elements with VNFs, non-virtualized network functions, or both (management of the VNFs may occur via the EM 310).

[0048] FIG. 4 is a block diagram illustrating components, according to some example embodiments, able to read instructions from a machine-readable or computer-readable medium (e.g., a non-transitory machine-readable storage medium) and perform any one or more of the methodologies discussed herein. Specifically, FIG. 4 shows a diagrammatic representation of hardware resources 400 including one or more processors (or processor cores) 410, one or more memory/storage devices 420, and one or more communication resources 430, each of which may be communicatively coupled via a bus 440. For embodiments where node virtualization (e.g., NFV) is utilized, a hypervisor 402 may be executed to provide an execution environment for one or more network slices/sub-slices to utilize the hardware resources 400.

[0049] The processors 410 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP) such as a baseband processor, an application specific integrated circuit (ASIC), a radio-frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 412 and a processor 414.

[0050] The memory/storage devices 420 may include main memory, disk storage, or any suitable combination thereof. The memory/storage devices 420 may include, but are not limited to any type of volatile or non-volatile memory such as dynamic random access memory (DRAM), static random-access memory (SRAM), erasable

programmable read-only memory (EPROM), electrically erasable programmable readonly memory (EEPROM), Flash memory, solid-state storage, etc.

[0051] The communication resources 430 may include interconnection or network interface components or other suitable devices to communicate with one or more peripheral devices 404 or one or more databases 406 via a network 408. For example, the communication resources 430 may include wired communication components (e.g., for coupling via a Universal Serial Bus (USB)), cellular communication components, NFC components, Bluetooth® components (e.g., Bluetooth® Low Energy), Wi-Fi® components, and other communication components.

[0052] Instructions 450 may comprise software, a program, an application, an applet, an app, or other executable code for causing at least any of the processors 410 to perform any one or more of the methodologies discussed herein. The instructions 450 may reside, completely or partially, within at least one of the processors 410 (e.g., within the processor's cache memory), the memory/storage devices 420, or any suitable combination thereof. Furthermore, any portion of the instructions 450 may be transferred to the hardware resources 400 from any combination of the peripheral devices 404 or the databases 406. Accordingly, the memory of processors 410, the memory/storage devices 420, the peripheral devices 404, and the databases 406 are examples of computer-readable and machine-readable media. [0053] In various embodiments discussed herein, techniques can be employed to facilitate collection of VNF (Virtualized Network Function) performance measurements related to virtualized resources.

[0054] NFV (Network Function Virtualization) migrates the execution of network functions from vertically integrated hardware to industry standard COTS (commercial of- the-shelf) servers in data center(s). Performance measurements that are independent of the migration (e.g. handover, tracking area update, and related measurements) are not impacted. Therefore, these performance measurements can be reused as VNF performance measurements. Other performance measurements that measure hardware usage (e.g., MME processor usage, or performance measurements that are tightly coupled to the hardware performance (e.g. data volume, GTP, etc. related

measurements)) are impacted. As the result, Virtualized Resource (VR) performance measurements can be employed to ensure that the VNFs (Virtualized Network

Functions) deployed on the NFV infrastructure can deliver a consistent and acceptable service quality to end users, as well as to isolate and correct failure conditions in a timely manner. These performance measurements can reflect the way VNFs are impacted by the NFVI services, and the inherent nature of the services being offered by the NFVI, for example, CPU, Virtual Machines, memory, and Virtual Networks.

[0055] In various aspects, performance measurements defined herein and associated techniques can be employed to allow 3GPP (Third Generation Partnership Project) operators to monitor the VNF performance impacted by the Virtualized

Resources (VR) (e.g., computing, storage, and network) in the NFV infrastructure (NFVI). In various aspects, techniques discussed herein can be employed to translate these performance measurements into VR performance measurements that can be implemented in VIM (Virtualized Infrastructure Manager) to monitor the performance of computing, storage, and/or networking resources in a NFVI. Additionally, techniques are discussed herein that can facilitate collection of these VR performance measurements.

[0056] Referring to FIG. 5, illustrated is a block diagram of a system 500 employable by a Network Manager (NM) that facilitates data notification in connection with a VNF related VR PM, according to various aspects described herein. System 500 can be implemented by NM 314 and can comprise one or more processors 51 0 (e.g., which can comprise one or more of processor(s) 410, for example, each of which can include processing circuitry and associated memory interface(s) (e.g., an interface to

send/receive data to/from memory external to the processing circuitry, etc.)), communication circuitry 520 (which can facilitate communication of data via one or more reference points, networks, etc., and can comprise communication resource(s) 430, etc.), and memory 530 (which can comprise any of a variety of storage mediums and can store instructions and/or data associated with at least one of the one or more processors 51 0 or communication circuitry 520, and can comprise memory/storage device(s) 420 and/or cache memory of processor(s) 510, etc.). In some aspects, the one or more processors 510, the communication circuitry 520, and the memory 530 can be included in a single device, while in other aspects, they can be included in different devices, such as part of a distributed architecture. As described in greater detail below, system 500 can be employed by a NM (e.g., NM 314) to perform one or more operations (e.g., those described herein in connection with FIGS. 1 1 -12, NM 314, etc.) that can facilitate creation of a VNF related VR PM job and associated data notification.

[0057] Referring to FIG. 6, illustrated is a block diagram of a system 600 employable by an Element Manager (EM) that facilitates data notification in connection with a VNF related VR PM, according to various aspects described herein. System 600 can be implemented by EM 310 and can comprise one or more processors 610 (e.g., which can comprise one or more of processor(s) 410, for example, each of which can include processing circuitry and associated memory interface(s) (e.g., an interface to send/receive data to/from memory external to the processing circuitry, etc.)), communication circuitry 620 (which can facilitate communication of data via one or more reference points, networks, etc., and can comprise communication resource(s) 430, etc.), and memory 630 (which can comprise any of a variety of storage mediums and can store instructions and/or data associated with at least one of the one or more processors 61 0 or communication circuitry 620, and can comprise memory/storage device(s) 420 and/or cache memory of processor(s) 610, etc.). In some aspects, the one or more processors 610, the communication circuitry 620, and the memory 630 can be included in a single device, while in other aspects, they can be included in different devices, such as part of a distributed architecture. As described in greater detail below, system 500 can be employed by an EM (e.g., EM 310) to perform one or more operations (e.g., those described herein in connection with FIGS. 1 1 and 13, EM 310, etc.) that can facilitate creation of a VNF related VR PM job and associated data notification.

[0058] Referring to FIG. 7, illustrated is a block diagram of a system 700 employable by a Virtual Network Function Manager (VNFM) that facilitates data notification in connection with a VNF related VR PM, according to various aspects described herein. System 700 can be implemented by VNFM 306 and can comprise one or more processors 71 0 (e.g., which can comprise one or more of processor(s) 410, for example, each of which can include processing circuitry and associated memory interface(s) (e.g., an interface to send/receive data to/from memory external to the processing circuitry, etc.)), communication circuitry 720 (which can facilitate

communication of data via one or more reference points, networks, etc., and can comprise communication resource(s) 430, etc.), and memory 730 (which can comprise any of a variety of storage mediums and can store instructions and/or data associated with at least one of the one or more processors 71 0 or communication circuitry 720, and can comprise memory/storage device(s) 420 and/or cache memory of processor(s) 710, etc.). In some aspects, the one or more processors 71 0, the communication circuitry 720, and the memory 730 can be included in a single device, while in other aspects, they can be included in different devices, such as part of a distributed architecture. As described in greater detail below, system 700 can be employed by a VNFM (e.g., VNFM 306) to perform one or more operations (e.g., those described herein in connection with FIGS. 1 1 and 14, VNFM 306, etc.) that can facilitate creation of a VNF related VR PM job and associated data notification.

[0059] Referring to FIG. 8, illustrated is a block diagram of a system 800 employable by a Virtualized Infrastructure Manager (VIM) that facilitates data notification in connection with a VNF related VR PM, according to various aspects described herein. System 800 can be implemented by VIM 302 and can comprise one or more processors 81 0 (e.g., which can comprise one or more of processor(s) 410, for example, each of which can include processing circuitry and associated memory interface(s) (e.g., an interface to send/receive data to/from memory external to the processing circuitry, etc.)), communication circuitry 820 (which can facilitate communication of data via one or more reference points, networks, etc., and can comprise communication resource(s) 430, etc.), and memory 830 (which can comprise any of a variety of storage mediums and can store instructions and/or data associated with at least one of the one or more processors 81 0 or communication circuitry 820, and can comprise memory/storage device(s) 420 and/or cache memory of processor(s) 810, etc.). In some aspects, the one or more processors 810, the communication circuitry 820, and the memory 830 can be included in a single device, while in other aspects, they can be included in different devices, such as part of a distributed architecture. As described in greater detail below, system 800 can be employed by a VIM (e.g., VIM 302) to perform one or more operations (e.g., those described herein in connection with FIGS. 1 1 and 15, VIM 302, etc.) that can facilitate creation of a VNF related VR PM job and associated data notification.

[0060] Referring to FIG. 9, illustrated is a block diagram of a system 900 employable by a Network Function Virtualization Infrastructure (NFVI) that facilitates data

notification in connection with a VNF related VR PM, according to various aspects described herein. System 900 can be implemented by NFVI 304 and can comprise one or more processors 91 0 (e.g., which can comprise one or more of processor(s) 410, for example, each of which can include processing circuitry and associated memory interface(s) (e.g., an interface to send/receive data to/from memory external to the processing circuitry, etc.)), communication circuitry 920 (which can facilitate

communication of data via one or more reference points, networks, etc., and can comprise communication resource(s) 430, etc.), and memory 930 (which can comprise any of a variety of storage mediums and can store instructions and/or data associated with at least one of the one or more processors 91 0 or communication circuitry 920, and can comprise memory/storage device(s) 420 and/or cache memory of processor(s) 910, etc.). In some aspects, the one or more processors 91 0, the communication circuitry 920, and the memory 930 can be included in a single device, while in other aspects, they can be included in different devices, such as part of a distributed architecture. As described in greater detail below, system 900 can be employed by a NFVI to perform one or more operations (e.g., those described herein in connection with FIGS. 1 1 and 16, NFVI 304, etc.) that can facilitate creation of a VNF related VR PM job and associated data notification.

[0061] Referring to FIG. 10, illustrated is a diagram of an example PM data collection technique for virtualized networks that can be employed in various aspects discussed herein. FIG. 10 shows PM data collection in connection with a system such as system 300 that supports NFV. As shown in FIG. 10, the various illustrated components can communicate (send and/or receive) with one another via various reference points. VIM 302 can communication with NFVI 304 via a Nf-Vi reference point, with VNFM 306 via a Vi-Vnfm reference point, and with NFVO 312 via a Or-Vi reference point. NFVI 304 can communicate with VIM 302 via the Nf-Vi reference point and with VNF instances 308, via a Vn-Nf reference point. VNFM 306 can communicate with VIM 302 via the Vi-Vnfm reference point, with VNF instances 308, via a Ve-Vnfm-vnf reference point, with EM 31 0 via a Ve-Vnfm-em reference point, and with NFVO 312 via an Or-Vnfm reference point. EM 310 can communicate with VNFM 306 via the Ve-Vnfm-em reference point, with NM 314 via a Itf-N reference point, and with VNF instances 308,. NFVO 312 can communicate with VIM 302 via the Or-Vi reference point, with VNFM 306 via the Or- Vnfm reference point, and with NM 314 via a Os-Ma-nfvo reference point. NM 314 can communicate with EM 31 0 via the Itf-N reference point and with NFVO 312 via the Os- Ma-nfvo reference point.

[0062] In the example scenario of FIG. 10, when PM data is to be collected, a NM (e.g., NM 314) can create a PM job at an EM (e.g., EM 310, wherein this PM job creation can comprise processor(s) 510 creating a request to create a PM job, sending the request via communication circuitry 520 via the Itf-N reference point, with the EM receiving the request via communication circuitry 620, and processor(s) 610 creating the PM job based on the request), so as to determine which measurement types, on which measured resources, at which times, are to be executed. For the NM (e.g., NM 314) is to collect the VNF related Virtualized Resource (VR) PM data, the EM (e.g., EM 31 0) can create a PM job at the VNFM (e.g., VNFM 306, wherein this PM job creation can comprise processor(s) 610 creating a request to create a PM job, sending the request via communication circuitry 620 via the Ve-Vnfm-em reference point, with the VNFM receiving the request via communication circuitry 720, and processor(s) 710 creating the PM job based on the request) that contains the measurement types, and the periods for which the collection is to be performed. Then, the VNFM (e.g., VNFM 306) can create a PM job at a VIM (e.g., VIM 302, wherein this PM job creation can comprise processor(s) 71 0 creating a request to create a PM job, sending the request via communication circuitry 720 via the Vi-Vnfm reference point, with the VIM receiving the request via communication circuitry 820, and processor(s) 810 creating the PM job based on the request) based on the information received from the EM (e.g., EM 310). The VIM (e.g., VIM 302) can configure the Ceilometer (or other data collection service 1002) in the NFVI (e.g., NFVI 304) to collect the VR PM data at the schedule and time period defined in the PM job (e.g., via processor(s) 810 creating a configuration file associated with the PM job, sending the configuration file via communication circuitry 820 via the Nf-Vi reference point, with the NFVI receiving the configuration file via communication circuitry 920, and processor(s) 910 configuring the data collection service 1 002 (e.g., Ceilometer, etc.) based on the configuration file). In such a scenario, the VIM (e.g., VIM 302) can act as a controller (e.g., an Openstack controller) to configure the Ceilometer (or other data collection service 1002) to collect the VR performance measurements. The Ceilometer (or other data collection service 1002) can report the VR PM data to the VIM (e.g., VIM 302) after it is collected (e.g., wherein the VR PM data can be collected by processor(s) 910, sent via communication circuitry 920 over the Nf-Vi reference point, received via communication circuitry 820, and processed by processor(s) 81 0). The VIM can send (e.g., via communication circuitry 820 over the Vi-Vnfm reference point) the data in a first PerformanceReport (Performance Report) information element (e.g., which can be generated by processor(s) 810 and wherein a PerformanceReport information element can define the format of a performance report provided by a producer to a consumer (e.g., associated with VNFM 306) and can comprise one or more performanceReportEntry (performance report entry) information elements that can list performance information entries) to the VNFM (e.g., which can receive the first performanceReport via communication circuitry 720 and process the PerformanceReport via processor(s) 710). The VNFM can send (e.g., via

communication circuitry 720 over the Ve-Vnfm-em reference point) the data in a second PerformanceReport (e.g., generated by processor(s) 710 based at least in part on the first PerformanceReport) information element to the EM (e.g., which can receive the second performanceReport via communication circuitry 620 and process the performanceReport via processor(s) 610). The NM can fetch (e.g., via a request, subscription, etc. generated by processor(s) 510 and sent via communication circuitry 520 over the Itf-N reference point to communication circuitry 620 for processing by processor(s) 610) the VNF PM data related to the VR(s) from the EM (e.g., which can generate the VNF related VR PM data via processor(s) 61 0, send the data via communication circuitry 620 over the Itf-N reference point to communication circuitry 520, for processing by processor(s) 510).

[0063] In various aspects, performance measurements discussed below can be defined at an EM (e.g., comprising 600) to monitor the performance of virtualized resources used by a VNF (e.g., as discussed herein). The EM can generate (e.g., via processor(s) 610) a performance measurement when the EM receives (e.g., via communication circuitry 620 over the Ve-Vnfm-em reference point) a corresponding performance measurement from the VNFM (e.g., generated by processor(s) 71 0 and transmitted by communication circuitry 720 via the Ve-Vnfm-em reference point). The EM can employ (e.g., via processor(s) 610) TF (Transparent Forwarding) techniques discussed herein to forward the performance measurement to the NM. [0064] In aspects discussed herein, Transparent Forwarding (TF) can involve a non- 3GPP (Third Generation Partnership Project) defined NE (Network Element)

maintaining a count based on an "externally defined collection method" of that NE. The 3GPP system can maintain a measurement count that is a snapshot and/or reading of the non-3GPP defined NE count at each granularity period (e.g., which can be defined for a given measurement job, etc.).

VIRTUALIZED RESOURCES

[0065] In various aspects, a VNF related VR PM job can comprise performance measurements associated with any of the virtualized resources discussed herein, which can comprise one or more of measurements for monitoring computing resources, measurements for monitoring storage resources, or data volume measurements for monitoring networking resources.

[0066] Measurements for monitoring computing resources can comprise

measurements of one or more of mean CPU (Central Processing Unit) usage or peak CPU usage.

[0067] Mean CPU usage measurements can have the following characteristics: (a) This measurement can provide the mean CPU usage of a VNF that can be the percentage of the total available CPU processing power; (b) can employ TF

(Transparent Forwarding); (c) this measurement can be created when an EM receives (e.g., via communication circuitry 620) the mean CPU usage measurement from a VNFM (e.g., sent via communication circuitry 720); (d) Each measurement can be an integer value (Unit: %); (e) can have an associated ID (e.g., VR.MeanCpuUsage.VnfID); (f) can be a ManagedElement (managed element); (g) Valid for packet switched; and (h) EPS.

[0068] Peak CPU usage measurements can have the following characteristics: (a) This measurement can provide the peak CPU usage of a VNF that can be the percentage of the total available CPU processing power; (b) can employ TF

(Transparent Forwarding); (c) This measurement can be created when an EM receives (e.g., via communication circuitry 620) the peak CPU usage measurement from VNFM (e.g., sent via communication circuitry 720); (d) Each measurement can be an integer value (Unit: %); (e) can have an associated ID (e.g., VR.PeakCpuUsage.VnfID); (f) can be a ManagedElement; (g) can be valid for packet switched; and (h) EPS. [0069] Measurements for monitoring storage resources can comprise measurements of one or more of mean memory usage, peak memory usage, mean disk usage, or peak disk usage.

[0070] Mean memory usage measurements can have the following characteristics: (a) This measurement can provide the mean memory usage of a VNF that can be the percentage of the total available memory; (b) can employ TF (Transparent Forwarding); (c) This measurement can be created when an EM receives (e.g., via communication circuitry 620) the mean memory usage measurement from VNFM (e.g., sent via communication circuitry 720); (d) Each measurement can be an integer value (Unit: %); (e) can have an associated ID (e.g., VR.MeanMemoryUsage.VnfID); (f) can be a ManagedElement; (g) can be valid for packet switched; and (h) EPS.

[0071] Peak memory usage measurements can have the following characteristics: (a) This measurement can provide the peak memory usage of a VNF that can be the percentage of the total available memory; (b) can employ TF (Transparent Forwarding); (c) This measurement can be created when an EM receives (e.g., via communication circuitry 620) the peak memory usage measurement from VNFM (e.g., sent via communication circuitry 720); (d) Each measurement can be an integer value (Unit: %); (e) can have an associated ID (e.g., VR.PeakMemoryUsage.VnfID); (f) can be a ManagedElement; (g) can be valid for packet switched; and (h) EPS.

[0072] Mean disk usage measurements can have the following characteristics: (a) This measurement can provide the mean disk usage of a VNF that can be the percentage of the total available disk storage; (b) can employ TF (Transparent

Forwarding); (c) This measurement can be created when an EM receives (e.g., via communication circuitry 620) the mean disk usage measurement from VNFM (e.g., sent via communication circuitry 720); (d) Each measurement can be an integer value (Unit: %); (e) can have an associated ID (e.g., VR.MeanDiskUsage.VnfID); (f) can be a ManagedElement; (g) can be valid for packet switched; and (h) EPS.

[0073] Peak disk usage measurements can have the following characteristics: (a) This measurement can provide the peak disk usage of a VNF that can be the percentage of the total available disk storage; (b) can employ TF (Transparent

Forwarding); (c) This measurement can be created when an EM receives (e.g., via communication circuitry 620) the peak disk usage measurement from a VNFM (e.g., sent via communication circuitry 720); (d) Each measurement can be an integer value (Unit: %); (e) can have an associated ID (e.g., VR.PeakDiskUsage.VnfID); (f) can be a ManagedElement; (g) can be valid for packet switched; and (h) EPS.

[0074] Data volume measurements for monitoring networking resources can comprise measurements of one or more of number of outgoing IP packets, number of incoming IP packets, number of octets of outgoing IP packets, and number of octets of incoming IP packets.

[0075] Measurements of the number of outgoing IP packets can have the following characteristics: (a) This measurement can provide the number of outgoing IP packets sent by a VNF; (b) can employ TF (Transparent Forwarding); (c) This measurement can be created when an EM receives (e.g., via communication circuitry 620) the number of outgoing IP packets measurement from a VNFM (e.g., sent via communication circuitry 720); (d) Each measurement can be an integer value (Unit: %); (e) can have an associated ID (e.g., VR.NumOfOutgoinglpPackets.VnfID); (f) can be a

ManagedElement; (g) can be valid for packet switched; and (h) EPS.

[0076] Measurements of the number of incoming IP packets can have the following characteristics: (a) This measurement can provides the number of incoming IP packets received by a VNF; (b) can employ TF (Transparent Forwarding); (c) This measurement can be created when an EM receives (e.g., via communication circuitry 620) the number of incoming IP packets measurement from VNFM (e.g., sent via communication circuitry 720); (d) Each measurement can be an integer value (Unit: %); (e) can have an associated ID (e.g., VR.NumOflncominglpPackets.VnfID); (f) can be a

ManagedElement; (g) can be valid for packet switched; (h) EPS.

[0077] Measurements of the number of octets of outgoing IP packets can have the following characteristics: (a) This measurement can provide the number of octets of outgoing IP packets sent by a VNF; (b) can employ TF (Transparent Forwarding); (c) This measurement can be created when an EM receives (e.g., via communication circuitry 620) the number of octets of outgoing IP packets measurement from a VNFM (e.g., sent via communication circuitry 720); (d) Each measurement can be an integer value (Unit: %); (e) can have an associated ID (e.g.,

VR.NumOfOctetsOfOutgoinglpPackets.VnfID); (f) can be a ManagedElement; (g) can be valid for packet switched; and (h) EPS.

[0078] Measurements of the number of octets of incoming IP packets can have the following characteristics: (a) This measurement can provide the number of octets of incoming IP packets received by a VNF; (b) can employ TF (Transparent Forwarding); (c) This measurement can be created when an EM receives (e.g., via communication circuitry 620) the number of octets of incoming IP packets measurement from a VNFM (e.g., sent via communication circuitry 720); (d) Each measurement can be an integer value (Unit: %); (e) can have an associated ID (e.g.,

VR.NumOfOctetsOflncominglpPackets.VnfID); (f) can be a ManagedElement; (g) can be valid for packet switched; and (h) EPS.

VIRTUALIZED RESOURCE PERFORMANCE MEASUREMENTS

[0079] In various aspects, VR performance measurements discussed below can be defined at a VNFM (e.g., employing system 700, etc.) to monitor the performance of virtualized resources used by a VNF. An EM can generate (e.g., via processor(s) 610) a performance measurement when EM receives (e.g., via communication circuitry 620 over the Ve-Vnfm-em reference point) the performance measurement from VNFM (e.g., generated by processor(s) 710 and transmitted by communication circuitry 720 via the Ve-Vnfm-em reference point).

[0080] Various performance measurements discussed herein can be defined to measure the performance of virtualized resources (e.g., computing, storage, and networking resources) at a NFVI (e.g., employing system 900). The specific virtualized resource where the measurement is to be collected can be identified by an objectType (object type, which can define the object type) attribute and an objectlnstanceld (object instance identifier, which can identify an object instance, e.g., one for which a performance metric is requested to be collected, is reported, or is available, one for which a threshold has been crossed, etc.) parameter that can be provided during the PM job creation. The virtualized resources can typically refer to VM(s).

[0081] When a performance measurement is collected, it can include

performanceValue (performance value, which can list performance value(s) (e.g., value(s) of metric(s) collected, that resulted in threshold crossing(s), etc.) with associated time stamp(s)) attribute(s) and timeStamp (time stamp, which can indicate when data was collected, when an alarm was raised/cleared, when a fault was observed, etc.) attribute(s) that can be contained in a PerformanceReport IE

(information element) (performance report, that can comprise performanceReportEntry (performance report entry) attribute(s) that that can list performance information entries) to be sent to the consumers. The PerformanceReport IE can also comprise objectType and objectlnstanceld attribute(s) that can identify the virtualized resource where the performance measurement is collected.

[0082] Collection methods can define how a performance measurement is collected. There are two types of collection methods discussed herein that can be employed to collect virtualized resource performance measurements: cumulative counter and status counter.

[0083] In cumulative counter methods, a virtualized resource can maintain a counter that can increment by one for each event detected. A performance measurement based on a cumulative counter method can be generated as follows: (1 ) a counter can be read at a beginning of a collectionPeriod (collection period, which can specify a periodicity at which a producer can collect information) attribute; (2) the counter can be read at the end of the collectionPeriod; and (3) a difference between the two values received in (1 ) and (2) can be calculated to generate a measurement. In some scenarios, the counter may wrap around (e.g., reset to 0) in a collectionPeriod. If so, appropriate measure(s) can be taken to correct the measurement. Cumulative counter methods can be used to measure, for example, the data volume of networking resources.

[0084] In status counter methods, a virtualized resource maintains a counter that reflects the status of the event. A performance measurement based on a status counter method can be obtained as follows: (1 ) the counter can be read continuously at each collectionPeriod interval; (2) all values received in (1 ) during the reportingPeriod (reporting period, an attribute that can specify the periodicity at which a producer will report to a consumer about performance information) interval that can equal a multiple (e.g., a positive integer (e.g., 1 or more) multiple) of the collectionPeriod; (3) the arithmetic mean of the values collected in (2) can be calculated to generate a mean measurement; and (4) a maximum value of the values collected in (2) can be selected to generate a peak measurement. Status counter methods can be used to measure, for example, mean and peak usage data of computing and storage resources.

[0085] Measurements for monitoring computing resources can include mean CPU usage or peak CPU usage. These measurements can be collected in a NFVI (e.g., via processor(s) 910). A VIM can configure (e.g., via a configuration file generated by processor(s) 810, sent by communication circuitry 820 via the Nf-Vi reference point, received by communication circuitry 920, and implemented by processor(s) 910) the OpenStack Ceilometer (or other data collection service) in the NFVI to enable the NFVI to collect and report (e.g., via processor(s) 910 and communication circuitry 920) these measurements.

[0086] A mean CPU usage measurement can provide a mean CPU usage of a VM that can be a percentage of the total available CPU processing power. This

measurement can be based on the status counter method. The mean CPU usage measurement can be obtained as follows: (1 ) the CPU usage of the VM can be read at each collectionPeriod interval, (2) values received during the reportingPeriod interval can be collected, and (3) the arithmetic mean of the values collected in (2) can be taken to compute the Mean CPU Usage measurement.

[0087] A peak CPU usage measurement can provide a peak CPU usage of a VM that can be a percentage of the total available CPU processing power. This

measurement can be based on the status counter method. The peak CPU usage measurement can be obtained as follows: (1 ) the CPU usage of the VM can be read at each collectionPeriod interval, (2) values received during the reportingPeriod interval can be collected, and (3) a maximum value of the values collected in (2) can be selected to compute the Peak CPU Usage measurement.

[0088] Measurements for monitoring storage resources can include mean memory usage, peak memory usage, mean disk usage, or peak disk usage. These

measurements can be collected in a NFVI (e.g., via processor(s) 910). A VIM can configure (e.g., via a configuration file generated by processor(s) 81 0, sent by communication circuitry 820 via the Nf-Vi reference point, received by communication circuitry 920, and implemented by processor(s) 910) the OpenStack Ceilometer (or other data collection service) in the NFVI to enable the NFVI to collect and report (e.g., via processor(s) 910 and communication circuitry 920) these measurements.

[0089] A mean memory usage measurement can provide a mean memory usage of a VM that can be a percentage of the total available memory. This measurement can be based on the status counter method. The mean memory usage measurement can be obtained as follows: (1 ) the memory usage of the VM can be read at each

collectionPeriod interval, (2) values received during the reportingPeriod interval can be collected, and (3) the arithmetic mean of the values collected in (2) can be taken to compute the Mean Memory Usage measurement.

[0090] A peak memory usage measurement can provide a peak memory usage of a VM that can be a percentage of the total available memory. This measurement can be based on the status counter method. The peak memory usage measurement can be obtained as follows: (1 ) the memory usage of the VM can be read at each collectionPeriod interval, (2) values received during the reportingPeriod interval can be collected, and (3) a maximum value of the values collected in (2) can be selected to compute the Peak Memory Usage measurement.

[0091] A mean disk usage measurement can provide a mean disk usage of a VM that can be a percentage of the total available disk storage. This measurement can be based on the status counter method. The mean disk usage measurement can be obtained as follows: (1 ) the disk usage of the VM can be read at each collectionPeriod interval, (2) values received during the reportingPeriod interval can be collected, and (3) the arithmetic mean of the values collected in (2) can be taken to compute the Mean Disk Usage measurement.

[0092] A peak disk usage measurement can provide a peak memory usage of a VM that can be a percentage of the total available disk storage. This measurement can be based on the status counter method. The peak disk usage measurement can be obtained as follows: (1 ) the disk usage of the VM can be read at each collectionPeriod interval, (2) values received during the reportingPeriod interval can be collected, and (3) a maximum value of the values collected in (2) can be selected to compute the Peak Disk Usage measurement.

[0093] Measurements for monitoring the data volume sent or received by networking resources can include number of outgoing IP packets, number of incoming IP packets, number of octets of outgoing IP packets, or number of octets of incoming IP packets. These measurements can be collected in a NFVI (e.g., via processor(s) 91 0). A VIM can configure (e.g., via a configuration file generated by processor(s) 810, sent by communication circuitry 820 via the Nf-Vi reference point, received by communication circuitry 920, and implemented by processor(s) 910) the OpenStack Ceilometer (or other data collection service) in the NFVI to enable the NFVI to collect and report (e.g., via processor(s) 910 and communication circuitry 920) these measurements.

[0094] A measurement of the number of outgoing IP packets can provide a number of outgoing IP packets sent by a VM. This measurement can be based on the cumulative counter method. The measurement of the number of outgoing IP packets can be obtained as follows: (1 ) the number of outgoing IP packets counter of a VM can be read at the beginning of each collectionPeriod interval, (2) the number of outgoing IP packets counter of a VM can be read at the end of each collectionPeriod interval, and (3) a difference between the values read in (1 ) and (2) can calculated to generate the Number of Outgoing IP Packets measurement.

[0095] A measurement of the number of incoming IP packets can provide a number of incoming IP packets received by a VM. This measurement can be based on the cumulative counter method. The measurement of the number of incoming IP packets can be obtained as follows: (1 ) the number of incoming IP packets counter of a VM can be read at the beginning of each collectionPeriod interval, (2) the number of incoming IP packets counter of a VM can be read at the end of each collectionPeriod interval, and (3) a difference between the values read in (1 ) and (2) can calculated to generate the Number of Incoming IP Packets measurement.

[0096] A measurement of the number of octets of outgoing IP packets can provide a number of octets of outgoing IP packets sent by a VM. This measurement can be based on the cumulative counter method. The measurement of the number of octets of outgoing IP packets can be obtained as follows: (1 ) the number of octets of outgoing IP packets counter of a VM can be read at the beginning of each collectionPeriod interval, (2) the number of octets of outgoing IP packets counter of a VM can be read at the end of each collectionPeriod interval, and (3) a difference between the values read in (1 ) and (2) can calculated to generate the Number of Octets of Outgoing IP Packets measurement.

[0097] A measurement of the number of octets of incoming IP packets can provide a number of octets of incoming IP packets received by a VM. This measurement can be based on the cumulative counter method. The measurement of the number of octets of incoming IP packets can be obtained as follows: (1 ) the number of octets of incoming IP packets counter of a VM can be read at the beginning of each collectionPeriod interval, (2) the number of octets of incoming IP packets counter of a VM can be read at the end of each collectionPeriod interval, and (3) a difference between the values read in (1 ) and (2) can calculated to generate the Number of Octets of Incoming IP Packets measurement.

[0098] Referring to FIG. 11 , illustrated is an example method 1 1 00 that can facilitate PM data notification, according to various aspects discussed herein. Example method 1 1 00 describes techniques that can facilitate creation of PM job(s) to collect VNF related VR performance measurements at 1 102-1 1 10, acknowledgement of PM job creation at 1 1 12-1 1 1 8, and generation of notification(s) when performance

measurement(s) is/are collected at 1 120-1 1 30. [0099] At 1 102, the NM can send (e.g., via communication circuitry 520 over the Itf-N reference point) a request (e.g., generated by processor(s) 510) to EM (e.g., which can receive the request via communication circuitry 620 and process it via processor(s) 610) to create a measurement job to collect VNF related VR PM data. The job can be defined by one or more parameters such as the following: (a) iOCName (information object class (IOC) name, which can specify one Managed Entity class name), iOCInstanceList (IOC instance list, which can specify the list of DNs of ManagedEntity instances whose measurementType(s) are to be collected), which can be object identifiers that can identify the VNF and VM where the measurements are to be collected; (b)

measurementCategoryList (measurement category list, which can specify the corresponding name of measurementType (measurement type) to be measured), which can include MeasuredAttribute (measured attribute, which can represent the name of the measurementType of the related ManagedEntity (managed entity) instance whose value is to be monitored and collected) and MeasurementTypeName (measurement type name, which can identify a name of one measurement type whose value is being collected and monitored) that can define the type of measurement(s) to be collected, for example, Mean CPU Usage, Peak Memory Usage, Number of Octets of Outgoing IP Packets, Number of Outgoing IP Packets, etc.; (c) granularityPeriod (granularity period, which can specify the period between two successive measurements or readings of a threshold value), which can define the granularity interval for the measurements; and (d) reportingPeriod (reporting period, which can specify the period between two successive notifications, such as of a file being ready, or an error in file preparation), which can define the reporting interval for the measurements.

[00100] At 1 104, the EM can use (e.g., via processor(s) 61 0) the iOCName and iOCInstanceList to identify the list of VNF instances where the PM job is to be created, and can send (e.g., via communication circuitry 620 over the Ve-Vnfm-em reference point) a request (e.g., generated by processor(s) 610) to the corresponding VNFM (e.g., which can receive the request via communication circuitry 720 and process it via processor(s) 710) to create a PM job with one or more parameters such as the following: (a) sourceSelector (source selector, which can define the VNF(s) and/or VNFC(s) for which performance information is requested to be collected), which can contain objectType (object type, which can indicate the object type of the VNF descriptor) and objectlnstanceld (object instance identifier, which can identify the object instance(s) of the VNF/VNFC for which performance is to be collected) attributes that can be mapped from iOCName and iOCInstanceList from 1 102; (b) performanceMetric (performance metric, which can define the type(s) of performance metric(s) for the specified VNF), which can identify the type of measurements to be collected (e.g., Mean CPU Usage, Peak Memory Usage, Number of Octets of Outgoing IP Packets, Number of Outgoing IP Packets, etc.), and can be mapped from the measurementCategoryList of 1 102; (c) collectionPeriod (collection period, which can specify the periodicity at which the VNFM will collect performance informance), which can be mapped from the granularityPeriod of 1 102; (d) reportingPeriod (reporting period, which can specify the periodicity at which the VNFM will report to the EM about performance information), which can be mapped from the reportingPeriod of 1 102.

[00101 ] At 1 106, the VNFM can identify (e.g., via processor(s) 71 0) the virtualized resources where the performance measurement is to be collected, based on the VNF identifiers received (e.g., via communication circuitry 720 over the Ve-Vnfm-em reference point) from the EM at 1 104. The VNFM can retain (e.g., via processor(s) 710 storing in memory 730) the mapping of VNF to virtualized resource through the VNF instantiation, wherein VNFM can request (e.g., via a request generated by processor(s) 71 0 and sent via communication circuitry 720 over the Vi-Vnfm reference point) VIM to allocate the virtualized resource for a VNF to be instantiated (the VIM can receive the request via communication circuitry 820 and process it via processor(s) 810).

[00102] At 1 108, VNFM can send (e.g., via communication circuitry 720 over the Vi- Vnfm reference point) a request (e.g., generated by processor(s) 71 0) to the VIM (e.g., which can receive the request via communication circuitry 820 and process it via processor(s) 810) to create a PM job that can include one or more attributes to collect the VR performance measurements such as the following, which can be received at 1 1 04: (a) resourceSelector (resource selector, which can define the VNF(s) and/or VNFC(s) for which performance information is requested to be collected), which can contain the objectType (object type, which can indicate the object type of virtualized resources) and objectlnstanceld (which can identify the object instances of the virtualised resource or Virtual Machine (VM) where the performance information is to be collected) attributes; (b) performanceMetric, which can identify the type(s) of measurement(s) to be collected; (c) collectionPeriod, which can indicate the periodicity at which the counter(s) are to be read; and (d) reportingPeriod, which can indicate the periodicity at which the measurement(s) will be reported. [00103] At 1 1 10, the VIM can configure (e.g., via a configuration file generated by processor(s) 810 and sent by communication circuitry 820 via the Nf-Vi reference point to communication circuitry 920 for implementation by processor(s) 910) the Ceilometer in the NFVI to configure (e.g., via processor(s) 91 0) the collection of a PM measurement (e.g., Mean CPU usage, etc.).

[00104] At 1 1 12, the Ceilometer in the NFVI can send (e.g., via communication circuitry 920 over the Nf-Vi reference point) an acknowledgement (e.g., generated by processor(s) 910) to VIM (e.g., which can receive the acknowledgement via

communication circuitry 820 and process it via processor(s) 81 0).

[00105] At 1 1 14, the VIM can send (e.g., via communication circuitry 820 over the Vi- Vnfm reference point) an acknowledgement (e.g., generated by processor(s) 810) to the VNFM (e.g., which can receive the acknowledgement via communication circuitry 720 and process it via processor(s) 710) with the PM job identifier - pmJobld (performance measurement job identifier, which can identify the PM job being created, deleted, etc.), indicating the PM job that has been created.

[00106] At 1 1 16, the VNFM can send (e.g., via communication circuitry 720 over the Ve-Vnfm-em reference point) a response (e.g., generated by processor(s) 710) to the EM (e.g., which can receive the response via communication circuitry 620 and process it via processor(s) 61 0) with the identifier of the PM job being created.

[00107] At 1 1 18, the EM can send (e.g., via communication circuitry 620 over the Itf-N reference point) a response (e.g., generated by processor(s) 61 0) to NM (e.g., which can receive the response via communication circuitry 520 and process it via

processor(s) 510) with a jobld (job identifier) that can be mapped (e.g., via processor(s) 61 0) from the pmJobld and a status = Success.

[00108] At 1 120, the Ceilometer can collect (e.g., via processor(s) 910) PM data.

[00109] At 1 122, the Ceilometer can report (e.g., via processor(s) 910 generating a message sent via communication circuitry 920 over the Nf-Vi reference point) the VR PM data to VIM (e.g., which can receive the data via communication circuitry 820 and process it via processor(s) 810).

[00110] At 1 124, in response to the VIM receiving (e.g., via communication circuitry 820 over the Nf-Vi interface) notification that the PM data is available (e.g., and processing via processor(s) 810), it can send (e.g., via communication circuitry 820 over the Vi-Vnfm interface) a PerformanceReport (e.g., generated by processor(s) 81 0) to the VNFM (e.g., which can receive the PerformanceReport via communication circuitry 720 and process it via processor(s) 710) that can includes one or more lEs (information elements) such as the following: (a) objectType, which can indicate the object type of virtualized resources; (b) objectlnstanceld, which can identify the object instances of the virtualised resource or Virtual Machine (VM) where the performance information is collected; and (c) performanceValueEntry (performance value entry), which can comprise one or more performance measurements that can comprise the timeStamp (time stamp, which can indicate when the measurement was collected( and

performanceValue (performance value, which can indicate the value of the performance measurement) information elements.

[00111 ] At 1 126, the VNFM can map (e.g., via processor(s) 710) the virtualized resource (e.g., VM) to the associated VNF/VNFC (VNF Component), and can convert (e.g., via processor(s) 710) the virtualized resource performance measurement into the VNF performance measurement (e.g., if a VNF has N (e.g., 2, etc.) VMs, processor(s) 71 0 can aggregate the N VMs performance measurements to create the VNF performance measurement). The VNFM can retain (e.g., via processor(s) 710 having sent to memory 730) the mapping of the VNF to virtualized resource through the VNF instantiation wherein the VNFM can request the VIM to allocate the virtualized resource for a VNF to be instantiated (e.g., via a request generated by processor(s) 710, sent via communication circuitry 720 over the Vi-Vnfm reference point and received by communication circuitry 820, and implemented by processor(s) 810).

[00112] At 1 128, the VNFM can send (e.g., via communication circuitry 720 over the Ve-Vnfm-em reference point) a PerformanceReport (e.g., generated by processor(s) 71 0) to EM that can comprise one or more information elements such as the following: (a) objectType, which can indicate the object type of a VNF descriptor; (b)

objectlnstanceld, which can identify the object instance(s) of the VNF or VNFC where the performance information was collected; and (c) performanceValueEntry, which can comprise one or more performance measurements that can comprise the timestamp (which can indicate when the measurement was collected) and performanceValue (which can indicate the value of the performance measurement) information elements.

[00113] At 1 130, the EM can use a transparent forwarding collection method to forward (e.g., via processor(s) 610) the VNF related VR PM data to NM (e.g., via communication circuitry 620 over the Itf-N reference point, wherein the VNF related VR PM data can be received via communication circuitry 520 and processed via

processor(s) 510). [00114] Referring to FIG. 12, illustrated is a flow diagram of a method 1200 that facilitates creation of a VNF related VR PM job and associated data notification by a NM according to various aspects described herein. In some aspects, method 1200 can be performed at a NM (e.g., NM 314). In other aspects, a machine readable medium can store instructions associated with method 1200 that, when executed, can cause a NM to perform the acts of method 1200.

[00115] At 1202, a request to create a measurement job (e.g., a VNF related VR PM job) to collect VNF related VR PM job can be sent to an EM, wherein the job can be defined by parameters discussed herein (e.g., those discussed at 1 102).

[00116] At 1204, a response can be received from the EM based on the request, wherein the response can comprise a job identifier associated with the VNF related VR

PM job (e.g., jobld) and a status (e.g., success) associated with the request.

[00117] At 1206, VNF related VR PM data can be received from the EM (e.g., which can be sent via transparent forwarding).

[00118] Additionally or alternatively, method 1200 can include one or more other acts described above in connection with system 500 and/or NM 314.

[00119] Referring to FIG. 13, illustrated is a flow diagram of a method 1300 that facilitates creation of a VNF related VR PM job and associated data notification by an EM according to various aspects described herein. In some aspects, method 1300 can be performed at an EM (e.g., EM 310). In other aspects, a machine readable medium can store instructions associated with method 1300 that, when executed, can cause a NM to perform the acts of method 1300.

[00120] At 1302, a first request to create a measurement job (e.g., a VNF related VR PM job) to collect VNF related VR PM job can be received from an EM, wherein the job can be defined by parameters discussed herein (e.g., those discussed at 1 102).

[00121 ] At 1304, a list of VNF instances where the VNF related VR PM job is to be created can be identified based on the first request.

[00122] At 1306, a second request to create a measurement job (e.g., a VNF related VR PM job) to collect VNF related VR PM job can be sent to a VNFM, wherein the second request can comprise parameters discussed herein (e.g., those discussed at 1 1 04).

[00123] At 1308, a first response comprising an identifier of the PM job being created can be received from the VNFM. [00124] At 1310, a second response can be sent to the EM, wherein the second response can comprise a jobld (e.g., mapped from the pmJobld) and a status (e.g., 'success').

[00125] At 1312, a PerformanceReport can be received from the VNFM that can comprise one or more information elements that indicate the VNF related VR PM data (e.g., objectType, objectlnstanceld, performanceValueEntry (e.g., which can comprise one or more timestamp lEs and one or more performanceValue lEs).

[00126] At 1314, a transparent forwarding collection method can be employed to forward VNF related VR PM data to the NM.

[00127] Additionally or alternatively, method 1300 can include one or more other acts described above in connection with system 600 and/or EM 310.

[00128] Referring to FIG. 14, illustrated is a flow diagram of a method 1400 that facilitates creation of a VNF related VR PM job and associated data notification by a VNFM according to various aspects described herein. In some aspects, method 1400 can be performed at a VNFM (e.g., VNFM 306). In other aspects, a machine readable medium can store instructions associated with method 1400 that, when executed, can cause a NM to perform the acts of method 1400.

[00129] At 1402, a first request to create a measurement job (e.g., a VNF related VR PM job) to collect VNF related VR PM job can be received from an EM, wherein the first request can comprise parameters discussed herein (e.g., those discussed at 1 104).

[00130] At 1404, virtualized resources where performance measurement(s) are to be collected can be identified based on the VNF identifier(s) of the first request received from the EM.

[00131 ] At 1406, a second request can be sent to a VIM to create a PM job (e.g., that can include one or more attributes, such as those discussed at 1 108) to collect VR performance measurements (e.g., from the identified VR(s)).

[00132] At 1408, an acknowledgement can be received from the VIM (e.g., comprising a PM job identifier) indicating that the PM job has been created.

[00133] At 1410, a response can be sent to the EM comprising an identifier of the PM job that was created.

[00134] At 1412, a first performance report can be received from the VIM comprising VR PM data (e.g., indicated via information elements such as those discussed at 1 124).

[00135] At 1414, the virtualized resources associated with the VR PM data can be mapped to one or more VNFs and/or VNFCs. [00136] At 1416, the VR PM data received via the first performance report can be converted into VNF PM data.

[00137] At 1418, a second performance report can be sent to the EM comprising the VNF PM data (e.g., comprising lEs indicated at 1 1 28).

[00138] Additionally or alternatively, method 1400 can include one or more other acts described above in connection with system 700 and/or VNFM 306.

[00139] Referring to FIG. 15, illustrated is a flow diagram of a method 1500 that facilitates creation of a VNF related VR PM job and associated data notification by a VIM according to various aspects described herein. In some aspects, method 1500 can be performed at a VIM (e.g., VIM 302). In other aspects, a machine readable medium can store instructions associated with method 1500 that, when executed, can cause a NM to perform the acts of method 1500.

[00140] At 1502, a request to create a PM job can be received from the VNFM, wherein the request comprises one or more attributes (e.g., attributes from 1 108 based on attributes from 1 1 04 to collect VR performance measurements).

[00141 ] At 1504, a data collection service (e.g., Ceilometer) in a NFVI can be configured to collect a VR performance measurement.

[00142] At 1506, a first acknowledgement can be received from the data collection service at the NFVI.

[00143] At 1508, a second acknowledgement can be sent to a VNFM (e.g., with a PM job identifier) indicating that the PM job has been created.

[00144] At 1510, VR PM data can be received from the data collection service at the NFVI.

[00145] At 1512, in response to receiving the VR PM data, a performance report can be sent to the VNFM comprising the VR PM data (e.g., indicated via lEs discussed at 1 124).

[00146] Additionally or alternatively, method 1500 can include one or more other acts described above in connection with system 800 and/or VIM 302.

[00147] Referring to FIG. 16, illustrated is a flow diagram of a method 1600 that facilitates creation of a VNF related VR PM job and associated data notification by a NFVI according to various aspects described herein. In some aspects, method 1600 can be performed at a NFVI (e.g., NFVI 304). In other aspects, a machine readable medium can store instructions associated with method 1600 that, when executed, can cause a NM to perform the acts of method 1 600. [00148] At 1602, configuration can be received from a VIM configuring collection of a PM measurement (e.g., CPU, memory, or data usage measurements discussed herein, etc.).

[00149] At 1604, an acknowledgement can be sent to the VIM.

[00150] At 1606, PM data (e.g., VR PM data) can be collected.

[00151 ] At 1608, the VR PM data can be reported to the VIM.

[00152] Additionally or alternatively, method 1600 can include one or more other acts described above in connection with system 900 and/or NFVI 304.

[00153] Examples herein can include subject matter such as a method, means for performing acts or blocks of the method, at least one machine-readable medium including executable instructions that, when performed by a machine (e.g., a processor with memory, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or the like) cause the machine to perform acts of the method or of an apparatus or system for concurrent communication using multiple communication technologies according to embodiments and examples described.

[00154] Example 1 is a machine readable medium comprising instructions that, when executed, cause a NM (Network Manager) to: send a request to an EM (Element

Manager) to create a VNF (Virtual Network Function) PM (Performance Measurement) job to collect VNF related VR (Virtualization Resource) PM data; receive a response from the EM comprising a jobld (job identifier) of the VNF PM job and a status, wherein the status equals success; and receive the VNF related VR PM data from the EM.

[00155] Example 2 comprises the subject matter of any variation of any of example(s)

1 , wherein the VNF PM job indicates a computing resources measurement.

[00156] Example 3 comprises the subject matter of any variation of any of example(s)

2, wherein the VNF related VR PM data comprises a mean CPU (Central Processing Unit) usage or a peak CPU usage.

[00157] Example 4 comprises the subject matter of any variation of any of example(s) 1 , wherein the VNF PM job indicates a storage resources measurement.

[00158] Example 5 comprises the subject matter of any variation of any of example(s) 4, wherein the VNF related VR PM data comprises a mean memory usage, a peak memory usage, a mean disk usage, or a peak disk usage.

[00159] Example 6 comprises the subject matter of any variation of any of example(s) 1 , wherein the VNF PM job indicates a data volume measurement. [00160] Example 7 comprises the subject matter of any variation of any of example(s) 6, wherein the VNF related VR PM data comprises a number of outgoing IP (Internet Protocol) packets, a number of incoming IP packets, a number of octets of outgoing IP packets, or a number of octets of incoming IP packets.

[00161 ] Example 8 is a machine readable medium comprising instructions that, when executed, cause an EM (Element Manager) to: receive a first request from a NM

(Network Manager) to create a VNF (Virtual Network Function) PM (Performance Measurement) job to collect VNF related VR (Virtualization Resource) PM data; identify a list of VNF instances where the VNF PM job is to be created based on the VNF PM job; send a second request to a VNFM (Virtual Network Function Manager) to create the VNF PM job; receive a first response from the VNFM comprising a pmJobld (PM job identifier) of the VNF PM job; send a second response to the EM comprising a jobld (job identifier) of the VNF PM job and a status that equals success, wherein the jobld is mapped from the pmJobld; receive a PerformanceReport (performance report) from the VNFM comprising one or more lEs (Information Elements) that indicate the VNF related VR PM data; and send the VNF related VR PM data to the NM.

[00162] Example 9 comprises the subject matter of any variation of any of example(s) 8, wherein the VNF related VR PM data is sent to the NM via transparent forwarding.

[00163] Example 10 comprises the subject matter of any variation of any of example(s) 8-9, wherein the VNF PM job indicates a computing resources

measurement.

[00164] Example 1 1 comprises the subject matter of any variation of any of example(s) 10, wherein the VNF related VR PM data comprises a mean CPU (Central Processing Unit) usage or a peak CPU usage.

[00165] Example 12 comprises the subject matter of any variation of any of example(s) 8, wherein the VNF PM job indicates a storage resources measurement.

[00166] Example 13 comprises the subject matter of any variation of any of example(s) 12, wherein the VNF related VR PM data comprises a mean memory usage, a peak memory usage, a mean disk usage, or a peak disk usage.

[00167] Example 14 comprises the subject matter of any variation of any of example(s) 8, wherein the VNF PM job indicates a data volume measurement.

[00168] Example 15 comprises the subject matter of any variation of any of example(s) 14, wherein the VNF related VR PM data comprises a number of outgoing IP (Internet Protocol) packets, a number of incoming IP packets, a number of octets of outgoing IP packets, or a number of octets of incoming IP packets.

[00169] Example 16 is a machine readable medium comprising instructions that, when executed, cause a VNFM (Virtual Network Function Manager) to: receive a first request from an EM (Element Manager) to create a VNF (Virtual Network Function) PM (Performance Measurement) job to collect VNF related VR (Virtualization Resource) PM data; identify one or more VRs (Virtualized Resources) where the VNF related VR PM data is to be collected; send a second request to a VIM (Virtualized Infrastructure Manager) to create a VR PM job to collect VR PM data from the one or more VRs;

receive an acknowledgement from the VIM indicating that the VR PM job has been created, wherein the acknowledgement comprises a pmJobld (PM job identifier) of the VR PM job; send a response to the EM comprising the pmJobld; receive a first

PerformanceReport (performance report) from the VIM comprising one or more lEs (Information Elements) that indicate the VR PM data; map the one or more VRs to a VNF or VNFC (VNF Component) associated with the VNF related VR PM job; convert the VR PM data to the VNF related VR PM data; and send a second

PerformanceReport to the EM comprising one or more lEs (Information Elements) that indicate the VNF related VR PM data.

[00170] Example 17 comprises the subject matter of any variation of any of example(s) 16, wherein the VNF PM job indicates a computing resources

measurement.

[00171 ] Example 18 comprises the subject matter of any variation of any of example(s) 17, wherein the VNF related VR PM data comprises a mean CPU (Central Processing Unit) usage or a peak CPU usage.

[00172] Example 19 comprises the subject matter of any variation of any of example(s) 16, wherein the VNF PM job indicates a storage resources measurement.

[00173] Example 20 comprises the subject matter of any variation of any of example(s) 19, wherein the VNF related VR PM data comprises a mean memory usage, a peak memory usage, a mean disk usage, or a peak disk usage.

[00174] Example 21 comprises the subject matter of any variation of any of example(s) 16, wherein the VNF PM job indicates a data volume measurement.

[00175] Example 22 comprises the subject matter of any variation of any of example(s) 21 , wherein the VNF related VR PM data comprises a number of outgoing IP (Internet Protocol) packets, a number of incoming IP packets, a number of octets of outgoing IP packets, or a number of octets of incoming IP packets.

[00176] Example 23 is a machine readable medium comprising instructions that, when executed, cause a VIM (Virtualized Infrastructure Manager) to: receive a request from a VNFM (VNF (Virtual Network Function) Manager) to create a VR (Virtualization Resource) PM (Performance Measurement) job to collect VR PM data from the one or more VRs; configure a data collection service in a NFVI (Network Function Virtualization Infrastructure) to configure collection of a PM; receive a first acknowledgement from the data collection service in the NFVI; send a second acknowledgement to the VNFM indicating that the VR PM job has been created, wherein the acknowledgement comprises a pmJobld (PM job identifier) of the VR PM job; receive VR PM data from the data collection service in the NFVI; and send a PerformanceReport (performance report) to the VNFM comprising one or more lEs (Information Elements) that indicate the VR PM data.

[00177] Example 24 comprises the subject matter of any variation of any of example(s) 23, wherein the VR PM job indicates a computing resources measurement.

[00178] Example 25 comprises the subject matter of any variation of any of example(s) 24, wherein the VR PM data comprises a mean CPU (Central Processing Unit) usage or a peak CPU usage.

[00179] Example 26 comprises the subject matter of any variation of any of example(s) 23, wherein the VR PM job indicates a storage resources measurement.

[00180] Example 27 comprises the subject matter of any variation of any of example(s) 26, wherein the VR PM data comprises a mean memory usage, a peak memory usage, a mean disk usage, or a peak disk usage.

[00181 ] Example 28 comprises the subject matter of any variation of any of example(s) 23, wherein the VR PM job indicates a data volume measurement.

[00182] Example 29 comprises the subject matter of any variation of any of example(s) 28, wherein the VR PM data comprises number of outgoing IP (Internet Protocol) packets, a number of incoming IP packets, a number of octets of outgoing IP packets, or a number of octets of incoming IP packets.

[00183] Example 30 is a machine readable medium comprising instructions that, when executed, cause a NFVI (Network Function Virtualization Infrastructure) to:

receive a configuration file from a VIM (Virtualization Infrastructure Manager) via a data collection service, wherein the configuration file configures collection of a VR (Virtualization Resource) PM (Performance Measurement); send an acknowledgement to the VIM; collect VR PM data in connection with the VR PM; and report the VR PM data to the VIM.

[00184] Example 31 comprises the subject matter of any variation of any of example(s) 30, wherein the VR PM data is collected based on a cumulative counter.

[00185] Example 32 comprises the subject matter of any variation of any of example(s) 31 , wherein the VR PM data comprises a mean memory usage, a peak memory usage, a mean disk usage, or a peak disk usage.

[00186] Example 33 comprises the subject matter of any variation of any of example(s) 30, wherein the VR PM data is collected based on a status counter.

[00187] Example 34 comprises the subject matter of any variation of any of example(s) 31 , wherein the VR PM data comprises a mean CPU (Central Processing Unit) usage, a peak CPU usage, a mean memory usage, a peak memory usage, a mean disk usage, or a peak disk usage.

[00188] Example 35 is an apparatus configured to be employed in a NM (Network Manager), comprising: means for sending a request to an EM (Element Manager) to create a VNF (Virtual Network Function) PM (Performance Measurement) job to collect VNF related VR (Virtualization Resource) PM data; means for receiving a response from the EM comprising a jobld (job identifier) of the VNF PM job and a status, wherein the status equals success; and means for receiving the VNF related VR PM data from the EM.

[00189] Example 36 comprises the subject matter of any variation of any of example(s) 35, wherein the VNF PM job indicates a computing resources

measurement.

[00190] Example 37 comprises the subject matter of any variation of any of example(s) 36, wherein the VNF related VR PM data comprises a mean CPU (Central Processing Unit) usage or a peak CPU usage.

[00191 ] Example 38 comprises the subject matter of any variation of any of example(s) 35, wherein the VNF PM job indicates a storage resources measurement.

[00192] Example 39 comprises the subject matter of any variation of any of example(s) 38, wherein the VNF related VR PM data comprises a mean memory usage, a peak memory usage, a mean disk usage, or a peak disk usage.

[00193] Example 40 comprises the subject matter of any variation of any of example(s) 35, wherein the VNF PM job indicates a data volume measurement. [00194] Example 41 comprises the subject matter of any variation of any of example(s) 40, wherein the VNF related VR PM data comprises a number of outgoing IP (Internet Protocol) packets, a number of incoming IP packets, a number of octets of outgoing IP packets, or a number of octets of incoming IP packets.

[00195] Example 42 is an apparatus configured to be employed in an EM (Element Manager), comprising: means for receiving a first request from a NM (Network

Manager) to create a VNF (Virtual Network Function) PM (Performance Measurement) job to collect VNF related VR (Virtualization Resource) PM data; means for identifying a list of VNF instances where the VNF PM job is to be created based on the VNF PM job; means for sending a second request to a VNFM (Virtual Network Function Manager) to create the VNF PM job; means for receiving a first response from the VNFM comprising a pmJobld (PM job identifier) of the VNF PM job; means for sending a second response to the EM comprising a jobld (job identifier) of the VNF PM job and a status that equals success, wherein the jobld is mapped from the pmJobld; means for receiving a

PerformanceReport (performance report) from the VNFM comprising one or more lEs (Information Elements) that indicate the VNF related VR PM data; and means for sending the VNF related VR PM data to the NM.

[00196] Example 43 comprises the subject matter of any variation of any of example(s) 42, wherein the VNF related VR PM data is sent to the NM via transparent forwarding.

[00197] Example 44 comprises the subject matter of any variation of any of example(s) 42-43, wherein the VNF PM job indicates a computing resources

measurement.

[00198] Example 45 comprises the subject matter of any variation of any of example(s) 44, wherein the VNF related VR PM data comprises a mean CPU (Central Processing Unit) usage or a peak CPU usage.

[00199] Example 46 comprises the subject matter of any variation of any of example(s) 42, wherein the VNF PM job indicates a storage resources measurement.

[00200] Example 47 comprises the subject matter of any variation of any of example(s) 46, wherein the VNF related VR PM data comprises a mean memory usage, a peak memory usage, a mean disk usage, or a peak disk usage.

[00201 ] Example 48 comprises the subject matter of any variation of any of example(s) 42, wherein the VNF PM job indicates a data volume measurement. [00202] Example 49 comprises the subject matter of any variation of any of example(s) 48, wherein the VNF related VR PM data comprises a number of outgoing IP (Internet Protocol) packets, a number of incoming IP packets, a number of octets of outgoing IP packets, or a number of octets of incoming IP packets.

[00203] Example 50 is an apparatus configured to be employed in a VNFM (Virtual Network Function Manager), comprising: means for receiving a first request from an EM (Element Manager) to create a VNF (Virtual Network Function) PM (Performance Measurement) job to collect VNF related VR (Virtualization Resource) PM data; means for identifying one or more VRs (Virtualized Resources) where the VNF related VR PM data is to be collected; means for sending a second request to a VIM (Virtualized Infrastructure Manager) to create a VR PM job to collect VR PM data from the one or more VRs; means for receiving an acknowledgement from the VIM indicating that the VR PM job has been created, wherein the acknowledgement comprises a pmJobld (PM job identifier) of the VR PM job; means for sending a response to the EM comprising the pmJobld; means for receiving a first PerformanceReport (performance report) from the VIM comprising one or more lEs (Information Elements) that indicate the VR PM data; means for mapping the one or more VRs to a VNF or VNFC (VNF Component) associated with the VNF related VR PM job; means for converting the VR PM data to the VNF related VR PM data; and means for sending a second PerformanceReport to the EM comprising one or more lEs (Information Elements) that indicate the VNF related VR PM data.

[00204] Example 51 comprises the subject matter of any variation of any of example(s) 50, wherein the VNF PM job indicates a computing resources

measurement.

[00205] Example 52 comprises the subject matter of any variation of any of example(s) 51 , wherein the VNF related VR PM data comprises a mean CPU (Central Processing Unit) usage or a peak CPU usage.

[00206] Example 53 comprises the subject matter of any variation of any of example(s) 50, wherein the VNF PM job indicates a storage resources measurement.

[00207] Example 54 comprises the subject matter of any variation of any of example(s) 53, wherein the VNF related VR PM data comprises a mean memory usage, a peak memory usage, a mean disk usage, or a peak disk usage.

[00208] Example 55 comprises the subject matter of any variation of any of example(s) 50, wherein the VNF PM job indicates a data volume measurement. [00209] Example 56 comprises the subject matter of any variation of any of example(s) 55, wherein the VNF related VR PM data comprises a number of outgoing IP (Internet Protocol) packets, a number of incoming IP packets, a number of octets of outgoing IP packets, or a number of octets of incoming IP packets.

[00210] Example 57 is an apparatus configured to be employed in a VIM (Virtualized Infrastructure Manager), comprising: means for receiving a request from a VNFM (VNF (Virtual Network Function) Manager) to create a VR (Virtualization Resource) PM (Performance Measurement) job to collect VR PM data from the one or more VRs; means for configuring a data collection service in a NFVI (Network Function

Virtualization Infrastructure) to configure collection of a PM; means for receiving a first acknowledgement from the data collection service in the NFVI; means for sending a second acknowledgement to the VNFM indicating that the VR PM job has been created, wherein the acknowledgement comprises a pmJobld (PM job identifier) of the VR PM job; means for receiving VR PM data from the data collection service in the NFVI; and means for sending a PerformanceReport (performance report) to the VNFM comprising one or more lEs (Information Elements) that indicate the VR PM data.

[00211 ] Example 58 comprises the subject matter of any variation of any of example(s) 57, wherein the VR PM job indicates a computing resources measurement.

[00212] Example 59 comprises the subject matter of any variation of any of example(s) 58, wherein the VR PM data comprises a mean CPU (Central Processing Unit) usage or a peak CPU usage.

[00213] Example 60 comprises the subject matter of any variation of any of example(s) 57, wherein the VR PM job indicates a storage resources measurement.

[00214] Example 61 comprises the subject matter of any variation of any of example(s) 60, wherein the VR PM data comprises a mean memory usage, a peak memory usage, a mean disk usage, or a peak disk usage.

[00215] Example 62 comprises the subject matter of any variation of any of example(s) 57, wherein the VR PM job indicates a data volume measurement.

[00216] Example 63 comprises the subject matter of any variation of any of example(s) 62, wherein the VR PM data comprises number of outgoing IP (Internet Protocol) packets, a number of incoming IP packets, a number of octets of outgoing IP packets, or a number of octets of incoming IP packets.

[00217] Example 64 is an apparatus configured to be employed in a NFVI (Network Function Virtualization Infrastructure), comprising: means for receiving a configuration file from a VIM (Virtualization Infrastructure Manager) via a data collection service, wherein the configuration file configures collection of a VR (Virtualization Resource) PM (Performance Measurement); means for sending an acknowledgement to the VIM; means for collecting VR PM data in connection with the VR PM; and means for reporting the VR PM data to the VIM.

[00218] Example 65 comprises the subject matter of any variation of any of example(s) 64, wherein the VR PM data is collected based on a cumulative counter.

[00219] Example 66 comprises the subject matter of any variation of any of example(s) 65, wherein the VR PM data comprises a mean memory usage, a peak memory usage, a mean disk usage, or a peak disk usage.

[00220] Example 67 comprises the subject matter of any variation of any of example(s) 64, wherein the VR PM data is collected based on a status counter.

[00221 ] Example 68 comprises the subject matter of any variation of any of example(s) 65, wherein the VR PM data comprises a mean CPU (Central Processing Unit) usage, a peak CPU usage, a mean memory usage, a peak memory usage, a mean disk usage, or a peak disk usage.

[00222] Example 69 comprises an apparatus comprising means for executing any of the described operations of examples 1 -68.

[00223] Example 70 comprises a machine readable medium that stores instructions for execution by a processor to perform any of the described operations of examples 1 - 68.

[00224] Example 71 comprises an apparatus comprising: a memory interface; and processing circuitry configured to: perform any of the described operations of examples 1 -68.

[00225] The above description of illustrated embodiments of the subject disclosure, including what is described in the Abstract, is not intended to be exhaustive or to limit the disclosed embodiments to the precise forms disclosed. While specific embodiments and examples are described herein for illustrative purposes, various modifications are possible that are considered within the scope of such embodiments and examples, as those skilled in the relevant art can recognize.

[00226] In this regard, while the disclosed subject matter has been described in connection with various embodiments and corresponding Figures, where applicable, it is to be understood that other similar embodiments can be used or modifications and additions can be made to the described embodiments for performing the same, similar, alternative, or substitute function of the disclosed subject matter without deviating therefrom. Therefore, the disclosed subject matter should not be limited to any single embodiment described herein, but rather should be construed in breadth and scope in accordance with the appended claims below.

[00227] In particular regard to the various functions performed by the above described components or structures (assemblies, devices, circuits, systems, etc.), the terms (including a reference to a "means") used to describe such components are intended to correspond, unless otherwise indicated, to any component or structure which performs the specified function of the described component (e.g., that is functionally equivalent), even though not structurally equivalent to the disclosed structure which performs the function in the herein illustrated exemplary implementations. In addition, while a particular feature may have been disclosed with respect to only one of several implementations, such feature may be combined with one or more other features of the other implementations as may be desired and advantageous for any given or particular application.

Claims

CLAIMS What is claimed is:

1 . A machine readable medium comprising instructions that, when executed, cause a NM (Network Manager) to:

send a request to an EM (Element Manager) to create a VNF (Virtual Network Function) PM (Performance Measurement) job to collect VNF related VR (Virtualization Resource) PM data;

receive a response from the EM comprising a jobld (job identifier) of the VNF PM job and a status, wherein the status equals success; and

receive the VNF related VR PM data from the EM.

2. The machine readable medium of claim 1 , wherein the VNF PM job indicates a computing resources measurement.

3. The machine readable medium of claim 2, wherein the VNF related VR PM data comprises a mean CPU (Central Processing Unit) usage or a peak CPU usage.

4. The machine readable medium of claim 1 , wherein the VNF PM job indicates a storage resources measurement.

5. The machine readable medium of claim 4, wherein the VNF related VR PM data comprises a mean memory usage, a peak memory usage, a mean disk usage, or a peak disk usage.

6. The machine readable medium of claim 1 , wherein the VNF PM job indicates a data volume measurement.

7. The machine readable medium of claim 6, wherein the VNF related VR PM data comprises a number of outgoing IP (Internet Protocol) packets, a number of incoming IP packets, a number of octets of outgoing IP packets, or a number of octets of incoming IP packets.

8. A machine readable medium comprising instructions that, when executed, cause an EM (Element Manager) to:

receive a first request from a NM (Network Manager) to create a VNF (Virtual Network Function) PM (Performance Measurement) job to collect VNF related VR (Virtualization Resource) PM data;

identify a list of VNF instances where the VNF PM job is to be created based on the VNF PM job;

send a second request to a VNFM (Virtual Network Function Manager) to create the VNF PM job;

receive a first response from the VNFM comprising a pmJobld (PM job identifier) of the VNF PM job;

send a second response to the EM comprising a jobld (job identifier) of the VNF PM job and a status that equals success, wherein the jobld is mapped from the pmJobld;

receive a PerformanceReport (performance report) from the VNFM comprising one or more lEs (Information Elements) that indicate the VNF related VR PM data; and send the VNF related VR PM data to the NM.

9. The machine readable medium of claim 8, wherein the VNF related VR PM data is sent to the NM via transparent forwarding.

10. The machine readable medium of any of claims 8-9, wherein the VNF PM job indicates a computing resources measurement.

1 1 . The machine readable medium of claim 10, wherein the VNF related VR PM data comprises a mean CPU (Central Processing Unit) usage or a peak CPU usage.

12. The machine readable medium of claim 8, wherein the VNF PM job indicates a storage resources measurement.

13. The machine readable medium of claim 12, wherein the VNF related VR PM data comprises a mean memory usage, a peak memory usage, a mean disk usage, or a peak disk usage.

14. The machine readable medium of claim 8, wherein the VNF PM job indicates a data volume measurement.

15. The machine readable medium of claim 14, wherein the VNF related VR PM data comprises a number of outgoing IP (Internet Protocol) packets, a number of incoming IP packets, a number of octets of outgoing IP packets, or a number of octets of incoming IP packets.

16. A machine readable medium comprising instructions that, when executed, cause a VNFM (Virtual Network Function Manager) to:

receive a first request from an EM (Element Manager) to create a VNF (Virtual Network Function) PM (Performance Measurement) job to collect VNF related VR (Virtualization Resource) PM data;

identify one or more VRs (Virtualized Resources) where the VNF related VR PM data is to be collected;

send a second request to a VIM (Virtualized Infrastructure Manager) to create a VR PM job to collect VR PM data from the one or more VRs;

receive an acknowledgement from the VIM indicating that the VR PM job has been created, wherein the acknowledgement comprises a pmJobld (PM job identifier) of the VR PM job;

send a response to the EM comprising the pmJobld;

receive a first PerformanceReport (performance report) from the VIM comprising one or more lEs (Information Elements) that indicate the VR PM data;

map the one or more VRs to a VNF or VNFC (VNF Component) associated with the VNF related VR PM job;

convert the VR PM data to the VNF related VR PM data; and

send a second PerformanceReport to the EM comprising one or more lEs (Information Elements) that indicate the VNF related VR PM data.

17. The machine readable medium of claim 16, wherein the VNF PM job indicates a computing resources measurement.

18. The machine readable medium of claim 17, wherein the VNF related VR PM data comprises a mean CPU (Central Processing Unit) usage or a peak CPU usage.

19. The machine readable medium of claim 16, wherein the VNF PM job indicates a storage resources measurement.

20. The machine readable medium of claim 19, wherein the VNF related VR PM data comprises a mean memory usage, a peak memory usage, a mean disk usage, or a peak disk usage.

21 . The machine readable medium of claim 16, wherein the VNF PM job indicates a data volume measurement.

22. The machine readable medium of claim 21 , wherein the VNF related VR PM data comprises a number of outgoing IP (Internet Protocol) packets, a number of incoming IP packets, a number of octets of outgoing IP packets, or a number of octets of incoming IP packets.

23. A machine readable medium comprising instructions that, when executed, cause a VIM (Virtualized Infrastructure Manager) to:

receive a request from a VNFM (VNF (Virtual Network Function) Manager) to create a VR (Virtualization Resource) PM (Performance Measurement) job to collect VR PM data from the one or more VRs;

configure a data collection service in a NFVI (Network Function Virtualization Infrastructure) to configure collection of a PM;

receive a first acknowledgement from the data collection service in the NFVI; send a second acknowledgement to the VNFM indicating that the VR PM job has been created, wherein the acknowledgement comprises a pmJobld (PM job identifier) of the VR PM job;

receive VR PM data from the data collection service in the NFVI; and

send a PerformanceReport (performance report) to the VNFM comprising one or more lEs (Information Elements) that indicate the VR PM data.

24. The machine readable medium of claim 23, wherein the VR PM job indicates a computing resources measurement.

25. The machine readable medium of claim 24, wherein the VR PM data comprises a mean CPU (Central Processing Unit) usage or a peak CPU usage.

26. The machine readable medium of claim 23, wherein the VR PM job indicates a storage resources measurement.

27. The machine readable medium of claim 26, wherein the VR PM data comprises a mean memory usage, a peak memory usage, a mean disk usage, or a peak disk usage.

28. The machine readable medium of claim 23, wherein the VR PM job indicates a data volume measurement.

29. The machine readable medium of claim 28, wherein the VR PM data comprises number of outgoing IP (Internet Protocol) packets, a number of incoming IP packets, a number of octets of outgoing IP packets, or a number of octets of incoming IP packets.