WO2022094224A1 - Online learning at a near-real time ric - Google Patents

Abstract

An apparatus for a Near real-time (Near-RT) radio access network intelligence controller (RIC) services for artificial intelligence (AI)/machine learning (ML) in an open radio access network (O-RAN), the apparatus including processing circuitry configure to send, to a Near-RT RIC via an R1 interface, an artificial intelligence (AI)/machine learning (ML) training service request, the AI/ML training service request comprising an indication of an AI/ML model structure, training preferences, and a training data description, and receive a training process identification (ID) from the Near-RT RIC, the training process ID identifying the AI/ML training service request. The processing circuitry may be further configured to return, by the training host, results of performing the training host function to the application, and modify, by the application, the AI/ML inference model based on the results.

Description

ONLINE LEARNING AT A NEAR-REAL TIME RIC

PRIORITY CLAIM

[0001] This application claims the benefit of priority to United States Provisional Patent Application 63/107,307, filed October 29, 2020, which is incorporated herein by reference in its entirety.

TECHNICAL FIELD

[0002] Aspects pertain to wireless communications. Some aspects relate to wireless networks including 3 GPP (Third Generation Partnership Project) networks, 3 GPP LTE (Long Term Evolution) networks, 3 GPP LTE- A (LTE Advanced) networks, (MulteFire, LTE-U), and fifth-generation (5G) networks including 5G new radio (NR) (or 5G-NR) networks, 5G networks such as 5G NR unlicensed spectrum (NR-U) networks and other unlicensed networks including Wi-Fi, CBRS (OnGo), etc. Other aspects are directed to Open R AN (O-RAN) architectures and, more specifically, techniques for model management for online learning at a near-real time (near-RT) RAN intelligence controller (RIC)(near-RT RIC).

BACKGROUND

[0003] Mobile communications have evolved significantly from early voice systems to today’s highly sophisticated integrated communication platform. With the increase in different types of devices communicating with various network devices, usage of 3GPP LTE systems has increased. The penetration of mobile devices (user equipment or UEs) in modern society has continued to drive demand for a wide variety of networked devices in many disparate environments. Fifth-generation (5G) wireless systems are forthcoming and are expected to enable even greater speed, connectivity, and usability. Next generation 5G networks are expected to increase throughput, coverage, and robustness and reduce latency and operational and capital expenditures. 5G new radio (5G-NR) networks will continue to evolve based on 3GPP LTE- Advanced with additional potential new⁷ radio access technologies (RATs) to enrich people’s lives with seamless wireless connectivity solutions delivering fast, rich content and sendees. As current cellular network frequency is saturated, higher frequencies, such as millimeter wave (mm Wave) frequency, can be beneficial due to their high bandwidth.

[0004] Potential LTE operation in the unlicensed spectrum includes (and is not limited to) the LTE operation in the unlicensed spectrum via dual connectivity (DC), or DC-based LAA, and the standalone LTE system in the unlicensed spectrum, according to which LTE-based technology solely operates in the unlicensed spectrum without requiring an “anchor” in the licensed spectrum, called MulteFire. MulteFire combines the performance benefits of LTE technology with the simplicity of Wi-Fi-like deployments.

[0005] Further enhanced operation of LTE and NR systems in the licensed, as well as unlicensed spectrum, is expected in future releases and 5G systems such as 0-RAN systems. Such enhanced operations can include techniques for Al and ML for 0-RAN networks.

BRIEF DESCRIPTION OF THE FIGURES

[0006] In the figures, which are not necessarily drawn to scale, like numerals may describe similar components in different view's. Like numerals having different letter suffixes may represent different instances of similar components. The figures illustrate generally, by way of example, but not by way of limitation, various aspects discussed in the present document.

[0007] FIG, 1 illustrates an example Open RAN (O-RAN) system architecture.

[0008] FIG. 2 illustrates a logical architecture of the O-RAN system of FIG. 1 .

[0009] FIG. 3 illustrates a system 300 for online learning at a Near-RT RIC, in accordance with some embodiments.

[0010] FIG. 4 illustrates a system 400 for online learning at a Near-RT RIC, in accordance with some embodiments.

[0011] FIG. 5 illustrates a method 500 for online training certification, in accordance with some embodiments. [0012] FIG. 6 illustrates a method 600 for online training certification, in accordance with some embodiments.

[0013] FIG. 7 illustrates a system 700 for online learning at a Near-RT RIC, in accordance with some embodiments.

[0014] FIG, 8 illustrates a method 800 for online training certification, in accordance with some embodiments.

DETAILED DESCRIPTION

[0015] The following description and the drawings sufficiently illustrate aspects to enable those skilled in the art to practice them. Other aspects may incorporate structural, logical, electrical, process, and other changes. Portions and features of some aspects may be included in or substituted for, those of other aspects.

Aspects outlined in the claims encompass all available equivalents of those claims.

[0016] FIG. 1 provides a high-level view of an Open RAN (O-RAN) architecture 100. The O-RAN architecture 100 includes four O-RAN defined interfaces - namely, the Al interface, the 01 interface, the 02 interface, and the Open

Fronthaul Management (M)-plane interface - which connect the Service Management and Orchestration (SMO) framework 102 to 0-RAN network functions (NFs) 104 and the O-Cloud 106. The SMO 102 (described in Reference [R01]) also connects with an external system 110, which provides enrichment data to the SMO 102. FIG. 1 also illustrates that the A1 interface terminates at an O-

RAN Non-Real Time (RT) RAN Intelligent Controller (RIC) 112 in or at the SMO 102 and at the O-RAN Near-RT RIC 114 in or at the O-RAN NFs 104. The O- RAN NFs 104 can be virtual network functions (UNFs) such as virtual machines (VMs) or containers, sitting above the O-Cloud 106 and/or Physical Network Functions (PNFs) utilizing customized hardware. All O-RAN NFs 104 are expected to support the 01 interface when interfacing with the SMO framework 102. The O-RAN NFs 104 connect to the NG-Core 108 via the NG interface (which is a 3GPP defined interface). The Open Fronthaul M-plane interface between the SMO 102 and the O-RAN Radio Unit (0-RU) 116 supports the O- RU 116 management in the O-RAN hybrid model as specified in Reference [R02],

The Open Fronthaul M-plane interface is an optional interface to the SMO 102 that is included for backward compatibility purposes as per Reference [R02] and is intended for management of the 0-RU 116 in hybrid mode only. The management architecture of flat mode (see Reference [R03]) and its relation to the 01 interface for the O-RU 116 is in development. The O-RU 1 16 termination of the 01 interface towards the SMO 102 as specified in Reference [R03],

[0017] FIG. 2 shows an O-RAN logical architecture 200 corresponding to the O- RAN architecture 100 of FIG. 1 . In FIG. 2, the SMO 202 corresponds to the SMO 102, O-Cloud 206 corresponds to the O-Cloud 106, the non-RT RIC 212 corresponds to the non-RT RIC 112, the near-RT RIC 214 corresponds to the near- RT RIC 114, and the O-RU 216 corresponds to the O-RU 116 of FIG. 2, respectively. The O-RAN logical architecture 200 includes a radio portion and a management portion. [0018] The management portion/side of the architectures 200 includes the SMO

Framework 202 containing the non-RT RIC 212, and may include the O-Cloud 206. The O-Cloud 206 is a cloud computing platform including a collection of physical infrastructure nodes to host the relevant O-RAN functions (e.g., the near- RT RIC 214, O-RAN Central Unit-Control Plane (O-CU-CP) 221, O-RAN Central Unit-User Plane O-CU-UP 222, and the O-RAN Distributed Unit (O-DU) 215, supporting software components (e.g., OSs, VA'IMs, container runtime engines, ML engines, etc.), and appropriate management and orchestration functions.

[0019] The radio portion/side of the logical architecture 200 includes the near-RT RIC 214, the O-DU 215, the O-RAN Radio Unit (O-RU) 216, the O-CU-CP 221 , and the O-CU-UP 222 functions. The radio portion/side of the logical architecture 200 may also include the O-e/gNB 210.

[0020] The O-DU 215 is a logical node hosting Radio Link Control (RLC), media access control (MAC), and higher physical (PHY) layer entities/elements (High- PHY layers) based on a lower layer functional split. The O-RU 216 is a logical node hosting lower PHY layer entities/elements (Low-PHY layer) (e.g., FFT/iFFT, PRACH extraction, etc.) and RF processing elements based on a lower layer functional split. Virtualization of O-RU 216 is FFS. The O-CU-CP 221 is a logical node hosting the RRC and the control plane (CP) part of the PDCP protocol. The O-CU-UP 222 is a logical node hosting the user plane part of the PDCP protocol and the SDAP protocol.

[0021] An E2 interface terminates at a plurality of E2 nodes. The E2 nodes are logical nodes/entities that terminate the E2 interface. For NR/5G access, the E2 nodes include the O-CU-CP 221, O-CU-UP 222, O-DU 215, or any combination of elements as defined in Reference [R04]. For E-UTRA access the E2 nodes include the O-e/gNB 210. As shown in FIG. 2, the E2 interface also connects the O-e/gNB 210 to the Near-RT RIC 214. The protocols over E2 interface are based exclusively on Control Plane (CP) protocols. The E2 functions are grouped into the following categories: (a) near-RT RIC 214 services (REPORT, INSERT, CONTROL and POLICY, as described in Reference [R04]); and (b) near-RT RIC 214 support functions, which include E2 Interface Management (E2 Setup, E2 Reset, Reporting of General Error Situations, etc.) and Near-RT RIC Sendee Update (e.g., capability exchange related to the list of E2 Node functions exposed over E2).

[0022] FIG. 2 shows the Uu interface between a UE 201 and O-e/gNB 210 as well as between the UE 201 and O-RAN components. The Uu interface is a 3 GPP defined interface (see e.g., sections 5.2 and 5.3 of Reference [R08]), which includes a complete protocol stack from LI to L..3 and terminates in the NG-RAN or E-UTRAN. The O-e/gNB 210 is an LIE eNB (see Reference [R05]), a 5G gNB or ng-eNB (see Reference [R07]) that supports the E2 interface. The O-e/gNB 210 may be the same or similar as discussed in FIGS. 3-8. The UE 201 may correspond to UEs discussed with respect to FIGS. 3-8 and/or the like. There may be multiple UEs 201 and/or multiple O-e/gNB 210, each of which may be connected to one another the via respective Uu interfaces. Although not shown in FIG. 2, the O- e/gNB 210 supports O-DU 215 and 0-R.U 216 functions with an Open Fronthaul interface between them.

[0023] The Open Fronthaul (OF) interface(s) is/are between O-DU 215 and O- RU 216 functions (see References [R02] and [R13].) The OF interface(s) includes the Control User Synchronization (CUS) Plane and Management (XI) Plane.

FIGS. 1 and 2 also show that the O-RU 216 terminates the OF M-Plane interface towards the O-DU 215 and optionally towards the SMO 202 as specified in Reference [R02], The O-RU 216 terminates the OF CUS-Plane interface towards the O-DU 215 and the SMO 202. [0024] The Fl-c interface connects the O-CU-CP 221 with the O-DU 215. As defined by 3 GPP, the Fl-c interface is between the gNB-CU-CP and gNB-DU nodes (see References [ROS] and [R11].) However, for purposes of O-RAN, the Fl-c interface is adopted between the O-CU-CP 221 with the O-DU 215 functions while reusing the principles and protocol stack defined by 3GPP and the definition of interoperability profile specifications.

[0025] The Fl-u interface connects the O-CU-UP 222 with the O-DU 215. As defined by 3GPP, the Fl-u interface is between the gNB-CU-UP and gNB-DU nodes (see References [R07] and [R10]). However, for purposes of O-RAN, the F1-u interface is adopted between the O-CU-UP 222 with the O-DU 215 functions while reusing the principles and protocol stack defined by 3GPP and the definition of interoperability profile specifi cati ons.

[0026] The NG-c interface is defined by 3GPP as an interface between the gNB- CU-CP and the AMF in the 5GC (see Reference [R07]). The NG-c is also referred as the N2 interface (see Reference [R07]). The NG-u interface is defined by 3GPP, as an interface between the gNB-CU-UP and the UPF in the 5GC (see Reference [R07]). The NG-u interface is referred as the N3 interface (see Reference [R07]). In O-RAN, NG-c and NG-u protocol stacks defined by 3 GPP are reused and may be adapted for O-RAN purposes. [0027] The X2-c interface is defined in 3GPP for transmitting control plane information between eNBs or between eNB and en-gNB in EN-DC. The X2-u interface is defined in 3 GPP for transmitting user plane information between eNBs or between eNB and en-gNB in EN-DC (see e.g., [005], [006]). In 0-RAN, X2- c and X2-u protocol stacks defined by 3GPP are reused and may be adapted for 0-RAN purposes .

[0028] The Xn-c interface is defined in 3GPP for transmitting control plane information between gNBs, ng-eNBs, or between an ng-eNB and gNB. The Xn-u interface is defined in 3GPP for transmitting user plane information between gNBs, ng-eNBs, or between ng-eNB and gNB (see e.g., References [ROS] and [R10]). In 0-RAN, Xn-c and Xn-u protocol stacks defined by 3GPP are reused and may be adapted for O-RAN purposes

[0029] The El interface is defined by 3GPP as being an interface between the gNB-CU-CP (e.g., gNB-CU-CP 3728) and gNB-CU-UP (see e.g., [007], [009]). In O-RAN, El protocol stacks defined by 3GPP are reused and adapted as being an interface between the O-CU-CP 221 and the O-CU-UP 222 functions.

[0030] The O-RAN Non-Real Time (RT) RAN Intelligent Controller (RIC) 212 is a logical function within the SMO framework 102, 202 that enables non-real- time control and optimization of RAN elements and resources; Al/machine learning (ML) workflow(s) including model training, inferences, and updates; and policy-based guidance of applications/features in the Near-RT RIC 214.

[0031] The O-RAN near-RT RIC 214 is a logical function that enables near-real- time control and optimization of RAN elements and resources via fine-grained data collection and actions over the E2 interface. The near-RT RIC 214 may include one or more AI/ML workflow's including model training, inferences, and updates.

[0032] The non-RT RIC 212 can be an ML training host to host the training of one or more ML models. The ML data can collected from one or more of the following: the Near-RT RIC 214, O-CLLCP 221, O-CU-UP 222, O-DU 215, O- RU 216, external enrichment source 110 of FIG. 1, and so forth. For supervised learning, and the ML training host and/or ML inference host/actor can be part of the non-RT RIC 212 and/or the near-RT RIC 214. For unsupervised learning, the ML training host and ML inference host/actor can be part of the non-RT RIC 212 and/or the near-RT RIC 214. For reinforcement learning, the ML. training host and ML inference host/actor are co-located as part of the near-RT RIC 214. In some implementations, the non-RT RIC 212 may request or trigger ML model training in the training hosts regardless of where the model is deployed and executed. ML models may be trained and not currently deployed.

[0033] In some implementations, the non-RT RIC 212 provides a query-able catalog for an ML designer/ developer to publish/install trained ML models (e.g., executable software components). In these implementations, the non-RT RIC 212 may provide discovery mechanism if a particular ML model can be executed in a target ML, inference host (MF), and what number and type of ML. models can be executed in the target ML inference host. The Near-RT RIC 214 is a managed function (MF). For example, there may be three types of ML catalogs made discoverable by the non-RT RIC 212: a design-time catalog (e.g., residing outside the non-RT RIC 212 and hosted by some other ML platform(s)), a training/ depl oym ent- time catalog (e.g., residing inside the non-RT RIC 212), and a run-time catalog (e.g., residing inside the non-RT RIC 212). The non-RT RIC 212 supports necessary capabilities for ML. model inference in support of ML assisted solutions running in the non-RT RIC 212 or some other ML inference host. These capabilities enable executable software to be installed such as VMs, containers, etc. The non-RT RIC 212 may also include and/or operate one or more ML engines, which are packaged software executable libraries that provide methods, routines, data types, etc., used to run ML models. The non-RT RIC 212 may also implement policies to switch and activate ML model instances under different operating conditions.

[0034] The non-RT RIC 22 is able to access feedback data (e.g., FM, PM, and network KPI statistics) over the 01 interface on ML model performance and perform necessary' evaluations. If the ML model fails during runtime, an alarm can be generated as feedback to the non-RT RIC 212. How well the ML model is performing in terms of prediction accuracy or other operating statistics it produces can also be sent to the non-RT RIC 212 over 01. The non-RT RIC 212 can also scale ML model instances running in a target MF over the 01 interface by observing resource utilization in MF. The environment where the ML model instance is running (e.g., the MF) monitors resource utilization of the running ML model. This can be done, for example, using an ORAN-SC component called ResourceMonitor in the near-RT RIC 214 and/or in the non-RT RIC 212, which continuously monitors resource utilization. If resources are low or fall below a certain threshold, the runtime environment in the near-RT RIC 214 and/or the non- RT RIC 212 provides a scaling mechanism to add more ML instances. The scaling mechanism may include a scaling factor such as an number, percentage, and/or other like data used to scale up/down the number of ML instances. ML model instances running in the target ML inference hosts may be automatically scaled by observing resource utilization in the MF. For example, the Kubemetes® (K.8s) runtime environment typically provides an auto-scaling feature.

[0035] The Al interface is between the non-RT RIC 212, which is within - the SMO 202) and the near-RT RIC 214. The Al interface supports three types of services as defined in Reference [R12], including a Policy Management Service, an Enrichment Information Service, and ML Model Management Service. Al policies have the following characteristics compared to persistent configuration as defined in Reference [R 12] : A l policies are not critical to traffic; Al policies have temporary validity; Al policies may handle individual UE or dynamically defined groups of UEs; Al policies act within and take precedence over the configuration; and Al policies are non-persistent, i.e., do not survive a restart of the near-RT RIC.

[0036] A technical problem is how to train and deploy AI/ML in the Near-RT RIC with lower latency and how to manage AI/ML to ensure adequate or better performance. Embodiments disclosed herein address these technical problems.

[0037] O-RAN currently considers model management for AI/ML models trained offline at SMO or Non-RT RIC, Without proper management mechanism, poorly designed online learning algorithm may jeopardize RAN performance. [0038] Artificial intelligence (Al) and machine learning (ML) may be used in RANs. RANs are often disaggregated and virtual, which enables more flexible RAN topologies and moves RAN intelligence towards the edge of the cloud. The non-RT RIC and Near-RT RIC may include AI/ML capability. The AI/ML- based RAN management solution may be under the control of the operators via the sendee management and orchestration (SMO) entity in the O-RAN framework.

[0039] In some embodiments, the O-RAN AI/ML deployment includes online learning located in Near-RT RIC. For online learning, the latency for SMO to perform model validation/certification before model deployment may be too long and may include management mechanisms to allow the model to be locally validated and then deployed at Near-RT RIC. In some embodiments an online training host at Near-RT RIC is disclosed and a new signaling and APIs for online learning model management are disclosed. [0040] In some embodiments an online training host at Near-RT RIC is a module that is part of Near-RT RIC platform. In some embodiment an online training host at Near-RT RIC is an xApp. In some embodiments, new⁷ signaling and APIs are disclosed for certifying/authorizing online training host at Near-RT RIC and for specifying detail model management configurations, such as model validation criteria, performance feedback message, and so forth.

[0041] FIG. 3 illustrates a system 300 for online learning at a Near-RT RIC, in accordance with some embodiments. Illustrated in FIG. 3 is SMO framework 302, Non-RT RIC 304, 01 306, Al 308, 01 termination 310, Al termination 312, Near-RT RIC 314, open APIs for xAPP 316, xAPPl 318, training host xAPP 320, xAPP N 322, messaging infrastructure 324, conflict management 326, xAPP subscription management (MGMT) 326, management sendees 330, security 332, shared data layer 334, database 336, E2 termination 338, E2 nodes 340, training host 342, and E2 344.

[0042] For O-RAN AI/ML deployment for online reinforcement learning in Near-RT RIC the training host 342, which is located in or resides in (or is logically part of) Near-RT RIC 314, and is configured to enable online learning. [0043] In some embodiments the training host 342 module provides functions in the Near-RT RIC platform 314 for managing AI/ML. In some embodiments, an xAPP that is training host xAPP 320 provides functions in the Near-RT RIC platform 314 for managing AI/ML.

[0044] Trained Al model needs to be validated and tested by the training host, e.g., training host xAPP 320 or training host 342, before being deployed to an inference host to perform network control/guidance actions according to observation from the network, e.g. E2 nodes 340. Network control actions are managed and controlled by the SMO framework 302. Management/control mechanisms from SMO framework 302 to the training host at the Near-RT RIC 314 ensure the training host is certified or authorized to deploy the trained Al models to the inference host. Management signaling between the SMO framework 302 and Near-RT RIC 314 and the training host using management APIs to enable online training processes that are manageable and controllable by SMO framework 302 are disclosed.

[0045] The non-RT RIC 304 and/or SMO framework 302 accesses feedback data (e.g., FM, PM, and network KPI statistics) over the 01 306 interface on ML model performance and performs necessary evaluations.

[0046] The AI/ML training in Near-RT RIC 314 offers training of applications (xApps) within Near-RT RIC 314. The AI/ML training provides generic and use case-independent capabilities to ALML-based applications, e.g., xApp requiring online learning 404.

[0047] FIG. 4 illustrates a system 400 for online learning at a Near-RT RIC, in accordance with some embodiments. Training host 342 is part of the Near-RT RIC 314 platform. The Near-RT RIC 314 implements a training host 342 function as part of the platform. The training host 342 offers training sendees to xApps, e.g., xAPP requiring online learning 404, that require online training at the Near-RT RIC 314. The SMO framework 302 manages training host 342 and xApps requiring online training 404 via 01 306 interface via 01 termination 310. The online training management xApp API 402 enables interaction between the training host 342 and xApp requiring online training 404. [0048] The training host 342 is part of the basic management services of the

Near-RT RIC 314. The basic management sendees include FCAPS (fault, configuration, accounting performance, security) and so forth and are supported as described in [R04], Embodiments disclose management mechanisms for supporting online learning. [0049] FIG. 5 illustrates a method 500 for online training certification, in accordance with some embodiments. Illustrated in FIG. 5 is SMO framework 302, Near-RT RIC 314, message 502, online training certificate 504, message 506, and online training certificate RSP 508. [0050] The training host 342 on Near-RT RIC 314 performs online training services to xAPP requiring online learning 404 that are certified by the SMO framework 302, The method 500 for certification can be initiated by the SMO framework 302 or by the training host 342 on Near-RT RIC 314. The message 502 for the certification process is transmitted over 01 306. [0051] The certification process may be initiated by the SMO framework 302 as follows. The SMO framework 302 sends the ‘Online Training Certificate’ 504 in message 502 to the Near-RT RIC 314. The Near-RT RIC 314 replies with ‘Online Training Certificate Response’ 508 in message 506.

[0052] FIG. 6 illustrates a method 600 for online training certification, in accordance with some embodiments. Illustrated in FIG. 6 is SMO framework 302, Near-RT RIC 314, message 602, online training certificate request (req) 604, message 606, online training certificate 608, message 610, and online training response (RSP) 612.

[0053] The certification process can be initiated by Near-RT RIC 314 as follows. The Near-RT RIC 314 sends the ‘Online Training Certificate Request’ 604 message 602 to SMO framework 302. The SMO framework 302 sends the ‘Online Training Certificate’ 608 message 606 to Near-RT RIC 314. The Near- RT RIC 314 replies with ‘Online Training Certificate Response (RSP)’ 612 message 610. [0054] The ‘Online Training Certificate’ 504, 608, and/or the message 502, 606 may include one or more of the following. The certificate ID, the online training services that the training host 342 on Near-RT RICs 314 are certified to perform. The training services may include one or more of the following: the certified online training algorithms to be performed, such as multi-arm bandit, deep-Q learning, deep deterministic policy gradient, and so forth; the certified online training libraries/packages; and, the xApp categories that are certified to use the online training sendees where the xApp category can be identified by an xApp category ID included in the xApp descriptor. [0055] The ‘Online Training Certificate’ 504, 608, and/or the message 502, 606 may further include one or more of the following. The time period that, this certificate is valid. The performance monitoring requirement for online training where the performance monitoring requirement may include training performance indication, such as training convergence speed, etc.; related E2 measurements to be monitored; and/or, the frequency to report performance monitoring metric from Near-RT RIC 314 back to SMO framework 302.

[0056] The report of the performance monitoring metrics reuses existing 01 specifications for performance management, in accordance with some embodiments.

[0057] The ‘Online Training Certificate Response’ 508, 612 and/or message

506, 610 may include one or more of the following. The certificate ID. The indication of whether the training host 342 is updated with the new certificate (ACK or NAK). Near-RT RIC 314 may be referenced when the training host 342 may be originating the messages 506, 602, 610.

[0058] The training host 342 at Near-RT RIC 314 may send the ‘Online

Training Certificate Request’ 604 in message 602 to the SMO framework 302.

For example, transmission of the ‘Online Training Certificate Request’ 604 message 602 can be triggered when the last received certificate expired. The ‘Online Training Certificate Request’ 604 in message 602 may be used to expose the online training capability of the Near-RT RIC 314 to the SMO framework 302. The ‘Online Training Certificate Request’ 604 in message 602 may include one or more of the following. The last issued certificate ID. The full content of the last certificate. The capability of the online training host 342 at the Near-RT RIC 314, The capability of the online training host 342 may include the online learning algorithm s/libraries/packages supported by the online training host 342, which may be in the form of online learning model catalogues.

[0059] The xAPP requiring online learning 404 may have one or more requirements or preconditions before it may use the training host 342. For example, an xApp requiring online learning 404 that utilizes the online training function at Near-RT RIC 314 is to be certified before being published from SMO framework 302 to the Near-RT RIC 314 run-time library. The lifecycle management for the xApp requiring online learning 404 is similar to typical xApps as disclosed in [ROS], except that, during runtime lifecycle, online training performance should also be monitored and online training may be deactivated/reactivated based on performance monitoring. The SMO framework 302 determines whether to certify an xApp requiring online learning 404 to run with the online training function at Near-RT RIC 314, the condition to activate/deactivate the online learning process for the xApp requiring online learning 404 and termination condition for the xApp requiring online learning 404.

[0060] The SMO framework 302 publishes the xApp requiring online learning 404 to a Near-RT RIC 314 that is certified to do the online training tasks requested or needed by the xApp requiring online learning 404. The SMO framework 302 and/or Near-RT RIC management service 328 monitors whether the certificate for performing the required online training task is still valid during the xApp requiring online learning 404 runtime.

[0061] The SMO framework 302 is aware of the online training certificate validity for the Near-RT RIC 314 based on the last ‘Online Training Certificate’ and ‘Online Training Certificate Response’ messages and can use 01 306 signaling to terminate the xApp requiring online learning 404 if the online training certificate expires. The Near-RT RIC 314 keeps a local copy of the latest online training certificate and is aware of the online training requirement for the xApp requiring online learning 404 based on the descriptor of the xApp requiring online learning 404.

[0062] The Near-RT RIC 314 may terminate the xApp requiring online learning 404 via online training management API, e.g., xAPP API 402, if the online training certificate expires. Or, alternatively, an xApp requiring online tearing 404 or training sendees from Near-RT RIC 314 may be deployed without being pre-certified by the SMO framework 302.

[0063] In this case, the Near-RT RIC 314 may reject to deploy an xApp requiring online learning 404 from the beginning if the required online training services described in the xApp descriptor cannot be supported by the training host 342.

[0064] Or, once successfully deployed, the online training management API can be used to subscribe to the requested or needed onl ine training service of the xApp to the training host, for which the training host may further request the SMO framework 302 to certify (if such service is not certified before or no longer valid) or to provide conditions for activate/deactivate/termination.

[0065] An xApp that utilizes the online training function at the Near-RT RIC

314 should include online training related information in the xApp descriptor. In addition to the xApp descriptor definition described in [R04], the descriptor for xApp requiring online learning may include one or more of the following: the online training xApp category ID; the type of online training algorithm the xApp plan to use; the online training libraries/packages the xApp prefers using; the compute/memory requirement, for online training; activation/deactivation condition for running online training for the xApp; and/or the Termination condition for the xApp based on online training performance.

[0066] /\n xApp that will or would like to utilize the online training function at the Near-RT RIC can interact with the training host 342 at the Near-RT RIC 314 via the xApp API 402. [0067] The API to support online training may include one or more of (A), (B),

(C), (D). and/or (E).

[0068] (A) API for subscribing online training sendee: xApp can use this API to register to the training host.

[0069] (B) API for performance monitoring: xApp can use the API to request training host to create log/traces for training performance to be reported to SMO via 01.

[0070] (C) API for training activation/deactivation and termination: The API can also define the target reward value for reinforcement learning and specify the relationship between the target reward and termination/activation/deactivation condition for training.

[0071] (D) API for model deployment: this API can be used for the training host to deploy the trained model to the xApp utilizing online training sendee.

[0072] (E) API for training configurations; API can be defined to specify training related configurations, e.g., Training data preparation: API to specify which online E2 measurements or data in R-NIB (Radio-Network Information Base) or UE-NIB should be mapped into training data traces. For reinforcement learning, there can be API specifying how the data is mapped to state and action and how reward is calculated. The training data preparation may include API to specify the batch size for training, batch sampling strategy, etc. The API for training may include APIs to interact with online training packages/library: specify neural net structure, and so forth.

[0073] FIG. 7 illustrates a system 700 for online learning at a Near-RT RIC, in accordance with some embodiments. An xAPP performing online learning 328 interacts with management sendees 328 via xAPP API 402 and SMO framework 302 via 01 & 02 termination 328. The training host is the xAPP performing online learning 328.

[0074] Online training is enabled by an xApp, e.g., xAPP performing online learning 328. In some embodiments there are two tightly coupled xApps, one handling online training, e.g., xAPP performing online learning 328, and the other handling online inference, or there can be one single xApp performing both training and inference, e.g., xAPP performing online learning 328 may be performing the training and the inferences on an inference host (not illustrated). The SMO framework 302 manages online training xApps, xAPP performing online learning 328, via 01 and 02 306. The SMO framework 302 may manage the xApp that is performing the inference with the inference host. A management API is configured to enable the management of online training xApp. An xApp with online training capability and an xApp utilizing online training capability from another xApp are certified before being published from the SMO framework 302 to the run-time library of the Near-RT RIC 314.

[0075] The lifecycle management for the xApp performing online training or the xApp utilizing online training capability from another xApp is similar to typical [0076] xApps as described in [R05]; however, during runtime lifecycle, online training performance is monitored and the termination condition for the xApp running or utilizing online training is related to online training performance. [0077] The SMO framework 302 determines whether to certify an xApp with online training capability and determines the termination condition for the xApp. The SMO framework 302 determines whether to certify an xApp utilizing online training capability from another xApp and the termination condition for the xApp utilizing online training capability from another xApp.

[0078] In some embodiments a training xApp or an xApp utilizing online training capability from another xApp can be deployed without, being pre- certified where a certification process between the xApp and SMO framework 302 follows, e.g., as disclosed in FIGS. 5 and 6, where there is a certification process or method between the training host, e.g., residing in the Near-RT RIC 314, and SMO framework 302 above.

[0079] ,An xApp with online training capability includes online training related information in the xApp descriptor. In addition to the xApp descriptor described in [R04], the descriptor for online training xApp includes one or more of (A), (B), and/or (C). (A) The types of online training algorithms the xApp supports. (B) Compute/memory requirement for online training. (C) Termination condition for the online training xApp based on online learning performance. [0080] The xApp API 402 supports an xApp with online training capability. For example, the xAPP API 402 may include one or more (A), (B), (C), (D), and/or (E). (A) API for subscribing and announcing as an “online training host” in the platform: an online training xApp uses this API to register itself as an online training host to the Near-RT RIC platform (so that it can be reached by online learning x.Apps if any). [0081] (B) API for online learning management: new API for the management function at Near-RT RIC to control online learning xApps according to messages received via 01 from SMO. E.g., an API for deactivating/reactivating online training and/or an API for updating online training hyperparameter setting (or Al from Near-RT RIC). [0082] (C) Performance monitoring: xApp can use the API to request training host to create log/traces for training performance to be reported to SMO via 01. [0083] (D) API for model deployment: this API can be used for the training host to deploy the trained model to the xApp utilizing online training service.

[0084] (E) API for training configurations: API can be defined to specify training related configurations, e.g,, one or more of the (I) and/or (2), [0085] 1) Training data preparation: an API to specify which online E2 measurements or data in R-NIB (Radio-Network Information Base) or UE-NIB should be mapped into training data traces. For reinforcement learning, there can be API specifying how the data is mapped to state and action and how reward is calculated and/or an API to specify the batch size for training, batch sampling strategy, and so forth. 2) An API to interact with online training packages/library: specify neural net structure, and so forth.

[0086] FIG. 8 illustrates a method 800 for online training certification, in accordance with some embodiments. The method 800 begins at operation 802 with invoking, by an application of the Near-RT RIC, a training host function of a training host, the application running an artificial intelligence (AI)/machine learning (ML) inference model. For example, the xAPP requiring online learning 404 may invoke a training host function of the training host 342 using the xAPP API 402.

[0087] The method 800 continues at operation 804 with performing, by the training host, the training host function, the training host function providing an online training service for the application. For example, the training host 342 may perform the training host function invoked by the XApp requiring online learning 404. The method 800 may include one more additional operations such as returning, by the training host, results of performing the training host function to the application, and modifying, by the application, the AI/ML inference model based on the results.

[0088] The methods described in conjunction with FIGS. 3-8 may include one or more additional operations. The operations of the methods described in conjunction with FIGS, 3-8 may be performed in a different order. One or more of the operations of the methods described in conjunction with FIGS. 3-8 may be optional. The operati ons of the methods described in conjunction with FIGS. 3-8 may be performed by an apparatus of a Near-RT RIC or an apparatus of a SMO, in accordance with some embodiments.

[0089] The following are additional examples. Example 1 includes A Near- RT RIC including a training host function performing online training services, and where an xApp uses services provided by the training host function and runs AI/ML inference model(s) trained by the training host function, and includes management services framework function providing management service for xApps, and an 01 termination for interaction with SMO.

[0090] In Example 2, the subject matter of Example 1 includes where the training host function monitors performance of online training processes and generates performance monitoring reports that are forwarded to the management services framework function and to SMO via the 01 Termination.

[0091] In Example 3, the subject matter of Examples 1 and 2 includes where the training host function is deployed as a native framework function within Near-RT RIC platform. [0092] In Example 4, the subject matter of Examples 1-3 includes where the training host function requires a valid online training certificate provided by SMO to perform online training sendees.

[0093] In Example 5, the subject matter of Examples 1-4 includes where 5 the 01 Termination receives the online training certificate from SMO, where the online training certificate further contains information about the online training service(s) that the training host function is certified to perform.

[0094] In Example 6, the subject matter of Examples 1-5 includes where the 01 Termination receives the online training certificate from SMO, where the online training certificate further contains information about the xApp categories that are certified to use the online training sendees provided by the training host function.

[0095] In Example 7, the subject matter of Examples 1-6 includes where the 01 Termination receives the online training certificate from SMO, where the online training certificate further contains information about conditions for the certificate to be valid.

[0096] In Example 8, the subject matter of Examples 1-7 includes where the 01 Termination sends an acknowledgement message to SMO in response to the reception of the online training certificate, where the acknowledgement message contains information about the current online training certificate stored at Near- RT RIC.

[0097] In Example 9, the subject matter of Examples 1-8 includes where 9 the 01 Termination sends a request message for a certificate to SMO prior to receiving the online training certificate from SMO, where the request message contains information about, the last certificate stored at Near-RT RIC.

[0098] In Example 10, the subject matter of Examples 1-9 includes where 10 the 01 Termination sends a request message for a certificate to SMO prior to receiving the online training certificate from SMO, where the request message further contains information about the online training capability for the training host function at Near-RT RIC.

[0099] In Example 11, the subject matter of Examples 1-10 includes where the xApp using training services from the training host function is certified and deployed to Near-RT RIC by SMO, where the online training certificate validity for the training services required for the xApp is verified by SMO. [00100] In Example 12, the subject matter of Examples 1-11 includes where the xApp requiring services from the training host function is admitted to be deployed when there is valid the online training certificate for the training sendees required for the xApp and is rejected to be deployed when there is no valid online training certificate for the training sendees required for the xApp.

[00101] 13. The Near-RT RIC in Claim 3, wherein the training host function receives instructions from the SMO via 01 termination to activate or deactivate a training process.

[00102] In Example 14, the subject matter of Examples 1-13 includes where the xApp using training sendees from the training host function is deployed along with an xApp descriptor, where the xApp descriptor includes online training information for the xApp.

[00103] In Example 15, the subject matter of Examples 1-14 includes where the xApp using training sendees from the training host function is deployed along with an xApp descriptor, where the xApp descriptor further includes an xApp category' information.

[00104] In Example 16, the subject matter of Examples 1-15 includes where the xApp using training services from the training host function is deployed along with an xApp descriptor, where the xApp descriptor further includes conditions to activate or deactivate a training process for the xApp and conditions to terminate the xApp.

[00105] In Example 17, the subject matter of Examples 1-16 includes where the xApp using training services from the training host function interacts with the training host function via online training management xApp API, where the interaction includes the xApp subscribing training services from the training host function and the training host function admitting or rejecting training services to the xApp.

[00106] In Example 18, the subject matter of Examples 1-17 includes where the training host function deployed trained AI/'ML inference models to the xApp and the xApp provide training configurations to the training host via online training management xApp API.

[00107] In Example 19, the subject matter of Examples 1-18 includes where the training host function is deployed as part of an xApp. [00108] In Example 20, the subject matter of Examples 1-19 includes where the xApp providing training host function is certified and deployed to Near-RT RIC by SMO, and an xApp requiring training sendees the xApp providing training host function is certified and deployed to Near-RT RIC by SMO, where online training certificate validity for the training services required for the xApp is verified by SMO.

[00109] In Example 21, the subject matter of Examples 1-20 includes where the xApp providing training host function is deployed with an xApp descriptor, where the xApp descriptor includes information about the online training services the xApp supports.

[00110] In Example 22, the subject matter of Examples 1-21 includes where the xApp providing training host function is deployed with an xApp descriptor, where the xApp descriptor further includes conditions to activate and deactivate a training process and conditions to terminate online training xApp. [00111] In Example 23, the subject matter of Examples 1-22 includes where the xApp providing training host function exchanges messages relating to online training operation with other Near-RT RIC platform functions and xApps via online training xApp API, where messages relating to online training operation include receiving training related configurations and deploying trained AI/ML models.

[00112] In Example 24, the subject matter of Examples 1 -23 includes where the xApp providing training host function exchanges messages relating to online training management with other Near-RT RIC platform functions and xApps via online training management xApp API, where the messages relating to online training management include control signaling to subscribe for online training sendees from the online training xApp and the control signaling for the online training xApp to announce its training capability to Near-RT platform.

[00113] In Example 25, the subject matter of Examples 1-24 includes where the xApp providing training host function exchanges messages relating to online training management with other Near-RT RIC platform functions and xApps via online training management xApp API, where the messages relating to online training management further include control signaling to activate and deactivate an online training process, control signaling to terminate the online training xApp and signaling to report online training performance. REFERENCES

[00114] The following references are included herein by reference.

[00115] [R01] O-RAN Alliance Working Group 1, O-RAN Operations and Maintenance Interface Specification, version 2.0 (Dec 2019) (“O-RAN- WG1 .01 -Interface-v02.00”).

[00116] [R02] O-RAN Alliance Working Group 4, O-RAN Fronthaul Management Plane Specification, version 2.0 (July 2019) (“ORAN-WG4.MP.O- v02.00.00”). [00117] [R03] O-RAN Alliance Working Group 1, O-RAN Operations and

Maintenance Architecture Specification, version 2.0 (Dec 2019) (“O-RAN- WGl .OAM-.Architecture-v02.00”).

[00118] [R04] O-RAN Alliance Working Group 3, Near-Real-time RAN Intelligent Controller Architecture & E2 General Aspects and Principles (“ORAN-WG3.E2GAP.0-vO.1”).

[00119] [R05] 3GPP TS 36.401 vlS. l .O (2019-01-09).

[00120] [R06] 3GPP TS 36.420 vl 5.2.0 (2020-01-09).

[00121] [R07] 3GPP TS 38.300 vl 6.0.0 (2020-01-08).

[00122] [R08] 3GPP TS 38.401 vl6.0.0 (2020-01-09). [00123] [R09] 3GPP TS 38.420 v!5.2.0 (2019-01-08).

[00124] [R10] 3GPP TS 38.460 v!6.0.0 (2020-01-09).

[00125] [R11] 3GPP TS 38.470 vl6.0.0 (2020-01-09).

[00126] [R12J O-RAN Alliance Working Group 2, O-RAN Al interface:

General Aspects and Principles Specification, version 1.0 (Oct 2019) (“ORAN- WG2.A1.GA&P-vOl .00”).

[00127] [ R 13 ] O-RAN Alliance Working Group (WG) 4, O-RAN Fronthaul

Control, User and Synchronization Plane Specification, version 2.0 (July 2019) (“ORAN-WG4.CUS.0-v02.00”).

[00128] [R14] O-RAN Working Group 3 Technical Specification, “Near-Real- time RAN Intelligent Controller; Near-RT RIC Architecture”, version 1.0.

[00129] [R15] O-RAN “Application Life Cycle Management (LCM)” [00130] [R 16] O-RAN WG1 , “O-RAN Architecture Description”.

[00131] [R17] O-RAN WG2, ““AI/ML Workflow Description and Requirements”. [00132] [R18] O-RAN WG2, “Non-RT RIC Functional Architecture’

TERMINOLOGY

[00133] The term “application” may refer to a complete and deployable package, environment to achieve a certain function in an operational environment.

The term “AI/ML application” or the like may be an application that contains some AI/ML models and application-level descriptions.

[00134] The term “machine learning” or “ML” refers to the use of computer systems implementing algorithms and/or statistical models to perform specific task(s) without using explicit instructions, but instead relying on patterns and inferences. ML algorithms build or estimate mathematical model(s) (referred to as “ML models” or the like) based on sample data (referred to as “training data,” “model training information,” or the like) in order to make predictions or decisions without being explicitly programmed to perform such tasks. Generally, an ML algorithm is a computer program that learns from experience with respect to some task and some performance measure, and an ML model may be any object or data structure created after an ML algorithm is trained with one or more training datasets. After training, an ML model may be used to make predictions on new datasets. Although the term “ML algorithm” refers to different concepts than the term “ML model,” these terms as discussed herein may be used interchangeably for the purposes of the present disclosure.

[00135] The term “machine learning model,” “ML model,” or the like may also refer to ML methods and concepts used by an ML. -assisted solution. An “ML- assisted solution” is a solution that addresses a specific use case using ML algorithms during operation. ML models include supervised learning (e.g., linear regression, k-nearest neighbor (KNN), descision tree algorithms, support machine vectors, Bayesian algorithm, ensemble algorithms, etc.) unsupervised learning (e.g., K-means clustering, principle component analysis ( PC A). etc.), reinforcement learning (e.g., Q-leaming, multi-armed bandit learning, deep RL, etc.), neural networks, and the like. Depending on the implementation a specific

ML model could have many sub-models as components and the ML model may train all sub-models together. Separately trained ML models can also be chained together in an AIL pipeline during inference. An “ML pipeline” is a set of functionalities, functions, or functional entities specific for an ML-assisted solution; an ML pipeline may include one or several data sources in a data pipeline, a model training pipeline, a model evaluation pipeline, and an actor. The “actor” is an entity that hosts an ML assisted solution using the output of the ML model inference). The term “ML- training host” refers to an entity, such as a network function, that hosts the training of the model. The term “ML inference host” refers to an entity, such as a network function, that hosts model during inference mode (which includes both the model execution as well as any online learning if applicable). The ML-host informs the actor about the output of the ML algorithm, and the actor takes a decision for an action (an “action” is performed by an actor as a result of the output of an ML assisted solution). The term “model inference information” refers to information used as an input to the ML model for determining inference(s); the data used to train an ML model and the data used to determine inferences may overlap, however, “training data” and “inference data” refer to different concepts. [00136] Although an aspect has been described with reference to specific exemplary aspects, it will be evident that various modifications and changes maybe made to these aspects without departing from the broader scope of the present disclosure. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. This Detailed Description, therefore, is not to be taken in a limiting sense, and the scope of various aspects is defined only by the appended claims, along with the full range of equivalents to which such claims are entitled.

Claims

CLAIMS What is claimed is:

1. An apparatus for a Near real-time (Near-RT) radio access network intelligence controller (RIC)(Near-RT RIC) in an open radio access network (O-RAN), the apparatus comprising: memory; and, processing circuitry coupled to the memory, the processing circuitry configured to: invoke, by an application of the Near-RT RIC, a training host function of a training host, the application running an artificial intelligence (AI)/machine learning (ML) inference model; and perform, by the training host, the training host function, the training host function providing an online training sendee for the application.

2. The apparatus of claim 1 wherein the application is an xAPP of the Near-RT RIC.

3. The apparatus of claim 1 wherein the processing circuitry is further configured to: return, by the training host, results of performing the training host function to the application; and modify, by the application, the AI/ML inference model based on the results.

4. The apparatus of claim 1 wherein the training host is a native framework function of the Near-RT RIC.

5. The apparatus of claim 1 wherein the processing circuitry is further configured to: perform, by a management sendees framework of the Near-RT RIC, a management service function for the application or the training host.

6. The apparatus of claim 1 wherein the processing circuitry is further configured to: receive, by art 01 termination of the Near-RT RIC, messages for a services management and organization (SMO), the messages received from the training host or the application, and send, by the 01 termination, the messages to the SMO via an 01 interface.

7. The apparatus of claim 6 wherein the processing circuitry is further configured to: monitor, by the training host, performance of online training processes of the application; generate, by the training host, performance monitoring reports based on the monitoring; and forward, by the training host, the monitoring reports to the SMO via the 01 termination.

8. The apparatus of claim 6 wherein the processing circuitry is further configured to: determine, by the application, whether the training host has a valid online training certificate provided by the SMO to perform online training services: and in response to the training host not having the valid online training certificate, refraining from invoking a training host function of the training host.

9. The apparatus of claim 6 wherein the processing circuitry' is further configured to: determine, by the Near-RT RIC, whether the training host has a valid online training certificate provided by the SMO to perform online training services; and in response to the training host not having the valid online training certificate, refraining from publishing the training host as available to the application.

10. The apparatus of claim 9 wherein the online training certificate comprises indications of which training host functions the training host is certified to perform and further comprises application categories that are certified to use the training host functions.

11. The apparatus of claim 6 wherein the processing circuitry' is further configured to: receive via the 01 termination, by the training host from the SMO, an online training certificate; and send via the 01 termination, by the training host, an online training certificate response to the SMO.

12. The apparatus of claim 11 wherein the processing circuitry is further configured to: before the receive, send via the 01 termination, by the training host, an online training certification request to the SMO.

13. The apparatus of claim 6 wherein the processing circuitry is further configured to: receive, from the SMO, an indication that, the application is certified to use training sendees from the Near-RT RIC

14. The apparatus of claim 6 wherein the processing circuitry is further configured to: receive, at the training host from the SMO via the 01 termination, an indication to activate or deactivate a training process, the training process a training of the AI/ML inference model.

15. The apparatus of claim 14 wherein the SMO deploys the application to the Near-RT RIC with an application descriptor, the application descriptor indicates online training information for the application and a category for the application.

16. The apparatus of claim 1 wherein the processing is further configured to: collect training data indicated by the training host function from the radio access network (RAN) via an E2 interface, wherein the application indicates training preferences to the training host, wherein the training preferences comprise one or more of the following: an indication of a loss function, an indication of an optimizer, an indication of a number of epochs, an indication of a number of interactions/batches, an indication of a learning rate, an indication of a split ratio of training and validation data, an indication of a gradient clipping, and an indication of a test data set, and wherein the training preferences comprise a training data description comprising a data type, a sampling rate, a sampling period, a granularity, and a condition.

17. A non-transitoiy computer-readable storage medium that stores instructions for execution by one or more processors of a Near real-time (Near- RT) radio access network intelligence controller (RIC)(Near-RT RIC) in an Open R,AN (O-RAN) network, the instructions to configure the one or more processors to perform the following operations: invoke, by an application of the Near-RT RIC, a training host function of a training host, the application running an artificial intelligence (AI)/machine learning (ML) inference model; and perform, by the training host, the training host function, the training host function providing an online training service for the application.

18. The non-transitory computer-readable storage medium of claim

17 wherein the application is an xAPP of the Near-RT RIC, and wherein the operations further comprise: return, by the training host, results of performing the training host function to the application; and modify, by the application, the AI/ML inference model based on the results.

19. A method performed on an apparatus of a Near real-time (Near- RT) radio access network intelligence controller (RIC)(Near-RT RIC) in an Open R,AN (O-RAN) network, the method comprising: invoking, by an application of the Near-RT RIC, a training host function of a ttaining host, the application running an artificial intelligence (AI)/machine learning (ML) inference model; and performing, by the training host, the training host function, the training host function providing an online training sendee for the application.

20. The method of claim 19 wherein the application is an xAPP of the Near-RT RIC, and wherein the method further comprises: returning, by the training host, results of performing the training host function to the application; and modifying, by the application, the AI/ML inference model based on the results.