CN115225478B - Network slice dynamic configuration method and device, electronic equipment and storage medium - Google Patents

Network slice dynamic configuration method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN115225478B
CN115225478B CN202110336480.9A CN202110336480A CN115225478B CN 115225478 B CN115225478 B CN 115225478B CN 202110336480 A CN202110336480 A CN 202110336480A CN 115225478 B CN115225478 B CN 115225478B
Authority
CN
China
Prior art keywords
slice
configuration
network
experience data
action
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110336480.9A
Other languages
Chinese (zh)
Other versions
CN115225478A (en
Inventor
黄永明
李进
张铖
尤肖虎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Network Communication and Security Zijinshan Laboratory
Original Assignee
Network Communication and Security Zijinshan Laboratory
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Network Communication and Security Zijinshan Laboratory filed Critical Network Communication and Security Zijinshan Laboratory
Priority to CN202110336480.9A priority Critical patent/CN115225478B/en
Publication of CN115225478A publication Critical patent/CN115225478A/en
Application granted granted Critical
Publication of CN115225478B publication Critical patent/CN115225478B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08—Configuration management of networks or network elements
    • H04L41/0803—Configuration setting
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08—Configuration management of networks or network elements
    • H04L41/0876—Aspects of the degree of configuration automation
    • H04L41/0886—Fully automatic configuration
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08—Configuration management of networks or network elements
    • H04L41/0889—Techniques to speed-up the configuration process
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08—Configuration management of networks or network elements
    • H04L41/0893—Assignment of logical groups to network elements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design
    • H04L41/142—Network analysis or design using statistical or mathematical methods
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Abstract

The invention provides a dynamic configuration method, a device, electronic equipment and a storage medium for network slices, which comprise the following steps: constructing a slice configuration experience data set; inputting the network state related to each slice configuration experience data into a slice pre-configuration model respectively, and obtaining a slice configuration action output by the slice pre-configuration model; based on Dropout calculation, acquiring variances and mean values of all slice configuration actions output by the slice pre-configuration model in various dimension variables, and assuming that the output of the slice pre-configuration model accords with normal distribution so as to construct a slice pre-configuration space; and determining the optimal slice configuration from the constructed slice pre-configuration space. The invention solves the problems that the traditional RAN slice configuration method is difficult to predict the flow and channel change, has low convergence speed and can not quickly provide the slice configuration decision oriented to SLA optimization, realizes the intellectualization and automation of the RAN slice, and improves the slice satisfaction rate and the slice configuration efficiency.

Description

Network slice dynamic configuration method and device, electronic equipment and storage medium
Technical Field
The present invention relates to the field of communications technologies, and in particular, to a method and apparatus for dynamically configuring a network slice, an electronic device, and a storage medium.
Background
With the wide commercial deployment of 5G mobile communication technology, it is important how to effectively guarantee differentiated service transmission requirements with low cost for digital vertical industry and industrial internet induced-growth of multiple emerging transmission services. As a 5G key enabling technology, network slicing utilizes network function virtualization and soft definition network control technology, and can operate multiple isolated logic networks on the same physical network to meet diversified service transmission requirements, and simultaneously improve network resource multiplexing gain.
The slice configuration of the traditional wireless access network (Wireless Access Network, RAN) mainly adopts a slice configuration method based on an optimization model, takes specific service quality and total resources as constraints, takes maximized network performance or operator benefits as targets, and solves the optimal slice resource configuration scheme through a classical iterative algorithm.
However, the slice resource management method based on the optimization model is too dependent on the prior flow model, and the computational complexity is significantly improved with the increase of the network scale. In the face of the great challenges caused by the random fluctuation of network traffic and channel quality and the sharp expansion of network scale caused by dense network deployment, which are caused by the mobility of users in a 5G (5 th generation mobile networks) wireless access network and the dynamic nature of complex wireless channels in the vertical industry, the conventional network slice configuration method cannot respond to the change of network environment rapidly because of the difficulty in predicting traffic distribution and channel change, and the difficulty in realizing optimal slice configuration with high efficiency.
Disclosure of Invention
Aiming at the defects existing in the prior art when the network slice configuration is realized, the embodiment of the invention provides a network slice dynamic configuration method, a device, electronic equipment and a storage medium.
The invention provides a dynamic configuration method of a network slice, which comprises the following steps: constructing a slice configuration experience data set; inputting the network state related to each slice configuration experience data in the slice configuration experience data set to a pre-trained slice pre-configuration model respectively, and obtaining a slice configuration action output by the slice pre-configuration model, wherein the slice configuration actions are in one-to-one correspondence with the network states; based on Dropout calculation, acquiring variances and mean values of all slice configuration actions output by the slice pre-configuration model in various dimension variables, and assuming that the output of the slice pre-configuration model accords with normal distribution so as to construct a slice pre-configuration space; and determining the optimal slice configuration from the constructed slice pre-configuration space.
According to the network slice dynamic configuration method provided by the invention, each group of slice configuration experience data in the slice configuration experience data set consists of a network state, an actual slice configuration action and a network rewards; the network state is composed of slice flow in a sliding slice window and average channel quality indexes under each slice window in the sliding slice window; the actual configuration action of the slice is the configuration combination of the number of physical resource blocks used for different slices in the current slice window; the network rewards are products of current network states and slice satisfaction rates of all slices in the current slice window under the configuration action.
According to the method for dynamically configuring the network slice, the method for constructing the slice configuration experience data set comprises the following steps:
setting up a network numerical simulation platform and setting the number of users adopted in the simulation; collecting a plurality of radio access network slices; calculating the slice satisfaction rate of each wireless access network slice under the corresponding resource allocation action; accumulating and counting the slice configuration experience under a plurality of groups of slice flows, and traversing a plurality of groups of different physical resource block slice configuration combinations under each group of slice flows; and constructing the slice configuration experience data set by taking each physical resource block slice configuration combination under each group of slice flow and the corresponding slice satisfaction rate as a group of slice configuration experience data.
According to the network slice dynamic configuration method provided by the invention, the plurality of radio access network slices comprise radio access network slices of at least three service types;
the wireless access network slices of the three service types comprise a mobile enhanced bandwidth service eMBB slice, a large-scale machine communication service mMTC slice and an ultra-low time delay ultra-high reliability service URLLC slice.
According to the method for dynamically configuring the network slice, after the slice configuration experience data set is constructed, the method further comprises the following steps: and normalizing each slice configuration experience data in the slice configuration experience data set so that the numerical value size of each dimension in each slice configuration experience data is mapped to an interval [0,1].
According to the network slice dynamic configuration method provided by the invention, after normalization processing is carried out on each slice configuration experience data in the slice configuration experience data set, the method further comprises the following steps: setting a satisfaction rate threshold; and screening all slice configuration experience data with network rewards larger than the satisfaction rate threshold from the slice configuration experience data set, and constructing a slice configuration experience effective data set.
According to the method for dynamically configuring the network slice, the slice pre-configuration model is a deep learning network model, and the method comprises the following steps: an input layer, a plurality of fully connected layers, and an output layer; the input layer is connected with a first full connection layer in the plurality of full connection layers; all the full connection layers in the plurality of full connection layers are connected in sequence; and the last full-connection layer is connected with the output layer.
According to the method for dynamically configuring the network slice provided by the invention, before inputting the network state related to each slice configuration experience data in the slice configuration experience data set into a pre-trained slice pre-configuration model, the method further comprises the following steps:
and pre-training a pre-constructed slice pre-configuration model by using the slice configuration experience data set so as to obtain the trained slice pre-configuration model.
According to the network slice dynamic configuration method provided by the invention, the optimal slice configuration is determined from the constructed slice pre-configuration space, and the method specifically comprises the following steps: and determining the optimal slice configuration from the constructed slice pre-configuration space based on a multi-arm gambling algorithm.
According to the method for dynamically configuring the network slice provided by the invention, the optimal slice configuration is determined from the constructed slice pre-configuration space based on the multi-arm gambling machine algorithm, and the method comprises the following steps:
step 1: initializing a network state, traversing the slice pre-configuration space, and determining the Q value and execution times of each configuration action in the multi-arm gambling machine algorithm, the initialization utilization rate epsilon and the updating period T1 of a slice pre-configuration model;
step 2: calculating average network rewards corresponding to each slice configuration action in the slice pre-configuration space in the previous t attempts according to the probability of epsilon based on the exploration-utilization strategy, selecting the slice configuration action with the maximum Q value, and randomly selecting the slice configuration action in the slice pre-configuration space according to 1 epsilon;
step 3: performing slice configuration action a of physical resource blocks t Acquiring configuration action a executed at time t t Instant network rewards R of (2) t (a) Transmitting the current slice configuration experience data to a database for updating the slice pre-configuration model;
step 4: judging whether the remainder function mod (T, T1) is 0, if so, updating the slice pre-configuration model; otherwise, directly entering step 5;
step 5: entering the next network state, and circularly executing the steps 2-4.
According to the network slice dynamic configuration method provided by the invention, in the previous t attempts, the average network rewards of the slice configuration action a are adoptedThe calculation formula of (2) is as follows:
wherein T is t (a) Representing the number of times of adopting the slice configuration action a in the current t attempts, 1 (·) is an indication function, R n (a) Representing an nth instant prize for employing slice configuration action a;
then at time t, a configuration action a is obtained using the exploration-utilization strategy t The calculation formula of (2) is as follows:
wherein A is a slice preconfigured action space.
According to the network slice dynamic configuration method provided by the invention, the instant network rewards R t (a) The calculation formula of (2) is as follows:
R t (a)＝SSR_eMBB(a)·SSR_mMTC(a)·SSR_URLLC(a)；
wherein ssr_embb (a) is the slice satisfaction rate of slice eMBB (a), ssr_emtc (a) is the slice satisfaction rate of slice ssr_emtc (a), and ssr_urllc (a) is the slice satisfaction rate of slice ssr_urllc (a).
The invention also provides a dynamic configuration device for the network slice, which mainly comprises: the data acquisition unit is used for constructing a slice configuration experience data set, wherein:
the model operation unit is used for inputting the network state related to each slice configuration experience data in the slice configuration experience data set to a pre-trained slice pre-configuration model respectively, and obtaining the slice configuration actions output by the slice pre-configuration model, wherein the slice configuration actions correspond to the network states one by one;
the pre-configuration space creation unit is mainly used for acquiring variances and mean values of all slice configuration actions output by the slice pre-configuration model in various dimension variables based on Dropout calculation, and assuming that the output of the slice pre-configuration model accords with normal distribution so as to construct a slice pre-configuration space;
the slice configuration determining unit is mainly used for determining the optimal slice configuration from the constructed slice pre-configuration space.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the network slice dynamic configuration method according to any one of the above are realized when the processor executes the program.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the network slice dynamic configuration method as described in any of the above.
The network slice dynamic configuration method, the device, the electronic equipment and the storage medium provided by the invention solve the problems that the traditional RAN slice configuration method is difficult to predict the flow and channel change, the convergence speed is low, and the slice configuration decision oriented to SLA optimization cannot be provided rapidly, realize the intellectualization and automation of the RAN slice, and improve the slice satisfaction rate and the slice configuration efficiency.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a network slice dynamic configuration method provided by the invention;
FIG. 2 is a second flow chart of the method for dynamically configuring network slices according to the present invention;
FIG. 3 is a schematic diagram of a change in network rewards (reward) with increasing network state space (epoode) under different experience utilization rates using the slice configuration method provided by the present invention;
FIG. 4 is a graph showing the variation of the enhanced mobile broadband (eMBB) slice satisfaction rate at different empirical utilization rates;
FIG. 5 is a graph showing changes in large-scale machine type communication (mMTC) slice satisfaction rates at different empirical utilization rates;
FIG. 6 is a graph showing the variation of ultra-reliable low latency (URLLC) slice satisfaction at different empirical utilization rates;
FIG. 7 is a schematic diagram of the change of reward with the increment of the epoode in a configuration method based on a Depth Q Network (DQN) algorithm;
FIG. 8 is a schematic diagram of a network slice dynamic configuration system provided by the present invention;
fig. 9 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be noted that in the description of embodiments of the present invention, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. The orientation or positional relationship indicated by the terms "upper", "lower", etc. are based on the orientation or positional relationship shown in the drawings, are merely for convenience of description and to simplify the description, and are not indicative or implying that the apparatus or elements in question must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as limiting the present invention. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to the specific circumstances.
Due to advanced algorithm, mass data and strong calculation power, the deep network learning can fully utilize historical slice configuration experience data and automatically learn the intra-slice traffic flow mode and channel variation trend. In addition, reinforcement learning is good at dynamically responding to the network state, and the slice pre-configuration model can be continuously optimized to adapt to complex vertical application scenes, so that slice dynamic configuration and slice service quality optimization are efficiently realized.
In view of the above, the network slice dynamic configuration method provided by the invention provides an intelligent wireless access network slice configuration method based on deep learning and reinforcement learning, which can effectively mine historical configuration experience gain, thereby guaranteeing differentiated 5G vertical industry service transmission requirements.
Generally, in order to cope with the challenges that random flow fluctuation, dynamic channel quality, dense network deployment and complex industry network scene bring about lower RAN slice configuration efficiency in a 5G wireless access network, and are difficult to provide stronger service level agreements (Service Level Agreement, SLA) guarantee, the network slice dynamic configuration method provided by the invention combines a Deep Neural Network (DNN) with a multi-arm gambling Machine Algorithm (MAB) by adopting a hierarchical intelligent control strategy, provides a hierarchical intelligent dynamic slice configuration method oriented to wireless access network SLA guarantee, fully learns effective historical slice configuration experience by adopting a slice pre-configuration model, and provides a slice pre-configuration range oriented to SLA guarantee by mining effective slice configuration experience data within a long time range. In addition, a classical multi-arm gambling machine algorithm is adopted, dynamic fine slice configuration adjustment is realized according to network state perception in a short time range, and a quick slice configuration solution for SLA guarantee is provided for dynamic complex RAN network slice management.
The method and system for dynamically configuring network slices provided by the embodiment of the invention are described below with reference to fig. 1 to 9.
Fig. 1 is a flow chart of a method for dynamically configuring a network slice according to the present invention, as shown in fig. 1, including but not limited to the following steps:
step S1: constructing a slice configuration experience data set;
step S2: inputting the network state related to each slice configuration experience data in the slice configuration experience data set into a pre-trained slice pre-configuration model respectively, and obtaining a slice configuration action output by the slice pre-configuration model, wherein the slice configuration actions correspond to the network states one by one;
step S3: based on Dropout calculation, acquiring variances and mean values of all slice configuration actions output by the slice pre-configuration model in various dimension variables, and assuming that the output of the slice pre-configuration model accords with normal distribution so as to construct a slice pre-configuration space;
step S4: and determining the optimal slice configuration from the constructed slice pre-configuration space.
Each group of slice configuration experience data in the slice configuration experience data set consists of a network state, an actual slice configuration action and a network reward; the network state is composed of the slice flow in the sliding slice window and the average channel quality index under each slice window in the sliding slice window; the actual configuration action of the slice is the configuration combination of the number of physical resource blocks used for different slices in the current slice window; the network rewards are products of current network states and slice satisfaction rates of all slices in a current slice window under the configuration action.
Specifically, each set of configuration experience data consists of a network state (state), a slice configuration action (action) and a network reward (reward).
The network state consists of the slice traffic within the sliding slice window and the average channel quality index CQI (CQI) under each slice window within the sliding slice window, i.e., state= (traffic, CQI). The time of each slice window is T seconds, and the time of the sliding slice window is the first H.T seconds of the current slice window.
The slice window refers to a time window in which the number of RBs in the slice remains unchanged from the current time T, and the time interval is T, for example: t=1s, slice window: [ t, t+1s ];
the sliding slice window is a time window formed by the first H adjacent slice windows at the current time T, and the time interval is H.T. For example: when h·t=10s, the sliding slice window is: [ t-10s, t ]; the sliding-slice window contains a plurality of slice windows before the current time t. The current slice window refers to a slice window corresponding to the current time t: [ t, t+1s ]; h refers to the number of slice windows in the sliding slice window; the first H.T seconds is the sliding-slice window at the current time t: [ t-H.T, t ].
The configuration action refers to a number configuration combination of physical Resource Blocks (RBs) for different slices within the current slice window, that is:
action＝(RB_slice_1,RB_slice_2,…,RB_slice_N)；
Where rb_slice_n is the number of RBs within slice n.
Since 5G contains three general classes of typical traffic, namely: ultra low latency ultra high reliability services (URLLC), large scale machine communication services (mctc), and mobile enhanced bandwidth services (eMBB), while different services have transmission requirements in terms of latency, rate, reliability. In order to meet the differentiated transmission requirements of different services, different slices are needed, and different numbers of RBs are configured for different slices so as to provide different service qualities in a targeted manner.
The slice window is a time concept, and refers to a time interval in which the number of resources in a slice remains unchanged.
The number configuration combination refers to the number of resource configurations in different slices in a slice window ([ t, t+1s ]) at the current time t;
the network rewarding report refers to the product of the counted Slice Satisfaction Rate (SSR) of each slice in the current slice window under the current network state and the configuration action, namely:
reward＝SSR_slice_1·SSR_slice_2·…·SSR_slice_N；
where N is the number of slices in the current slice window, the number of slices being related to the number of traffic types, each slice serving a class of traffic. The definition of SSR is that in the current slice window, the ratio of the number of data packets which simultaneously meet the requirements of time delay, user rate and bit error rate in the data packets generated by the service to the total number of the data packets generated in the current slice window is 0, 1.
The step S1 of constructing the slice configuration experience data set can be realized by the following steps:
a network numerical simulation platform is built based on C++, and the number of users adopted in the simulation is set; collecting a plurality of radio access network slices comprising three service types, including an emmbc slice, an emtc slice and a URLLC slice; traversing resource allocation actions of physical resource blocks in each radio access network slice; counting the slice flow of each wireless access network slice under the corresponding slice window; calculating the slice satisfaction rate of each wireless access network slice under the corresponding resource allocation action; accumulating and counting the slice configuration experience under a plurality of groups of slice flows, and traversing a plurality of groups of different physical resource block slice configuration combinations under each group of slice flows; and constructing the slice configuration experience data set by taking each physical resource block slice configuration combination under each group of slice flow and the corresponding slice satisfaction rate as a group of slice configuration experience data.
As an optional embodiment, the step of collecting the slice configuration experience dataset may specifically be:
based on C++, a network numerical simulation platform is built, 100 users are adopted in the simulation, and assuming that each user generates 1 type (eMBB, mMTC or URLLC) of service traffic, three types of RAN slices, namely eMBB slices, mMTC slices and URLLC slices, are included in the embodiment, and each type of slice serves one type of specific service (namely eMBB service, mMTC service and URLLC service).
Optionally, the number of users of the emmbc service, the mMTC service and the URLLC service are respectively: 8,46 and 46.
Further, a scenario using a single base station and multiple users is taken as an example for explanation:
for a sliced RB resource configuration, the total number of RBs is N B =270. The number of medium RBs per slice remains constant within a slice window. Within the slice window, the intra-slice RB resources are allocated to intra-slice active users at each transmission time interval (Transmission Time Interval, TTI) using a polling scheduling algorithm.
To collect slice configuration experience data, the present example traverses the slice RB resource configuration actions rb_config, rb_config= (rb_embb, rb_emtc, rb_urllc) of eMBB slice, emtc slice, and URLLC.
Within each TTI time (typically 1ms in length), the poll scheduler allocates in-slice RB resources for active users accessing a particular slice.
After a slice window time (e.g., 1000 ms), the time delay (including queuing time delay and processing time delay), the corresponding user rate, the packet loss rate, the slice flow and the spectrum efficiency of each service data packet in each slice are counted. And by comparing the SLA time delay of the slices with the SLA rate requirement, calculating the slice satisfaction rate SSR of each slice as follows:
Wherein, SSR n A slice satisfaction rate indicating a slice n; u (U) n Representing a set of users accessing slice n; q (Q) u Representing a set of data packets generated by user u accessing slice n within a slice window; k (K) q And (3) indicating whether the data packet q meets the service SLA requirement, if so, taking a value of 1, and if not, taking a value of 0.
Optionally, the criterion for judging whether the service SLA requirement is met is: service time delay t q SLA time delay requirement not higher than slice nAnd the user rate is not lower than the rate requirement of SLA for slice n +.>
Different slice windows have corresponding sliding windows (the length of the sliding window is L slice windows), and the network traffic in the sliding window t is:
traffic＝(traffic_t-L，traffic_t-L-1,…，traffic_t-1)；
accordingly, the flow rate at the slice window n is:
traffic_n＝(traffic_n_eMBB,traffic_n_mMTC,traffic_n_URLLC)。
through traversing RB resource combinations, the network slice satisfaction rates under different RB slice configuration actions under the corresponding network traffic are counted:
SSR＝(SSR_eMBB,SSR_mMTC,SSR_URLLC)；
so that slice configuration experience data can be obtained: (traffic, RB_config, SSR).
The expression of the resource allocation action rb_config is:
RB_config＝(RB_eMBB,RB_mMTC,RB_URLLC)。
based on the network simulation platform, the embodiment accumulates and counts the slice configuration experience under 30 groups of slice flows, and traverses 36046 groups of different RB slice configuration combinations under each group of flows, wherein the minimum RB allocation granularity is 1 RB, each slice is allocated with 1 RB at least, and the total RB number is 270. Here, each RB slice configuration combination at each set of slice flows and the corresponding slice satisfaction rate are taken as one set of configuration experience data, and therefore, the whole slice configuration experience data set is obtained by adopting the permutation and combination method, wherein 1081380 sets of slice configuration experience data are included in total.
Accordingly, before inputting the network state associated with each slice configuration experience data in the slice configuration experience data set into the pre-trained slice pre-configuration model, further comprising: and pre-training a pre-constructed slice pre-configuration model by using the slice configuration experience data set so as to obtain the trained slice pre-configuration model.
As an alternative embodiment, the slice pre-configuration model is a deep learning network model (DNN), comprising: an input layer, a plurality of fully connected layers, and an output layer; the input layer is connected with a first full connection layer in the plurality of full connection layers; all the full connection layers in the plurality of full connection layers are connected in sequence; and the last full-connection layer is connected with the output layer.
Equivalently, the structure of the deep neural network mainly includes: an input layer, m full connection layers (F 1 ，F 2 ，…，F m ) An output layer. Input of the Input layer is user CQI and slice flow traffic of sliding slice window in slice configuration experience data, namely input= (traffic, CQI), and the Input layer and full connection layer F 1 Is connected with each other; full connection layer F 1 Is connected with the next full connection layer F 2 Is fully connected with the neuron nodes of the (a); but according to the connection mode, the last full connection layer F is connected in sequence through m full connection layers m With the output layerFull connection; the output layer outputs the RB resource quantity combination information of each slice, that is, the configured quantity of RBs in different slices can be output through DNN, which is expressed as: output= (rb_slice_1, rb_slice_2, …, rb_slice_n).
As an alternative, in order to simplify the calculation, traffic may be used as Input only in the actual configuration process. Correspondingly, in step S2, the network state related to each slice configuration experience data in the slice configuration experience data set is input to a pre-trained slice pre-configuration model, and a slice configuration action output by the slice pre-configuration model is obtained, which specifically includes:
taking Taffic in the effective slice configuration experience data set as Input of DNN, wherein traffic= (traffic_eMBB, traffic_mMTC, traffic_URLLC); the three-dimensional slice configuration action rb_config within the corresponding valid slice configuration experience may be taken as the Output of the slice pre-configuration model, where:
RB_config＝(RB_eMBB,RB_mMTC,RB_URLLC)。
the network structure of the DNN adopted in this embodiment is "Input-L2-L3-L4-Output", and after nonlinear conversion of the DNN layer by layer, a mapping between the network traffic state traffic and the slice RB configuration action rb_config in the effective configuration experience is established. The trained DNN can learn the history effective slice configuration experience, and further can output RB_config that the SSR meets a specific threshold according to the predicted network state traffic.
Fig. 2 is a second flow chart of the dynamic configuration method of network slices provided by the invention, as shown in fig. 2, the invention discloses a hierarchical intelligent dynamic slice configuration method for guaranteeing wireless access network SLAs, which mainly comprises the following steps:
step one: collecting a slice configuration experience data set;
step two: preprocessing slice configuration experience;
step three: effective slice configuration experience screening;
step four: slice configuration experience mapping based on a deep neural network;
step five: calculating uncertainty of the trained deep neural network output by using Dropout, and outputting a slice pre-configuration space;
step six: and selecting a slice configuration action in a preconfigured action space by adopting a multi-arm gambling machine algorithm, and outputting the optimal slice configuration.
Specifically, in the second step, in order to increase the convergence rate of the deep neural network algorithm, normalization processing is performed on the RB slice configuration experience data collected in the first step, including: the numerical size of each dimension in each slice configuration experience data is mapped to interval [0,1].
Under the definition of the slice satisfaction rate SSR, the value range of the SSR is [0,1], so that normalization processing is not needed. Therefore, in the slice configuration experience data, the objects to be normalized are only the traffic and rb_config of the network.
Wherein, the expression of each configuration experience is: (state, action, rewind) wherein:
state＝(traffic，CQI)；
action＝(RB_slice_1,RB_slice_2,…,RB_slice_N)；
both state and action are vectors, so that a piece of configuration experience is not just three-dimensional data, but its dimension is much higher. Because there is a large amount of slice configuration experience data, each dimension may have different values, for example, the traffic range is 100M to 1000Mbits, normalization is needed, and the result of normalization processing is that the data of each dimension in the configuration experience has a value of [0,1].
As shown in fig. 2, after normalizing each slice configuration experience data in the slice configuration experience data set, the method further includes: setting a satisfaction rate threshold; and screening all slice configuration experience data with network rewards larger than the satisfaction rate threshold from the slice configuration experience data set, and constructing a slice configuration experience effective data set.
Specifically, in the third step, according to the slice satisfaction rate SSR, configuration experience meeting the service SLA requirement is screened out from the original RB slice configuration experience data set.
Optionally, in the method for dynamically configuring a network slice provided by the present invention, the slice satisfaction rate threshold is set as follows: SSR_threshold=0.99, and the slice configuration experience of SSR_eMBB, SSR_mMTC and SSR_URLLC in the configuration experience which are simultaneously larger than SSR_threshold is extracted and is included in the effective slice configuration experience data set.
Another possible case is that, under a specific traffic, if SSR corresponding to all rb_configs fails to meet the ssr_threshold requirement, at this time, slice experience (traffic, rb_config, SSR) corresponding to the ssr_embb, ssr_mctc, ssr_urllc maximum value is saved to the valid slice configuration experience data set.
In step S3, the calculating based on Dropout obtains the variance and the mean of all the slice configuration actions outputted by the slice pre-configuration model in each dimension variable, and presumes that the output of the slice pre-configuration model accords with normal distribution to construct a specific implementation method of the slice pre-configuration space, mainly refers to calculating the uncertainty of the DNN output by Dropout, and in the step five of outputting the slice pre-configuration space, the example calculates the mean and the variance of the DNN output trained to provide the distribution of the effective slice RB configuration actions in specific network states, and finally outputs the slice RB pre-configuration action space.
Specifically, the implementation of the deep learning model output uncertainty calculation by adopting Dropout includes:
and (3) calculating variances and mean values of RB resource allocation quantity on each slice by testing the outputs of a plurality of groups of DNNs and calculating RB slice allocation actions outputted by the trained slice pre-allocation model (DNN).
Wherein, the expression of the Mean of Variance is:
Mean＝(Mean_RB_slice_1，Mean_RB_slice_2,…,Mean_RB_slice_N)Variance＝(Var_RB_slice_1，Var_RB_slice_2,…,Var_RB_slice_N)。
in addition, assuming that the variable of each dimension in the RB slice configuration action accords with normal distribution, finally, the slice pre-configuration space can be output according to the distribution of the slice configuration action.
As an alternative embodiment, for the trained DNN model, a Dropout strategy is adopted, and the output of the DNN is perturbed with a probability of p=0.01 for each neuron in the hidden layer without incorporating the forward transmission of the DNN, so as to test the uncertainty of the output of the DNN model.
With the Dropout strategy described above, the present embodiment tests the RB slice configuration actions rb_config of 100 sets of DNNs output, and calculates the Mean (mean_rb_embb, mean_rb_emtc, mean_rb_urllc) and variance (var_rb_embb, var_rb_emtc, var_rb_urllc) of each dimension of DNNs output.
Assuming that the output of DNN obeys normal distribution, then the slice RB preconfigured action space k= (action_1, action_2, …, action_k) is obtained.
The invention provides a dynamic configuration method of network slices, which aims at optimizing SLA indexes such as time delay, user speed and the like according to an effective slice configuration experience data set and providing a slice pre-configuration action space.
Further, the multi-arm gambling Machine Algorithm (MAB) based determination of the optimal slice configuration from the build slice pre-configuration space described in step S4 includes, but is not limited to, the following steps:
Step 1: initializing a network state, traversing the slice pre-configuration space, and determining the Q value and execution times of each configuration action in the multi-arm gambling machine algorithm, the initialization utilization rate epsilon and the updating period T1 of a slice pre-configuration model;
step 2: calculating average network rewards corresponding to each slice configuration action in the slice pre-configuration space in the previous t attempts according to an epsilon-Greedy strategy, selecting the slice configuration action with the maximum Q value, and randomly selecting the slice configuration action in the slice pre-configuration space according to 1 epsilon;
in the first t attempts, the average network reward of action a is configured with slicesThe calculation formula of (2) is as follows:
wherein T is t (a) Representing the number of times of adopting the slice configuration action a in the current t attempts, 1 (·) is an indication function, R n (a) Representing an nth instant prize for employing slice configuration action a;
then at time t, configuration action a obtained by adopting E-Greedy algorithm t The calculation formula of (2) is as follows:
step 3: performing slice configuration action a of physical resource blocks t Acquiring configuration action a executed at time t t Instant network rewards R of (2) t (a) Transmitting the current slice configuration experience data to a database for updating the slice pre-configuration model;
R t (a)＝SSR_eMBB(a)·SSR_mMTC(a)·SSR_URLLC(a)；
Wherein ssr_embb (a) is the slice satisfaction rate of slice eMBB (a), ssr_emtc (a) is the slice satisfaction rate of slice ssr_emtc (a), ssr_urllc (a) is the slice satisfaction rate of slice ssr_urllc (a);
step 4: judging whether the remainder function mod (T, T1) is 0, if so, updating the slice pre-configuration model; otherwise, directly entering step 5;
and 5, entering the next network state, and circularly executing the steps 2-4.
The method for determining the optimal slice configuration provided by the invention is that the optimal slice configuration action is selected from the RB slice pre-configuration action space K output in the step S3 by utilizing an exploration-utilization (E-Greedy) strategy through an MAB algorithm.
Specifically, the MAB algorithm adopts an epsilon-Greedy strategy, selects an RB slice configuration action, and outputs a group of slice RB configuration quantity combinations, and the expression is as follows:
(RB_slice_1,RB_slice_2,…,RB_slice_N)。
then, the network environment feeds back the statistical information such as the slice satisfaction rate, the spectral efficiency, the error rate and the like in the slice window, calculates the network rewards report=ssr_slice_1, ssr_slice_2, …, ssr_slice_n, and further updates the utility value and the execution times of the slice configuration action.
As an alternative embodiment, the flow of the MAB-greeny based dynamic slice RB resource allocation algorithm is as follows:
Sub-step 1: initializing a network state, traversing an RB configuration action space, and calculating the Q value and the action execution times of each configuration action in a multi-arm gambling machine algorithm; in addition, initializing the utilization rate E and DNN updating period T1;
sub-step 2: inputting the network state traffic into a trained DNN model, and calculating a slice pre-configuration action space K;
wherein traffic= (traffic_embb, traffic_emtc, traffic).
Sub-step 3: calculating the average value of the reward corresponding to each slice configuration action in the preconfigured action space under the previous t attempts according to the E-Greedy strategy and using the E probability, and selecting the slice configuration action with the maximum Q value; in addition, the slice configuration actions in the preconfigured action space K are randomly selected with 1-E.
In the first t attempts, the average reward for employing slice configuration action a isThen at time t, configuration action a obtained by adopting E-Greedy algorithm t ；
Sub-step 4: performing RB slice configuration action a t Record action a performed at time t t Instant prize R for environmental feedback t (a) A. The invention relates to a method for producing a fibre-reinforced plastic composite Meanwhile, transmitting the current slice configuration experience to a database for updating DNN;
sub-step 5: judging whether the remainder function mod (T, T1) is 0, if so, updating DNN, otherwise, directly entering a substep 6;
And 6, entering the next network state, and circularly executing the substep 2 to the substep 5.
In the above dynamic slice configuration algorithm based on MAB-e-greeny, the slice configuration action is output in the form of a 3-dimensional vector rb_config= (rb_embb, rb_mctc, rb_urllc).
FIG. 3 is a schematic diagram showing the incremental change of the reward along with the epinode under different experience utilization rates, where the epinode refers to traversing the network state space once, so as to better illustrate the effectiveness of the method provided by the invention in dynamic configuration of network slices, as shown in FIG. 3, based on the method provided by the invention, since the RB slice configuration experience is fully utilized by the slice pre-configuration model based on the depth network, the pre-configuration action space of the MAB_E_Greedy algorithm is greatly reduced, so as to obtain a better initial configuration point, and the algorithm convergence efficiency is improved.
When the experience utilization rate is larger, the probability of randomly attempting to configure actions is smaller, and the corresponding reward is smoother. When e=0.95, the exploration of new actions and the utilization of history configuration experience are well compromised, and the average reward is higher than 0.98, i.e. the slice satisfaction rate of emmtc and URLLC is higher than 98%.
Fig. 4 is a schematic diagram of the change of the slice satisfaction rate of the enhanced mobile broadband (emmbb) under different experience utilization rates, fig. 5 is a schematic diagram of the change of the slice satisfaction rate of the large-scale machine type communication (mMTC) under different experience utilization rates, and fig. 6 is a schematic diagram of the change of the slice satisfaction rate of the ultra-reliable low-delay (URLLC) under different experience utilization rates, and the conditions of the change of the slice satisfaction rates of the emmbb slice, the emtc slice and the URLLC slice with the epicode under different experience utilization rates e are respectively shown in fig. 4 to 6.
When e=0.95, the mean values of the emmbc, mMTC, and URLLC slice satisfaction rates were 98.5%,99.88%, and 99.83%, respectively.
In the configuration method of the Depth Q Network (DQN) based algorithm, the change schematic diagram of the reward along with the increment of the epoode is shown in fig. 7, and compared with the hierarchical intelligent radio access network slice configuration algorithm based on DNN and MAB provided by the invention, the convergence rate of the scheme based on DQN is slower, and the reward=ssr_embb, ssr_mctc, ssr_urllc corresponding to the output slice result has larger volatility. The reason for the performance difference is mainly that the DQN is not fully utilized for the effective history configuration experience, and extensive trial and error is required to be performed on the DQN in the original configuration action space to maintain the Q value tables of different configuration actions.
Table 1 slice satisfaction rate comparison list
Table 1 is a table of comparison of the slice satisfaction rates of the slice configuration methods provided by the present invention with the slice satisfaction rates of the 7 slice configuration methods provided by other classical RAN slice configuration methods, as shown in Table 1, the slice satisfaction rates of the 7 slice configuration methods, MAB-e-Greedy, MAB-UCB, MAB-random, DQN, traffic_sliding, hard_sliding, etc., are counted together, and the first three are three typical classes of E-Greedy strategies, confidence region upper bound calculation strategies and random strategies in a multi-arm gambling Machine (MAB); DQN refers to a deep Q network; traffic_sliding refers to a slice configuration method for distributing total RB resources according to the proportion of the flow in each slice to the total flow; hard_sliding refers to a slice configuration method for equally dividing total RB resources according to the number of slices.
From table 1, it is evident that: the average value of the corresponding slice satisfaction rate of the MAB-E-Greedy algorithm is highest, and the value is 99.23%; in addition, DQN is difficult to converge quickly, especially for large motion space situations, requiring more new motions to be tried to obtain experience to maintain a Q-value table for the complete motion space; the slicing based on the flow proportion is less in consideration of the time delay requirement of the service, more sufficient RB resources cannot be reserved for the URLLC service, and when the total RB resources are limited in quantity, lower time delay guarantee is difficult to realize; for the average slicing strategy, RB resources are evenly distributed to different slices, the method is simple and easy to operate, but the slicing flow and the service SLA requirements cannot be captured, and the slice satisfaction rate is difficult to guarantee. The corresponding slice of the algorithm has the highest satisfaction rate average value and takes the value of 99.23 percent; in addition, DQN is difficult to converge quickly, especially for large motion space situations, requiring more new motions to be tried to obtain experience to maintain a Q-value table for the complete motion space; the slicing based on the flow proportion is less in consideration of the time delay requirement of the service, more sufficient RB resources cannot be reserved for the URLLC service, and when the total RB resources are limited in quantity, lower time delay guarantee is difficult to realize; for the average slicing strategy, RB resources are evenly distributed to different slices, the method is simple and easy to operate, but the slicing flow and the service SLA requirements cannot be captured, and the slice satisfaction rate is difficult to guarantee.
By integrating the contents shown in fig. 3-7, it can be proved that the dynamic slice configuration method based on DNN and MAB provided by the invention adopts a hierarchical intelligent control mechanism, fully utilizes the history effective configuration experience in a long time range, and dynamically responds to the network state change in a short time range, and compared with the classical RAN slice method, can realize faster algorithm convergence and higher slice SLA satisfaction rate, can effectively be used as an intelligent RAN slice management module in a wireless network management platform, and completes the intellectualization and automation of slice management so as to realize efficient dynamic slice resource configuration.
Fig. 8 is a dynamic configuration system for network slices, which is provided by the present invention, and fig. 8 mainly includes: the data acquisition unit 801 is mainly used for constructing a slice configuration experience data set; the model operation unit 802 is mainly configured to input network states related to each slice configuration experience data in the slice configuration experience data set to a pre-trained slice pre-configuration model, and obtain slice configuration actions output by the slice pre-configuration model, where the slice configuration actions correspond to the network states one by one; the pre-configuration space creating unit 803 is mainly used for obtaining variances and mean values of all slice configuration actions output by the slice pre-configuration model in various dimension variables based on Dropout calculation, and assuming that the output of the slice pre-configuration model accords with normal distribution so as to construct a slice pre-configuration space; the slice configuration determining unit 804 is mainly configured to determine an optimal slice configuration from the build slice pre-configuration space based on a multi-arm gambling machine algorithm.
Further, the network slice dynamic configuration system provided by the invention adopts a hierarchical intelligent control strategy, on one hand, a network flow and channel quality change mode in a longer time range is captured through an operation depth neural network model, and a slice pre-configuration action space oriented to slice satisfaction rate optimization is provided; on the other hand, a multi-arm gambling machine algorithm is operated, the state change of the network in a short time range is dynamically responded, and the optimal slice configuration combination is searched in the pre-configuration action space so as to dynamically optimize the SLA guarantee level of the service, and meanwhile, the slice configuration efficiency is improved.
The method is mainly used for executing the following configuration steps in the specific operation process of the network slice dynamic configuration system:
step one, collecting a slice configuration experience data set; step two, preprocessing slice configuration experience; step three, screening effective slice configuration experience data; fourthly, mapping the slice configuration experience data based on DNN; step five, calculating uncertainty of trained DNN by adopting Dropout, and outputting a slice pre-configuration space; and step six, adopting a multi-arm gambling machine algorithm to select configuration combinations in a pre-configuration space and outputting the optimal slice configuration actions.
The network slice dynamic configuration system provided by the invention solves the problems that the traditional RAN slice configuration method is difficult to predict flow and channel change, has low convergence speed and can not quickly provide slice configuration decisions for SLA optimization, realizes the intellectualization and automation of the RAN slice, and improves the slice satisfaction rate and the slice configuration efficiency.
It should be noted that, when the network slice dynamic configuration system provided in the embodiment of the present invention is specifically implemented, the network slice dynamic configuration system may be implemented based on the network slice dynamic configuration method described in any one of the foregoing embodiments, which is not described in detail in this embodiment.
Fig. 9 is a schematic structural diagram of an electronic device provided by the present invention, and as shown in fig. 9, the electronic device may include: processor 910, communication interface (Communications Interface), memory 930, and communication bus 940, wherein processor 910, communication interface 920, and memory 930 communicate with each other via communication bus 940. Processor 910 may invoke logic instructions in memory 930 to perform a network slice dynamic configuration method comprising: constructing a slice configuration experience data set; inputting the network state related to each slice configuration experience data in the slice configuration experience data set to a pre-trained slice pre-configuration model respectively, and obtaining a slice configuration action output by the slice pre-configuration model, wherein the slice configuration actions are in one-to-one correspondence with the network states; based on Dropout calculation, acquiring variances and mean values of all slice configuration actions output by the slice pre-configuration model in various dimension variables, and assuming that the output of the slice pre-configuration model accords with normal distribution so as to construct a slice pre-configuration space; and determining the optimal slice configuration from the constructed slice pre-configuration space based on a multi-arm gambling algorithm.
Further, the logic instructions in the memory 930 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the network slice dynamic configuration method provided by the above methods, the method comprising: constructing a slice configuration experience data set; inputting the network state related to each slice configuration experience data in the slice configuration experience data set to a pre-trained slice pre-configuration model respectively, and obtaining a slice configuration action output by the slice pre-configuration model, wherein the slice configuration actions are in one-to-one correspondence with the network states; based on Dropout calculation, acquiring variances and mean values of all slice configuration actions output by the slice pre-configuration model in various dimension variables, and assuming that the output of the slice pre-configuration model accords with normal distribution so as to construct a slice pre-configuration space; and determining the optimal slice configuration from the constructed slice pre-configuration space based on a multi-arm gambling algorithm.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor is implemented to perform the network slice dynamic configuration method provided by the above embodiments, the method comprising: constructing a slice configuration experience data set; inputting the network state related to each slice configuration experience data in the slice configuration experience data set to a pre-trained slice pre-configuration model respectively, and obtaining a slice configuration action output by the slice pre-configuration model, wherein the slice configuration actions are in one-to-one correspondence with the network states; based on Dropout calculation, acquiring variances and mean values of all slice configuration actions output by the slice pre-configuration model in various dimension variables, and assuming that the output of the slice pre-configuration model accords with normal distribution so as to construct a slice pre-configuration space; and determining the optimal slice configuration from the constructed slice pre-configuration space based on a multi-arm gambling algorithm.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (15)

1. A method for dynamically configuring a network slice, comprising:
constructing a slice configuration experience data set;
inputting the network state related to each slice configuration experience data in the slice configuration experience data set to a pre-trained slice pre-configuration model respectively, and obtaining a slice configuration action output by the slice pre-configuration model, wherein the slice configuration actions are in one-to-one correspondence with the network states;
based on Dropout calculation, acquiring variances and mean values of all slice configuration actions output by the slice pre-configuration model in various dimension variables, and assuming that the output of the slice pre-configuration model accords with normal distribution so as to construct a slice pre-configuration space;
and determining the optimal slice configuration from the constructed slice pre-configuration space.
2. The method of dynamic configuration of network slices as claimed in claim 1, wherein,
each group of slice configuration experience data in the slice configuration experience data set consists of a network state, an actual slice configuration action and a network reward;
the network state is composed of slice flow in a sliding slice window and average channel quality indexes under each slice window in the sliding slice window;
The actual configuration action of the slice is the configuration combination of the number of physical resource blocks used for different slices in the current slice window;
the network rewards are products of current network states and slice satisfaction rates of all slices in the current slice window under the configuration action.
3. The network slice dynamic configuration method of claim 2, wherein constructing a slice configuration experience dataset comprises:
setting up a network numerical simulation platform and setting the number of users adopted in the simulation;
collecting a plurality of radio access network slices;
traversing resource allocation actions of physical resource blocks in each radio access network slice;
counting the slice flow of each wireless access network slice under the corresponding slice window;
calculating the slice satisfaction rate of each wireless access network slice under the corresponding resource allocation action;
accumulating and counting the slice configuration experience under a plurality of groups of slice flows, and traversing a plurality of groups of different physical resource block slice configuration combinations under each group of slice flows;
and constructing the slice configuration experience data set by taking each physical resource block slice configuration combination under each group of slice flow and the corresponding slice satisfaction rate as a group of slice configuration experience data.
4. The network slice dynamic configuration method of claim 3, wherein the plurality of radio access network slices comprises radio access network slices of at least three traffic types;
the wireless access network slices of the three service types comprise a mobile enhanced bandwidth service eMBB slice, a large-scale machine communication service mMTC slice and an ultra-low time delay ultra-high reliability service URLLC slice.
5. The network slice dynamic configuration method of claim 3, further comprising, after constructing the slice configuration experience dataset: and normalizing each slice configuration experience data in the slice configuration experience data set so that the numerical value size of each dimension in each slice configuration experience data is mapped to an interval [0,1].
6. The network slice dynamic configuration method of claim 5, further comprising, after normalizing each slice configuration experience data in the slice configuration experience data set:
setting a satisfaction rate threshold;
and screening all slice configuration experience data with network rewards larger than the satisfaction rate threshold from the slice configuration experience data set, and constructing a slice configuration experience effective data set.
7. The network slice dynamic configuration method according to claim 1, wherein the slice pre-configuration model is a deep learning network model, comprising: an input layer, a plurality of fully connected layers, and an output layer;
the input layer is connected with a first full connection layer in the plurality of full connection layers;
all the full connection layers in the plurality of full connection layers are connected in sequence;
and the last full-connection layer is connected with the output layer.
8. The network slice dynamic configuration method of claim 1, further comprising, prior to inputting the network state associated with each slice configuration experience data in the slice configuration experience data set into a pre-trained slice pre-configuration model:
and pre-training a pre-constructed slice pre-configuration model by using the slice configuration experience data set so as to obtain the trained slice pre-configuration model.
9. The method of claim 1, wherein determining an optimal slice configuration from the build slice pre-configuration space comprises:
and determining the optimal slice configuration from the constructed slice pre-configuration space based on a multi-arm gambling algorithm.
10. The network slice dynamic configuration method of claim 9, wherein the determining the optimal slice configuration from the build slice pre-configuration space based on a multi-arm gambling algorithm comprises:
step 1: initializing a network state, traversing the slice pre-configuration space, and determining the Q value and execution times of each configuration action in the multi-arm gambling machine algorithm, the initialization utilization rate epsilon and the updating period T1 of a slice pre-configuration model;
step 2: calculating average network rewards corresponding to each slice configuration action in the slice pre-configuration space in the previous t attempts according to the probability of epsilon based on the exploration-utilization strategy, selecting the slice configuration action with the maximum Q value, and randomly selecting the slice configuration action in the slice pre-configuration space according to 1 epsilon;
step 3: performing slice configuration action a of physical resource blocks t Acquiring configuration action a executed at time t t Instant network rewards R of (2) t (a) Transmitting the current slice configuration experience data to a database for updating the slice pre-configuration model;
step 4: judging whether the remainder function mod (T, T1) is 0, if so, updating the slice pre-configuration model; otherwise, directly entering step 5;
And 5, entering the next network state, and circularly executing the steps 2-4.
11. The network slice dynamic configuration method of claim 10, which isCharacterized in that in the first t attempts, an average network reward of slice configuration action a is employedThe calculation formula of (2) is as follows:
wherein T is t (a) Representing the number of times of adopting the slice configuration action a in the current t attempts, 1 (·) is an indication function, R n (a) Representing an nth instant prize for employing slice configuration action a;
then at time t, a configuration action a is obtained using the exploration-utilization strategy t The calculation formula of (2) is as follows:
wherein A is a slice preconfigured action space.
12. The network slice dynamic configuration method of claim 10, wherein the instant network rewards R t (a) The calculation formula of (2) is as follows:
R t (a)＝SSR_eMBB(a)·SSR_mMTC(a)·SSR_URLLC(a)；
wherein, SSR_eMBB (a) is the slice satisfaction rate of slice eMBB (a), SSR_mMTC (a) is the slice satisfaction rate of slice mMTC (a), and SSR_URLLC (a) is the slice satisfaction rate of slice URLLC (a).
13. A network slice dynamic configuration device, comprising:
the data acquisition unit is used for constructing a slice configuration experience data set;
the model operation unit is used for respectively inputting the network state related to each slice configuration experience data in the slice configuration experience data set into a pre-trained slice pre-configuration model, and obtaining the slice configuration actions output by the slice pre-configuration model, wherein the slice configuration actions correspond to the network states one by one;
The pre-configuration space creating unit is used for acquiring variances and mean values of all slice configuration actions output by the slice pre-configuration model in various dimension variables based on Dropout calculation, and assuming that the output of the slice pre-configuration model accords with normal distribution so as to construct a slice pre-configuration space;
and the slice configuration determining unit is used for determining the optimal slice configuration from the constructed slice pre-configuration space.
14. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the network slice dynamic configuration method steps of any one of claims 1 to 12 when the computer program is executed.
15. A non-transitory computer readable storage medium, having stored thereon a computer program, which when executed by a processor, implements the network slice dynamic configuration method steps of any one of claims 1 to 12.
CN202110336480.9A 2021-03-29 2021-03-29 Network slice dynamic configuration method and device, electronic equipment and storage medium Active CN115225478B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110336480.9A CN115225478B (en) 2021-03-29 2021-03-29 Network slice dynamic configuration method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110336480.9A CN115225478B (en) 2021-03-29 2021-03-29 Network slice dynamic configuration method and device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN115225478A CN115225478A (en) 2022-10-21
CN115225478B true CN115225478B (en) 2023-07-28

Family

ID=83604932

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110336480.9A Active CN115225478B (en) 2021-03-29 2021-03-29 Network slice dynamic configuration method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN115225478B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109495907A (en) * 2018-11-29 2019-03-19 北京邮电大学 A kind of the wireless access network-building method and system of intention driving
WO2020143384A1 (en) * 2019-01-08 2020-07-16 华为技术有限公司 Method for scheduling resources for network slices and device
WO2020258920A1 (en) * 2019-06-26 2020-12-30 华为技术有限公司 Network slice resource management method and apparatus
CN112511342A (en) * 2020-11-16 2021-03-16 北京邮电大学 Network slicing method and device, electronic equipment and storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109495907A (en) * 2018-11-29 2019-03-19 北京邮电大学 A kind of the wireless access network-building method and system of intention driving
WO2020143384A1 (en) * 2019-01-08 2020-07-16 华为技术有限公司 Method for scheduling resources for network slices and device
WO2020258920A1 (en) * 2019-06-26 2020-12-30 华为技术有限公司 Network slice resource management method and apparatus
CN112511342A (en) * 2020-11-16 2021-03-16 北京邮电大学 Network slicing method and device, electronic equipment and storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于AI的5G网络切片管理技术研究;徐丹;王海宁;袁祥枫;朱雪田;;电子技术应用(01);全文 *

Also Published As

Publication number Publication date
CN115225478A (en) 2022-10-21

Similar Documents

Publication Publication Date Title
Yuan et al. Meta-reinforcement learning based resource allocation for dynamic V2X communications
CN111866954B (en) User selection and resource allocation method based on federal learning
Chergui et al. Zero-touch AI-driven distributed management for energy-efficient 6G massive network slicing
Setayesh et al. Resource slicing for eMBB and URLLC services in radio access network using hierarchical deep learning
Ko et al. Joint client selection and bandwidth allocation algorithm for federated learning
US11343155B2 (en) Machine learning algorithms for quality of service assurance in network traffic
Wen et al. An overview of data-importance aware radio resource management for edge machine learning
Gu et al. Ganslicing: A gan-based software defined mobile network slicing scheme for iot applications
Wang et al. LinkSlice: Fine-grained network slice enforcement based on deep reinforcement learning
Saraiva et al. Deep reinforcement learning for QoS-constrained resource allocation in multiservice networks
Sun et al. Accelerating convergence of federated learning in mec with dynamic community
Hlophe et al. AI meets CRNs: A prospective review on the application of deep architectures in spectrum management
Robinson et al. Downlink scheduling in LTE with deep reinforcement learning, LSTMs and pointers
Qu et al. Stochastic cumulative DNN inference with RL-aided adaptive IoT device-edge collaboration
CN105530203B (en) The connection control method and system of D2D communication links
CN113543160B (en) 5G slice resource allocation method, device, computing equipment and computer storage medium
CN115225478B (en) Network slice dynamic configuration method and device, electronic equipment and storage medium
Mei et al. Semi-decentralized network slicing for reliable V2V service provisioning: A model-free deep reinforcement learning approach
Peng et al. Data-driven spectrum partition for multiplexing URLLC and eMBB
CN116484976A (en) Asynchronous federal learning method in wireless network
WO2023011371A1 (en) Method and system for configuring a threshold value for a handover parameter of a wireless communication system
Mamode et al. Machine Learning-Based Scheduling in 5G/6G Communication Systems
Falahatraftar et al. A multiple linear regression model for predicting congestion in heterogeneous vehicular networks
Binghui et al. NPRA: A Novel Predictive Resource Allocation Mechanism for Next Generation Network Slicing
González et al. Network Selection over 5G-Advanced Heterogeneous Networks Based on Federated Learning and Cooperative Game Theory

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant