CN117971511A

CN117971511A - Collaborative visual simulation platform

Info

Publication number: CN117971511A
Application number: CN202410389306.4A
Authority: CN
Inventors: 左良; 周岩; 程阿良
Original assignee: Qingdao Eurasian Feng Technology Development Co ltd
Current assignee: Qingdao Eurasian Feng Technology Development Co ltd
Priority date: 2024-04-02
Filing date: 2024-04-02
Publication date: 2024-05-03
Anticipated expiration: 2044-04-02
Also published as: CN117971511B

Abstract

The invention belongs to the technical field of simulation platforms, and discloses a collaborative visual simulation platform; comprising the following steps: the resource library module is used for storing the allocable resources; the task library module is used for acquiring tasks to be distributed and acquiring characteristic data to be distributed according to the tasks to be distributed; the scheme generating module is used for searching the allocable resources and the characteristic data to be allocated by using an improved genetic algorithm to obtain a resource allocation scheme; establishing a soft mapping relation between the resources and the tasks according to a resource allocation scheme; the relation scoring module is used for scoring the soft mapping relation between the resource and the task; the method realizes the intellectualization, the dynamics and the visualization of the resource allocation process, can make better resource allocation decisions in a complex environment, improves the resource utilization efficiency and better serves task demands.

Description

Collaborative visual simulation platform

Technical Field

The invention relates to the technical field of simulation platforms, in particular to a collaborative visual simulation platform.

Background

The patent with the application publication number of CN104850700A discloses a collaborative visual simulation platform, which comprises at least four computers, at least three-dimensional display screens, a mobile information terminal and a remote consultation terminal; the first computer receives experimental data and monitoring images, extracts driving data and transmits the driving data to the second computer, and the second computer performs space motion reproduction virtual simulation to generate video images; the third computer performs ground test site virtual simulation to generate a video image; a fourth computer performs drawing processing to generate a video image; the first computer sends experimental data and video images to the mobile information terminal and the remote consultation terminal, the cooperation mode of the simulation platform is changed from a general software mode to a software-hardware cooperation mode, real data are fully utilized on a simulation scene to construct three-dimensional simulation environments such as ground scenes, moon scenes, atmosphere, stars and the like, the authenticity of the visual simulation environment is enhanced, the cooperation operation between software and hardware is realized, and the simulation efficiency and the authenticity are improved.

However, at present, with the rapid increase of the demands of various tasks and the complexity of the resource environment, the resource allocation scheme is formulated only by means of manual experience, so that the scene change is difficult to adapt; for example, in a big data processing system, massive data analysis tasks and machine learning model training tasks arrive at the same time, and the tasks have requirements on computing resources and storage resources; the resource manager firstly meets the requirement of analysis tasks through past experience judgment, so that the subsequent model training tasks have serious delay when waiting for the resources; on the other hand, the existing resource allocation scheme is difficult to adjust once determined, and cannot quickly respond to the changes of resources and task environments; for example, when a large amount of bandwidth resources are newly added, if the original network transmission task cannot automatically sense and use the new resources, the available bandwidth is wasted; the current resource allocation evaluation also mainly relies on manual statistics of various indexes, so that the situation of insufficient resource utilization is difficult to intuitively find; for example, there is a state that the storage resource is busy and the computing resource is idle, and it is difficult to identify the situation of unbalanced matching in time by periodic manual statistical analysis; in summary, the current resource allocation mechanism relies on experience judgment, lacks dynamic adjustment, and is not well evaluated to intuitively systemize, so that the complexity and the dynamic performance of tasks and resource environments are difficult to deal with, and the resource allocation efficiency is low.

In view of the above, the present invention provides a collaborative visual simulation platform to solve the above-mentioned problems.

Disclosure of Invention

In order to overcome the defects in the prior art, the invention provides the following technical scheme for achieving the purposes: a collaborative visual simulation platform, comprising: the resource library module is used for storing the allocable resources;

The task library module is used for acquiring tasks to be distributed and acquiring characteristic data to be distributed according to the tasks to be distributed;

The scheme generating module is used for searching the allocable resources and the characteristic data to be allocated by using an improved genetic algorithm to obtain a resource allocation scheme; establishing a soft mapping relation between the resources and the tasks according to a resource allocation scheme;

The relation scoring module is used for scoring the soft mapping relation between the resource and the task to obtain a scoring result; all the modules are connected in a wired and/or wireless mode, so that data transmission among the modules is realized.

Further, the allocable resources include computing resources, storage resources, and network resources; storing corresponding specific information of the allocable resources;

The corresponding specific information of the allocable resources comprises the health state of the resources, the cost information of the resources, the current load state of the resources and the service type of the resources.

Further, the method for acquiring the health status of the resource includes:

The method comprises the steps of acquiring resource utilization rate data in real time by installing a lightweight probe program on a resource; the resource utilization data comprises CPU, memory and disk IO of the resource,

Inputting the resource utilization rate data acquired in real time into a pre-trained resource health evaluation model, and predicting to obtain a resource health score;

Presetting a resource health degree threshold value, and comparing the resource health degree score with the resource health degree threshold value; if the resource health score is smaller than the resource health threshold, judging that the resource health state of the resource is sub-health; if the resource health score is greater than or equal to the resource health threshold, judging that the resource health state of the resource is healthy;

The resource cost information obtaining mode comprises the following steps:

constructing a resource cost database and recording static cost data; the static cost data comprises power consumption and maintenance cost of resources with different models and configurations;

Calculating to obtain the operation cost of the resource according to the static cost data, the current electricity price and the use time; i.e. resource cost information;

the method for acquiring the current load state of the resource comprises the following steps:

Installing a monitoring agent program on a resource server, and collecting utilization rate data in real time; the utilization rate data comprises CPU utilization rate, memory utilization rate and disk utilization rate;

calculating the overall utilization LY of the resources by using a weighted average algorithm;

; wherein, CP is CPU utilization rate; NC is the memory utilization rate; IO is the disk utilization rate; w1, w2 and w3 are weight parameters;

presetting a utilization rate threshold interval; judging the load state of the resource according to the utilization rate threshold interval and the overall utilization rate, wherein the load state comprises an idle state, a normal state and a busy state;

the resource service type is directly obtained through a monitoring interface or monitoring management software provided by the equipment.

Further, the training mode of the resource health evaluation model includes:

step 1, collecting historical resource utilization rate data, manually marking corresponding health degrees, and constructing a marking data set; dividing the labeling data set into a training set and a verification set;

Step 2, presetting an improved recurrent neural network as an infrastructure of a resource health evaluation model;

step 3, extracting characteristics of historical resource utilization rate data, wherein the characteristics are maximum, minimum, average or variance; meanwhile, the health degree of the manual marking is converted into a continuous numerical value;

step 4, training a resource health degree evaluation model by using the training set; taking the extracted characteristics as input, and taking the marked health degree value as expected output;

Step 5, presetting a characteristic loss function; testing the effect of the resource health degree evaluation model on the verification set every fixed training iteration times, and recording the value of the characteristic loss function; if the value of the characteristic loss function is not reduced for P iterations continuously, stopping training; obtaining a trained resource health degree evaluation model;

feature loss function ; Wherein kl is the total number of samples in the training set or validation set; s is the sum index; wr _s is the weight of the resource class to which the s-th sample belongs; s _s is the prediction output of the model to the S-th sample; /(I)A true tag for the s-th sample; lambda is a regularization parameter; r (ρ) is a regularization term for the model parameter λ, which takes the L1 or L2 norm of the parameter vector.

Further, the preset improved recurrent neural network mode includes:

the improved recurrent neural network adds an improved double-layer GRU network and a multi-scale time sequence convolution unit on the basis of the basic framework of the recurrent neural network;

Improving the double-layer GRU network to add a gating unit between the double-layer GRU networks;

the formula for adding the gating cell is: ；/>；

Wherein gg is the output of the gating unit; sigma is a Sigmoid function; w _gg is a gating unit weight matrix of the forward GRU; h ₁ is the output state of the forward GRU cells; For element-by-element multiplication; h ₂ is the output state of the backward GRU cells; the GRU ₂ is a computing function of a layer 2 GRU network in the double-layer GRU network;

Extracting expansion time characteristics by a multi-scale time sequence convolution unit;

The extraction formula is as follows: ; wherein hh is the input timing sequence; ww is the convolution window size; dd is the expansion ratio; dilatedConv1D is a time series dilation convolution operation function.

Further, the tasks to be allocated comprise manual creation tasks, scheduling tasks from a third party system, automatic triggering tasks of business processes and respective corresponding task attributes; the task attributes comprise business attributes, priorities, expiration dates and resource consumption descriptions;

Constructing a task queue for storing tasks to be distributed; the tasks to be distributed are delivered to the task queue in the form of message text description.

Further, the obtaining manner of the feature data to be distributed includes:

Constructing a task feature extraction model based on deep learning, wherein the task feature extraction model comprises a word vector layer, a convolution layer, a pooling layer and a full connection layer; the word vector layer converts the message text description into a word vector sequence by adopting a word vector training mode with semantic supervision;

defining a word vector layer into an objective function Lc;

；

Wherein v _w is a word vector of the word w; v _s is a word vector corresponding to word s for which word w has a dependency relationship; v _w ^T is the special transpose of v _w; sigma is a Sigmoid activation function; y is a label, y is 1, and represents that the words have dependency relationship, and y is 0, and represents no relationship; i is an index;

f convolution units with residual errors are arranged in the convolution layer, and each convolution unit is used for outputting a residual error characteristic diagram;

the output formula of the output residual characteristic diagram is as follows: ; wherein H _f is a characteristic diagram of convolution operation output in the f convolution unit; x is an input feature map; o _f is a residual characteristic diagram output by the f convolution unit;

the pooling layer uses an attention mechanism to calculate the influence degree of each residual feature map on the loss function in classification, and the influence degree is used as the weight of the residual feature map;

The calculation formula is as follows: ; wherein score _f is the weight of residual feature map O _f; α _f is the influence of O _f on the loss function in classification; loss (O _f) is the Loss function value of residual feature map O _f;

; representing the partial derivative of the Loss function Loss to the residual feature map O _f;

Loss function ; Wherein Loss _cate is a classification Loss function; loss _reg is a regularized Loss function;

; wherein N is the total number of samples; a is a sample index; u _a is the one-hot code of the a sample true class; p _a is the prediction probability of the task feature extraction model for the class of the a sample;

; wherein λ1 is a regularization coefficient; /(I) Extracting L1 norms or L2 norms of all parameters of the model for the task features;

collecting v groups of message text descriptions of the history completion task, and manually labeling feature labels to construct a feature label data set; dividing the feature tag data set into a feature training set and a feature verification set; iteratively training a task feature extraction model using the training set; verifying the effect of the task feature extraction model by using the verification set; if the loss function is not reduced in the continuous Y-time iteration process, ending the model convergence training; obtaining a task feature extraction model after training is completed;

for a task to be distributed, analyzing text description of the task by using a task feature extraction model which is completed by training, and outputting a feature data vector to be distributed; the feature data vector to be allocated is a vector formed by probability values of z feature categories;

And selecting the characteristic category with the maximum probability value and the probability value thereof as the characteristic data to be allocated of the task to be allocated.

Further, the method for searching to obtain the resource allocation scheme by using the improved genetic algorithm comprises the following steps:

Step 201, generating initial w resource allocation schemes according to allocable resources and characteristic data to be allocated, and constructing an initial population by taking the initial resource allocation schemes as initial population individuals;

Step 202, evaluating performance indexes of each resource allocation scheme through simulation, and taking the performance indexes as the fitness of initial population individuals;

Step 203, selecting and retaining population individuals with high fitness by adopting a mode of combination selection of roulette selection and elite retention strategy;

step 204, intersecting the reserved population individuals pairwise according to fixed probability, generating new population individuals and adding the new population individuals into the population; when crossing, applying a coding crossing method based on task priority and resource matching degree;

Step 205, carrying out mutation operation on the population individuals obtained after crossing according to mutation probability, and guiding the mutation operation by utilizing constraint conditions to generate new population individuals;

repeating the iterative operation of the steps 202-205, and outputting a resource allocation scheme corresponding to the population individuals with the highest final fitness when the preset iteration times or the fitness convergence of the population individuals are reached.

Further, the method for establishing the soft mapping relation between the resource and the task includes:

Defining a two-dimensional matrix Mapping [ M ] [ N ], wherein M is the number of resources, and N is the number of tasks;

Traversing a resource allocation scheme, and analyzing each task and allocated resources to obtain an analysis result; setting the corresponding element value in the Mapping [ M ] [ N ] matrix to be 1 according to the analysis result; and actively triggering the redistribution of the soft mapping relation according to the state of the resource and the change of the task priority.

Further, the method for obtaining the scoring result includes:

Defining a scoring index system, wherein the scoring index system comprises scoring items of task satisfaction, resource utilization rate, task delay time and system throughput;

Constructing a scoring sample matrix according to a scoring index system, wherein the rows of the scoring sample matrix represent scoring samples at each moment, and the columns of the scoring sample matrix represent four scoring items;

Normalizing the scoring sample matrix and calculating a covariance matrix of the scoring sample matrix; decomposing the characteristic values of the covariance matrix, and reserving the first M characteristic vectors with larger characteristic values; constructing a transformation matrix by taking the reserved M eigenvectors as column vectors; mapping the original scoring samples to a new space by using a transformation matrix to obtain main components of the dimensionality reduction scoring; and carrying out linear weighting on the main components to obtain a comprehensive score, namely a scoring result.

The collaborative visual simulation platform has the technical effects and advantages that:

By establishing a soft mapping relation between the resources and the tasks, dynamic optimization of resource allocation is realized, and the reallocation can be actively triggered according to the real-time states of the resources and the tasks, so that the resource allocation scheme can rapidly respond to dynamic changes of the resources and the tasks; meanwhile, task characteristics are automatically extracted by adopting a model based on deep learning, and an optimal resource allocation scheme is searched by adopting an improved genetic algorithm, so that data-driven automatic decision is realized, and excessive dependence on expert experience is avoided; in addition, the effect of the resource allocation scheme is intuitively and systematically monitored and evaluated by using the scoring sample matrix, the condition of insufficient resource utilization can be found and adjusted so as to improve the resource utilization rate; the performance of different resource allocation schemes can be rapidly and efficiently evaluated through the simulation platform; the method realizes the intellectualization, the dynamics and the visualization of the resource allocation process, can make better resource allocation decisions in a complex environment, improves the resource utilization efficiency and better serves task demands.

Drawings

FIG. 1 is a schematic diagram of a collaborative visual simulation platform according to the present invention;

fig. 2 is a schematic diagram of an electronic device according to the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Example 1: referring to fig. 1, the collaborative visual simulation platform according to the present embodiment includes:

the resource library module is used for storing the allocable resources;

The corresponding specific information of the allocable resources comprises the health state of the resources, the cost information of the resources, the current load state of the resources and the service type of the resources;

The resource health state obtaining method comprises the following steps:

the method comprises the steps of acquiring resource utilization rate data in real time by installing a lightweight probe program on a resource; the resource utilization data includes but is not limited to CPU, memory and disk IO of the resource,

Inputting the resource utilization rate data acquired in real time into a pre-constructed resource health degree evaluation model, and predicting to obtain a resource health degree score between 0 and 100;

The training mode of the resource health evaluation model comprises the following steps:

Step 1, collecting historical resource utilization rate data, manually marking corresponding health degrees, and constructing a marking data set; dividing the labeling data set into a training set and a verification set; the specific dividing ratio can be determined according to actual conditions; for example, an 80% training set and a 20% validation set;

Step 3, extracting characteristics of historical resource utilization rate data, wherein the characteristics are maximum, minimum, average or variance; meanwhile, the health degree of the manual marking is converted into a continuous numerical value of 0-100;

feature loss function ; Wherein kl is the total number of samples in the training set or validation set; s is the sum index; wr _s is the weight of the resource class to which the s-th sample belongs; the sensitivity degree of different types of resources to the loss function is reflected, and the sensitivity degree can be manually set; s _s is the prediction output of the model to the S-th sample; /(I)A true tag for the s-th sample; lambda is a regularization parameter; r (ρ) is a regularization term for the model parameter λ, λ taking either the L1 or L2 norms of the parameter vector; is used for enhancing the generalization capability of the model and avoiding overfitting.

The feature loss function adds the weight of the resource category and the feature regularization, so that the model focuses on the sample size in the training process, the category of the resource and the complexity of the model are considered, and a better resource health evaluation effect can be obtained.

The preset improved recurrent neural network mode comprises the following steps:

the double-layer GRU network is improved, and a gating unit is added between the double-layer GRU networks and is used for realizing the selection of the characteristics;

the formula for adding the gating cell is: ；/>；

Wherein gg is the output of the gating unit; sigma is a Sigmoid function; for limiting the output of the gating cell to between 0 and 1; w _gg is a gating unit weight matrix of the forward GRU; h ₁ is the output state of the forward GRU cells; For element-by-element multiplication; for feature selection; h ₂ is the output state of the backward GRU cells; semantic information representing the whole; the GRU ₂ is a computing function of a layer 2 GRU network in the double-layer GRU network;

It should be noted that the gating unit may be regarded as a mask or a filter for controlling the flow of information; the second layer GRU network can be focused on useful features only, so that the expression capacity of the model is improved; a multi-scale sequential convolution unit; the time sequence expansion convolution can extract characteristic modes under different time scales by adjusting the expansion rate of a convolution kernel; for example, a small expansion rate may capture a detailed pattern, and a large expansion rate may cover a longer time-series pattern of a wider area; the modeling ability of the model to time series is enhanced.

The basic framework of the Recurrent Neural Network (RNN) consists of an input layer, a recurrent hidden layer, an output layer, a weight matrix and an activation function; the recursive hidden layer is used to process the sequence by recursively concatenating itself; the weight matrix is used for linear mapping of matrix parameters of the input, such as an input weight matrix Win, a recursive weight matrix Wrec and the like; the activation function generally uses a tanh or ReLU function; training of RNNs typically uses BPTT algorithms, calculates error gradients by time-expansion, and updates the weight matrix by gradient descent.

the preset mode of the resource health degree threshold comprises the following steps:

Determining key split points according to the type and purpose of the resource, e.g., 60 may be divided into key thresholds for a certain class of core database server resources; evaluating the impact of different partitioning points, e.g., a threshold of 60 minutes, will determine how many resources are unhealthy; the comprehensive evaluation determines a proper threshold value, so that the resource health condition can be accurately reflected, and the false alarm rate is not too high or too low; in a preferred embodiment, the resource health threshold is 68.

The resource cost information obtaining mode comprises the following steps:

constructing a resource cost database for recording static cost data; the static cost data comprises power consumption and maintenance cost of resources with different models and configurations;

Preferably, the method comprises the steps of accessing a fee inquiry interface of a public cloud management platform to obtain the on-demand charging price of public cloud resources in real time; comparing the public cloud resource price with the cost calculated by the private resource, selecting resource cost information with lower cost to store in a form of a table, and configuring a strategy for periodic manual checking and updating to ensure the accuracy of data;

It should be explained that the operation cost of the resource c=cpurchase+cpower+c maintenance;

wherein, C purchases the current electricity price as the resource; c, power is the power cost of the resource; c, maintaining the resource as maintenance cost;

If the resource using time is T; cpower=current electricity price×power consumption×t; maintenance = maintenance fee annual rate x T/total number of hours per year; the annual maintenance cost rate is obtained according to the maintenance cost;

For example, a certain server has a purchase price of 10000 yuan, power consumption of 500W, current electricity price of 1 yuan/degree, and maintenance annual rate of 10% of the value of the server; when used for 100 hours, the operating cost is 10500 yuan.

the weight parameters are obtained by fitting experimental data through regression analysis and a least square method; the experimental data are a large amount of utilization rate data acquired in an experimental environment;

It should be noted that, fitting data is performed using a least square method; the goal of the least squares method is to minimize the sum of squares of the differences between the model predictions and the actual observations; this fitting may be performed using mathematical software or programming language; in the above process, the values of w1, w2 and w3 are adjusted to minimize the error, and the adjustment can be implemented by an iterative algorithm, and the values of w1, w2 and w3 are continuously adjusted until the minimum error is found; the least squares method may use statistical software or programming languages such as SciTy library of PyThon or MATLAB, etc. to perform the fitting operation.

the preset mode of the utilization rate threshold interval comprises the following steps:

Collecting historical resource utilization rate data, and counting the utilization rate distribution conditions of different time periods; setting a utilization rate segmentation point, dividing the utilization rate into a plurality of sections, wherein each section represents a load state, for example, a 0-30% section represents an idle state, a 30-70% section represents a normal state, and a 70-100% section represents a busy state; aiming at different types of resources, adjusting the partition points of the interval; for example, there may be differences in the fit-for intervals of computing resources and storage resources; comprehensively considering the purpose and importance of the resources, and determining the corresponding relation of utilization rate sections of different resources, namely a utilization rate threshold section; for example, a more relaxed threshold interval is set for critical traffic resources.

Preferably, the monitoring agent program actively reports the resource utilization rate and the load state to the resource library module every t seconds in a pushing mode;

Constructing a task queue for storing tasks to be distributed; delivering tasks to be distributed to a task queue in the form of message text description; the queue of tasks to be allocated may be implemented in the form of a message queue, e.g., kafka, rabbitMQ, etc.;

It should be noted that, manually creating a task refers to providing a human-computer interaction interface by the system, allowing a user to manually create and submit the task to the system; for example, creating a data analysis task through a Web interface, and submitting the task to a system for scheduling and executing; scheduling tasks from a third party system refers to interfacing with other scheduling systems or workflow engines and accepting tasks that they schedule for distribution; data processing tasks generated by a big data processing system such as Spark, airflow and a dispatching platform can be accessed; reporting tasks of the enterprise business system or the ERP system can be accessed;

The automatic triggering task of the business process refers to automatically generating and triggering the task when a specific business event occurs through modeling and analyzing the business process; for example, training and deployment tasks of a product recommendation model are automatically triggered to be executed when order payment is successful; the coupling of the business process and the task flow is realized through the execution of the business event associated task;

business attributes such as data analysis tasks mapped to "analysis" attributes and model training tasks mapped to "training" attributes;

The priority refers to a priority value assigned by a task sender when submitting a task; a larger value indicates a higher priority;

The expiration date refers to the expiration time that the task sender designates to be completed when submitting the task; if not, defaulting to K hours after the submission time;

the resource consumption description acquisition mode comprises the following steps: performing resource consumption evaluation on the content and task characteristics of the task; for example, a statistical model is adopted to evaluate the resource requirement of the type task, namely, the resource consumption description; for example, some type of analysis task consumes 4-core 8G memory;

Further, the method for obtaining the feature data to be distributed includes:

extracting characteristic fields of the tasks to be distributed according to the tasks to be distributed as characteristic data to be distributed; the feature fields include, but are not limited to, task type, priority, business attributes, resource consumption description, expiration date;

constructing a task feature extraction model based on deep learning, wherein the task feature extraction model automatically learns and extracts task features by analyzing task text descriptions;

The task feature extraction model comprises a word vector layer, a convolution layer, a pooling layer and a full connection layer; the word vector layer converts the message text description into a word vector sequence by adopting a word vector training mode with semantic supervision;

defining a word vector layer into an objective function Lc;

；

Wherein v _w is a word vector of the word w; v _s is a word vector corresponding to word s for which word w has a dependency relationship; v _w ^T is the special transpose of v _w; sigma is a Sigmoid activation function; y is a label, y is 1, and represents that the words have dependency relationship, and y is 0, and represents no relationship; i is an index; the existence of the dependency relationship refers to the existence of the dependency relationship on the semantics or the convincing of the words in a sentence or a text;

the input feature map refers to the feature representation of the current convolution module layer; a feature map is obtained after convolution, activation and other operations; feature abstraction representing the input text; then A feature map representing the network operation output of the previous layer, which is used as the input feature map of the current convolution module layer; for example, at the first convolution module layer,/>The text vector sequence output by the word vector layer is subjected to vector arrangement and arrangement to form a primary feature map; input feature map/>, by residual connectionThe characteristic is directly added with the output characteristic diagram of the current convolution module, so that characteristic re-correction and characteristic enhancement are realized;

for a new task to be allocated, analyzing text description of the new task to be allocated by using a task feature extraction model which is completed by training, and outputting a feature data vector to be allocated; the feature data vector to be allocated is a vector formed by probability values of z feature categories;

selecting the characteristic category with the maximum probability value and the probability value thereof as the characteristic data to be allocated of the task to be allocated;

Further, the method for searching the resource allocation scheme by using the improved genetic algorithm comprises the following steps:

Step 205, carrying out mutation operation on the population individuals obtained after crossing according to mutation probability, guiding the mutation operation by utilizing constraint conditions, and controlling the mutation range and direction to avoid generating illegal individuals; generating new population individuals by replacing part of the segments in the resource allocation scheme;

The method for generating the initial w resource allocation schemes comprises the following steps:

Inquiring a resource library module to acquire allocable resource information; the task library inquiring module is used for acquiring feature data to be distributed of the task to be distributed; for example, the feature data to be assigned for a task is [ analysis task, 0.8] [ training task, 0.15] [ storage task, 0.05], which means that the probability that the task is judged to be the "analysis task" is highest, reaching 0.8; extracting the characteristic of the analysis task and the probability value thereof as characteristic data to be allocated for the task;

traversing all tasks to be allocated, and pre-allocating corresponding types of resources for the tasks to be allocated according to the characteristic data to be allocated of the tasks to be allocated, namely pre-allocating the tasks to be allocated; for example, for "analysis tasks," CPU and memory resources are allocated in advance in a matched manner;

On the basis of pre-allocation, constructing an initial population individual to complete initial random mapping of resources-tasks; checking whether each population individual accords with the resource constraint condition; if the initial population size is not consistent with the initial population size, the regeneration is abandoned, and the population with the initial population size of w is obtained;

the acquisition mode of the resource constraint condition comprises the following steps:

only selecting resources with healthy state of resources to participate in task allocation; preferentially selecting resources with lower cost in the resource cost information; when the cost of a plurality of resources is similar, other constraint conditions are considered; avoiding selecting resources with busy load states for allocation; resources with load states of idle and normal are preferentially considered; tasks can only be allocated to resources capable of providing corresponding service types; for example, storage tasks can only be allocated to storage class resources and cannot be allocated to computing resources; the sum of the amount of resources occupied by tasks allocated on a single resource cannot exceed the maximum provisioning capacity of that resource.

The process of the simulation evaluation of the performance index comprises the following steps:

Establishing a software mapping relation between resources and tasks according to a resource allocation scheme, and forming a virtual execution environment; the execution environment does not need to actually run tasks, and then the execution process of the simulation tasks is directly simulated in software; the software mapping relation, which tasks are allocated to which resources to be executed, forms a logical mapping relation, but the mapping is in a software space, does not actually run the tasks, and is only used for constructing a simulation environment;

Constructing a simulation platform by taking a discrete event simulation technology as a core, and setting simulation parameters, for example, simulation parameters including simulation time length, time step and the like;

the discrete event simulation technology is a computer simulation technology, and describes the process of the system state change along with time in a discrete independent event mode according to a time sequence; in discrete event simulation, the variable state is changed only when a discrete event occurs and remains unchanged within the event interval; for example, simulations of the resource allocation system, events may be set to points in time when tasks arrive, tasks begin to execute, tasks complete, etc., at which discrete event states change;

Writing a simulation scene, setting logic of task operation, dependency relationship among tasks, a time model of resource providing service and the like to form a simulation model;

running a simulation model, and recording performance indexes in the simulation execution process, wherein the performance indexes comprise resource utilization rate, task circulation time, task scheduling overhead and the like;

And repeatedly running the simulation model for N times, and counting out an average value or a distribution value of each performance index, namely the performance index of the resource allocation scheme.

The roulette wheel wager selection process includes:

Calculating the cumulative probability of the fitness value of each population individual in the total fitness, namely the selected probability; generating a random number between 0 and 1, and selecting population individuals with high fitness according to probability;

The reason for the generation of the random number is that random sampling property is ensured in order to introduce randomness; specifically, in roulette selection, the probability that an individual is selected is proportional to the value of the fitness, and if the individual is selected directly according to the value of the fitness, the effect of random sampling is lost; the introduction of the random number can enable the individual to be selected according to the adaptability probability, and meanwhile, certain randomness is maintained, so that the situation of sinking into a local optimal solution is avoided; for example, assuming that there are 3 individuals, the fitness values are 0.1, 0.2, 0.7, respectively, and the total fitness is 1; the probabilities of their selection should be 10%, 20%, 70%, respectively; if the random number is not used, selecting the 3 rd individual according to the fitness, wherein the method is obviously unreasonable;

the elite retention selection process comprises the following steps:

Sorting all population individuals according to the fitness value, and reserving the highest P% of population individuals as elite to directly enter the next generation population, for example, reserving the highest P value as 10; thus, the condition that the optimal individual is eliminated in each iteration can be prevented;

the combination selection is to calculate a certain proportion (for example, 70%) of population individuals to select through a roulette manner, and another proportion (for example, 30%) is obtained through elite retention selection, so that new population individuals are finally formed, thereby ensuring random diversity and preventing optimal individuals from losing.

Step 4, specifically, randomly selecting individuals with higher partial fitness according to the cross probability to pair each other, and then performing cross operation on each pair of individuals to generate new individuals;

the crossing mode of the crossing operation is single-point crossing or multi-point crossing, specifically, the two parent individuals are divided by randomly selecting the crossing point, and then the segments in the two parent individuals are interchanged to form two new child individuals;

Examples are as follows:

parent A is |a1|a2|a3|; parent B is |b1|b2|b3|; the offspring A after single-point crossing is |a1|a2|b3|; the filial generation B is |b1|b2|a3|;

The traditional code crossing can generate illegal subunits, and the code crossing method based on task priority and resource matching degree, which accords with the service characteristics, is designed in consideration of the characteristics of the resource allocation problem

The coding ordering is carried out according to the priority of the tasks, and the tasks with high priority can be well inherited into the sub-units when being crossed in the front section of the coding; the resource matching degree of each task and the resource is preset to be increased, and the task and the resource are guided to be distributed through the resource matching degree when the tasks and the resource are crossed, so that illegal child generation is avoided;

The mutation probability is that for new individuals generated by crossing, part of individuals are randomly selected to carry out mutation operation according to a preset low mutation probability (for example, 10 percent);

The mutation operation refers to randomly selecting a position in the resource allocation scheme code, changing the task allocation resource value of the position, and generating a new population individual;

the method for guiding the mutation operation by using the constraint condition comprises the following steps:

The mutated points can only be modified among tasks, and the resource types of the tasks are not allowed to be directly changed; the newly allocated resource type must be within its selectable range; the allocation task of a single resource cannot exceed its upper limit of working capacity.

defining a two-dimensional matrix Mapping [ M ] [ N ] for representing a soft Mapping relation, wherein M is the number of resources, N is the number of tasks, and the matrix is constructed only in a software simulation space;

traversing a resource allocation scheme, and analyzing each task and allocated resources to obtain an analysis result; setting the corresponding element value in the Mapping [ M ] [ N ] matrix to be 1 according to the analysis result; for example, task 1 is allocated to resource 3 and resource 5, then Mapping [3] [1] =1, mapping [5] [1] =1;

actively triggering the redistribution of the soft mapping relation according to the state of the resource and the change of the task priority; the comprehensive test of the simulation scene is achieved through the mapping of the disturbance task and the resource;

Preferably, an interface and an interaction module are established, the mapping relation between the resources and the tasks is displayed in a visual mode, interactive modification is allowed, and evaluation is intuitively assisted;

Further, the method for obtaining the scoring result includes:

defining a scoring index system, wherein the scoring index system comprises scoring items of four dimensions of task satisfaction, resource utilization, task delay time and system throughput;

Normalizing the scoring sample matrix and calculating a covariance matrix of the scoring sample matrix; decomposing the characteristic values of the covariance matrix, and reserving the first M characteristic vectors with larger characteristic values; constructing a transformation matrix by taking the reserved M eigenvectors as column vectors; mapping the original scoring samples to a new space by using a transformation matrix to obtain main components of the dimensionality reduction scoring; the principal components are subjected to linear weighting to obtain comprehensive scores, namely scoring results; the linear weighted weighting coefficient adopts the decreasing weight of the corresponding characteristic value;

the scoring samples refer to scoring data obtained after each scoring of the soft mapping relation; for example, the soft mapping relationship is scored at time t1, and the obtained task satisfaction is 0.8; the resource utilization rate is 0.7; the task delay time is 0.9; the system throughput is 0.6; the four scoring data form a scoring sample at time t 1;

The way to score the task satisfaction, a scoring term, includes:

The resource requirement of the task ii is Rii, and the actual obtained resource quantity is Pii; task ii satisfaction Sii =1 when Pii is not less than ri; task ii satisfaction Sii = Pii/ri when Pii < ri;

Converting the number of non-dimensional resources into satisfaction quantitative indexes in the range of 0-1;

the way to score the resource utilization rate scoring item includes:

The usage amount of the resource jj is Ujj, the total amount is Hjj, and the utilization rate RUjj = Ujj/Hjj;

the way to score the task delay time, the scoring term, includes:

counting the total use of each task from entering the system to finishing, and recording the total delay time Tii of each task; collecting and averaging Tii of all tasks to obtain average delay time Ta;

Presetting a delay time tolerance threshold Tt, and when Ta < Tt, a task delay time score Rd=1; when Ta is greater than or equal to Tt, task delay time score rd=tt/Ta;

The delay time tolerance threshold Tt includes:

collecting historical task operation data, and counting actual total delay time from submission to completion of different types of tasks; respectively carrying out delay time distribution analysis on different types of tasks and confirming a peak value distribution interval; comprehensively considering task priority, and selecting a lower value of a distribution interval as a threshold value for a high-priority task; for low priority tasks, selecting a higher value as the threshold; considering the maximum delay time acceptable by the task, and determining the longest tolerance time of various tasks; and integrating the statistical analysis and the experience value, and presetting recommended delay time tolerance thresholds of various tasks.

The scoring of the scoring term of the system throughput Ts includes:

counting the number NT of tasks completed by the system in unit time; presetting an ideal throughput Tt, namely an ideal processing task rate of the system; when NT is equal to or greater than Tt, the system throughput score rs=1; when NT < Tt, the system throughput score rs=nt/Tt;

The method for presetting the ideal throughput comprises the following steps:

Comprehensively considering future service growth expectations, and determining an ideal throughput target which is stable and has a certain elastic buffer; in a preferred embodiment, the ideal throughput is 80% of the current maximum throughput;

The task satisfaction degree scoring item quantifies the satisfaction condition of a task party by comparing the resource demand description of the task with the obtained resource quantity; the resource utilization rate directly uses the resource utilization rate data in the simulation scene; the task delay time scoring item counts the waiting and executing time of the task; the system throughput scoring item counts the number of tasks completed per unit time.

According to the embodiment, the dynamic optimization of resource allocation is realized by establishing the soft mapping relation between the resources and the tasks, and the reallocation can be actively triggered according to the real-time states of the resources and the tasks, so that the resource allocation scheme can rapidly respond to the dynamic changes of the resources and the tasks; meanwhile, task characteristics are automatically extracted by adopting a model based on deep learning, and an optimal resource allocation scheme is searched by adopting an improved genetic algorithm, so that data-driven automatic decision is realized, and excessive dependence on expert experience is avoided; in addition, the effect of the resource allocation scheme is intuitively and systematically monitored and evaluated by using the scoring sample matrix, the condition of insufficient resource utilization can be found and adjusted so as to improve the resource utilization rate; the performance of different resource allocation schemes can be rapidly and efficiently evaluated through the simulation platform; the method realizes the intellectualization, the dynamics and the visualization of the resource allocation process, can make better resource allocation decisions in a complex environment, improves the resource utilization efficiency and better serves task demands.

Example 2: referring to fig. 2, the disclosure provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the above-provided running mode of the collaborative visual simulation platform when executing the computer program.

Since the electronic device described in this embodiment is an electronic device used for implementing a collaborative visual simulation platform in the embodiment of the present application, based on the collaborative visual simulation platform described in the embodiment of the present application, a person skilled in the art can understand a specific implementation manner of the electronic device and various modifications thereof, so how to implement the method in the embodiment of the present application for the electronic device will not be described in detail herein. As long as the person skilled in the art implements the electronic device adopted by the collaborative visual simulation platform in the embodiment of the application, the electronic device belongs to the scope of protection required by the application.

The above formulas are all formulas with dimensionality removed and numerical calculation, the formulas are formulas with the latest real situation obtained by software simulation through collecting a large amount of data, and preset parameters and threshold selection in the formulas are set by those skilled in the art according to the actual situation.

The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above examples, and all technical solutions belonging to the concept of the present invention belong to the protection scope of the present invention. It should be noted that modifications and adaptations to those skilled in the art without departing from the principles of the present invention are intended to be comprehended within the scope of the present invention.

Claims

1. A collaborative visual simulation platform, comprising: the resource library module is used for storing the allocable resources;

2. The collaborative visual simulation platform of claim 1, wherein the allocatable resources comprise computing resources, storage resources, and network resources; storing corresponding specific information of the allocable resources;

3. The collaborative visual simulation platform according to claim 2, wherein the method for obtaining the health status of the resource comprises:

The resource cost information obtaining mode comprises the following steps:

4. A collaborative visual simulation platform according to claim 3, wherein the training of the resource health assessment model comprises:

5. The collaborative visual simulation platform of claim 4, wherein the predetermined improved recurrent neural network approach comprises:

the formula for adding the gating cell is: ；/>；

6. The collaborative visual simulation platform of claim 5, wherein the tasks to be distributed include manually created tasks, automatically triggered tasks from a third party system scheduling task and business process, and respective corresponding task attributes; the task attributes comprise business attributes, priorities, expiration dates and resource consumption descriptions;

7. The collaborative visual simulation platform according to claim 6, wherein the obtaining manner of the feature data to be distributed comprises:

defining a word vector layer into an objective function Lc;

；

8. The collaborative visual simulation platform according to claim 7, wherein the searching for a resource allocation scheme using an improved genetic algorithm comprises:

9. The collaborative visual simulation platform according to claim 8, wherein the manner of establishing a soft mapping relationship between resources and tasks comprises:

10. The collaborative visual simulation platform according to claim 9, wherein the manner of obtaining the scoring result comprises: