CN117474464B - Multi-service processing model training method, multi-service processing method and electronic equipment - Google Patents
Multi-service processing model training method, multi-service processing method and electronic equipment Download PDFInfo
- Publication number
- CN117474464B CN117474464B CN202311274812.0A CN202311274812A CN117474464B CN 117474464 B CN117474464 B CN 117474464B CN 202311274812 A CN202311274812 A CN 202311274812A CN 117474464 B CN117474464 B CN 117474464B
- Authority
- CN
- China
- Prior art keywords
- service
- sample data
- feature
- model
- features
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012549 training Methods 0.000 title claims abstract description 63
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000012545 processing Methods 0.000 title claims abstract description 34
- 238000003672 processing method Methods 0.000 title claims abstract description 13
- 238000000605 extraction Methods 0.000 claims abstract description 46
- 230000008859 change Effects 0.000 claims description 33
- 230000004927 fusion Effects 0.000 claims description 23
- 230000003068 static effect Effects 0.000 claims description 21
- 238000004891 communication Methods 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 10
- 238000012423 maintenance Methods 0.000 claims description 8
- 238000012544 monitoring process Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 claims description 6
- 238000003709 image segmentation Methods 0.000 claims description 5
- 230000007246 mechanism Effects 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 4
- 238000013145 classification model Methods 0.000 claims description 3
- 230000004913 activation Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 239000013598 vector Substances 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 230000003213 activating effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
- G06Q10/103—Workflow collaboration or project management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0475—Generative networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/094—Adversarial learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/16—Real estate
- G06Q50/163—Real estate management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Strategic Management (AREA)
- Human Resources & Organizations (AREA)
- General Health & Medical Sciences (AREA)
- Tourism & Hospitality (AREA)
- Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Marketing (AREA)
- Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- General Business, Economics & Management (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Primary Health Care (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a multi-service processing model training method, a multi-service processing method and electronic equipment, wherein the multi-service processing model comprises a multi-branch feature extraction model, a multi-head attention module and a service identification model; the training method comprises the following steps: acquiring multi-service sample data with service labels; inputting the multi-service sample data into the multi-branch feature extraction model, and respectively extracting specific features in each service sample data and common features among a plurality of services; inputting the specific features into the multi-head attention module, and respectively carrying out feature enhancement on the specific features, wherein the attention feature of each specific feature; and training the service model for multiple times based on the attention features, the commonality features and the service labels to obtain a trained identification model. The method can enable one model to identify the states of multiple services, and further can realize overall management of service coordination by multiple service departments.
Description
Technical Field
The application relates to the technical field of area management, in particular to a multi-service processing model training method, a multi-service processing method and electronic equipment.
Background
The intelligent park is to use various information technologies or innovative concepts to open and integrate the systems and services of the park, so as to improve the efficiency of resource utilization, optimize park management and services, and improve the quality of life of citizens. The intelligent park fully utilizes the new generation information technology in the park information advanced form of each industry based on the next generation innovation of the knowledge society, realizes the deep integration of informatization, industrialization and towns, improves the towns quality, realizes the fine and dynamic management, improves the park management effect and improves the living quality of citizens.
However, the event occurring in the intelligent park is often not an isolated event, for example, may also be accompanied by a public security event in some events, and may also be accompanied by a public security event in some public facility monitoring events. The management among the business departments is discrete, and when multiple types of events occur, resource sharing and business coordination are difficult to achieve in time.
Therefore, how to perform overall management of business collaboration for multiple business departments becomes a technical problem to be solved.
Disclosure of Invention
The application provides a multi-service processing model training method, a multi-service processing method and electronic equipment, which at least solve the technical problem of how to carry out overall management of service collaboration on a plurality of service departments in the related technology.
According to a first aspect of the present application, there is provided a multi-business process model training method, the multi-business process model including a multi-branch feature extraction model, a multi-head attention module and a business identification model; the training method comprises the following steps: acquiring multi-service sample data with service labels; inputting the multi-service sample data into the multi-branch feature extraction model, and respectively extracting specific features in each service sample data and common features among a plurality of services; inputting the specific features into the multi-head attention module, and respectively carrying out feature enhancement on the specific features to obtain the attention features of each specific feature; and training the service model for multiple times based on the attention features, the commonality features and the service labels to obtain a trained identification model.
Optionally, the multi-branch feature extraction model includes a discriminator and a feature generation network corresponding to the service one to one; the inputting the multi-service sample data into the multi-branch feature extraction model, and extracting the specific feature and the common feature among the plurality of services in each service sample data respectively comprises the following steps: extracting initial specificity characteristics and initial commonality characteristics among a plurality of services in each service sample data by utilizing the characteristic generating network; training the discriminator by using the initial common characteristic and the initial specific characteristic, so that the trained discriminator can perform modal classification on the initial specific characteristic, and cannot perform modal classification on the initial common characteristic, and the distance between the common characteristic and the specific characteristic is larger than a preset distance.
Optionally, the multi-service sample data includes: park operation and maintenance sample data, park emergency event sample data and park public facility monitoring sample data.
Optionally, the multi-service sample data includes video sample data and passive sensor sample data; before inputting the multi-business sample data into the multi-branch feature extraction model, comprising: image segmentation is carried out on the video sample data to obtain a plurality of static feature sequences corresponding to the service; filtering the passive sensor sample data to obtain a plurality of sensing signal characteristic sequences corresponding to the service; extracting time sequence image change characteristics in a static characteristic sequence corresponding to each service respectively; extracting time sequence sensing signal change characteristics in a sensing signal characteristic sequence corresponding to each service respectively; performing time sequence and dimension alignment on the time sequence image change characteristics and the time sequence sensing signal change characteristics; clustering the aligned post-time sequence image change characteristics and the time sequence sensing signal change characteristics to obtain clustering service characteristics corresponding to each service.
Optionally, the inputting the specific features into the multi-head attention module performs feature enhancement on the specific features, and the attention features of each specific feature include: the attention between the specific feature and the corresponding business is learned using a multi-headed attention mechanism.
Optionally, the identification model includes a plurality of identification network branches corresponding to the services one to one; the training the service model for multiple times based on the attention feature, the commonality feature and the service label, and obtaining a trained identification model comprises the following steps: performing feature fusion on the attention feature and the commonality feature to obtain a fusion feature; and training the corresponding recognition network branches by utilizing the fusion characteristics and the service labels respectively to obtain a trained classification model, wherein model parameters can be shared among each recognition network branch.
According to a second aspect, an embodiment of the present application provides a multi-service processing method, including: acquiring data to be processed; inputting the data to be processed into the multi-service processing model according to any one of the first aspect, and obtaining a service identification result.
According to a third aspect, an embodiment of the present application provides a multi-service processing model training apparatus, where the multi-service processing model includes a multi-branch feature extraction model, a multi-head attention module, and a service identification model; the training device comprises: the acquisition module is used for acquiring multi-service sample data with service labels; the first feature extraction module is used for inputting the multi-service sample data into the multi-branch feature extraction model and respectively extracting specific features in each service sample data and common features among a plurality of services; the second feature extraction module is used for inputting the specific features into the multi-head attention module, and respectively carrying out feature enhancement on the specific features to obtain the attention features of each specific feature; and the recognition model training module is used for training the business model for multiple times based on the attention characteristic, the commonality characteristic and the business label to obtain a trained recognition model.
According to a fourth aspect, an embodiment of the present application provides an electronic device according to the second aspect, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus, and the memory is configured to store a computer program; the processor is configured to execute the multi-service processing model training method according to any one of the first aspect and/or the multi-service processing method according to the second aspect by executing the computer program stored on the memory.
According to a fifth aspect, an embodiment of the present application provides a computer readable storage medium, characterized in that the storage medium has stored therein a computer program, wherein the computer program is arranged to execute the multi-service processing model training method according to any one of the first aspect and/or the multi-service processing method according to the second aspect at run-time.
According to the multi-service processing model training method, multi-service sample data with service labels are obtained; inputting the multi-service sample data into the multi-branch feature extraction model, and respectively extracting specific features in each service sample data and common features among a plurality of services; inputting the specific features into the multi-head attention module, and respectively carrying out feature enhancement on the specific features, wherein the attention feature of each specific feature; and training the service model for multiple times based on the attention features, the commonality features and the service labels to obtain a trained identification model. The recognition model can learn the characteristics related to the business according to different attention weights, and can learn the characteristics in other businesses. And a model can be used for identifying the states of various services, so that the overall management of service coordination by a plurality of service departments can be realized.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
FIG. 1 is a schematic diagram of a multi-service processing model according to the present application;
FIG. 2 is a schematic flow chart of a multi-service processing model training method provided by the application;
fig. 3 is a schematic flow chart of a multi-service processing method provided by the application;
FIG. 4 is a schematic diagram of a multi-service processing model training device provided by the application;
fig. 5 is a schematic diagram of an electronic device provided by the present application.
Detailed Description
In order to more clearly illustrate the general inventive concept, a detailed description is given below by way of example with reference to the accompanying drawings.
In order that those skilled in the art will better understand the present application, a technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The application provides a multi-service processing model training method, referring to fig. 1, wherein the multi-service processing model comprises a multi-branch feature extraction model, a multi-head attention module and a service identification model, referring to fig. 2, the method can comprise the following steps:
s10, acquiring multi-service sample data with service labels.
As an exemplary embodiment, the multi-service sample data includes: park operation and maintenance sample data, park emergency event sample data and park public facility monitoring sample data. The sample data may be video data collected by a camera, or may be a sensing signal collected by some passive sensor, for example, a vibration signal collected by a vibration cable, a vibration fiber, a ground wave sensor, or the like.
In this embodiment, the data collected by each service department may be obtained as multi-service sample data.
S20, inputting the multi-service sample data into the multi-branch feature extraction model, and respectively extracting specific features in each service sample data and common features among a plurality of services. As an exemplary embodiment, the number of branches of the feature extraction model may be the same as the number of services, that is, one feature extraction network branch corresponding to each service, so as to extract a specific feature in each service sample data and a common feature between a plurality of services, respectively. In this embodiment, each feature extraction network branch may include a specific feature extraction sub-branch and a common feature extraction sub-branch to extract common features and specific features in the corresponding service sample data, respectively.
Wherein the specific features may include key features characterizing the current service, for example, corresponding specific vibration features or image change features may be generated when a utility in the utility monitoring service is disturbed or triggered; or the characteristic vibration characteristic or the characteristic image change characteristic of operation and maintenance equipment or operation and maintenance personnel when operation and maintenance is carried out, or the characteristic vibration characteristic or the characteristic image change characteristic corresponding to an emergency event in the emergency event. The commonality feature may include a feature common among a plurality of services, for example, a dynamic feature of a person in video data, a dynamic feature of a vehicle or device, or a dynamic feature of a utility, or the like, or a vibration feature of a person in a sensor signal, a vibration feature of a vehicle or device, or a vibration feature of a utility, or the like. In this embodiment, the extracted common features may include features that are common to a plurality of services when they occur.
As an exemplary embodiment, the multi-branch feature extraction model includes a discriminator and a feature generation network in one-to-one correspondence with the traffic; the inputting the multi-service sample data into the multi-branch feature extraction model, and extracting the specific feature and the common feature among the plurality of services in each service sample data respectively comprises the following steps: extracting initial specificity characteristics and initial commonality characteristics among a plurality of services in each service sample data by utilizing the characteristic generating network; training the discriminator by using the initial common characteristic and the initial specific characteristic, so that the trained discriminator can perform modal classification on the initial specific characteristic, and cannot perform modal classification on the initial common characteristic, and the distance between the common characteristic and the specific characteristic is larger than a preset distance.
In this embodiment, the discriminant assist features may be utilized to generate training of the network. And carrying out at least one round of alternating training on the discriminant and the characteristic generating network, wherein each round of alternating training comprises at least one discriminant training stage and at least one characteristic generating network training stage. Illustratively, the training process may be: the discriminant training stage-the feature generation network training stage … … are used as alternate training of one round, and then the alternate training is repeated for a plurality of times, so that the finally obtained common feature mode extracted by the feature generation network is irrelevant, namely the mode classification cannot be carried out by a front-mounted discriminant, and the obtained common feature and the obtained specific feature are independent linearly.
The method comprises the steps of constructing a feature generation network for extracting specific features and common features, carrying out at least one round of alternate training on a discriminator and the feature generation network, minimizing a discriminator loss function in a discriminator training stage, enabling the discriminator to carry out modal classification on the specific features, enabling the discriminator to not classify the common features, and enabling the distance between the common features and the specific features to be larger than a preset distance; maximizing the loss function of the discriminator in the training stage of the feature generation network until the common feature generated by the feature generation network cannot be subjected to modal classification by the discriminator, wherein the common feature and the specific feature are linearly independent, the common feature cannot be distinguished by the modal classifier through a maximum-minimum countermeasure learning strategy, and meanwhile, the specific feature and the common feature are linearly independent.
S30, inputting the specific features into the multi-head attention module, and respectively carrying out feature enhancement on the specific features to obtain the attention features of each specific feature. As an exemplary embodiment, each specific feature is weighted by a multi-head attention module, that is, each specific feature is marked according to its corresponding service, and the multi-head attention module learns the attention between the specific feature and the corresponding service by using a multi-head attention mechanism, so that the multi-head attention module can identify key features focused by each service.
S40, training the service model for multiple times based on the attention features, the commonality features and the service labels to obtain a trained identification model. As an exemplary embodiment, feature fusion is performed on the attention feature and the commonality feature to obtain a fusion feature; the fusion features fuse the attention weights of key features focused by each service, and through fusion with the common features, the fusion features not only have wider feature dimensions, but also have attention mechanisms corresponding to the services, and when the model is trained, the recognition model can learn the key features focused by each service, and has wider recognition capability.
And training the identification model by utilizing the fusion feature and the service tag in turn by utilizing the fusion feature and the service tag to obtain fusion model parameters, wherein the fusion model parameters and the corresponding service and the attention weights in the corresponding fusion feature have corresponding weights, so that the identification model can learn the features related to the service in a targeted manner according to different attention weights and learn the features in other services.
Or the identification model comprises a plurality of identification network branches corresponding to the services one by one; and training the corresponding recognition network branches by utilizing the fusion characteristics and the service labels respectively to obtain a trained classification model, wherein model parameters can be shared among each recognition network branch. Through a model parameter sharing mechanism, after one of the identification models identifies the characteristics of other services, the model can be endowed with model parameters corresponding to the other services, so that one model can identify the states of multiple services, and further, the overall management of service coordination by multiple service departments can be realized.
As an alternative embodiment, the multi-service sample data includes video sample data and passive sensor sample data; before inputting the multi-business sample data into the multi-branch feature extraction model, comprising:
Image segmentation is carried out on the video sample data to obtain a plurality of static feature sequences corresponding to the service; and filtering the passive sensor sample data to obtain a plurality of sensing signal characteristic sequences corresponding to the service. As an exemplary embodiment, when image segmentation is performed, the requirement of each business department can be referred, and if any entity corresponding to the business exists in the same image, the entity can be segmented and extracted. Classifying according to the service, and constructing a static feature sequence according to the time sequence by the classified static features. Likewise, a plurality of sensor signal feature sequences corresponding to the service are also constructed for the sensor data samples in the manner of image data samples.
And extracting time sequence image change characteristics in the static characteristic sequence corresponding to each service respectively. Extracting image data or screened key frames through a pre-trained image feature extraction network image feature, in the embodiment, extracting through a cascade convolution network and a time sequence feature extraction network image feature, extracting static features through the convolution network, constructing a static feature sequence after the static features are obtained, and extracting time sequence related features in the static feature sequence through the time sequence feature extraction network, namely extracting dynamic image features. To ensure that specific static features and features of dynamic states can be extracted.
As an alternative embodiment, in a static feature extraction network, video data or key frame data is input into the network in units of pixels, and features are extracted in one-dimensional convolution and activation operations. The input image is activated as a LeakyReLU function after passing through the convolutional layer. In the first convolution layer, batch normalization is also used to enhance the processing effect of the activation function, accelerate the convergence of the model, and prevent the appearance of gradient vanishing.
Specifically, in the static feature extraction network, two-dimensional operation is performed on video data or key frame data. Firstly, inputting data into a convolution layer, wherein the convolution kernel size is 3 multiplied by 3, and the filling parameter is 1, and activating by using LeakyReLU activation layers after batch standardization operation; then input into the convolution layer with the convolution kernel size of 1×1, and use LeakyReLU activation layer to make secondary activation, finally use the largest pooling layer with the pooling size of 2×2 to pool the feature map.
Illustratively, in a static feature extraction network, video data or key frame data is input into the network in units of regions of radius r, where r is a super parameter, centered at each pixel. Thus, the convolution operation performed thereon and the activation operation are two-dimensional. In this way, the network learns the spatial information contained in the video data or key frame data by examining the information in each pixel and its neighborhood. To extract static features in the video data or key frame data that are related to spatial information.
After the static features are obtained, time-series related features in the static feature sequence are extracted through a cyclic neural network (such as LSTM, GRU) to characterize the image change features.
And respectively extracting time sequence sensing signal change characteristics in the sensing signal characteristic sequences corresponding to each service. In this embodiment. Taking the sensing signal as a vibration signal for example for explanation, in this embodiment, the original signal may be subjected to wavelet analysis to obtain wavelet coefficients under multiple scales, and then the time domain and frequency domain information may be extracted simultaneously by the change of the wavelet coefficients. The timing related features of the vibrations are extracted by a recurrent neural network (e.g., LSTM, GRU).
And carrying out time sequence and dimension alignment on the time sequence image change characteristic and the time sequence sensing signal change characteristic. The sensor signal features and the image change features have time sequence association, that is, when the state of an object changes, sensor signals are correspondingly generated, so that the features can be aligned based on a time sequence relationship, in addition, the image change features and the sensor signal features are different in feature dimension, in this embodiment, the two features are also required to be aligned in dimension, in this embodiment, the sensor signal features are taken as vibration features, vibration features can be described by taking a wavelet packet decomposition tree as an example, for example, a plurality of frequency band signal components are extracted, energy of signals of each frequency band is obtained, feature vectors are constructed by taking the energy of each frequency band of the signals as elements, and the vector is normalized to obtain a histogram of the feature vectors. And similarly, binarizing the image change features, and obtaining a feature vector histogram of the image change features to further realize dimension alignment.
Clustering the aligned post-time sequence image change characteristics and the time sequence sensing signal change characteristics to obtain clustering service characteristics corresponding to each service. One or more cluster centers are determined based on semantic units of the plurality of vibration feature entities. The clusters (or clusters) generated by the clustering operation are a collection of data objects that are similar to each other and different from the objects in the other clusters. The cluster center is the most important one of the objects in the cluster, which is the most representative of the cluster and the most interpretable for the other objects in the cluster. In some embodiments, a cluster has only one cluster center. In some embodiments, the cluster center may be one or more vibration feature entities selected from a plurality of vibration feature entities, each cluster center serving as a reference object in calculating a similarity between the cluster center and other vibration feature entities of the plurality of vibration feature entities. After the cluster center is obtained, the vibration characteristic of the cluster center is taken as the specific vibration characteristic corresponding to the embodiment.
The first clustering module is used for respectively obtaining the vibration characteristics and the non-vibration characteristics so that the vibration characteristics are more similar, the vibration characteristics and the non-vibration characteristics are less similar, and the recognition model is easier to distinguish the vibration characteristics and the non-vibration characteristics.
The embodiment of the application also provides a multi-service processing method, as shown in fig. 3, which can include:
S100, acquiring data to be processed;
s200, inputting the data to be processed into a multi-service processing model to obtain a service identification result. In this embodiment, the multi-service processing model is obtained based on the training method in the above embodiment.
As an exemplary embodiment, the plurality of services may include a campus operation and maintenance service, a campus emergency service, and a campus public facility monitoring service, which in this embodiment are described by way of example as follows:
After the data to be processed of any business department is obtained, the data to be processed is input into a multi-business processing model, specific features and common features are extracted, attention weights are given to the specific features through a multi-head attention module, the specific features with the attention weights and the common features are fused to obtain fusion features, and in the embodiment, feature fusion can be performed in a feature splicing mode, for example, feature splicing is performed through a full-connection layer.
The fusion features are input into an identification model, and the identification model can simultaneously identify different service states in the data to be processed. In this embodiment, attention weights of the parameters of the recognition model are given in real time by fusing attention weights of the features, so that the model can adaptively recognize a service state with high weight.
As another optional embodiment, the fusion feature is input into at least one identification network branch, in this embodiment, the fusion feature may be input into an identification network branch corresponding to a service department that collects the data to be processed to perform service state identification, and when an output result of other service states exists in the identification result, a model parameter of the identification network branch may be replaced with a model parameter corresponding to other services, or the model parameter and a model parameter corresponding to other services are weighted and fused according to the attention weight of the fusion feature, so that the identification model can identify multiple service states at the same time. And further, the comprehensive management of business cooperation by a plurality of business departments can be realized.
The embodiment of the application also provides a multi-service processing model training device, as shown in fig. 4, comprising:
The multi-service processing model comprises a multi-branch feature extraction model, a multi-head attention module and a service identification model; the training device comprises:
An acquisition module 41, configured to acquire multi-service sample data with service labels;
a first feature extraction module 42, configured to input the multi-service sample data into the multi-branch feature extraction model, and extract a specific feature in each service sample data and a common feature between a plurality of services;
A second feature extraction module 43, configured to input the specific features into the multi-head attention module, and perform feature enhancement on the specific features to obtain attention features of each specific feature;
the recognition model training module 44 is configured to train the service model for multiple times based on the attention feature, the commonality feature and the service tag, to obtain a trained recognition model.
It should be noted that, the acquiring module 41 in this embodiment may be used to perform the step S10, the first feature extracting module 42 in this embodiment may be used to perform the step S20, the second feature extracting module 43 in this embodiment may be used to perform the step S30, and the recognition model training module 44 in this embodiment may be used to perform the step S40.
It should be noted that the above modules are the same as examples and application scenarios implemented by the corresponding steps, but are not limited to what is disclosed in the above embodiments. It should be noted that the above modules may be implemented in software or in hardware as part of the apparatus as shown in fig. 5, where the hardware environment includes a network environment.
Thus, according to yet another aspect of an embodiment of the present application, there is also provided an electronic device for implementing the above method, which may be a server, a terminal, or a combination thereof.
Fig. 5 is a block diagram of an alternative electronic device, according to an embodiment of the application, as shown in fig. 5, comprising a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502 and the memory 503 communicate with each other via the communication bus 504, wherein,
A memory 503 for storing a computer program;
the processor 501, when executing the computer program stored on the memory 503, performs the following steps:
acquiring multi-service sample data with service labels;
Inputting the multi-service sample data into the multi-branch feature extraction model, and respectively extracting specific features in each service sample data and common features among a plurality of services;
Inputting the specific features into the multi-head attention module, and respectively carrying out feature enhancement on the specific features, wherein the attention feature of each specific feature;
and training the service model for multiple times based on the attention features, the commonality features and the service labels to obtain a trained identification model.
Alternatively, in the present embodiment, the above-described communication bus may be a PCI (PERIPHERAL COMPONENT INTERCONNECT, peripheral component interconnect standard) bus, or an EISA (Extended Industry Standard Architecture ) bus, or the like. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, only one thick line is shown in fig. 5, but not only one bus or one type of bus.
The communication interface is used for communication between the electronic device and other devices.
The memory may include RAM or may include non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
The processor may be a general purpose processor and may include, but is not limited to: CPU (Central Processing Unit ), NP (Network Processor, network processor), etc.; but may also be a DSP (DIGITAL SIGNAL Processing), ASIC (Application SPECIFIC INTEGRATED Circuit), FPGA (Field-Programmable gate array) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components.
Alternatively, specific examples in this embodiment may refer to examples described in the foregoing embodiments, and this embodiment is not described herein.
It will be appreciated by those skilled in the art that the structure shown in fig. 5 is only illustrative, and the device implementing the method may be a terminal device, and the terminal device may be a smart phone (such as an Android Mobile phone, an iOS Mobile phone, etc.), a tablet computer, a palm computer, a Mobile internet device (Mobile INTERNET DEVICES, MID), a PAD, etc. Fig. 5 is not limited to the structure of the electronic device. For example, the terminal device may also include more or fewer components (e.g., network interfaces, display devices, etc.) than shown in fig. 5, or have a different configuration than shown in fig. 5.
Those of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be implemented by a program for instructing a terminal device to execute in association with hardware, the program may be stored in a computer readable storage medium, and the storage medium may include: flash disk, ROM, RAM, magnetic or optical disk, etc.
According to yet another aspect of an embodiment of the present application, there is also provided a storage medium. Alternatively, in the present embodiment, the above-described storage medium may be used for executing the program code of the above-described method.
Alternatively, in this embodiment, the storage medium may be located on at least one network device of the plurality of network devices in the network shown in the above embodiment.
Alternatively, in the present embodiment, the storage medium is configured to store program code for performing the steps of:
acquiring multi-service sample data with service labels;
Inputting the multi-service sample data into the multi-branch feature extraction model, and respectively extracting specific features in each service sample data and common features among a plurality of services;
Inputting the specific features into the multi-head attention module, and respectively carrying out feature enhancement on the specific features, wherein the attention feature of each specific feature;
and training the service model for multiple times based on the attention features, the commonality features and the service labels to obtain a trained identification model.
Alternatively, specific examples in the present embodiment may refer to examples described in the above embodiments, which are not described in detail in the present embodiment.
Alternatively, in the present embodiment, the storage medium may include, but is not limited to: various media capable of storing program codes, such as a U disk, ROM, RAM, a mobile hard disk, a magnetic disk or an optical disk.
The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
The integrated units in the above embodiments may be stored in the above-described computer-readable storage medium if implemented in the form of software functional units and sold or used as separate products. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, comprising several instructions for causing one or more computer devices (which may be personal computers, servers or network devices, etc.) to perform all or part of the steps of the method described in the embodiments of the present application.
In the foregoing embodiments of the present application, the descriptions of the embodiments are emphasized, and for a portion of this disclosure that is not described in detail in this embodiment, reference is made to the related descriptions of other embodiments.
In several embodiments provided by the present application, it should be understood that the disclosed client may be implemented in other manners. The above-described embodiments of the apparatus are merely exemplary, and the division of the units, such as the division of the units, is merely a logical function division, and may be implemented in another manner, for example, multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interfaces, units or modules, or may be in electrical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution provided in the present embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The application can be realized by adopting or referring to the prior art at the places which are not described in the application.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and variations of the present application will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the application are to be included in the scope of the claims of the present application.
Claims (7)
1. The multi-service processing model training method is characterized in that the multi-service processing model comprises a multi-branch feature extraction model, a multi-head attention module and a service identification model; the training method comprises the following steps:
acquiring multi-service sample data with service labels;
Inputting the multi-service sample data into the multi-branch feature extraction model, and respectively extracting specific features in each service sample data and common features among a plurality of services;
Inputting the specific features into the multi-head attention module, and respectively carrying out feature enhancement on the specific features to obtain the attention features of each specific feature;
Training the service model for multiple times based on the attention feature, the commonality feature and the service label to obtain a trained identification model;
The multi-branch feature extraction model comprises discriminators and feature generation networks which are in one-to-one correspondence with the services;
The inputting the multi-service sample data into the multi-branch feature extraction model, and extracting the specific feature and the common feature among the plurality of services in each service sample data respectively comprises the following steps:
Extracting initial specificity characteristics and initial commonality characteristics among a plurality of services in each service sample data by utilizing the characteristic generating network;
Training the discriminator by utilizing the initial common characteristic and the initial specific characteristic, so that the trained discriminator can perform modal classification on the initial specific characteristic, and cannot perform modal classification on the initial common characteristic, and the distance between the common characteristic and the specific characteristic is larger than a preset distance;
the multi-service sample data includes: park operation and maintenance sample data, park emergency event sample data and park public facility monitoring sample data;
the multi-service sample data includes video sample data and passive sensor sample data;
before inputting the multi-business sample data into the multi-branch feature extraction model, comprising:
Image segmentation is carried out on the video sample data to obtain a plurality of static feature sequences corresponding to the service; filtering the passive sensor sample data to obtain a plurality of sensing signal characteristic sequences corresponding to the service;
extracting time sequence image change characteristics in a static characteristic sequence corresponding to each service respectively;
extracting time sequence sensing signal change characteristics in a sensing signal characteristic sequence corresponding to each service respectively;
Performing time sequence and dimension alignment on the time sequence image change characteristics and the time sequence sensing signal change characteristics;
clustering the aligned post-time sequence image change characteristics and the time sequence sensing signal change characteristics to obtain clustering service characteristics corresponding to each service.
2. The multi-business process model training method of claim 1, wherein said inputting said specific features into said multi-headed attention module performs feature enhancement on said specific features, respectively, and the attention features of each of said specific features comprises:
the attention between the specific feature and the corresponding business is learned using a multi-headed attention mechanism.
3. The multi-service processing model training method of claim 1, wherein the recognition model comprises a plurality of recognition network branches corresponding to services one to one;
the training the service model for multiple times based on the attention feature, the commonality feature and the service label, and obtaining a trained identification model comprises the following steps:
Performing feature fusion on the attention feature and the commonality feature to obtain a fusion feature;
And training the corresponding recognition network branches by utilizing the fusion characteristics and the service labels respectively to obtain a trained classification model, wherein model parameters can be shared among each recognition network branch.
4. A multi-service processing method, comprising:
Acquiring data to be processed;
Inputting the data to be processed into the multi-service processing model according to any one of claims 1-3 to obtain a service identification result.
5. The multi-service processing model training device is characterized in that the multi-service processing model comprises a multi-branch feature extraction model, a multi-head attention module and a service identification model; the training device comprises:
the acquisition module is used for acquiring multi-service sample data with service labels;
the first feature extraction module is used for inputting the multi-service sample data into the multi-branch feature extraction model and respectively extracting specific features in each service sample data and common features among a plurality of services;
The second feature extraction module is used for inputting the specific features into the multi-head attention module, and respectively carrying out feature enhancement on the specific features to obtain the attention features of each specific feature;
The recognition model training module is used for training the business model for multiple times based on the attention features, the commonality features and the business labels to obtain a trained recognition model;
The multi-branch feature extraction model comprises discriminators and feature generation networks which are in one-to-one correspondence with the services;
The first feature extraction module is further used for extracting initial specific features and initial common features among a plurality of services in each service sample data by utilizing the feature generation network; training the discriminator by utilizing the initial common characteristic and the initial specific characteristic, so that the trained discriminator can perform modal classification on the initial specific characteristic, and cannot perform modal classification on the initial common characteristic, and the distance between the common characteristic and the specific characteristic is larger than a preset distance;
the multi-service sample data includes: park operation and maintenance sample data, park emergency event sample data and park public facility monitoring sample data;
the multi-service sample data includes video sample data and passive sensor sample data;
before inputting the multi-business sample data into the multi-branch feature extraction model, comprising: image segmentation is carried out on the video sample data to obtain a plurality of static feature sequences corresponding to the service; filtering the passive sensor sample data to obtain a plurality of sensing signal characteristic sequences corresponding to the service; extracting time sequence image change characteristics in a static characteristic sequence corresponding to each service respectively; extracting time sequence sensing signal change characteristics in a sensing signal characteristic sequence corresponding to each service respectively; performing time sequence and dimension alignment on the time sequence image change characteristics and the time sequence sensing signal change characteristics; clustering the aligned post-time sequence image change characteristics and the time sequence sensing signal change characteristics to obtain clustering service characteristics corresponding to each service.
6. An electronic device comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus, characterized in that,
The memory is used for storing a computer program;
The processor is configured to execute the multi-service processing model training method according to any one of claims 1 to 3 and/or the multi-service processing method according to claim 4 by running the computer program stored on the memory.
7. A computer readable storage medium, characterized in that the storage medium has stored therein a computer program, wherein the computer program is arranged to execute the multi-service processing model training method of any of claims 1 to 3 and/or the multi-service processing method of claim 4 at run-time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311274812.0A CN117474464B (en) | 2023-09-28 | 2023-09-28 | Multi-service processing model training method, multi-service processing method and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311274812.0A CN117474464B (en) | 2023-09-28 | 2023-09-28 | Multi-service processing model training method, multi-service processing method and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117474464A CN117474464A (en) | 2024-01-30 |
| CN117474464B true CN117474464B (en) | 2024-05-07 |
Family
ID=89626571
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311274812.0A Active CN117474464B (en) | 2023-09-28 | 2023-09-28 | Multi-service processing model training method, multi-service processing method and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117474464B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110032551A (en) * | 2019-03-12 | 2019-07-19 | 阿里巴巴集团控股有限公司 | Model generates system, method and forecasting system |
| CN112819073A (en) * | 2021-02-01 | 2021-05-18 | 上海明略人工智能(集团)有限公司 | Classification network training method, image classification device and electronic equipment |
| CN113515942A (en) * | 2020-12-24 | 2021-10-19 | 腾讯科技（深圳）有限公司 | Text processing method and device, computer equipment and storage medium |
| WO2021218828A1 (en) * | 2020-04-27 | 2021-11-04 | 支付宝(杭州)信息技术有限公司 | Training for differential privacy-based anomaly detection model |
| CN113688313A (en) * | 2021-08-12 | 2021-11-23 | 北京三快在线科技有限公司 | Training method of prediction model, information pushing method and device |
| CN114372564A (en) * | 2022-01-12 | 2022-04-19 | 腾讯科技（深圳）有限公司 | Model training method for object classification, object classification method and device |
| CN115439442A (en) * | 2022-08-31 | 2022-12-06 | 湖北星盛电气装备研究院有限公司 | Industrial product surface defect detection and positioning method and system based on commonality and difference |
-
2023
- 2023-09-28 CN CN202311274812.0A patent/CN117474464B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110032551A (en) * | 2019-03-12 | 2019-07-19 | 阿里巴巴集团控股有限公司 | Model generates system, method and forecasting system |
| WO2021218828A1 (en) * | 2020-04-27 | 2021-11-04 | 支付宝(杭州)信息技术有限公司 | Training for differential privacy-based anomaly detection model |
| CN113515942A (en) * | 2020-12-24 | 2021-10-19 | 腾讯科技（深圳）有限公司 | Text processing method and device, computer equipment and storage medium |
| CN112819073A (en) * | 2021-02-01 | 2021-05-18 | 上海明略人工智能(集团)有限公司 | Classification network training method, image classification device and electronic equipment |
| CN113688313A (en) * | 2021-08-12 | 2021-11-23 | 北京三快在线科技有限公司 | Training method of prediction model, information pushing method and device |
| CN114372564A (en) * | 2022-01-12 | 2022-04-19 | 腾讯科技（深圳）有限公司 | Model training method for object classification, object classification method and device |
| CN115439442A (en) * | 2022-08-31 | 2022-12-06 | 湖北星盛电气装备研究院有限公司 | Industrial product surface defect detection and positioning method and system based on commonality and difference |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117474464A (en) | 2024-01-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP2020504358A (en) | Image-based vehicle damage evaluation method, apparatus, and system, and electronic device | |
| Ajagbe et al. | Investigating the efficiency of deep learning models in bioinspired object detection | |
| EP3983953A1 (en) | Understanding deep learning models | |
| CN113287120A (en) | Vehicle driving environment abnormity monitoring method and device, electronic equipment and storage medium | |
| CN114244611A (en) | Abnormal attack detection method, device, equipment and storage medium | |
| Sun et al. | Image steganalysis based on convolutional neural network and feature selection | |
| CN114419363A (en) | Target classification model training method and device based on label-free sample data | |
| CN110851488A (en) | Multi-source-based multi-modal data fusion analysis processing method and platform | |
| Hossain et al. | Verifying the images authenticity in cognitive internet of things (CIoT)-oriented cyber physical system | |
| Buttar et al. | Toward trustworthy human suspicious activity detection from surveillance videos using deep learning | |
| Venkatesvara Rao et al. | Real-time video object detection and classification using hybrid texture feature extraction | |
| CN117474464B (en) | Multi-service processing model training method, multi-service processing method and electronic equipment | |
| CN115953650B (en) | Training method and device for feature fusion model | |
| Karpagam et al. | Leveraging CNN Deep Learning Model for Smart Parking | |
| McCoy et al. | Ensemble Learning for UAV Detection: Developing a Multi-Class Multimodal Dataset | |
| Goel et al. | Transfer Learning-based Driver Distraction Detection | |
| CN113569879A (en) | Training method of abnormal recognition model, abnormal account recognition method and related device | |
| Anwer et al. | Smart traffic incident reporting system in e-government | |
| CN117312828B (en) | Public facility monitoring method and system | |
| Luna et al. | License Plate Recognition for Stolen Vehicles Using Optical Character Recognition | |
| CN111310806B (en) | Classification network, image processing method, device, system and storage medium | |
| CN116778534B (en) | Image processing method, device, equipment and medium | |
| CN117314391B (en) | Operation and maintenance job management method and device, electronic equipment and storage medium | |
| CN115550014B (en) | Application program protection method and related equipment | |
| CN115984635B (en) | Multi-source remote sensing data classification model training method, classification method and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |