CN113342933A - Multi-feature interactive network recruitment text classification method similar to double-tower model - Google Patents

Multi-feature interactive network recruitment text classification method similar to double-tower model Download PDF

Info

Publication number
CN113342933A
CN113342933A CN202110600441.5A CN202110600441A CN113342933A CN 113342933 A CN113342933 A CN 113342933A CN 202110600441 A CN202110600441 A CN 202110600441A CN 113342933 A CN113342933 A CN 113342933A
Authority
CN
China
Prior art keywords
feature
module
text
output end
convolution
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110600441.5A
Other languages
Chinese (zh)
Other versions
CN113342933B (en
Inventor
高尚兵
张骏强
李文婷
相林
陈浩霖
于永涛
周君
朱全银
张正伟
汪长春
蔡创新
郝明阳
胡序洋
李少凡
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangsu Kesheng Xuanyi Technology Co ltd
Original Assignee
Huaiyin Institute of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huaiyin Institute of Technology filed Critical Huaiyin Institute of Technology
Priority to CN202110600441.5A priority Critical patent/CN113342933B/en
Publication of CN113342933A publication Critical patent/CN113342933A/en
Application granted granted Critical
Publication of CN113342933B publication Critical patent/CN113342933B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/3331—Query processing
    • G06F16/334—Query execution
    • G06F16/3344—Query execution using natural language analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Databases & Information Systems (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention relates to a multi-feature interaction network recruitment text classification method of a kind of double-tower model, which comprises the steps of preprocessing a recruitment sample text and processing the preprocessed recruitment sample text into a sparse feature text and a dense feature text with a mapping relation; respectively carrying out sequence vectorization on the two feature texts through a pre-training language model and a one-hot coding mechanism; then, respectively sending the two output sequence vectors into a preset first feature extraction model and a second feature extraction model for secondary feature extraction, simultaneously constructing a feature interaction model for performing multi-feature interaction and output on two paths of feature extraction networks, and finally fusing and splicing feature vectors output by the three, performing attention weighting and performing dimension reduction and classification output; the whole design scheme introduces two recruitment texts with different feature distributions, carries out differentiation processing, and then constructs a multi-feature interaction mechanism between networks, so that the diversity of features brought by the difference of the fully learned data is improved, and the classification precision of the recruitment texts is improved.

Description

Multi-feature interactive network recruitment text classification method similar to double-tower model
Technical Field
The invention relates to a multi-feature interactive network recruitment text classification method of a similar double-tower model, and belongs to the technical field of natural language text processing.
Background
On-line recruitment and resume delivery are becoming the main approaches for enterprise job recruitment and employment of young people. The requirements of a certain industry on talent skills in the current and future fields can be known through the network recruitment text, so that a special talent culture plan meeting the requirements of enterprises can be well made for colleges and universities, the pressure of difficulty in employment of graduates is relieved, meanwhile, the direction of recruiting talents in the future can be provided for the enterprises in the fields to a certain extent, and the development of the enterprises is promoted.
The occupation types which are not available for all walks of life and the characteristics of large data volume and quick updating of the network recruitment text make manual classification extremely inconvenient, and the requirement for realizing automatic classification of the network recruitment text so as to facilitate subsequent analysis is increased day by day. The traditional text classification model depends on the performance of the classifier in the model, and the performance of the classifier depends on the richness of knowledge possessed by an expert who designs the classifier. Therefore, the performance exhibited by the model is often greatly affected by human factors.
Disclosure of Invention
The invention aims to solve the technical problem of providing a multi-feature interaction network recruitment text classification method similar to a double-tower model, designing a personalized secondary feature extraction network and a feature interaction model based on the division of sparse feature texts and dense feature texts corresponding to the recruitment texts, realizing multi-feature fusion, effectively improving the classification precision of the recruitment texts and providing accurate basis for further analysis of employment data.
The invention adopts the following technical scheme for solving the technical problems: the invention designs a multi-feature interaction network recruitment text classification method of a kind of double-tower model, which comprises the following steps of I to VI, obtaining a recruitment text classification probability model, and A to B, realizing the classification of a target recruitment text;
step I, collecting each recruitment sample text, determining that each recruitment sample text corresponds to a real classification category in preset recruitment classification categories respectively, dividing the recruitment sample text into a sparse feature text and a dense feature text according to preset dense attributes and preset sparse attributes aiming at each recruitment sample text, further obtaining a sparse feature text and a dense feature text corresponding to each recruitment sample text respectively, and entering a step II;
step II, respectively aiming at each recruitment sample text, obtaining a word vector sequence corresponding to the sparse feature text and a one-hot vector sequence corresponding to the dense feature text, further obtaining a word vector sequence and a one-hot vector sequence corresponding to each recruitment sample text, and then entering step III;
step III, based on a preset first feature extraction model corresponding to a word vector sequence, a preset second feature extraction model corresponding to a one-hot vector sequence, a feature self-interaction model respectively corresponding to each feature extraction model and a global feature interaction model between the two feature extraction models, presetting a first feature extraction model input end and a preset second feature extraction model input end as input ends, presetting a first feature extraction model output end, presetting a second feature extraction model output end and butting the feature self-interaction model output ends with the input ends of a feature fusion layer, sequentially connecting an attention layer and a softmax layer in series at the output end of the feature fusion layer, constructing a text classification initial probability model, and then entering step VI;
step VI, inputting a word vector sequence and a one-hot vector sequence which respectively correspond to each recruitment sample text, outputting the probability that each recruitment sample text respectively corresponds to each preset recruitment classification, training aiming at a text classification initial probability model by combining the fact that each recruitment sample text respectively corresponds to a real classification in each preset recruitment classification, and obtaining a recruitment text classification probability model;
step A, obtaining a word vector sequence and a one-hot vector sequence corresponding to the target recruitment text according to the steps I to II, and then entering the step B;
and step B, applying a recruitment text classification probability model, processing a word vector sequence and a one-hot vector sequence corresponding to the target recruitment text to obtain the probability that the target text object respectively corresponds to each preset recruitment classification category, and selecting the classification category corresponding to the maximum probability as the classification category corresponding to the target text object to realize the classification of the target recruitment text.
As a preferred technical scheme of the invention: the step I comprises the following steps I1 to I3;
step I1., collecting each recruitment sample text, determining that each text sample object corresponds to a real classification category in preset recruitment classification categories respectively, and entering step I2;
step I2, deleting each preset nonsense word in each recruitment sample text, updating each recruitment sample text, and then entering step I3;
and step I3, aiming at each recruitment sample text, dividing the recruitment sample text into a sparse feature text and a dense feature text according to each preset dense attribute and each preset sparse attribute, further obtaining the sparse feature text and the dense feature text corresponding to each recruitment sample text, and then entering the step II.
As a preferred technical scheme of the invention: in the step II, respectively aiming at each recruitment sample text, obtaining a word vector sequence corresponding to the sparse feature text according to the following steps II-I-I1 to II-I-I2;
step II-I-I1., aiming at the sparse feature text of the recruitment sample text, applying a pre-training language model to obtain word vectors corresponding to each word in the sparse feature text, and then entering step II-I-I2;
and II-I2. forming a word vector sequence corresponding to the sparse feature text by word vectors corresponding to all the words in the sparse feature text.
As a preferred technical scheme of the invention: in the step II, respectively aiming at each recruitment sample text, obtaining a word vector sequence corresponding to the sparse feature text according to the following steps II-I-II1 to II-I-II 3;
II-I-II1, performing word segmentation on the sparse feature text of the recruitment sample text, deleting conjunctions in the sparse feature text according to a preset conjunction library to obtain each sparse feature word in the sparse feature text, and then entering the step II-I-II 2;
II-I-II2, respectively aiming at each sparse feature word in the sparse feature text, applying word2vec algorithm to obtain word vectors corresponding to the sparse feature words, and then entering the step II-I-II 3;
and II-I-II3. forming a word vector sequence corresponding to the sparse feature text by the word vectors corresponding to the sparse feature participles in the sparse feature text.
As a preferred technical scheme of the invention: in the step II, respectively aiming at each recruitment sample text, obtaining a one-hot vector sequence corresponding to the dense feature text according to the following steps II-II-1 to II-II-3;
II-II-1, performing word segmentation on the dense feature text of the recruitment sample text, deleting corresponding characters in the dense feature text according to a preset word list to obtain each dense feature word in the dense feature text, and then entering the step II-II-2;
II-II-2, selecting each non-repeated dense feature participle in the dense feature text, sequencing each non-repeated dense feature participle according to the position of each non-repeated dense feature participle in the dense feature text for the first time, and then entering step II-II-3;
and II-II-3, obtaining vectors corresponding to the non-repeated dense feature participles respectively, and combining the sequence of the non-repeated dense feature participles to form a one-hot vector sequence corresponding to the dense feature text.
As a preferred technical scheme of the invention: in the step III, based on a preset first feature extraction model corresponding to the word vector sequence, a preset second feature extraction model corresponding to the one-hot vector sequence, a feature self-interaction model corresponding to each of the two feature extraction models, and a global feature interaction model between the two feature extraction models, an input end of the preset first feature extraction model and an input end of the preset second feature extraction model are used as input ends, an output end of the preset first feature extraction model, an output end of the preset second feature extraction model, and a fusion output end of each feature self-interaction model output and a fusion output end of the global feature interaction model output are further connected with an input end of the feature fusion layer, an output end of the feature fusion layer is sequentially connected with the attention layer and the softmax layer in series, and a text classification initial probability model is constructed.
As a preferred technical scheme of the invention: the preset first feature extraction model is characterized in that the input end of the first feature extraction model is formed by three convolution kernels with the heights of 6, 7 and 8 and the widths consistent with the dimension of an input word vector sequence from the input end to the output end of the preset first feature extraction model, the input end of a first convolution module is formed by the input ends of the first convolution modules, the output ends of the three first convolution modules are in butt joint with the input end of a first fusion module, the output end of the first fusion module is in butt joint with the input end of an LSTM network and the input end of a second convolution module with the convolution kernel size of 1 x 1; the output end of the LSTM network is in butt joint with the input end of a third convolution module with the convolution kernel size of 1 multiplied by 1, and the output end of the second convolution module is in butt joint with the input end of the first pooling module; the output end of the second fusion module is respectively butted with the input end of a fifth convolution module with convolution kernel size of 1 multiplied by 1; the output end of the fifth convolution module and the output ends of the two series-connected fourth convolution modules are respectively butted with the input end of the fourth fusion module, the output end of the fourth fusion module is butted with the input end of the global average pooling module, and the output end of the global average pooling module forms the output end of the first feature extraction model;
in the structure of the first feature self-interaction model corresponding to the first feature extraction model, the input ends of six convolution modules with convolution kernels of 1 × 1 form six input ends of the first feature self-interaction model, and the six input ends are respectively connected with six connection features of pairwise arrangement and combination among the output end features of the first fusion module, the output end features of the second fusion module, the output end features of the third fusion module and the output end features of the fourth fusion module; the six sixth convolution modules form a group of two sixth convolution modules to form three sixth convolution module groups, each sixth convolution module group respectively corresponds to each fifth fusion module and each first K-value maximum pooling module one by one, the output ends of the two sixth convolution modules in each sixth convolution module group are in butt joint with the input end of the corresponding fifth fusion module, and the output end of the fifth fusion module is in butt joint with the input end of the corresponding first K-value maximum pooling module; the output end of each first K-value maximum pooling module is in butt joint with the input end of the sixth fusion module, the output end of the sixth fusion module is in butt joint with the input end of the first sub-attention module, and the output end of the first sub-attention module forms the output end of the first characteristic self-interaction model.
As a preferred technical scheme of the invention: the preset second feature extraction model is in a direction from the input end to the output end of the preset second feature extraction model, the input end of a Region embedding module in the DPCNN model forms the input end of the second feature extraction model, the output end of the Region embedding module is respectively butted with the input end of a seventh fusion module, the input end of an eighth fusion module and the input end of two series-connected seventh convolution modules with convolution kernels of 5 multiplied by 5, the output ends of the two series-connected seventh convolution modules are simultaneously butted with the input end of the seventh fusion module, and the output end of the seventh fusion module is respectively butted with the input end of a circular convolution pooling module and the input end of the eighth fusion module; the output end of the circulation convolution pooling module is in butt joint with the input end of a ninth convolution module, the output end of the ninth convolution module is in butt joint with the input end of the ninth convolution module with the convolution kernel size of 1 multiplied by 1, the output end of the eighth fusion module is in butt joint with the input end of an eighth convolution module with the convolution kernel size of 1 multiplied by 1, the output ends of the eighth convolution module and the ninth convolution module are in butt joint with the input end of a tenth fusion module respectively, and the output end of the tenth fusion module is in butt joint with the input end of a second K-value maximum pooling module; the output end of the circular convolution pooling module and the output end of the second K-value maximum pooling module are respectively butted with the input end of an eleventh fusion module, the output end of the eleventh fusion module is butted with the input end of the maximum pooling module, and the output end of the maximum pooling module forms the output end of the second feature extraction model;
in the structure of the second feature self-interaction model corresponding to the second feature extraction model, the input ends of a tenth convolution module with convolution kernels of 1 × 1 form three input ends of the second feature self-interaction model, and the three input ends are respectively connected with three connection features of pairwise arrangement and combination among the output end feature of an eighth fusion module, the output end feature of the tenth fusion module and the output end feature of an eleventh fusion module; the output ends of the three tenth convolution modules are respectively in one-to-one correspondence with the input end of the third K-value maximum pooling module, the input end of the fourth K-value maximum pooling module and the input end of the fifth K-value maximum pooling module, the output end of the third K-value maximum pooling module, the output end of the fourth K-value maximum pooling module and the output end of the fifth K-value maximum pooling module are respectively in butt joint with the input end of the twelfth fusion module, the output end of the twelfth fusion module is in butt joint with the input end of the second sub-attention module, and the output end of the second sub-attention module forms the output end of the second characteristic self-interaction model.
As a preferred technical scheme of the invention: the input end of the second pooling module with the step length of 2 forms the input end of the circulating convolution pooling module from the input end to the output end of the circulating convolution pooling module, the output end of the second pooling module is in butt joint with the input ends of eleventh convolution modules with convolution kernels of 5 multiplied by 5 in series, the output ends of the eleventh convolution modules with two series are in butt joint with the input end of a thirteenth fusion module, and the output ends of the thirteenth fusion module form the output end of the circulating convolution pooling module.
As a preferred technical scheme of the invention: in the structure of the global feature interaction model, the input ends of six fourteenth fusion modules form six input ends of the global feature interaction model, of the six input ends, the first input end is respectively butted with the output end of a first fusion module in the first feature extraction model and the output end of a Region embedding module in the second feature extraction model, the second input end is respectively butted with the output end of a second convolution module in the first feature extraction model and the output end of a second seventh convolution module in two seventh convolution modules which are connected in series in the second feature extraction model, the third input end is respectively butted with the output end of a first pooling module in the first feature extraction model and the output end of a seventh fusion module in the second feature extraction model, and the fourth input end is respectively butted with the output end of a third fusion module in the first feature extraction model and the output end of a second pooling module in the last circulation of the circular convolution pooling modules in the second feature extraction model, the fifth input end is respectively butted with the output end of a first fourth convolution module in two series-connected fourth convolution modules in the first feature extraction model and the output end of a first eleventh convolution module in two series-connected eleventh convolution modules when the last circulation is performed in the circular convolution pooling module in the second feature extraction model, and the sixth input end is respectively butted with the output end of a second fourth convolution module in two series-connected fourth convolution modules in the first feature extraction model and the output end of a second eleventh convolution module in two series-connected eleventh convolution modules when the last circulation is performed in the circular convolution pooling module in the second feature extraction model; and the fourteenth fusion modules respectively correspond to the average pooling modules one by one, the output ends of the fourteenth fusion modules are respectively in butt joint with the input ends of the average pooling modules, the output ends of the average pooling modules are respectively in butt joint with the input end of the fifteenth fusion module, and the output ends of the fifteenth fusion modules form the output end of the global feature interaction model.
Compared with the prior art, the multi-feature interactive network recruitment text classification method similar to the double-tower model has the following technical effects:
(1) the invention designs a multi-feature interaction network recruitment text classification method of a double-tower-like model, which adopts a processing combination design of a similar double-tower model, designs parallel branches to respectively extract two kinds of feature vectors based on two word vector sequences with different semantic dense feature degrees generated after processing a recruitment text object, performs feature interaction on the two kinds of features in the process, then sequentially performs feature fusion, attention weighting, linear dimension reduction and classification to obtain the prediction probability of the text object corresponding to each preset recruitment classification category, finally considers the label category corresponding to the maximum prediction probability as the prediction classification corresponding to the target text object to realize the prediction classification of the target text object, introduces two coding mechanisms with different complexity degrees to differentially process recruitment texts with different feature distributions in the whole design scheme, then a multi-feature interaction mechanism is constructed among networks to better capture feature information, so that the recruitment text classification precision is improved;
(2) in the multi-feature interactive network recruitment text classification method of the double-tower-like model, the ALBERT model is applied to sparse feature texts, so that the word vectors are obtained, compared with the BERT model, the method has the advantages of small quantity of parameters and high speed, and the ALBER model inherits the characteristic of the BERT model in extracting context semantic information;
(3) in the multi-feature interactive network recruitment text classification method of the double-tower-like model, one-hot coding is applied to the dense feature sub-texts, effective feature distribution is concentrated in view of the fact that a one-hot coding mechanism is friendly to short text coding, the dense feature sub-texts are often short in text length (mostly within twenty words), and the method can also play a role in feature expansion compared with the existing sequence vector acquisition mode by applying the one-hot coding;
(4) in the multi-feature interactive network recruitment text classification method of the class double-tower model, a multi-level residual DPCNN model, namely an MSC-DPCNN model, is provided on the basis of the original DPCNN when dense features are processed. The original DPCNN model only carries out simple residual connection between the levels, while in the MSC-DPCNN, two composite residual networks are provided, and semantic feature interaction between feature channels is more sufficient through 1 multiplied by 1 convolution in the composite residual networks; between the composite residual error networks, the connection between the residual error networks is carried out, and the residual error information of the whole network can be utilized by the network when the network is finally output through the maximum pooling operation.
(5) In the multi-feature interactive network recruitment text classification method similar to the double-tower model, when sparse features are processed, a new composite multi-convolution downstream network first feature self-interaction model is designed to perform secondary feature extraction on word vectors output by an ALBERT pre-training language model, a residual error network layer is added between convolution modules, feature extraction and fusion are performed inside the residual error network through LSTM, and feature interaction is performed between the residual error networks. The newly constructed convolutional neural network can prevent gradient explosion and gradient disappearance as much as possible while keeping the depth enough, and the sensitivity of the network to local features is enhanced.
(6) In the multi-feature interaction network recruitment text classification method of the double-tower-like model, a new feature interaction model is constructed and consists of three submodels, the output of the three submodels is jointly formed by the outputs of the three submodels, and the submodels comprise a first feature self-interaction model, a second feature self-interaction model and a global feature interaction model. The first characteristic self-interaction model is specific to the first characteristic self-interaction model, rich semantic characteristic information of the first characteristic self-interaction model is utilized and fused through extracting a characteristic fusion layer of a trunk network and a residual error network, meanwhile, in order to fully keep K pooling instead of maximum pooling of useful characteristics used after convolution, finally, a two-feature interaction mode is carried out on each extracted characteristic by means of an attention mechanism, and finally, weighted output is carried out after characteristic fusion. The second feature self-interaction model is specific to the MSC-DPCNN model, features of each key level of the trunk network and the composite residual error network are extracted, feature interaction is carried out between every two feature self-interaction models, multiple features are reserved through K pooling, and finally the features are output in an attention weighting mode. The global feature interactive model simultaneously extracts the features of the first feature self-interactive model and the MSC-DPCNN, the features of corresponding levels are extracted, then in order to enable dense features and sparse features to be better interactively fused, the features of the corresponding levels are overlapped and averaged, and finally the output is carried out in the form of attention weighting. By respectively extracting and fusing the features of the two feature extraction networks and performing corresponding level fusion interaction on the dense features and the sparse features of the two networks, semantic information obtained by the final model feature fusion layer is richer;
(7) in the multi-feature interaction network recruitment text classification method similar to the double-tower model, dense features are extracted from dense feature short texts and feature distribution is narrow, and sparse features are extracted from sparse feature long texts and feature distribution is comprehensive. Therefore, an attention mechanism is introduced, and the characteristics of the two texts extracted by the attention mechanism are fully fused, so that after attention weighting, the dense characteristics and the sparse characteristics complement each other, the characteristics are more effectively expressed, and the accuracy of the recruitment text classification is integrally improved;
(8) in the multi-feature interactive network recruitment text classification method similar to the double-tower model, because the network is deep, residual error connection is introduced, but the simple residual error connection logic is too simple and the information between the residual error connections is not fully utilized, so that respective composite residual error networks are respectively constructed aiming at the data characteristics and the model structure characteristics processed by the branch model, the feature interaction between the levels is improved, and the problem of gradient dispersion is alleviated.
(9) The invention designs a multi-feature interactive network recruitment text classification method of a kind of double-tower model, which is based on the idea of integrated learning. By constructing a model, the model integrates two sub-models, and the two sub-models respectively extract the features of the text data with different feature distributions but with mapping relations, so that the improvement of the features brought by the difference of the data on diversity is fully utilized, the diversity improvement effect brought by the feature interaction model is further enhanced by constructing the feature interaction model between the two sub-models, and finally, the improvement of the fusion interaction and the text classification on the accuracy is achieved.
Drawings
FIG. 1 is a flow chart diagram of a multi-feature interactive network recruitment text classification method of a double-tower-like model designed by the invention;
FIG. 2 is a schematic diagram of a first feature self-interaction model application in the design of the present invention;
FIG. 3 is a diagram of a second feature self-interaction model application in the design of the present invention;
FIG. 4 is a schematic diagram of a global feature interaction model application in the design of the present invention;
FIG. 5 is a schematic diagram of the application of the LSTM model in the design of the present invention.
FIG. 6 is a partial model of the ALBERT pre-training language model Encoder in the design of the present invention.
FIG. 7 is a block diagram of an overall model in the design of the present invention.
Detailed Description
The following description will explain embodiments of the present invention in further detail with reference to the accompanying drawings.
The invention designs a multi-feature interaction network recruitment text classification method similar to a double-tower model, and in practical application, as shown in fig. 1, the following steps I to VI are specifically executed to obtain a recruitment text classification probability model.
Step I, collecting each recruitment sample text, determining that each recruitment sample text corresponds to a real classification category in preset recruitment classification categories respectively, dividing the recruitment sample text into a sparse feature text and a dense feature text according to preset dense attributes and preset sparse attributes aiming at each recruitment sample text, further obtaining the sparse feature text and the dense feature text corresponding to each recruitment sample text respectively, and entering the step II.
The recruitment sample text is divided into a sparse feature text and a dense feature text, specifically, the dense feature text is each piece of information directly or most related to the recruitment information, and the sparse feature text is each piece of information less related to the recruitment information.
In practical applications, the step I specifically executes the following steps I1 to I3.
Step I1. is to collect the recruitment sample texts, determine that the text sample objects respectively correspond to the real classification categories in the preset recruitment classification categories, and then enter step I2.
Step I2, deleting each preset nonsense word in each recruitment sample text, updating each recruitment sample text, and then entering step I3; wherein each preset meaningless type comprises a messy code type, an empty type, a tab type, a carriage return type and specific repeated words, including but not limited to 'position description', 'position responsibility', 'function requirement', etc.
And step I3, aiming at each recruitment sample text, dividing the recruitment sample text into a sparse feature text and a dense feature text according to each preset dense attribute and each preset sparse attribute, further obtaining the sparse feature text and the dense feature text corresponding to each recruitment sample text, and then entering the step II.
And II, respectively aiming at each recruitment sample text, obtaining a word vector sequence corresponding to the sparse feature text and a one-hot vector sequence corresponding to the dense feature text, further obtaining the word vector sequence and the one-hot vector sequence corresponding to each recruitment sample text, and then entering the step III.
In the actual implementation of the step II, two sets of execution schemes may be specifically designed for obtaining the word vector sequence corresponding to the sparse feature text, where the first scheme is to obtain the word vector sequence corresponding to the sparse feature text according to the following steps II-I1 to II-I2 for each recruitment sample text.
And II-I-I1., aiming at the sparse feature text of the recruitment sample text, applying a pre-training language model to obtain word vectors corresponding to each word in the sparse feature text, and then entering the step II-I-I2.
And II-I2. forming a word vector sequence corresponding to the sparse feature text by word vectors corresponding to all the words in the sparse feature text.
In practical application, the pre-training language model specifically adopts an ALBERT model, that is, a feature text sample is used as input, word vectors corresponding to each word in the feature text sample are used as output, and the pre-training language model is obtained for the ALBERT model and is used for processing a sparse feature text of a recruitment sample text to obtain the word vectors corresponding to each word in the sparse feature text.
ALBERT (A Lite BERT For Self-contained Learning Of Language representation, a simplified version Of BERT For Language characterization For Self-Supervised Learning) Language model: in order to solve the problem of difficulty in large training of the BERT model, ALBERT is proposed. The ALBERT inherits the bidirectionality of the BERT model, and inherits the Encoder part of the Transformer model on the model structure, as shown in FIG. 6. Each word can still utilize the context information of the word at the same time. And the number of parameters is much smaller than the conventional BERT architecture. ALBERT overcomes the major obstacles faced by expanding pre-trained language models through two parameter reduction techniques. The first technique is to factorize the embedding parameterization, decomposing the large vocabulary embedding matrix into two small matrices, thereby separating the size of the hidden layer from the size of the vocabulary embedding. The second technique is cross-layer parameter sharing. This technique can avoid the increase in the number of parameters with the increase in the depth of the network. The two technologies obviously reduce the parameter quantity of BERT, and simultaneously do not cause obvious influence on the performance of the BERT, and in addition, an order prediction task SOP is provided to replace the traditional NSP pre-training language task. The ALBERT language model finally realizes the further enhancement of semantic information of the text vector on the basis of the BERT, and accelerates the training speed of the model. In the Encoder section used by the ALBERT model, the most important is the Self-Attention layer of Self-Attention, and in practical application, in order to obtain more excellent word vector representation, the model uses the Multi-Head Attention which is a variant form of Self-Attention. The multi-head self-attention uses various linear transformations to project Q, K and V, and the main calculation formula is as follows:
Figure BDA0003092754580000091
Figure BDA0003092754580000092
MultiHead(Q，K，V)＝
Concat(head1，head2，…，headk)Wo
Figure BDA0003092754580000093
wherein, in the formula (1), Q (query), K (Key) and V (value) are obtained by multiplying a word embedding vector and a matrix, and dkRepresenting the dimensions of the Query and Key vectors for each word, Softmax () being a normalized activation function; (z) in formula (2)1,z2,…,zn) Representing an N-dimensional vector; concat () in equation (3) is a splicing function, Wo represents an additional weight matrix, so that the dimension of the matrix after splicing is compressed to the sequence length,
Figure BDA0003092754580000101
then are the weight matrices representing Q, K, V, respectively.
Regarding the second scheme for obtaining the word vector sequence corresponding to the sparse feature text, specifically, the word vector sequence corresponding to the sparse feature text is obtained according to the following steps II-I-II1 to II-I-II3 for each recruitment sample text.
And II-I-II1, performing word segmentation on the sparse feature text of the recruitment sample text, deleting conjunctions in the sparse feature text according to a preset conjunction library to obtain each sparse feature word in the sparse feature text, and then entering the step II-I-II2.
And II-I-II2, respectively aiming at each sparse feature word in the sparse feature text, applying a word2vec algorithm to obtain a word vector corresponding to the sparse feature word, and then entering the step II-I-II3.
And II-I-II3. forming a word vector sequence corresponding to the sparse feature text by the word vectors corresponding to the sparse feature participles in the sparse feature text.
Thus, in practical application, the two design schemes can be adopted to realize the acquisition of the word vector sequence and the one-hot vector sequence corresponding to each recruitment sample text in the step II.
In addition, regarding the acquisition of the one-hot vector sequence corresponding to the dense feature text in the step II, the one-hot vector sequence corresponding to the dense feature text is acquired according to the following steps II-II-1 to II-II-3 by specifically designing each recruitment sample text.
And II-II-1, performing word segmentation on the dense feature text of the recruitment sample text, deleting corresponding characters in the dense feature text according to a preset word list to obtain each dense feature word in the dense feature text, and then entering the step II-II-2.
And II-II-2, selecting each non-repeated dense feature word in the dense feature text, sequencing each non-repeated dense feature word according to the position of each non-repeated dense feature word appearing in the dense feature text for the first time, and then entering the step II-II-3.
And II-II-3, obtaining vectors corresponding to the non-repeated dense feature participles respectively, and combining the sequence of the non-repeated dense feature participles to form a one-hot vector sequence corresponding to the dense feature text.
Step iii, as shown in fig. 7, based on the preset first feature extraction model corresponding to the word vector sequence, the preset second feature extraction model corresponding to the one-hot vector sequence, the feature self-interaction model corresponding to each of the two feature extraction models, and the global feature interaction model between the two feature extraction models, the preset first feature extraction model input end, the preset second feature extraction model input end are used as input ends, the preset first feature extraction model output end, the preset second feature extraction model output end, and the fusion output end of each feature self-interaction model output and the global feature interaction model output end are further connected to the input end of the feature fusion layer, the output end of the feature fusion layer is sequentially connected in series with the attention layer and the softmax layer, a text classification initial probability model is constructed, and then the process proceeds to step VI.
Specifically, the preset first feature extraction model has an input end of a first feature extraction model formed by three convolution kernels with respective heights of 6, 7 and 8 and widths consistent with the dimensionality of the input word vector sequence from an input end to an output end thereof, output ends of the three first convolution modules are in butt joint with an input end of a first fusion module, and an output end of the first fusion module is in butt joint with an input end of an LSTM network and an input end of a second convolution module with convolution kernel size of 1 × 1; the output end of the LSTM network is in butt joint with the input end of a third convolution module with the convolution kernel size of 1 multiplied by 1, and the output end of the second convolution module is in butt joint with the input end of the first pooling module; in practical application, the structure of the LSTM network is as shown in fig. 5, where the output end of the third convolution module is respectively connected to the input end of the second fusion module and the input end of the third fusion module, the output end of the first pooling module is connected to the input end of the third fusion module, the output end of the third fusion module is respectively connected to the input end of the second fusion module and the input ends of the two fourth convolution modules with convolution kernels of 5 × 5 in series, and the output end of the second fusion module is connected to the input end of the fifth convolution module with convolution kernel of 1 × 1; the output end of the fifth convolution module and the output ends of the two series-connected fourth convolution modules are respectively butted with the input end of the fourth fusion module, the output end of the fourth fusion module is butted with the input end of the global average pooling module, and the output end of the global average pooling module forms the output end of the first feature extraction model.
The first feature extraction model is used for obtaining three feature vectors through three high convolution kernels in the process of processing a word vector sequence, performing feature fusion on the three feature vectors, and then performing the following two operations on the fused feature vectors: firstly, inputting the fused feature vector into an LSTM network in a first residual error network layer; secondly, inputting the fused feature vector into a second convolution module, then inputting the output of the second convolution module into a first pooling module, performing feature fusion on the output of the first pooling module and the output of a third convolution module in a first residual network layer in a third fusion module, and inputting the fused feature vector into two series-connected fourth convolution modules and a fifth convolution module; finally, the output of the fifth convolution module and the output of the two series-connected fourth convolution modules are subjected to feature fusion through the fourth fusion module, input into the global average pooling layer, and output into a feature vector F finally output by the model1Each convolution in the above operation is followed by activation using the nonlinear activation function prilu.
As shown in fig. 2, in the structure of the first feature self-interaction model corresponding to the first feature extraction model, six input ends of a sixth convolution module with a convolution kernel size of 1 × 1 form six input ends of the first feature self-interaction model, and the six input ends are respectively connected with six connection features of pairwise arrangement and combination among output end features of the first fusion module, output end features of the second fusion module, output end features of the third fusion module, and output end features of the fourth fusion module; the six sixth convolution modules form a group of two sixth convolution modules to form three sixth convolution module groups, each sixth convolution module group respectively corresponds to each fifth fusion module and each first K-value maximum pooling module one by one, the output ends of the two sixth convolution modules in each sixth convolution module group are in butt joint with the input end of the corresponding fifth fusion module, and the output end of the fifth fusion module is in butt joint with the input end of the corresponding first K-value maximum pooling module; the output end of each first K-value maximum pooling module is in butt joint with the input end of the sixth fusion module, the output end of the sixth fusion module is in butt joint with the input end of the first sub-attention module, and the output end of the first sub-attention module forms the output end of the first characteristic self-interaction model.
The preset second feature extraction model is in a direction from the input end to the output end of the preset second feature extraction model, the input end of a Region embedding module in the DPCNN model forms the input end of the second feature extraction model, the output end of the Region embedding module is respectively butted with the input end of a seventh fusion module, the input end of an eighth fusion module and the input end of two series-connected seventh convolution modules with convolution kernels of 5 multiplied by 5, the output ends of the two series-connected seventh convolution modules are simultaneously butted with the input end of the seventh fusion module, and the output end of the seventh fusion module is respectively butted with the input end of a circular convolution pooling module and the input end of the eighth fusion module; the output end of the circulation convolution pooling module is in butt joint with the input end of a ninth convolution module, the output end of the ninth convolution module is in butt joint with the input end of the ninth convolution module with the convolution kernel size of 1 multiplied by 1, the output end of the eighth fusion module is in butt joint with the input end of an eighth convolution module with the convolution kernel size of 1 multiplied by 1, the output ends of the eighth convolution module and the ninth convolution module are in butt joint with the input end of a tenth fusion module respectively, and the output end of the tenth fusion module is in butt joint with the input end of a second K-value maximum pooling module; the output end of the circular convolution pooling module and the output end of the second K-value maximum pooling module are respectively butted with the input end of the eleventh fusion module, the output end of the eleventh fusion module is butted with the input end of the maximum pooling module, and the output end of the maximum pooling module forms the output end of the second feature extraction model.
The input end of the second pooling module with the step length of 2 forms the input end of the circulating convolution pooling module from the input end to the output end of the circulating convolution pooling module, the output end of the second pooling module is in butt joint with the input ends of eleventh convolution modules with convolution kernels of 5 multiplied by 5 in series, the output ends of the eleventh convolution modules with two series are in butt joint with the input end of a thirteenth fusion module, and the output end of the thirteenth fusion module forms the output end of the circulating convolution pooling module.
The second feature extraction model is used for carrying out secondary feature extraction operation on the word vector formed by the one-hot coding by the MSC-DPCNN model in the process of processing the word vector sequence. Firstly, after the processing of the Region embedding layer, the Region embedding layer convolutes a n-gram Region, and an obtained feature graph is used as the output of the layer. Two convolutions are then performed and the Region embedding layer is residual connected to a second convolution module and then input to the circular convolution pooling layer. The cyclic convolution module reduces the length of the cyclic sequence by half every time when the cyclic convolution module performs one-time operation until the sequence length is equal to 1, the cyclic convolution module stops the cyclic convolution until the output characteristic vector and the output of the second composite residual error network layer are fused and input into the maximum pooling layer, and finally the one-dimensional characteristic vector F is output2After each convolution in the above operations, a nonlinear activation function prilu is used for activation and padding operation is used for boundary padding.
As shown in fig. 3, in the structure of the second feature self-interaction model corresponding to the second feature extraction model, the input ends of the tenth convolution module with convolution kernel size of 1 × 1 form three input ends of the second feature self-interaction model, and the three input ends are respectively connected to three connection features of pairwise arrangement and combination among the output end feature of the eighth fusion module, the output end feature of the tenth fusion module, and the output end feature of the eleventh fusion module; the output ends of the three tenth convolution modules are in one-to-one correspondence respectively and are in butt joint with the input end of a third K-value maximum pooling module, the input end of a fourth K-value maximum pooling module and the input end of a fifth K-value maximum pooling module, the output end of the third K-value maximum pooling module, the output end of the fourth K-value maximum pooling module and the output end of the fifth K-value maximum pooling module are in butt joint with the input end of a twelfth fusion module respectively, the output end of the twelfth fusion module is in butt joint with the input end of a second sub-attention module, and the output end of the second sub-attention module forms the output end of a second characteristic self-interaction model.
And as shown in fig. 4, in the structure of the global feature interaction model application, the input ends of six fourteenth fusion modules constitute six input ends of the global feature interaction model, of the six input ends, the first input end is respectively connected with the output end of a first fusion module in the first feature extraction model and the output end of a Region embedding module in the second feature extraction model, the second input end is respectively connected with the output end of a second convolution module in the first feature extraction model and the output end of a second seventh convolution module in two series-connected seventh convolution modules in the second feature extraction model, the third input end is respectively connected with the output end of a first pooling module in the first feature extraction model and the output end of a seventh fusion module in the second feature extraction model, the fourth input end is respectively connected with the output end of a third fusion module in the first feature extraction model and the output end of a second pooling module in the last cycle of the circular convolution pooling modules in the second feature extraction model, the fifth input end is respectively butted with the output end of a first fourth convolution module in two series-connected fourth convolution modules in the first feature extraction model and the output end of a first eleventh convolution module in two series-connected eleventh convolution modules when the last circulation is performed in the circular convolution pooling module in the second feature extraction model, and the sixth input end is respectively butted with the output end of a second fourth convolution module in two series-connected fourth convolution modules in the first feature extraction model and the output end of a second eleventh convolution module in two series-connected eleventh convolution modules when the last circulation is performed in the circular convolution pooling module in the second feature extraction model; and the fourteenth fusion modules respectively correspond to the average pooling modules one by one, the output ends of the fourteenth fusion modules are respectively in butt joint with the input ends of the average pooling modules, the output ends of the average pooling modules are respectively in butt joint with the input end of the fifteenth fusion module, and the output ends of the fifteenth fusion modules form the output end of the global feature interaction model.
For the text classification initial probability model constructed in step III, the word vector sequence and the one-hot vector from the pre-training language model are received by the first feature self-interaction model and the second feature self-interaction model, respectively, and are processed to obtain the feature vector F output by the first feature self-interaction model1And obtaining a feature vector F of the second feature output from the interaction model2Finally, the feature vector F output by the feature interaction model3、F4、F5. And splicing the five eigenvectors for attention weighting, and sending the eigenvectors into a softmax classifier for classification after dimension reduction.
And VI, inputting a word vector sequence and a one-hot vector sequence which respectively correspond to each recruitment sample text, outputting the probability that each recruitment sample text respectively corresponds to each preset recruitment classification, and training a text classification initial probability model by combining the fact that each recruitment sample text respectively corresponds to a real classification in each preset recruitment classification to obtain a recruitment text classification probability model.
And (4) after the recruitment text classification probability model is obtained based on the execution of the steps I to VI, executing the following steps A to B to realize the classification of the target recruitment text.
Step A, obtaining a word vector sequence and a one-hot vector sequence corresponding to the target recruitment text according to the steps I to II, and then entering the step B;
and step B, applying a recruitment text classification probability model, processing a word vector sequence and a one-hot vector sequence corresponding to the target recruitment text to obtain the probability that the target text object respectively corresponds to each preset recruitment classification category, and selecting the classification category corresponding to the maximum probability as the classification category corresponding to the target text object to realize the classification of the target recruitment text.
Aiming at the designed multi-feature interactive network recruitment text classification method of the double-tower-like model, the invention further designs computer equipment which comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, and is characterized in that the multi-feature interactive network recruitment text classification method of the double-tower-like model is realized when the processor executes the computer program.
In order to better explain the effectiveness of the method, the multi-feature interactive network recruitment text classification method of the double-tower-like model designed by the invention is applied to practice, as shown in fig. 1, the data of the network are analyzed, such as Crawl Spider is applied to crawl the network recruitment data to obtain 500004 pieces of recruitment information text data, 500004 pieces of recruitment information text data are further cleaned, the sentence is normalized, and if abnormal conditions such as messy codes and return line and exchange symbols or tabulation symbols occur in the sentence, corresponding modification or deletion is performed so as to ensure the correctness of the text content and the specification of some high-frequency meaningless specific recruitment field words occurring in the sentence, including but not limited to 'post description', 'post responsibility', 'functional requirement' and the like. And then processing the recruitment text into two text data sets with a mapping relation, namely a sparse feature text data set and a dense feature text data set, vectorizing the sparse feature data set by using a pre-training language model, and vectorizing the dense feature text by using one-hot coding. And then, respectively carrying out depth feature extraction on the two vector sequences through two feature extraction networks, processing sequence vectors formed by sparse feature texts with longer text lengths by using a first feature self-interaction model, processing one-hot sequence vectors formed by dense feature texts with shorter texts by using a second feature self-interaction model, and obtaining three feature vectors through the feature interaction model. Then, performing feature fusion on the obtained multiple feature vectors, then performing attention weighting, and fully utilizing dense features in the dense features to help a model to learn and inhibit distribution imbalance possibly existing in the features extracted from the sparse feature text, so as to enrich advantages brought by feature diversity; and finally, outputting the final prediction label through the full connection layer and softmax.
In the design scheme, the characteristic texts in the text data are classified, the pre-training language model with rich extracted characteristics is selected for sparse characteristic texts with long text lengths to carry out vectorization processing, one-hot codes are selected for dense characteristic texts with short texts to carry out processing, and the one-hot codes can play a role in feature expansion. The two sub-texts are coded in different forms, then deep semantic feature extraction is carried out, finally, feature fusion and attention weighting are carried out, on the premise that the two sub-texts have a mapping relation, learning of feature information of the sub-texts is carried out by means of features of the other sub-texts, and therefore classification accuracy of the recruitment texts is improved on the whole.
The embodiments of the present invention will be described in detail with reference to the drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the gist of the present invention.

Claims (10)

1. A multi-feature interactive network recruitment text classification method similar to a double-tower model is characterized by comprising the following steps of: executing the following steps I to VI to obtain a recruitment text classification probability model, and executing the following steps A to B to realize the classification of the target recruitment text;
step I, collecting each recruitment sample text, determining that each recruitment sample text corresponds to a real classification category in preset recruitment classification categories respectively, dividing the recruitment sample text into a sparse feature text and a dense feature text according to preset dense attributes and preset sparse attributes aiming at each recruitment sample text, further obtaining a sparse feature text and a dense feature text corresponding to each recruitment sample text respectively, and entering a step II;
step II, respectively aiming at each recruitment sample text, obtaining a word vector sequence corresponding to the sparse feature text and a one-hot vector sequence corresponding to the dense feature text, further obtaining a word vector sequence and a one-hot vector sequence corresponding to each recruitment sample text, and then entering step III;
step III, based on a preset first feature extraction model corresponding to a word vector sequence, a preset second feature extraction model corresponding to a one-hot vector sequence, a feature self-interaction model respectively corresponding to each feature extraction model and a global feature interaction model between the two feature extraction models, presetting a first feature extraction model input end and a preset second feature extraction model input end as input ends, presetting a first feature extraction model output end, presetting a second feature extraction model output end and butting the feature self-interaction model output ends with the input ends of a feature fusion layer, sequentially connecting an attention layer and a softmax layer in series at the output end of the feature fusion layer, constructing a text classification initial probability model, and then entering step VI;
step VI, inputting a word vector sequence and a one-hot vector sequence which respectively correspond to each recruitment sample text, outputting the probability that each recruitment sample text respectively corresponds to each preset recruitment classification, training aiming at a text classification initial probability model by combining the fact that each recruitment sample text respectively corresponds to a real classification in each preset recruitment classification, and obtaining a recruitment text classification probability model;
step A, obtaining a word vector sequence and a one-hot vector sequence corresponding to the target recruitment text according to the steps I to II, and then entering the step B;
and step B, applying a recruitment text classification probability model, processing a word vector sequence and a one-hot vector sequence corresponding to the target recruitment text to obtain the probability that the target text object respectively corresponds to each preset recruitment classification category, and selecting the classification category corresponding to the maximum probability as the classification category corresponding to the target text object to realize the classification of the target recruitment text.
2. The multi-feature interactive network recruitment text classification method of the twin-tower-like model according to claim 1, wherein: the step I comprises the following steps I1 to I3;
step I1, collecting each recruitment sample text, determining that each text sample object corresponds to a real classification category in preset recruitment classification categories respectively, and then entering step I2;
step I2, deleting each preset nonsense word in each recruitment sample text, updating each recruitment sample text, and then entering the step I3;
step I3, aiming at each recruitment sample text, dividing the recruitment sample text into a sparse feature text and a dense feature text according to preset dense attributes and preset sparse attributes, further obtaining the sparse feature text and the dense feature text corresponding to each recruitment sample text, and then entering the step II.
3. The multi-feature interactive network recruitment text classification method of the twin-tower-like model according to claim 1, wherein: in the step II, respectively aiming at each recruitment sample text, obtaining a word vector sequence corresponding to the sparse feature text according to the following steps II-I-I1 to II-I-I2;
step II-I-I1., aiming at the sparse feature text of the recruitment sample text, applying a pre-training language model to obtain word vectors corresponding to each word in the sparse feature text, and then entering step II-I-I2;
and II-I2. forming a word vector sequence corresponding to the sparse feature text by word vectors corresponding to all the words in the sparse feature text.
4. The multi-feature interactive network recruitment text classification method of the twin-tower-like model according to claim 1, wherein: in the step II, respectively aiming at each recruitment sample text, obtaining a word vector sequence corresponding to the sparse feature text according to the following steps II-I-II1 to II-I-II 3;
step II-I-II1, performing word segmentation processing on the sparse feature text of the recruitment sample text, deleting conjunctions in the sparse feature text according to a preset conjunction library to obtain each sparse feature word in the sparse feature text, and then entering step II-I-II 2;
II-I-II2, respectively aiming at each sparse feature participle in the sparse feature text, applying word2vec algorithm to obtain a word vector corresponding to the sparse feature participle, and then entering the step II-I-II 3;
and II-I-II3, forming a word vector sequence corresponding to the sparse feature text by the word vectors corresponding to the sparse feature participles in the sparse feature text.
5. The multi-feature interactive network recruitment text classification method of the twin-tower-like model according to claim 1, wherein: in the step II, respectively aiming at each recruitment sample text, obtaining a one-hot vector sequence corresponding to the dense feature text according to the following steps II-II-1 to II-II-3;
II-II-1, performing word segmentation on the dense feature text of the recruitment sample text, deleting corresponding characters in the dense feature text according to a preset word list to obtain each dense feature word in the dense feature text, and then entering the step II-II-2;
II-II-2, selecting each non-repeated dense feature participle in the dense feature text, sequencing each non-repeated dense feature participle according to the position of each non-repeated dense feature participle in the dense feature text for the first time, and then entering step II-II-3;
and II-II-3, obtaining vectors corresponding to the non-repeated dense feature participles respectively, and combining the sequence of the non-repeated dense feature participles to form a one-hot vector sequence corresponding to the dense feature text.
6. The multi-feature interactive network recruitment text classification method of the twin-tower-like model according to claim 1, wherein: in the step III, based on a preset first feature extraction model corresponding to the word vector sequence, a preset second feature extraction model corresponding to the one-hot vector sequence, a feature self-interaction model corresponding to each of the two feature extraction models, and a global feature interaction model between the two feature extraction models, an input end of the preset first feature extraction model and an input end of the preset second feature extraction model are used as input ends, an output end of the preset first feature extraction model, an output end of the preset second feature extraction model, and a fusion output end of each feature self-interaction model output and a fusion output end of the global feature interaction model output are further connected with an input end of the feature fusion layer, an output end of the feature fusion layer is sequentially connected with the attention layer and the softmax layer in series, and a text classification initial probability model is constructed.
7. The multi-feature interactive network recruitment text classification method of the twin-tower-like model according to claim 6, wherein: the preset first feature extraction model comprises three input ends of a first convolution module, wherein the input ends of the first convolution module are respectively 6, 7 and 8 in height and are respectively formed by convolution kernels with the width consistent with the dimension of the input word vector sequence, the input ends of the first convolution module are respectively formed by the input ends of the first convolution module, the output ends of the three first convolution modules are in butt joint with the input end of a first fusion module, the output end of the first fusion module is in butt joint with the input end of an LSTM network, and the convolution kernels are 1 in size
Figure DEST_PATH_IMAGE001
1, an input of a second convolution module; the output end of the LSTM network is in butt joint with a convolution kernel with the size of 1
Figure 28439DEST_PATH_IMAGE001
1, the output end of the second convolution module is in butt joint with the input end of the first pooling module; the output end of the third convolution module is respectively butted with the input end of the second fusion module and the input end of the third fusion module, the output end of the first pooling module is butted with the input end of the third fusion module, the output end of the third fusion module is respectively butted with the input end of the second fusion module, and the sizes of two convolution kernels are 5
Figure 476738DEST_PATH_IMAGE001
The input end of a fourth convolution module in 5-phase series connection and the output end of a second fusion module are in butt joint with a convolution kernel with the size of 1
Figure 803945DEST_PATH_IMAGE001
1, an input of a fifth convolution module; the output end of the fifth convolution module and the output ends of the two series-connected fourth convolution modules are respectively butted with the input end of the fourth fusion module, and the output end of the fourth fusion module is butted with the input end of the fourth fusion moduleThe input end of the local average pooling module and the output end of the global average pooling module form the output end of the first feature extraction model;
in the structure of the first feature self-interaction model corresponding to the first feature extraction model, the sizes of six convolution kernels are 1
Figure 995892DEST_PATH_IMAGE001
The input end of a sixth convolution module of 1 forms six input ends of a first characteristic self-interaction model, and the six input ends are respectively butted with six connection characteristics of pairwise arrangement and combination among the output end characteristics of a first fusion module, the output end characteristics of a second fusion module, the output end characteristics of a third fusion module and the output end characteristics of a fourth fusion module; the six sixth convolution modules form a group of two sixth convolution modules to form three sixth convolution module groups, each sixth convolution module group respectively corresponds to each fifth fusion module and each first K-value maximum pooling module one by one, the output ends of the two sixth convolution modules in each sixth convolution module group are in butt joint with the input end of the corresponding fifth fusion module, and the output end of the fifth fusion module is in butt joint with the input end of the corresponding first K-value maximum pooling module; the output end of each first K-value maximum pooling module is in butt joint with the input end of the sixth fusion module, the output end of the sixth fusion module is in butt joint with the input end of the first sub-attention module, and the output end of the first sub-attention module forms the output end of the first characteristic self-interaction model.
8. The multi-feature interactive network recruitment text classification method of the twin-tower-like model according to claim 7, wherein: the preset second feature extraction model is in a direction from the input end to the output end of the preset second feature extraction model, the input end of a Region embedding module in the DPCNN model forms the input end of the second feature extraction model, and the output end of the Region embedding module is respectively in butt joint with the input end of a seventh fusion module, the input end of an eighth fusion module and two convolution kernels with the size of 5
Figure 341422DEST_PATH_IMAGE001
Input end of a seventh convolution module with 5 phases connected in series, two phasesThe output end of the seventh convolution module is connected in series and is simultaneously connected with the input end of the seventh fusion module, and the output end of the seventh fusion module is respectively connected with the input end of the cyclic convolution pooling module and the input end of the eighth fusion module; the pooling characteristic circulation output end of the circulation convolution pooling module is in butt joint with the input end of a ninth fusion module, and the output end of the ninth fusion module is in butt joint with a convolution kernel with the size of 1
Figure 206961DEST_PATH_IMAGE001
1, the output end of the eighth fusion module is in butt joint with a convolution kernel with the size of 1
Figure 270732DEST_PATH_IMAGE001
1, the output end of the eighth convolution module and the output end of the ninth convolution module are respectively butted with the input end of a tenth fusion module, and the output end of the tenth fusion module is butted with the input end of a second K-value maximum pooling module; the output end of the circular convolution pooling module and the output end of the second K-value maximum pooling module are respectively butted with the input end of an eleventh fusion module, the output end of the eleventh fusion module is butted with the input end of the maximum pooling module, and the output end of the maximum pooling module forms the output end of the second feature extraction model;
in the structure of the second feature self-interaction model corresponding to the second feature extraction model, the sizes of three convolution kernels are 1
Figure 17102DEST_PATH_IMAGE001
The input end of the tenth convolution module of 1 forms three input ends of a second characteristic self-interaction model, and the three input ends are respectively butted with three connection characteristics of pairwise arrangement and combination among the output end characteristics of an eighth fusion module, the output end characteristics of a tenth fusion module and the output end characteristics of an eleventh fusion module; the output ends of the three tenth convolution modules are respectively in one-to-one butt joint with the input end of the third K-value maximum pooling module, the input end of the fourth K-value maximum pooling module, the input end of the fifth K-value maximum pooling module, the output end of the third K-value maximum pooling module, the output end of the fourth K-value maximum pooling module and the output end of the fifth K-value maximum pooling moduleThe output end of the five K-value maximum pooling module is respectively butted with the input end of a twelfth fusion module, the output end of the twelfth fusion module is butted with the input end of a second sub-attention module, and the output end of the second sub-attention module forms the output end of a second characteristic self-interaction model.
9. The multi-feature interactive network recruitment text classification method of the twin-tower-like model according to claim 8, wherein: the input end of the second pooling module with the step length of 2 forms the input end of the circulating convolution pooling module from the input end to the output end of the circulating convolution pooling module, the output end of the second pooling module is in butt joint with the input ends of eleventh convolution modules with convolution kernels of 5 multiplied by 5 in series, the output ends of the eleventh convolution modules with two series are in butt joint with the input end of a thirteenth fusion module, and the output ends of the thirteenth fusion module form the output end of the circulating convolution pooling module.
10. The multi-feature interactive network recruitment text classification method of the twin-tower-like model according to claim 9, wherein: in the structure of the global feature interaction model, the input ends of six fourteenth fusion modules form six input ends of the global feature interaction model, of the six input ends, the first input end is respectively butted with the output end of a first fusion module in the first feature extraction model and the output end of a Region embedding module in the second feature extraction model, the second input end is respectively butted with the output end of a second convolution module in the first feature extraction model and the output end of a second seventh convolution module in two seventh convolution modules which are connected in series in the second feature extraction model, the third input end is respectively butted with the output end of a first pooling module in the first feature extraction model and the output end of a seventh fusion module in the second feature extraction model, and the fourth input end is respectively butted with the output end of a third fusion module in the first feature extraction model and the output end of a second pooling module in the last circulation of the circular convolution pooling modules in the second feature extraction model, the fifth input end is respectively butted with the output end of a first fourth convolution module in two series-connected fourth convolution modules in the first feature extraction model and the output end of a first eleventh convolution module in two series-connected eleventh convolution modules when the last circulation is performed in the circular convolution pooling module in the second feature extraction model, and the sixth input end is respectively butted with the output end of a second fourth convolution module in two series-connected fourth convolution modules in the first feature extraction model and the output end of a second eleventh convolution module in two series-connected eleventh convolution modules when the last circulation is performed in the circular convolution pooling module in the second feature extraction model; and the fourteenth fusion modules respectively correspond to the average pooling modules one by one, the output ends of the fourteenth fusion modules are respectively in butt joint with the input ends of the average pooling modules, the output ends of the average pooling modules are respectively in butt joint with the input end of the fifteenth fusion module, and the output ends of the fifteenth fusion modules form the output end of the global feature interaction model.
CN202110600441.5A 2021-05-31 2021-05-31 Multi-feature interactive network recruitment text classification method similar to double-tower model Active CN113342933B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110600441.5A CN113342933B (en) 2021-05-31 2021-05-31 Multi-feature interactive network recruitment text classification method similar to double-tower model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110600441.5A CN113342933B (en) 2021-05-31 2021-05-31 Multi-feature interactive network recruitment text classification method similar to double-tower model

Publications (2)

Publication Number Publication Date
CN113342933A true CN113342933A (en) 2021-09-03
CN113342933B CN113342933B (en) 2022-11-08

Family

ID=77472740

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110600441.5A Active CN113342933B (en) 2021-05-31 2021-05-31 Multi-feature interactive network recruitment text classification method similar to double-tower model

Country Status (1)

Country Link
CN (1) CN113342933B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114579869A (en) * 2022-05-05 2022-06-03 腾讯科技（深圳）有限公司 Model training method and related product
CN115809334A (en) * 2022-11-22 2023-03-17 北京百度网讯科技有限公司 Training method of event relevance classification model, text processing method and text processing device
CN116662676A (en) * 2023-06-09 2023-08-29 北京华品博睿网络技术有限公司 Online recruitment bidirectional reciprocity recommendation system and method based on multi-behavior modeling
WO2024060066A1 (en) * 2022-09-21 2024-03-28 京东方科技集团股份有限公司 Text recognition method, and model and electronic device

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110232127A (en) * 2019-06-17 2019-09-13 重庆紫光华山智安科技有限公司 File classification method and device
US20190370394A1 (en) * 2018-05-31 2019-12-05 Fmr Llc Automated computer text classification and routing using artificial intelligence transfer learning
CN110874410A (en) * 2019-11-01 2020-03-10 河南理工大学 Text classification method based on long-time and short-time memory network and convolutional neural network
CN111428038A (en) * 2020-03-26 2020-07-17 国网浙江杭州市萧山区供电有限公司 Self-attention mechanism-based electric power complaint work order multi-label text classification method
CN111985369A (en) * 2020-08-07 2020-11-24 西北工业大学 Course field multi-modal document classification method based on cross-modal attention convolution neural network
CN112732915A (en) * 2020-12-31 2021-04-30 平安科技（深圳）有限公司 Emotion classification method and device, electronic equipment and storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190370394A1 (en) * 2018-05-31 2019-12-05 Fmr Llc Automated computer text classification and routing using artificial intelligence transfer learning
CN110232127A (en) * 2019-06-17 2019-09-13 重庆紫光华山智安科技有限公司 File classification method and device
CN110874410A (en) * 2019-11-01 2020-03-10 河南理工大学 Text classification method based on long-time and short-time memory network and convolutional neural network
CN111428038A (en) * 2020-03-26 2020-07-17 国网浙江杭州市萧山区供电有限公司 Self-attention mechanism-based electric power complaint work order multi-label text classification method
CN111985369A (en) * 2020-08-07 2020-11-24 西北工业大学 Course field multi-modal document classification method based on cross-modal attention convolution neural network
CN112732915A (en) * 2020-12-31 2021-04-30 平安科技（深圳）有限公司 Emotion classification method and device, electronic equipment and storage medium

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
李军炜: "短文本用户评论的分类***设计与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *
涂文博 等: "针对文本分类的神经网络模型", 《计算机***应用》 *
马晨峰: "混合深度学习模型在新闻文本分类中的应用", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114579869A (en) * 2022-05-05 2022-06-03 腾讯科技（深圳）有限公司 Model training method and related product
CN114579869B (en) * 2022-05-05 2022-07-22 腾讯科技（深圳）有限公司 Model training method and related product
WO2024060066A1 (en) * 2022-09-21 2024-03-28 京东方科技集团股份有限公司 Text recognition method, and model and electronic device
CN115809334A (en) * 2022-11-22 2023-03-17 北京百度网讯科技有限公司 Training method of event relevance classification model, text processing method and text processing device
CN115809334B (en) * 2022-11-22 2023-11-10 北京百度网讯科技有限公司 Training method of event relevance classification model, text processing method and device
CN116662676A (en) * 2023-06-09 2023-08-29 北京华品博睿网络技术有限公司 Online recruitment bidirectional reciprocity recommendation system and method based on multi-behavior modeling

Also Published As

Publication number Publication date
CN113342933B (en) 2022-11-08

Similar Documents

Publication Publication Date Title
CN113342933B (en) Multi-feature interactive network recruitment text classification method similar to double-tower model
CN110597735B (en) Software defect prediction method for open-source software defect feature deep learning
CN107239446B (en) A kind of intelligence relationship extracting method based on neural network Yu attention mechanism
CN112115238B (en) Question-answering method and system based on BERT and knowledge base
CN111831789B (en) Question-answering text matching method based on multi-layer semantic feature extraction structure
CN107943784B (en) Relationship extraction method based on generation of countermeasure network
CN110442684A (en) A kind of class case recommended method based on content of text
CN112732916B (en) BERT-based multi-feature fusion fuzzy text classification system
CN108280064A (en) Participle, part-of-speech tagging, Entity recognition and the combination treatment method of syntactic analysis
CN111966812B (en) Automatic question answering method based on dynamic word vector and storage medium
CN111400494B (en) Emotion analysis method based on GCN-Attention
CN112733866A (en) Network construction method for improving text description correctness of controllable image
CN111046661A (en) Reading understanding method based on graph convolution network
CN112732921B (en) False user comment detection method and system
CN116304066A (en) Heterogeneous information network node classification method based on prompt learning
CN113094502A (en) Multi-granularity takeaway user comment sentiment analysis method
CN116662565A (en) Heterogeneous information network keyword generation method based on contrast learning pre-training
CN115168579A (en) Text classification method based on multi-head attention mechanism and two-dimensional convolution operation
CN113822061B (en) Small sample patent classification method based on feature map construction
CN114817307A (en) Low-sample NL2SQL method based on semi-supervised learning and meta-learning
CN113051904A (en) Link prediction method for small-scale knowledge graph
CN117349311A (en) Database natural language query method based on improved RetNet
CN117113937A (en) Electric power field reading and understanding method and system based on large-scale language model
CN117152573A (en) Transformer and data enhancement based network media multi-mode information extraction method
CN115359486A (en) Method and system for determining custom information in document image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20221209

Address after: 223001 Room 201, building 7, 83 Beijing North Road, qingjiangpu District, Huai'an City, Jiangsu Province

Patentee after: Jiangsu Kesheng Xuanyi Technology Co.,Ltd.

Address before: 210000 No. 89, Beijing North Road, Huai'an City, Jiangsu Province

Patentee before: HUAIYIN INSTITUTE OF TECHNOLOGY

TR01 Transfer of patent right