CN116662598B - Character portrait information management method based on vector index and electronic equipment - Google Patents

Character portrait information management method based on vector index and electronic equipment Download PDF

Info

Publication number
CN116662598B
CN116662598B CN202310933367.8A CN202310933367A CN116662598B CN 116662598 B CN116662598 B CN 116662598B CN 202310933367 A CN202310933367 A CN 202310933367A CN 116662598 B CN116662598 B CN 116662598B
Authority
CN
China
Prior art keywords
sequence
output
vector
module
track
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310933367.8A
Other languages
Chinese (zh)
Other versions
CN116662598A (en
Inventor
梅雨
杨广学
孙禄明
李柏
蒋铭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panorama Zhilian Tianjin Technology Co ltd
Panoramic Zhilian Wuhan Technology Co ltd
Beijing Panorama Zhilian Technology Co ltd
Original Assignee
Panorama Zhilian Tianjin Technology Co ltd
Panoramic Zhilian Wuhan Technology Co ltd
Beijing Panorama Zhilian Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Panorama Zhilian Tianjin Technology Co ltd, Panoramic Zhilian Wuhan Technology Co ltd, Beijing Panorama Zhilian Technology Co ltd filed Critical Panorama Zhilian Tianjin Technology Co ltd
Priority to CN202310933367.8A priority Critical patent/CN116662598B/en
Publication of CN116662598A publication Critical patent/CN116662598A/en
Application granted granted Critical
Publication of CN116662598B publication Critical patent/CN116662598B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/51—Indexing; Data structures therefor; Storage structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/55—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • Databases & Information Systems (AREA)
  • Library & Information Science (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Processing Or Creating Images (AREA)
  • Image Analysis (AREA)

Abstract

The application relates to the technical field of data management, and discloses a character portrait information management method and electronic equipment based on vector indexes. The method may include: acquiring face data and track data, and generating a face feature sequence and a track feature sequence; constructing a first model, and inputting a face feature sequence and a track feature sequence into the first model; establishing a mapping relation between the output of the first model at the t-th time step and face data and/or track data from the t-th row vector source of the first output feature matrix, wherein the vector output by the cyclic neural unit at the t-th time step is used as an index vector of the mapped face data and/or track data; and matching the index vector with the code vector to establish a vectorized index. The application can meet the service requirements of information acquisition of the associated target information and the homomorphic mode object through the index.

Description

Character portrait information management method based on vector index and electronic equipment
Technical Field
The application relates to the technical field of data management, in particular to a character portrait information management method based on vector indexes and electronic equipment.
Background
The traditional face data vectorization index directly carries out vectorization through a face image, and then the vectorization index is established on the face image by searching out the code vector which is most matched with the input, so that the establishment of the vectorization index of the face image is based on the code vector, the code vector can also be regarded as a vectorized face image, the established vectorization index can only provide the retrieval result of similar face images and can only meet the service requirement of a person image comparison determination target, and for the service requirement of police and the like, the simple search of people with similar facial features is difficult to meet the requirement, and the retrieval of multiple associated dimensions is required to provide the face image which meets the requirement better.
Disclosure of Invention
The application provides a character portrait information management method based on vector indexes and electronic equipment, and solves the technical problem that the vectorization indexes of face images in the related technology can only meet the business requirements of a portrait comparison determination target.
According to one aspect of the present application, there is provided a character image information management method based on vector index, including the following.
Step S101, face data and track data are collected, a face feature is generated based on the face data collected at a time point, and a track feature is generated based on the track data collected at a time point; and respectively sequencing the face features and the track features according to the sequence of the acquisition time to generate a face feature sequence and a track feature sequence.
Step S102, constructing a first model, wherein the first model comprises a first module, a second module, a first middle module, a second middle module, a third module and a fourth module, the first module comprises a plurality of convolution layers, the second module comprises more than one convolution layer, a track feature matrix is generated based on a track feature sequence, and the element of the (e) th row of the track feature matrix represents the (e) th vector component of the (q) th track feature, wherein q is more than 1 and e is more than 1; inputting the track feature matrix into a first module and outputting the first output feature matrix by the first module; inputting the face feature sequence into a second module and outputting a second output feature by the second module; the first output feature matrix is input to the first intermediate module and the first intermediate module generates a first output sequence based on the first output feature matrix, one sequence element in the first output sequence representing one row vector in the first output feature matrix.
The sequence units of the first output sequence and the second output characteristics are synthesized to generate a second output sequence, the second output sequence is input into a second intermediate module, the second intermediate module generates a position vector for the sequence units of the second output sequence, and the position vector and the corresponding sequence units in the second output sequence are synthesized to generate the sequence units of the third output sequence.
The third output sequence is input into a third module, the third module comprises more than two hidden layers, an output characteristic matrix of the last hidden layer of the third module is input into a fourth module as a second output characteristic matrix, and a calculation formula of the s-th hidden layer of the third module is as follows:the method comprises the steps of carrying out a first treatment on the surface of the Wherein->Representing the output feature matrix of the s-th hidden layer, wherein s.gtoreq.1, softmax represents the normalized exponential function,>、/>、/>a first input matrix, a second input matrix and a third input matrix representing respectively the s-th hidden layer,/or->，，/>The method comprises the steps of carrying out a first treatment on the surface of the Wherein->、/>、/>Respectively representing a first, a second and a third weight matrix +.>，/>Representing a matrixed third output sequence, s > 1,/if s > 1>，Representing the output feature matrix of the hidden layer of the s-1 layer.
The fourth module comprises a cyclic nerve unit and a full-connection layer, wherein the cyclic nerve unit inputs one row vector of the second output characteristic matrix at each time step, and the input sequence is input according to the sequence number of the row vectors of the second output characteristic matrix.
Step S103, a mapping relation is established between the output of the cyclic neural unit at the t-th time step and the face data and/or track data from the t-th row vector of the second output feature matrix, and the vector of the output of the cyclic neural unit at the t-th time step is used as an index vector of the mapped face data and/or track data, wherein t is more than 1.
Step S104, matching the index vector with the code vector to establish a vectorization index.
According to the character image information management method based on the vector index of at least one embodiment of the present application, the face data is image data, and the track data is address data.
According to the character image information management method based on the vector index of at least one embodiment of the present application, the track characteristics are obtained by performing the word segmentation and the word vector generation processing based on the track data.
According to the character image information management method based on the vector index in at least one embodiment of the application, when the track feature matrix is generated, the track features are aligned by taking the track feature with the largest dimension in the track feature sequence as the standard, and the dimensions of all the track features after the alignment are consistent.
According to a vector index-based portrait information management method according to at least one embodiment of the present application, a first module performs a dilation convolution in which the following is defined.The method comprises the steps of carrying out a first treatment on the surface of the Wherein->And->Represents the maximum expansion rate that can be selected for the ith and i+1th convolution layers, respectively, where i.gtoreq.1,/and->Indicating the selected expansion ratio of the ith convolution layer, the maximum expansion ratio that the 2 nd convolution layer can select +.>K is the size of the convolution kernel of the convolution layer, and when i=n, the maximum expansion rate that the nth convolution layer can select +.>，/>Representing the selected expansion ratio of the nth convolution layer, n being the total number of convolution layers of the first module.
According to the character image information management method based on the vector index, when the first module performs training, the output of the first module is connected with a training classifier, and the output of the training classifier is classified into two categories which correspond to a normal path and an illegal path respectively.
According to at least one embodiment of the present application, the formula of combining the sequence unit of the first output sequence and the second output feature isWhereinAnd->The j-th sequence unit representing the second output sequence and the first output sequence, respectively,/->And a second output feature generated by the jth face feature of the face feature sequence is represented, wherein Concat represents a Concat function, and j is more than 1.
According to at least one embodiment of the present application, the formula for combining the position vector and the corresponding sequence unit in the second output sequence isWhereinAnd->The j-th sequence unit representing the third output sequence and the second output sequence, respectively,/->A position vector representing a j-th sequence element of the second output sequence, concat representing a Concat function, where j > 1.
According to the character portrait information management method based on vector index according to at least one embodiment of the present application, the output of the cyclic neural unit is connected with the training classifier, the output of the training classifier is mapped to the classification space, and the labels in the classification space represent different behavior types of characters.
According to at least one embodiment of the present application, a vector index-based portrait information management method includes: generating a face feature sequence and a track feature sequence from face data and track data to be queried, inputting a first model, and outputting a plurality of index vectors to be queried; and matching the obtained index vector to be queried with the code vector of the vectorization index, and extracting face data and/or track data mapped by the index vector matched with the matched code vector as an index result.
According to another aspect of the present application, there is also provided an electronic device comprising a memory, a processor and instructions stored on the memory, the processor executing the instructions to implement the steps of the above method.
According to another aspect of the present application there is also provided a computer readable storage medium having stored thereon a computer program/instruction which when executed by a processor performs the steps of the above method.
The application has the beneficial effects that: according to the application, the human face data and the track data are processed in a combined way, the time relevance and the spatial characteristics of the track data are integrated to perform comprehensive vectorization processing on the human data, the vectorization index is constructed on the basis, and the service requirements of the information acquisition of the related target information and the homomorphic mode object can be met through the index.
Drawings
The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the disclosure and together with the description serve to explain the principles of the disclosure.
Fig. 1 is a flowchart of a character image information management method based on vector index according to an embodiment of the present application.
Fig. 2 is a flowchart of an indexing method of vectorized index according to an embodiment of the present application.
FIG. 3 is an exemplary diagram of an apparatus employing a hardware implementation of a processing system in accordance with a method of an embodiment of the application.
Detailed Description
The present disclosure is described in further detail below with reference to the drawings and the embodiments. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant content and not limiting of the present disclosure. It should be further noted that, for convenience of description, only a portion relevant to the present disclosure is shown in the drawings.
In addition, embodiments of the present disclosure and features of the embodiments may be combined with each other without conflict. The technical aspects of the present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Unless otherwise indicated, the exemplary implementations/embodiments shown are to be understood as providing exemplary features of various details of some ways in which the technical concepts of the present disclosure may be practiced. Thus, unless otherwise indicated, features of the various implementations/embodiments may be additionally combined, separated, interchanged, and/or rearranged without departing from the technical concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Furthermore, when the terms "comprises" and/or "comprising," and variations thereof, are used in the present specification, the presence of stated features, integers, steps, operations, elements, components, and/or groups thereof is described, but the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof is not precluded. It is also noted that, as used herein, the terms "substantially," "about," and other similar terms are used as approximation terms and not as degree terms, and as such, are used to explain the inherent deviations of measured, calculated, and/or provided values that would be recognized by one of ordinary skill in the art.
Fig. 1 illustrates a flowchart of a character image information management method based on vector index according to an embodiment of the present application. As shown in fig. 1, the portrait information management method may include step S101, step S102, step S103, and step S104. The respective steps will be described in detail below.
In step S101, face data and trajectory data are collected, a face feature is generated based on face data collected at a time point, and a trajectory feature is generated based on trajectory data collected at a time point. Thus, some face features and track features are generated according to the data acquired at the time points. And respectively sequencing the face features and the track features according to the sequence of the acquisition time to generate a face feature sequence and a track feature sequence.
In the present application, the time node dividing the character data is generally equivalent to the acquisition time of the character data. In various embodiments of the present application, the face data is referred to as image data of a face, and the track data is referred to as address data.
In the present application, a specific manner of generating the trajectory feature by the trajectory data may be to perform the word segmentation and the word vector generation processing based on the trajectory data to obtain the trajectory feature. This embodiment is a conventional means in the art, and will not be described in detail in the present application.
In step S102, a first model is constructed. The first model comprises a first module, a second module, a first middle module, a second middle module, a third module and a fourth module.
A track feature matrix is generated based on the track feature sequence, wherein elements of an e-th column of a q-th row of the track feature matrix represent an e-th vector component of the q-th track feature, where q > 1 and e > 1. When the track feature matrix is generated, the track features are aligned by taking the track feature with the largest dimension in the track feature sequence as a standard, and the dimensions of all the track features after the alignment are consistent. The alignment process may include interpolating dimensions at the tail of the track feature, with a dimension value of 0.
The trajectory feature matrix may be input into a first module that outputs a first output feature matrix, which may include a plurality of convolution layers defining:. Wherein->And->Represents the maximum expansion rate that can be selected for the ith and i+1th convolution layers, respectively, where i.gtoreq.1,/and->Indicating the selected expansion ratio of the ith convolution layer, the maximum expansion ratio that the 2 nd convolution layer can select +.>K is the size of the convolution kernel of the convolution layer, and when i=n, the maximum expansion rate that the nth convolution layer can select +.>，/>Representing the selected expansion ratio of the nth convolution layer, n being the total number of convolution layers of the first module.
The size of the convolution kernels of the multiple convolution layers is uniform. By matrixing the track feature sequence and then performing expansion convolution, the key feature extraction is performed on the lengthy track feature while the sequence structure is maintained.
When the first module is trained, the output of the first module is connected with a first training classifier, and the output of the first training classifier is classified into two categories which respectively correspond to a normal path and an illegal path.
The normal path indicates that the track generated by the track data from the input track feature matrix does not exceed the normal path. The illegal path indicates that the track generated by the track data from the input track feature matrix exceeds the normal path.
The face feature sequence is input to the second module. The second module includes more than one convolution layer and outputs a second output feature based on the input face feature sequence.
The second module is connected with a second training classifier during training, and the classification label output by the second training classifier can correspond to the expression type of the face data of the input face features. The definition of the expression type may be set according to the actual situation, which is not described herein.
The first output feature matrix is input to a first intermediate module, which generates a first output sequence based on the first output feature matrix. The first intermediate module is configured such that one sequence element in the first output sequence represents one row vector in the first output feature matrix, that is to say each sequence element in the first output sequence represents one row vector in the first output feature matrix, respectively.
The sequence unit of the first output sequence and the second output characteristic are synthesized to generate a second output sequence, and the synthesized formula can be set as followsWherein->And->Respectively represent the second inputThe jth sequence unit of the output sequence and the first output sequence,>and a second output feature generated by the jth face feature representing the face feature sequence, wherein Concat represents a Concat function.
The second output sequence is input into a second intermediate module, the second intermediate module generates a position vector for the sequence units in the second output sequence, and the position vector and the corresponding sequence units in the second output sequence are synthesized to generate the sequence units of the third output sequence. The second intermediate module may be a conventional module in the prior art capable of generating a position vector from the sequence unit.
The formula for combining the position vector and the corresponding sequence unit in the second output sequence isWherein->And->The j-th sequence unit representing the third output sequence and the second output sequence, respectively,/->A position vector representing a j-th sequence element of the second output sequence, concat representing a Concat function.
In one embodiment of the application, the position vector is generated based on position coordinates of the trajectory data of the source of the sequence unit in the second output sequence. Since the position coordinates are typically represented in numerical values, they can be directly stitched as a position vector.
The third output sequence is input into a third module, the third module comprises more than two hidden layers, a second output characteristic matrix of the last hidden layer of the third module is input into a fourth module, and a calculation formula of the s-th hidden layer of the third module is as follows:the method comprises the steps of carrying out a first treatment on the surface of the Wherein->Representing the output feature matrix of the s-th hidden layer, wherein s.gtoreq.1, softmax represents the normalized exponential function,>、/>、/>a first input matrix, a second input matrix and a third input matrix representing respectively the s-th hidden layer,/or->，/>，The method comprises the steps of carrying out a first treatment on the surface of the Wherein->、/>、/>Respectively representing a first weight matrix, a second weight matrix and a third weight matrix +.>，/>Representing a matrixed third output sequence, s > 1,/if s > 1>，A second output feature matrix representing a hidden layer of the s-1 layer.
The fourth module comprises a cyclic nerve unit and a full-connection layer, wherein the cyclic nerve unit inputs one row vector of the second output characteristic matrix at each time step, and the input sequence is input according to the sequence number of the row vectors of the second output characteristic matrix.
In step S103, a mapping relationship is established between the output of the recurrent neural unit and the face data and/or the trajectory data. According to the application, the human face data and the track data are processed in a combined way, the time relevance and the spatial characteristics of the track data are integrated to perform comprehensive vectorization processing on the human data, the vectorization index is constructed on the basis, and the service requirements of the information acquisition of the related target information and the homomorphic mode object can be met through the index.
The input symbolized representation of the recurrent neural unit at the t-th time step is:wherein->Input representing the cyclic neural unit at time step t,/->The t-th row vector representing the second output feature matrix.
Step S103, a mapping relation is established between the output of the cyclic neural unit at the t-th time step and the face data and/or track data from the t-th row vector of the second output feature matrix, and the vector output by the cyclic neural unit at the t-th time step is used as an index vector of the mapped face data and/or track data.
The t-th row vector of the second output feature matrix is derived from the t-th sequence element of the third output sequence.
During training, the output of the cyclic neural unit is connected with a training classifier, the output of the training classifier is mapped to a classification space, and labels in the classification space represent different behavior types of characters.
As an example, the classification space includes two classification labels, which respectively indicate that the behavior type of the person is legal and illegal, for example, that an area where legal person enters without authority is defined as illegal, that illegal person enters is defined as illegal, and that an area where legal person enters with authority is defined as legal.
As one example, the classification space includes a plurality of classification tags that respectively represent the behavior types of a person as walking, running, using a non-motor vehicle, using a motor vehicle, and the like.
In the present application, the behavior type may be selected according to a specific application scenario of the method of the embodiment of the present application, and the above is merely an exemplary description.
The first module and the second module perform independent training, the independent training is combined into the first model after the independent training is completed, the third module and the fourth module perform combined training, and the weight parameters of the first module and the second module are not updated when the third module and the fourth module perform training.
In step S104, the index vector and the code vector are matched to create a vectorized index. Among them, a specific embodiment thereof will be described in detail below with reference to fig. 2.
Fig. 2 shows a flowchart of an indexing method of vectorized indexing, wherein the indexing method may include step S201, step S202, and step S203.
In step S201, face feature sequences and track feature sequences may be generated from face data and track data to be queried. The face data and the track data to be queried can be acquired face data and track data, a face feature is generated based on the face data acquired at a time point, and a track feature is generated based on the track data acquired at a time point. Thus, some face features and track features are generated according to the data acquired at the time points. And respectively sequencing the face features and the track features according to the sequence of the acquisition time to generate a face feature sequence and a track feature sequence. The specific manner of generating the trajectory feature by the trajectory data may be to perform the word segmentation and the word vector generation processing based on the trajectory data to obtain the trajectory feature.
Inputting the face feature sequence and the track feature sequence into a first model, so that the first model outputs a plurality of index vectors to be queried; and matching the obtained index vector to be queried with the code vector of the vectorization index, and extracting face data and/or track data mapped by the index vector matched with the matched code vector as an index result. Specifically, in step S202, the face data is input to the second module of the first model, the output second output feature is used as the index vector to be queried, the obtained index vector to be queried matches the code vector of the vectorization index, and the face data and/or the track data mapped by the index vector matched by the matched code vector are extracted as the index result. In step S203, the track data is input into a first module of the first model, a first output feature matrix output by the first module is used as an index vector to be queried, the obtained index vector to be queried matches the code vector of the vectorization index, and face data and/or track data mapped by the index vector matched by the matched code vector are extracted as index results. In the above embodiment, the index vector matched by extracting the matched code vector does not include the index vector with the query.
FIG. 3 illustrates an example diagram of an apparatus in which the above method employs a hardware implementation of a processing system. The apparatus may include corresponding modules that perform the steps of the flowcharts described above. Thus, each step or several steps in the flowcharts described above may be performed by respective modules, and the apparatus may include one or more of these modules. A module may be one or more hardware modules specifically configured to perform the respective steps, or be implemented by a processor configured to perform the respective steps, or be stored within a computer-readable medium for implementation by a processor, or be implemented by some combination.
The hardware architecture may be implemented using a bus architecture. The bus architecture may include any number of interconnecting buses and bridges depending on the specific application of the hardware and the overall design constraints. Bus 1100 connects together various circuits including one or more processors 1200, memory 1300, and/or hardware modules. Bus 1100 may also connect various other circuits 1400, such as peripherals, voltage regulators, power management circuits, external antennas, and the like.
Bus 1100 may be an industry standard architecture (ISA, industry Standard Architecture) bus, a peripheral component interconnect (PCI, peripheral Component) bus, or an extended industry standard architecture (EISA, extended Industry Standard Component) bus, among others. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, only one connection line is shown in the figure, but not only one bus or one type of bus.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and further implementations are included within the scope of the preferred embodiment of the present disclosure in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present disclosure. The processor performs the various methods and processes described above. For example, method embodiments in the present disclosure may be implemented as a software program tangibly embodied on a machine-readable medium, such as a memory. In some embodiments, part or all of the software program may be loaded and/or installed via memory and/or a communication interface. One or more of the steps of the methods described above may be performed when a software program is loaded into memory and executed by a processor. Alternatively, in other embodiments, the processor may be configured to perform one of the methods described above in any other suitable manner (e.g., by means of firmware).
Logic and/or steps represented in the flowcharts or otherwise described herein may be embodied in any readable storage medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions.
For the purposes of this description, a "readable storage medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable read-only memory (CDROM). In addition, the readable storage medium may even be paper or other suitable medium on which the program can be printed, as the program can be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner if necessary, and then stored in a memory.
It should be understood that portions of the present disclosure may be implemented in hardware, software, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
Those of ordinary skill in the art will appreciate that all or a portion of the steps implementing the methods of the embodiments described above may be implemented by a program to instruct related hardware. The program may be stored in a readable storage medium. The program, when executed, includes one or a combination of steps for implementing the method.
Furthermore, each functional unit in each embodiment of the present disclosure may be integrated into one processing module, or each unit may exist alone physically, or two or more units may be integrated into one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules may also be stored in a readable storage medium if implemented in the form of software functional modules and sold or used as a stand-alone product. The storage medium may be a read-only memory, a magnetic disk or optical disk, etc.
As shown in fig. 3, a vector index-based character image information management apparatus 1000 according to an embodiment of the present disclosure may include an acquisition module 1002, a generation module 1004, a first model module 1006, a mapping module 1008, and a matching construction module 1010.
In the acquisition module 1002, face data and trajectory data are acquired.
In the generating module 1004, a face feature is generated based on face data collected at a time point, and a track feature is generated based on track data collected at a time point. Thus, some face features and track features are generated according to the data acquired at the time points. And respectively sequencing the face features and the track features according to the sequence of the acquisition time to generate a face feature sequence and a track feature sequence. Wherein, the related specific content can refer to the previous description, and the description is omitted herein for brevity.
In the first model module 1006, the first model described above may be formed. Wherein the first model may comprise a first module, a second module, a first intermediate module, a second intermediate module, a third module, and a fourth module as described above. The trajectory feature matrix may be input into a first module and the face feature sequence may be input into a second module. The first output feature matrix output by the first module is input into the first intermediate module. The second output sequence generated by combining the sequence unit of the first output sequence and the second output characteristic is input into a second middle module, the third output sequence generated by combining the position vector and the corresponding sequence unit in the second output sequence is input into a third module, and the second output characteristic matrix of the last hidden layer of the third module is input into a fourth module. The input and output of each module and the construction manner may refer to the previous description, and are not repeated herein.
In the mapping module 1008, a mapping relationship is established between the output of the cyclic neural unit of the fourth module at the t-th time step and the face data and/or the track data from the t-th row vector of the second output feature matrix, and the vector output by the cyclic neural unit at the t-th time step is used as an index vector of the mapped face data and/or track data.
In the match construction module 1010, the index vector is matched with the code vector to create a vectorized index.
It should be noted that, the relevant content in each module may refer to the relevant content of the portrait information management method based on vector index according to the present application, and for brevity, the description of the relevant content of each module is simplified.
The present disclosure also provides an electronic device, including: a memory storing execution instructions; and a processor or other hardware module that executes the memory-stored execution instructions, causing the processor or other hardware module to perform the method described above.
The present disclosure also provides a readable storage medium having stored therein execution instructions which when executed by a processor are adapted to carry out the above-described method.
In the description of the present specification, reference to the terms "one embodiment/manner," "some embodiments/manner," "example," "a particular example," "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment/manner or example is included in at least one embodiment/manner or example of the application. In this specification, the schematic representations of the above terms are not necessarily for the same embodiment/manner or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments/modes or examples. Furthermore, the various embodiments/modes or examples described in this specification and the features of the various embodiments/modes or examples can be combined and combined by persons skilled in the art without contradiction.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present application, the meaning of "plurality" means at least two, for example, two, three, etc., unless specifically defined otherwise.
It will be appreciated by those skilled in the art that the above-described embodiments are merely for clarity of illustration of the disclosure, and are not intended to limit the scope of the disclosure. Other variations or modifications will be apparent to persons skilled in the art from the foregoing disclosure, and such variations or modifications are intended to be within the scope of the present disclosure.

Claims (10)

1. A character portrayal information management method based on vector index, comprising:
step S101, face data and track data are collected, a face feature is generated based on the face data collected at a time point, and a track feature is generated based on the track data collected at a time point; the face features and the track features are respectively sequenced according to the sequence of the acquisition time to generate a face feature sequence and a track feature sequence;
step S102, constructing a first model, wherein the first model comprises a first module, a second module, a first middle module, a second middle module, a third module and a fourth module, the first module comprises a plurality of convolution layers, the second module comprises more than one convolution layer, a track feature matrix is generated based on a track feature sequence, and the element of the (e) th row of the track feature matrix represents the (e) th vector component of the (q) th track feature, wherein q is more than 1 and e is more than 1; inputting the track feature matrix into a first module and outputting the first output feature matrix by the first module; inputting the face feature sequence into a second module and outputting a second output feature by the second module; the first output feature matrix is input into the first intermediate module and the first intermediate module generates a first output sequence based on the first output feature matrix, one sequence unit in the first output sequence representing one row vector in the first output feature matrix; synthesizing the sequence units of the first output sequence and the second output characteristics to generate a second output sequence, inputting the second output sequence into a second intermediate module, generating a position vector for the sequence units of the second output sequence by the second intermediate module, and synthesizing the position vector and the corresponding sequence units in the second output sequence to generate the sequence units of a third output sequence; the third output sequence is input into a third module, the third module comprises more than two hidden layers, and an output characteristic matrix of the last hidden layer of the third module is input into a fourth module as a second output characteristic matrix; the fourth module comprises a circulating nerve unit and a full-connection layer, wherein the circulating nerve unit inputs one row vector of the second output characteristic matrix at each time step, and the input sequence is input according to the sequence number of the row vector of the second output characteristic matrix;
step S103, establishing a mapping relation between the output of the cyclic neural unit at the t-th time step and the face data and/or track data from the t-th row vector of the second output feature matrix, wherein the vector of the output of the cyclic neural unit at the t-th time step is used as an index vector of the mapped face data and/or track data, and t is more than 1; and
step S104, matching the index vector with the code vector to establish a vectorization index.
2. The character image information management method according to claim 1, wherein the face data is image data and the track data is address data.
3. The character image information management method based on vector index according to claim 1, wherein the track feature is obtained by performing word segmentation and word vector generation processing based on track data.
4. The character image information management method according to claim 3, wherein when the track feature matrix is generated, the track features are aligned with the track feature with the largest dimension in the track feature sequence as a standard, and the dimensions of all the track features after the alignment process are consistent.
5. The character image information management method according to claim 1, wherein when the first module performs training, an output of the first module is connected to a training classifier, and an output of the training classifier is classified into two categories, which correspond to a normal path and an illegal path, respectively.
6. The vector index-based portrait information management method according to claim 1 where a formula of a combination of a sequence unit of a first output sequence and a second output feature is as follows:
wherein->And->The j-th sequence unit representing the second output sequence and the first output sequence, respectively,/->And a second output feature generated by the jth face feature representing the face feature sequence, wherein Concat represents a Concat function.
7. The vector index-based portrait information management method according to claim 1 where a formula of combining a position vector with a corresponding sequence unit in a second output sequence is as follows:
wherein->And->The j-th sequence unit representing the third output sequence and the second output sequence, respectively,/->A position vector representing a j-th sequence element of the second output sequence, concat representing a Concat function.
8. The vector index-based persona information management method of claim 1, wherein an output of the recurrent neural unit is connected to a training classifier, an output of the training classifier is mapped to a classification space, and labels within the classification space represent different behavioral types of the persona.
9. The vector index-based portrait information management method according to claim 1, wherein said vector index indexing method includes: generating a face feature sequence and a track feature sequence from face data and track data to be queried, inputting a first model, and outputting a plurality of index vectors to be queried; and matching the obtained index vector to be queried with the code vector of the vectorization index, and extracting face data and/or track data mapped by the index vector matched with the matched code vector as an index result.
10. An electronic device comprising a memory, a processor, and instructions stored on the memory, wherein the processor executes the instructions to implement the steps of the persona information management method of claim 1.
CN202310933367.8A 2023-07-27 2023-07-27 Character portrait information management method based on vector index and electronic equipment Active CN116662598B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310933367.8A CN116662598B (en) 2023-07-27 2023-07-27 Character portrait information management method based on vector index and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310933367.8A CN116662598B (en) 2023-07-27 2023-07-27 Character portrait information management method based on vector index and electronic equipment

Publications (2)

Publication Number Publication Date
CN116662598A CN116662598A (en) 2023-08-29
CN116662598B true CN116662598B (en) 2023-09-26

Family

ID=87724532

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310933367.8A Active CN116662598B (en) 2023-07-27 2023-07-27 Character portrait information management method based on vector index and electronic equipment

Country Status (1)

Country Link
CN (1) CN116662598B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110532432A (en) * 2019-08-21 2019-12-03 深圳供电局有限公司 A kind of personage's trajectory retrieval method and its system, computer readable storage medium
CN111046757A (en) * 2019-11-27 2020-04-21 西安电子科技大学 Training method and device for face portrait generation model and related equipment
KR20200060942A (en) * 2018-11-23 2020-06-02 주식회사 리얼타임테크 Method for face classifying based on trajectory in continuously photographed image

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20200060942A (en) * 2018-11-23 2020-06-02 주식회사 리얼타임테크 Method for face classifying based on trajectory in continuously photographed image
CN110532432A (en) * 2019-08-21 2019-12-03 深圳供电局有限公司 A kind of personage's trajectory retrieval method and its system, computer readable storage medium
CN111046757A (en) * 2019-11-27 2020-04-21 西安电子科技大学 Training method and device for face portrait generation model and related equipment

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
视频监控场景下基于深度学习的多人脸跟踪;李荔等;电子技术与软件工程;全文 *

Also Published As

Publication number Publication date
CN116662598A (en) 2023-08-29

Similar Documents

Publication Publication Date Title
CN107291871B (en) Matching degree evaluation method, device and medium for multi-domain information based on artificial intelligence
Tang et al. A multi-stage framework with context information fusion structure for skin lesion segmentation
CN112084331A (en) Text processing method, text processing device, model training method, model training device, computer equipment and storage medium
Xu et al. Attribute hashing for zero-shot image retrieval
CN115147598B (en) Target detection segmentation method and device, intelligent terminal and storage medium
CN108764280B (en) Medical data processing method and system based on symptom vector
CN108985133B (en) Age prediction method and device for face image
CN112651940B (en) Collaborative visual saliency detection method based on dual-encoder generation type countermeasure network
US20230334893A1 (en) Method for optimizing human body posture recognition model, device and computer-readable storage medium
CN112287656B (en) Text comparison method, device, equipment and storage medium
CN111160049B (en) Text translation method, apparatus, machine translation system, and storage medium
CN107392191B (en) Method for judging identity, device and electronic equipment
CN115186133A (en) Video generation method and device, electronic equipment and medium
Livieris A new ensemble self-labeled semi-supervised algorithm
CN115410717A (en) Model training method, data retrieval method, image data retrieval method and device
CN113722507B (en) Hospitalization cost prediction method and device based on knowledge graph and computer equipment
CN108460335B (en) Video fine-granularity identification method and device, computer equipment and storage medium
CN116662598B (en) Character portrait information management method based on vector index and electronic equipment
CN112183303A (en) Transformer equipment image classification method and device, computer equipment and medium
Marik et al. A hybrid deep feature selection framework for emotion recognition from human speeches
Livieris A New Ensemble Semi-supervised Self-labeled Algorithm.
Silva et al. Arithmetic circuit classification using convolutional neural networks
CN111680132A (en) Noise filtering and automatic classifying method for internet text information
CN107220317B (en) Matching degree evaluation method, device, equipment and storage medium based on artificial intelligence
CN112699907A (en) Data fusion method, device and equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant