CN113947763A - Pavement arrow recognition method and device based on template self-supervision - Google Patents

Pavement arrow recognition method and device based on template self-supervision Download PDF

Info

Publication number
CN113947763A
CN113947763A CN202111220342.0A CN202111220342A CN113947763A CN 113947763 A CN113947763 A CN 113947763A CN 202111220342 A CN202111220342 A CN 202111220342A CN 113947763 A CN113947763 A CN 113947763A
Authority
CN
China
Prior art keywords
template
arrow
supervision
self
data set
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111220342.0A
Other languages
Chinese (zh)
Inventor
王连涛
王力民
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Changzhou Campus of Hohai University
Original Assignee
Changzhou Campus of Hohai University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Changzhou Campus of Hohai University filed Critical Changzhou Campus of Hohai University
Priority to CN202111220342.0A priority Critical patent/CN113947763A/en
Publication of CN113947763A publication Critical patent/CN113947763A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a pavement arrow recognition method and a device based on template self-supervision, which comprises the steps of training on a pre-constructed expansion data set by using a pre-constructed self-supervision template comparison learning model containing an encoder and a projection layer to obtain a trained network, wherein the expansion data set is an arrow template and an expansion data set of a corresponding example sample; identifying the category of the arrow image by using the trained network; the invention can finish the training of the arrow recognition model only by utilizing the template without any manual marking, obtains the arrow pictures simulating various visual angles through the automatic transformation of the template, then inputs the model in pairs to train a depth model which can judge whether the instance is matched with the template, namely can be used for the recognition of the arrow, can effectively avoid the dependence of the manual marking through the self-supervision contrast learning, and finishes the reliable recognition model through the training without the manual supervision.

Description

Pavement arrow recognition method and device based on template self-supervision
Technical Field
The invention relates to a pavement arrow recognition method and device based on template self-supervision, and belongs to the technical field of image processing.
Background
The invention discloses a more advanced road surface guiding arrow recognition method which is applied to the fields of intelligent driving, intelligent transportation facilities and the like and is required to automatically recognize guiding arrows of lanes. The existing method for recognizing the road surface arrow needs to perform perspective transformation on an image acquired by a sensor to obtain a top view without deformation, and then recognize the arrow by adopting a template matching mode; or training a classifier to recognize after manually collecting a large number of arrow image labels. The template self-supervision recognition method can finish the training of the arrow recognition model only by using the template without any manual marking. Arrow pictures simulating various visual angles are obtained through automatic transformation of the template, and then the images are input into the model in pairs to train a depth model capable of judging whether the instance is matched with the template, so that the method can be used for identifying the arrow.
Disclosure of Invention
The invention aims to overcome the defects in the prior art and provides a pavement arrow recognition method and device based on template self-supervision.
In order to achieve the purpose, the invention is realized by adopting the following technical scheme:
in a first aspect, the invention provides a pavement arrow recognition method based on template self-supervision, which comprises the following steps:
training on a pre-constructed extended data set by using a pre-constructed self-supervision template contrast learning model containing an encoder and a projection layer to obtain a trained network, wherein the extended data set is an arrow template and an extended data set of a corresponding example sample;
and identifying the category of the arrow image by using the trained network.
Further, the specific data expansion method adopted during the construction of the expansion data set is as follows:
each template image is transformed by perspective to obtain 1200 arrow images with different angles, and the transformation comprises the following steps: azimuth angle is 0 to 360 degrees, step interval is 45 degrees, 8 types in total; the pitch angle is 0 to-70 degrees, the stepping interval is-5 degrees, and 15 kinds are adopted; the visual angle is 42 degrees to 132 degrees, the stepping interval is 10 degrees, and the total number is 10;
and forming a positive pair with the corresponding arrow template and forming a negative pair with the other arrow templates.
Further, the specific network structure of the self-supervision template comparison learning model is as follows:
selecting any neural network with feature extraction capability as an encoder to extract features, and calculating an image through the encoder to obtain a feature vector:
h＝f(x)；
using a two-layer fully-connected network as a projection layer, projecting the feature map into a space for contrast loss, wherein a projection vector z is the output of a feature vector h in the projection layer:
z＝g(h)＝W(2)σ(W(1)h)；
wherein σ represents a ReLu activation function;
given a pair of images { x, T }, the similarity between them is defined by cosine similarity:
Figure BDA0003312357590000021
constructing a loss function to be optimized:
Figure BDA0003312357590000022
wherein tau is a hyper parameter and is interpreted as a temperature parameter, so that the repulsion effect between negative pairs can be enhanced, the optimization capability of a loss function is improved, and the convergence speed is accelerated.
Further, the training on the pre-constructed augmented data set by using the pre-constructed self-supervised template contrast learning model comprising the encoder and the projection layer comprises the following steps:
inputting the positive pair and the negative pair obtained by combination into a pre-constructed self-supervision template comparison learning model, and respectively obtaining a feature vector h by calculating the image and the template through an encoderi、hT:
hi＝f(xi)；hT＝f(xT)；
Projecting the obtained feature vectors into a mapping space for contrast loss, zi、zTAre respectively image feature vectors hiTemplate feature hTOutput at projection layer:
zi＝g(hi)＝W(2)σ(W(1)hi)；zT＝g(hT)＝W(2)σ(W(1)hT)；
the network automatically adjusts the parameters based on the back propagation of the loss function.
Further, the identifying the category of the arrow image by using the trained network includes:
sending the arrow image to be recognized into a trained feature extraction network, and obtaining a feature vector of the arrow image;
calculating the similarity between the feature vector of the image and each template vector by using a similarity calculation formula;
the arrow image is identified as the arrow category having the greatest similarity.
In a second aspect, the present invention provides a pavement arrow recognition apparatus based on template self-supervision, including:
the training unit is used for training on a pre-constructed extended data set by using a pre-constructed self-supervision template comparison learning model containing an encoder and a projection layer to obtain a trained network, wherein the extended data set is an arrow template and an extended data set of a corresponding example sample;
and the identification unit is used for identifying the category of the arrow image by using the trained network.
In a third aspect, the invention provides a road surface arrow recognition device based on template self-supervision, comprising a processor and a storage medium;
the storage medium is used for storing instructions;
the processor is configured to operate in accordance with the instructions to perform the steps of the method according to any of the above.
In a fourth aspect, the invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of any of the methods described above.
Compared with the prior art, the invention has the following beneficial effects:
the method realizes the classification of the arrow images by learning the images of the template arrows in the self-supervision contrast learning and national standards, can finish the training of the arrow recognition model only by utilizing the templates without any manual marking, obtains the arrow images simulating various visual angles through the automatic transformation of the templates, then inputs the models in pairs to train a depth model which can judge whether the examples are matched with the templates, can be used for recognizing the arrows, can effectively avoid the dependence of the manual marking through the self-supervision contrast learning, and trains and finishes the reliable recognition model without the manual supervision.
Drawings
Fig. 1 is a flowchart of a method for identifying a road surface arrow based on template self-supervision according to an embodiment of the present invention;
FIG. 2 is 11 arrow template images in national standard of China according to an embodiment of the present invention;
FIG. 3 is a partially expanded data presentation diagram provided by an embodiment of the present invention;
FIG. 4 is a diagram of a neural network training process for template self-supervised contrast learning provided by an embodiment of the present invention;
fig. 5 is a left-turn and u-turn composite direction arrow of actual shooting and a template image successfully matched by the method provided by the embodiment of the invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and the protection scope of the present invention is not limited thereby.
Example 1
The embodiment introduces a pavement arrow recognition method based on template self-supervision, which comprises the following steps:
training on a pre-constructed extended data set by using a pre-constructed self-supervision template contrast learning model containing an encoder and a projection layer to obtain a trained network, wherein the extended data set is an arrow template and an extended data set of a corresponding example sample;
and identifying the category of the arrow image by using the trained network.
The following drawings illustrate the technical scheme of the present invention in further detail by taking 11 arrows in national standard of China as examples.
The application process of the pavement arrow recognition method based on template self-supervision provided by the embodiment specifically relates to the following steps:
step S1: constructing an arrow template and an expansion data set of a corresponding example sample;
step S2: constructing an automatic supervision template comparison learning model comprising an encoder and a projection layer;
step S3: training on the augmented data set of step S2 using the model of step S1;
step S4: the category of the arrow image is identified using the network trained in step S3.
The method for recognizing a road surface arrow based on template self-supervision according to claim 1, as shown in fig. 1, wherein the specific data expansion method in step S1 is as follows:
s1.1, subjecting each template image to perspective transformation to obtain 1200 arrow images at different angles, as shown in fig. 2, 11 template images in the national standard, and fig. 3 is a sample of a selected partial transformation image, where the transformation includes: azimuth angle is 0 to 360 degrees, step interval is 45 degrees, 8 types in total; the pitch angle is 0 to-70 degrees, the stepping interval is-5 degrees, and 15 kinds are adopted; the visual angle is 42 degrees to 132 degrees, the step interval is 10 degrees, and the total number is 10.
S1.2: and forming a positive pair with the corresponding arrow template and forming a negative pair with the other arrow templates.
The method for recognizing the road surface arrow based on the template self-supervision as claimed in claim 2, wherein the self-supervision template comparison learning specific network structure in the step S2 is as follows:
s2.1: any neural network having a feature extraction capability is selected as the encoder extraction feature, and any network in which a convolution layer and a pooling layer are combined, including VGGNet, *** lenet, ResNet, densneet, and the like, can be used. The image is calculated by an encoder to obtain a feature vector:
h＝f(x)；
s2.2: the feature vectors are projected into the space for contrast loss using two layers of nonlinear MLPs as projection layers. The projection vector z is the output of the feature vector h in the projection layer:
z＝g(h)＝W(2)σ(W(1)h)；
wherein σ represents a ReLu activation function;
s2.3: given a pair of images { x, T }, the similarity between them is defined by cosine similarity:
Figure BDA0003312357590000061
s2.4: and (3) performing back propagation by using a loss function, and optimizing network parameters:
Figure BDA0003312357590000062
wherein tau is a temperature parameter and takes a value of 0.1.
The method for recognizing the road surface arrow based on the template self-supervision as claimed in claim 3, as shown in fig. 4, wherein the concrete steps of training the network in step S3 are as follows:
s3.1: inputting the positive and negative pairs obtained by combining the step S1 into the network structure constructed in the step S2, and respectively obtaining a feature vector h by calculating the image and the template through an encoderi、hT:
hi＝f(xi)；hT＝f(xT)；
S3.2: projecting the obtained feature vectors into a mapping space for contrast loss, zi、zTAre respectively image feature vectors hiTemplate feature hTOutput at projection layer:
zi＝g(hi)＝W(2)σ(W(1)hi)；zT＝g(hT)＝W(2)σ(W(1)hT)；
s3.3: the network automatically adjusts parameters according to the back propagation of the loss function, and deep learning tool boxes such as PyTorch and Tensorflow can be used.
The method for recognizing the road surface arrow based on the template self-supervision as claimed in claim 4, wherein the trained network in the step S3 is applied to a specific classification task, and the specific steps are as follows:
s4.1: sending the arrow image to be recognized into the feature extraction network trained in the step S3, and obtaining a feature vector of the arrow image;
s4.2: calculating the similarity between the feature vector of the image and each template vector by using the similarity calculation formula in the step S2.3;
s4.3: the arrow image is identified as the arrow category having the greatest similarity.
The left side of fig. 5 is a road surface left-turn and turn-around composite arrow image which is actually shot, and after the arrow image is compared with the arrow in the template library by using the method of the present invention, the left-turn and turn-around composite arrow in the template library is successfully paired.
Example 2
The embodiment provides a road surface arrow recognition device based on template self-supervision, includes:
the training unit is used for training on a pre-constructed extended data set by using a pre-constructed self-supervision template comparison learning model containing an encoder and a projection layer to obtain a trained network, wherein the extended data set is an arrow template and an extended data set of a corresponding example sample;
and the identification unit is used for identifying the category of the arrow image by using the trained network.
Example 3
The embodiment provides a pavement arrow recognition device based on template self-supervision, which comprises a processor and a storage medium;
the storage medium is used for storing instructions;
the processor is configured to operate in accordance with the instructions to perform the steps of the method according to any one of:
training on a pre-constructed extended data set by using a pre-constructed self-supervision template contrast learning model containing an encoder and a projection layer to obtain a trained network, wherein the extended data set is an arrow template and an extended data set of a corresponding example sample;
and identifying the category of the arrow image by using the trained network.
Example 4
The present embodiment provides a computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, carries out the steps of the method of any one of the following;
training on a pre-constructed extended data set by using a pre-constructed self-supervision template contrast learning model containing an encoder and a projection layer to obtain a trained network, wherein the extended data set is an arrow template and an extended data set of a corresponding example sample;
and identifying the category of the arrow image by using the trained network.
The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.

Claims (8)

1. A pavement arrow recognition method based on template self-supervision is characterized by comprising the following steps:
training on a pre-constructed extended data set by using a pre-constructed self-supervision template contrast learning model containing an encoder and a projection layer to obtain a trained network, wherein the extended data set is an arrow template and an extended data set of a corresponding example sample;
and identifying the category of the arrow image by using the trained network.
2. The template-based self-supervision pavement arrow recognition method according to claim 1, characterized in that: the specific data expansion method adopted during the construction of the expansion data set comprises the following steps:
each template image is transformed by perspective to obtain 1200 arrow images with different angles, and the transformation comprises the following steps: azimuth angle is 0 to 360 degrees, step interval is 45 degrees, 8 types in total; the pitch angle is 0 to-70 degrees, the stepping interval is-5 degrees, and 15 kinds are adopted; the visual angle is 42 degrees to 132 degrees, the stepping interval is 10 degrees, and the total number is 10;
and forming a positive pair with the corresponding arrow template and forming a negative pair with the other arrow templates.
3. The template-based self-supervision pavement arrow recognition method according to claim 2, characterized in that: the specific network structure of the self-supervision template comparison learning model is as follows:
selecting any neural network with feature extraction capability as an encoder to extract features, and calculating an image through the encoder to obtain a feature vector:
h＝f(x)；
using a two-layer fully-connected network as a projection layer, projecting the feature map into a space for contrast loss, wherein a projection vector z is the output of a feature vector h in the projection layer:
z＝g(h)＝W(2)σ(W(1)h)；
wherein σ represents a ReLu activation function;
given a pair of images { x, T }, the similarity between them is defined by cosine similarity:
Figure FDA0003312357580000021
constructing a loss function to be optimized:
Figure FDA0003312357580000022
wherein tau is a hyper parameter and is interpreted as a temperature parameter, so that the repulsion effect between negative pairs can be enhanced, the optimization capability of a loss function is improved, and the convergence speed is accelerated.
4. The template-based self-supervision pavement arrow recognition method of claim 3, characterized in that: the training on the pre-constructed augmented data set using a pre-constructed self-supervised template contrast learning model comprising an encoder and a projection layer comprises:
inputting the positive pair and the negative pair obtained by combination into a pre-constructed self-supervision template comparison learning model, and respectively obtaining a feature vector h by calculating the image and the template through an encoderi、hT：
hi＝f(xi)；hT＝f(xT)；
Projecting the obtained feature vectors into a mapping space for contrast loss, zi、zTAre respectively image feature vectors hiTemplate feature hTOutput at projection layer:
zi＝g(hi)＝W(2)σ(W(1)hi)；zT＝g(hT)＝W(2)σ(W(1)hT)；
the network automatically adjusts the parameters based on the back propagation of the loss function.
5. The template-based self-supervision pavement arrow recognition method of claim 4, characterized in that: the identifying the category of the arrow image by using the trained network comprises:
sending the arrow image to be recognized into a trained feature extraction network, and obtaining a feature vector of the arrow image;
calculating the similarity between the feature vector of the image and each template vector by using a similarity calculation formula;
the arrow image is identified as the arrow category having the greatest similarity.
6. A road surface arrow head recognition device based on template self-supervision, characterized by that includes:
the training unit is used for training on a pre-constructed extended data set by using a pre-constructed self-supervision template comparison learning model containing an encoder and a projection layer to obtain a trained network, wherein the extended data set is an arrow template and an extended data set of a corresponding example sample;
and the identification unit is used for identifying the category of the arrow image by using the trained network.
7. The utility model provides a road surface arrow point recognition device based on template is from supervision which characterized in that: comprising a processor and a storage medium;
the storage medium is used for storing instructions;
the processor is configured to operate in accordance with the instructions to perform the steps of the method according to any one of claims 1 to 5.
8. A computer-readable storage medium having stored thereon a computer program, characterized in that: the program when executed by a processor implements the steps of the method of any one of claims 1 to 5.
CN202111220342.0A 2021-10-20 2021-10-20 Pavement arrow recognition method and device based on template self-supervision Pending CN113947763A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111220342.0A CN113947763A (en) 2021-10-20 2021-10-20 Pavement arrow recognition method and device based on template self-supervision

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111220342.0A CN113947763A (en) 2021-10-20 2021-10-20 Pavement arrow recognition method and device based on template self-supervision

Publications (1)

Publication Number Publication Date
CN113947763A true CN113947763A (en) 2022-01-18

Family

ID=79331753

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111220342.0A Pending CN113947763A (en) 2021-10-20 2021-10-20 Pavement arrow recognition method and device based on template self-supervision

Country Status (1)

Country Link
CN (1) CN113947763A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115240036A (en) * 2022-09-22 2022-10-25 武汉珈鹰智能科技有限公司 Training method, application method and storage medium of crack image recognition network

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115240036A (en) * 2022-09-22 2022-10-25 武汉珈鹰智能科技有限公司 Training method, application method and storage medium of crack image recognition network

Similar Documents

Publication Publication Date Title
WO2020186678A1 (en) Three-dimensional map constructing method and apparatus for unmanned aerial vehicle, computer device, and storage medium
CN109359526B (en) Human face posture estimation method, device and equipment
Vazquez et al. Virtual and real world adaptation for pedestrian detection
CN110135249B (en) Human behavior identification method based on time attention mechanism and LSTM (least Square TM)
CN110728209A (en) Gesture recognition method and device, electronic equipment and storage medium
CN107423760A (en) Based on pre-segmentation and the deep learning object detection method returned
CN107808129A (en) A kind of facial multi-characteristic points localization method based on single convolutional neural networks
WO2020062433A1 (en) Neural network model training method and method for detecting universal grounding wire
CN105740899A (en) Machine vision image characteristic point detection and matching combination optimization method
CN113361636B (en) Image classification method, system, medium and electronic device
CN106599810B (en) A kind of head pose estimation method encoded certainly based on stack
CN111178170B (en) Gesture recognition method and electronic equipment
CN113643329B (en) Twin attention network-based online update target tracking method and system
WO2015165227A1 (en) Human face recognition method
CN104751463B (en) A kind of threedimensional model optimal viewing angle choosing method based on sketch outline feature
CN111881716A (en) Pedestrian re-identification method based on multi-view-angle generation countermeasure network
CN105760879A (en) Fourier-Mellin transform-based image geometric matching method
CN113393524A (en) Target pose estimation method combining deep learning and contour point cloud reconstruction
CN109920050A (en) A kind of single-view three-dimensional flame method for reconstructing based on deep learning and thin plate spline
CN115497122A (en) Method, device and equipment for re-identifying blocked pedestrian and computer-storable medium
CN107330363B (en) Rapid internet billboard detection method
CN113947763A (en) Pavement arrow recognition method and device based on template self-supervision
CN106203446B (en) Three dimensional object recognition positioning method for augmented reality auxiliary maintaining system
CN107644203A (en) A kind of feature point detecting method of form adaptive classification
CN112668662B (en) Outdoor mountain forest environment target detection method based on improved YOLOv3 network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination