CN114241203B - Workpiece length measuring method and system - Google Patents
Workpiece length measuring method and system Download PDFInfo
- Publication number
- CN114241203B CN114241203B CN202210168746.8A CN202210168746A CN114241203B CN 114241203 B CN114241203 B CN 114241203B CN 202210168746 A CN202210168746 A CN 202210168746A CN 114241203 B CN114241203 B CN 114241203B
- Authority
- CN
- China
- Prior art keywords
- network
- feature map
- layer
- workpiece
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B21/00—Measuring arrangements or details thereof, where the measuring technique is not covered by the other groups of this subclass, unspecified or not relevant
- G01B21/02—Measuring arrangements or details thereof, where the measuring technique is not covered by the other groups of this subclass, unspecified or not relevant for measuring length, width, or thickness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a method and a system for measuring the length of a workpiece, and belongs to the field of workpiece dimension measurement. The method comprises the following steps: constructing a coding path; constructing a characteristic splicing path comprising an ECA module and two down-sampling modules; the activation layer of the ECA module activates the feature vector input into the activation layer by adopting a sigmoid activation function, and the element-level multiplication layer is used for multiplying the feature pattern output by the first coding network and the feature vector output by the activation layer to obtain a cross-channel interactive feature pattern; and constructing a decoding path, training the constructed image segmentation network, inputting the image of the workpiece to be detected into an image segmentation network model, performing skeleton refinement on the workpiece region in the output segmentation characteristic diagram, and determining the length of the workpiece to be detected according to the pixel parameters and the camera calibration parameters of the image of the workpiece to be detected. The invention can save labor and time cost, improve the efficiency of the whole process of measuring the length of the workpiece and improve the accuracy of the measuring result.
Description
Technical Field
The invention relates to the field of workpiece dimension measurement, in particular to a method and a system for measuring the length of a workpiece.
Background
The accurate and efficient workpiece size measurement is realized, and the method has important significance for guaranteeing the product quality and improving the production efficiency and the operation profit of a factory. In recent years, with the increasing level of industrialization, machine vision measurement technology has become one of the mainstream methods for measuring the size of a workpiece by virtue of its advantages of high precision, high speed, non-contact and the like. However, the existing machine vision measurement technology mainly depends on the traditional image processing algorithm, and a large amount of manual feature design is required for acquiring specific workpiece information in an image, so that the labor and time cost is too high, the robustness is poor, and the workpiece dimension measurement cost is high and the effect is poor.
Disclosure of Invention
The invention aims to provide a method and a system for measuring the length of a workpiece, which are used for reducing the cost of measuring the size of the workpiece and improving the measuring effect.
In order to achieve the purpose, the invention provides the following scheme:
a workpiece length measurement method comprising:
constructing a coding path of an image segmentation network; the coding path sequentially comprises a first coding network, a second coding network, a third coding network and a fourth coding network; the first coding network, the second coding network and the third coding network each comprise two convolutional layers and a pooling layer, and the pooling layer is located behind the two convolutional layers; the fourth encoding network comprises two convolutional layers;
constructing a characteristic splicing path of the image segmentation network; the characteristic splicing path sequentially comprises an ECA module and two down-sampling modules; the ECA module receives the feature map output by the first coding network, the ECA module comprises an average pooling layer, a convolution layer, an activation layer and an element-level multiplication layer, the activation layer activates the feature vector input into the activation layer by adopting a sigmoid activation function, and the element-level multiplication layer is used for multiplying the feature map output by the first coding network and the feature vector output by the activation layer to obtain a cross-channel interactive feature map; the two downsampling modules have the same structure and comprise two convolution layers and a pooling layer, and the pooling layer is positioned between the two convolution layers;
constructing a decoding path of the image segmentation network to complete the construction of the image segmentation network; the decoding path sequentially comprises: the device comprises a first decoding network, a second decoding network, a third decoding network, a channel conversion layer and a pixel conversion layer; the first decoding network, the second decoding network and the third decoding network have the same structure and respectively comprise an upsampling layer, a splicing layer and two convolutional layers; the channel conversion layer is a convolution layer, and the pixel conversion layer outputs a binary image;
training the constructed image segmentation network to obtain an image segmentation network model;
inputting the image of the workpiece to be detected into the image segmentation network model to obtain a segmentation feature map;
and thinning a framework of the workpiece region in the segmentation characteristic diagram, and determining the length of the workpiece to be detected according to the pixel parameters after framework thinning and the camera calibration parameters of the image of the workpiece to be detected.
Optionally, convolution kernels of convolutional layers in the first coding network, the second coding network, the third coding network and the fourth coding network are all 1 × 1; pooling layers in the first, second, and third coding networks are all used for maximum pooling operations.
Optionally, the constructing a feature stitching path of the image segmentation network specifically includes:
constructing the ECA module; the average pooling layer of the ECA module is used for performing global average pooling operation on the feature map output by the first coding network, and the convolution layer of the ECA module is used for performing 1-dimensional convolution operation with convolution kernel of 5 on the feature vector output by the average pooling layer;
constructing a first downsampling module and a second downsampling module; the first downsampling module is used for downsampling the cross-channel interactive feature map output by the ECA module, and the second downsampling module is used for downsampling the feature map output by the first downsampling module; the convolution kernels of the two convolution layers in the first downsampling module and the second downsampling module are both 1 x 1, and the pooling layers in the first downsampling module and the second downsampling module are used for performing maximum pooling operation.
Optionally, the convolution kernels of the two convolution layers of the first decoding network, the second decoding network and the third decoding network are all 1 x 1;
the up-sampling layer of the first decoding network is used for performing up-sampling operation on the feature map output by the fourth coding network to obtain a first up-sampling feature map; the splicing layer of the first decoding network is used for splicing the first up-sampling feature map and the feature map output by the feature splicing path to obtain a first spliced feature map; two convolution layers of the first decoding network are used for carrying out continuous convolution operation twice on the first splicing feature map;
the up-sampling layer of the second decoding network is used for performing up-sampling operation on the feature map output by the first decoding network to obtain a second up-sampling feature map; the splicing layer of the second decoding network is used for splicing the second up-sampling feature map and the feature map output by the first down-sampling module to obtain a second spliced feature map; two convolution layers of the second decoding network are used for carrying out continuous convolution operation twice on the second splicing feature map;
the up-sampling layer of the third decoding network is used for performing up-sampling operation on the feature map output by the second decoding network to obtain a third up-sampling feature map; the splicing layer of the third decoding network is used for splicing the third up-sampling feature map and the cross-channel interactive feature map output by the ECA module to obtain a third spliced feature map; two convolutional layers of the third decoding network are used for performing two consecutive convolution operations on the third splicing feature map.
Optionally, the skeleton refining is performed on the workpiece region in the segmentation feature map, and the length of the workpiece to be measured is determined according to the pixel parameter after skeleton refining and the camera calibration parameter of the image of the workpiece to be measured, which specifically includes:
performing skeleton refinement on a workpiece region in the segmentation characteristic diagram until the width of the workpiece in the segmentation characteristic diagram is a pixel point;
determining the number of pixel points in the length direction of the workpiece after skeleton refinement;
determining the length of the workpiece to be measured by using a formula D = z × D; wherein D is the length of the workpiece to be measured; z is a camera calibration parameter and represents a scale factor; d is the number of pixel points in the length direction of the workpiece after the skeleton is thinned.
The present invention also provides a workpiece length measuring system, comprising:
the encoding path construction module is used for constructing an encoding path of the image segmentation network; the coding path sequentially comprises a first coding network, a second coding network, a third coding network and a fourth coding network; the first coding network, the second coding network and the third coding network each comprise two convolutional layers and a pooling layer, and the pooling layer is located behind the two convolutional layers; the fourth encoding network comprises two convolutional layers;
the characteristic splicing path construction module is used for constructing a characteristic splicing path of the image segmentation network; the characteristic splicing path sequentially comprises an ECA module and two down-sampling modules; the ECA module receives the feature map output by the first coding network, the ECA module comprises an average pooling layer, a convolution layer, an activation layer and an element-level multiplication layer, the activation layer activates the feature vector input into the activation layer by adopting a sigmoid activation function, and the element-level multiplication layer is used for multiplying the feature map output by the first coding network and the feature vector output by the activation layer to obtain a cross-channel interactive feature map; the two downsampling modules have the same structure and comprise two convolution layers and a pooling layer, and the pooling layer is positioned between the two convolution layers;
the decoding path construction module is used for constructing a decoding path of the image segmentation network to complete construction of the image segmentation network; the decoding path sequentially comprises: the device comprises a first decoding network, a second decoding network, a third decoding network, a channel conversion layer and a pixel conversion layer; the first decoding network, the second decoding network and the third decoding network have the same structure and respectively comprise an upsampling layer, a splicing layer and two convolutional layers; the channel conversion layer is a convolution layer, and the pixel conversion layer outputs a binary image;
the training module is used for training the constructed image segmentation network to obtain an image segmentation network model;
the input module is used for inputting the image of the workpiece to be detected into the image segmentation network model to obtain a segmentation characteristic diagram;
and the length determining module of the workpiece to be detected is used for carrying out skeleton refinement on the workpiece area in the segmentation characteristic diagram and determining the length of the workpiece to be detected according to the pixel parameters after skeleton refinement and the camera calibration parameters of the image of the workpiece to be detected.
Optionally, convolution kernels of convolutional layers in the first coding network, the second coding network, the third coding network and the fourth coding network are all 1 × 1; pooling layers in the first, second, and third coding networks are all used for maximum pooling operations.
Optionally, the feature splicing path constructing module specifically includes:
an ECA module construction unit for constructing the ECA module; the average pooling layer of the ECA module is used for performing global average pooling operation on the feature map output by the first coding network, and the convolution layer of the ECA module is used for performing 1-dimensional convolution operation with convolution kernel of 5 on the feature vector output by the average pooling layer;
a downsampling module construction unit for constructing a first downsampling module and a second downsampling module; the first downsampling module is used for downsampling the cross-channel interactive feature map output by the ECA module, and the second downsampling module is used for downsampling the feature map output by the first downsampling module; the convolution kernels of the two convolution layers in the first downsampling module and the second downsampling module are both 1 x 1, and the pooling layers in the first downsampling module and the second downsampling module are used for performing maximum pooling operation.
Optionally, the convolution kernels of the two convolution layers of the first decoding network, the second decoding network and the third decoding network are all 1 x 1;
the up-sampling layer of the first decoding network is used for performing up-sampling operation on the feature map output by the fourth coding network to obtain a first up-sampling feature map; the splicing layer of the first decoding network is used for splicing the first up-sampling feature map and the feature map output by the feature splicing path to obtain a first spliced feature map; two convolution layers of the first decoding network are used for carrying out continuous convolution operation twice on the first splicing feature map;
the up-sampling layer of the second decoding network is used for performing up-sampling operation on the feature map output by the first decoding network to obtain a second up-sampling feature map; the splicing layer of the second decoding network is used for splicing the second up-sampling feature map and the feature map output by the first down-sampling module to obtain a second spliced feature map; two convolution layers of the second decoding network are used for carrying out continuous convolution operation twice on the second splicing feature map;
the up-sampling layer of the third decoding network is used for performing up-sampling operation on the feature map output by the second decoding network to obtain a third up-sampling feature map; the splicing layer of the third decoding network is used for splicing the third up-sampling feature map and the cross-channel interactive feature map output by the ECA module to obtain a third spliced feature map; two convolutional layers of the third decoding network are used for performing two consecutive convolution operations on the third splicing feature map.
Optionally, the module for determining the length of the workpiece to be measured specifically includes:
the skeleton thinning unit is used for thinning the skeleton of the workpiece region in the segmentation characteristic diagram until the width of the workpiece in the segmentation characteristic diagram is a pixel point;
the number of pixel points determining unit is used for determining the number of pixel points in the length direction of the workpiece after the skeleton is refined;
a length determination unit for determining the length of the workpiece to be measured by using the formula D = z × D; wherein D is the length of the workpiece to be measured; z is a camera calibration parameter and represents a scale factor; d is the number of pixel points in the length direction of the workpiece after the skeleton is thinned.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
according to the invention, by constructing the image segmentation network model for efficiently extracting the image features, the efficiency and precision of extracting the workpiece are improved, the labor and time cost are saved, the efficiency of the whole process of measuring the length of the workpiece is finally improved, and the accuracy of the measurement result is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a schematic flow chart of a method for measuring the length of a workpiece according to the present invention;
FIG. 2 is a schematic diagram of an image segmentation network model according to the present invention;
FIG. 3 is a schematic diagram of the ECA module of the present invention;
FIG. 4 is a schematic diagram of a workpiece length measuring system according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention aims to provide a workpiece length measuring method and a workpiece length measuring system, which can realize accurate and rapid segmentation of a workpiece in an image by constructing an image segmentation network for efficiently extracting image features, thereby improving the efficiency of measuring the size of the whole workpiece.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
Fig. 1 is a schematic flow chart of a method for measuring the length of a workpiece according to the present invention, as shown in fig. 1, including the following steps:
step 100: and constructing an encoding path of the image segmentation network. The coding path sequentially comprises a first coding network, a second coding network, a third coding network and a fourth coding network; the first coding network, the second coding network and the third coding network respectively comprise two convolution layers and a pooling layer, convolution kernels of the convolution layers are all 1 x 1, the pooling layer is used for performing maximum pooling operation, and the pooling layer is located behind the two convolution layers. Since each convolution layer is followed by one activation operation, performing two consecutive 1 x 1 convolutions of the feature map helps to introduce more non-linearity, thereby enhancing the feature expression capability of the network. And extracting the most main characteristic information from the subsequent pooling layer through characteristic dimension reduction. The fourth coding network comprises two convolutional layers.
Step 200: and constructing a characteristic splicing path of the image segmentation network. The feature stitching path includes, in order, an ECA module and two downsampling modules.
The ECA module receives a feature map output by a first coding network, the ECA module comprises an average pooling layer, a convolution layer, an active layer and an element-level multiplication layer, the average pooling layer of the ECA module is used for carrying out global average pooling operation on the feature map output by the first coding network, the convolution layer is used for carrying out 1-dimensional convolution operation with convolution kernel of 5 on the feature vector output by the average pooling layer, the active layer adopts a sigmoid active function to activate the feature vector input into the active layer, and the element-level multiplication layer is used for multiplying the feature map output by the first coding network and the feature vector output by the active layer to obtain a cross-channel interactive feature map.
The two down-sampling modules have the same structure and respectively comprise two convolution layers and a pooling layer, convolution kernels of the convolution layers are all 1 x 1, the pooling layer is located between the two convolution layers, and the pooling layer is used for performing maximum pooling operation. The two down-sampling modules are respectively a first down-sampling module and a second down-sampling module, the first down-sampling module performs down-sampling operation on the cross-channel interactive feature map output by the ECA module, and the second down-sampling module is used for performing down-sampling operation on the feature map output by the first down-sampling module.
Step 300: and constructing a decoding path of the image segmentation network. The decoding path comprises in sequence: the device comprises a first decoding network, a second decoding network, a third decoding network, a channel conversion layer and a pixel conversion layer; the first decoding network, the second decoding network and the third decoding network have the same structure and respectively comprise an upsampling layer, a splicing layer and two convolution layers, and convolution kernels of the convolution layers are all 1 x 1.
The up-sampling layer of the first decoding network is used for performing up-sampling operation on the feature map output by the fourth coding network to obtain a first up-sampling feature map; the splicing layer of the first decoding network is used for splicing the first up-sampling feature map and the feature map output by the feature splicing path to obtain a first splicing feature map; two convolutional layers of the first decoding network are used to perform two consecutive convolutional operations on the first stitched feature map.
The up-sampling layer of the second decoding network is used for performing up-sampling operation on the feature map output by the first decoding network to obtain a second up-sampling feature map; the splicing layer of the second decoding network is used for splicing the second up-sampling feature map and the feature map output by the first down-sampling module to obtain a second spliced feature map; the two convolution layers of the second decoding network are used to perform two successive convolution operations on the second stitched feature map.
The up-sampling layer of the third decoding network is used for performing up-sampling operation on the feature map output by the second decoding network to obtain a third up-sampling feature map; the splicing layer of the third decoding network is used for splicing the third up-sampling feature map and the cross-channel interactive feature map output by the ECA module to obtain a third spliced feature map; the two convolutional layers of the third decoding network are used to perform two consecutive convolution operations on the third stitched feature map.
Since the purpose of the image segmentation network is to separate the artifacts from the background, the third decoding network is input into the last convolutional layer, and the number of channels thereof is converted into 2 (representing the artifacts and the background, respectively), so as to obtain a feature map G5 with the size of 1024 × 1024 × 2.
The channel conversion layer is a convolutional layer, and the feature map output by the third decoding network is input to the channel conversion layer and converted into a multi-channel feature map, so that the workpiece can be distinguished from the background.
The pixel conversion layer converts the image output by the channel conversion layer into a binary image and outputs the binary image, wherein the pixel point value of which the pixel value is greater than the threshold value is set as 1, namely the foreground, and the pixel point value of which the pixel value is less than the threshold value is set as 0, namely the background.
The image segmentation network is constructed in the steps 100 to 300, and in order to avoid that the accuracy of the workpiece segmentation effect is affected by excessive loss of detail information, the invention only performs down-sampling operation three times in the encoding path. As shown in fig. 2, an example of inputting a workpiece image a1 with dimensions 1024 × 1024 × 1 to be detected is taken. The above process is further explained.
(1) An encoding path: inputting a1 into a convolution layer with convolution kernel of 1 × 1 to obtain a feature map a2 with size of 1024 × 1024 × 64, inputting a2 into a convolution layer with convolution kernel of 1 × 1 to obtain a feature map A3 with size of 1024 × 1024 × 64, and performing maximum pooling operation on A3 to obtain a feature map B1 with size of 512 × 512 × 64.
The encoding path of B1, namely the path of the first encoding network, is obtained from A1, B1 is input into the second encoding network to obtain C1, and C1 is input into the third encoding network to obtain D1. The coding paths of the three coding networks are consistent, namely, the amplification of the channel dimension is realized by carrying out continuous convolution operation twice, and then the dimension reduction of the spatial characteristics is carried out by carrying out pooling operation. Finally, the fourth coding network is used to perform convolution operation on the D1 for two more times, and a feature map D3 with the size of 128 × 128 × 512 is obtained.
(2) Characteristic splicing path: because the continuous feature map size amplification in the subsequent decoding path can result in lower image resolution, the invention adopts the feature splicing path to introduce more fine-grained surface information into the feature map in the decoding path, thereby making up the semantic and resolution difference between the low-level feature and the high-level feature and generating the high-quality decoding feature map.
An Efficient Channel Attention (ECA) module can perform feature information interaction between channels in a lightweight manner, so as to extract effective Channel Attention features from a feature map. As shown in fig. 3, a feature map A3 with a size of 1024 × 64 is used as an input of the ECA module, and a global average pooling operation is performed on A3 to obtain a feature vector with a size of 1 × 64. And then carrying out 1-dimensional convolution operation with convolution kernel of 5 on the feature vector to complete cross-channel feature information interaction. And then, activating the same by using a sigmoid activation function to obtain activated 1 x 64 feature vectors. And then, element-level multiplication is carried out on the original feature map A3 and the feature vector to obtain a final cross-channel interactive feature map G1. The feature map G1 at this time not only contains rich fine-grained information, but also extracts sufficient channel attention features through the ECA module.
Next, downsampling is performed based on the feature map G1. The feature map G1 is input into a down-sampling module, the number of channels is expanded from 64 to 128 by a convolution layer with convolution kernel 1 × 1, and then the spatial dimension is compressed from 1024 to 512 by a layer of maximum pooling operation, thereby obtaining a feature map F1 with a size of 512 × 128. The spatial dimension reduction of the feature map is realized through a structure of convolution-pooling-convolution. Finally, the feature map F1 is input into the next down-sampling module to obtain a feature map E1 with a size of 256 × 256.
As shown in fig. 2, the feature stitching path passes through the ECA module and the downsampling module to transfer the fine-grained information enriched in the low-level feature map a3 to the high-level feature map.
(3) Decoding path: inputting the feature map D3 with the size of 128 × 128 × 512 output by the encoding path into an upsampling layer to obtain a feature map E2 with the size of 256 × 256 × 256, and performing feature splicing operation of channel dimension on the E2 and the E1 output by the feature splicing path to obtain a feature map with the number of channels being 2 times of the original number, namely 256 × 256 × 512. The feature map is input into a convolution layer with convolution kernel 1 × 1, and the number of channels is compressed from 512 to 256, so as to obtain E3. And inputting the E3 into a convolution layer with convolution kernel of 1 × 1 to obtain an equal-size characteristic graph E4.
The above is the process of obtaining E4 from D3 by using the first decoding network, and the decoding manner from E4 to F4 by using the second decoding network and from F4 to G4 by using the third decoding network is similar, that is, the upsampling operation is performed first, then the feature concatenation is performed with F1 or G1, and then the further feature extraction is performed by the convolutional layer.
Since the objective of this segmentation task is to separate the workpiece from the background, G4 is input into the last convolutional layer, i.e., the channel conversion layer, and the number of channels is converted into 2 (representing the workpiece and the background, respectively), so as to obtain a feature map G5 with dimensions 1024 × 1024 × 2.
Finally, the feature map G5 is input to the pixel conversion layer, and the channel image is converted into a binary map and output, where the pixel point values with pixel values greater than the threshold are set to 1, i.e., foreground, and the pixel point values with pixel values less than the threshold are set to 0, i.e., background.
Step 400: and training the constructed image segmentation network to obtain an image segmentation network model. Specifically, the method comprises the steps of establishing a workpiece segmentation data set and training. When a workpiece segmentation data set is established, firstly, an industrial camera is adopted to continuously shoot workpiece images on a production line, so that the optical axis of the camera is always vertical to a shooting plane and is kept unchanged, and in consideration of poor network edge area prediction effect possibly caused by convolution operation, a workpiece target is ensured to be positioned in the central area of the image in the shooting process; and then, labeling by adopting labeling software, and labeling the region of each workpiece in each workpiece image so as to obtain a labeling file corresponding to each image. And finally, based on the workpiece image and the corresponding label file, obtaining a workpiece segmentation data set, and dividing the workpiece segmentation data set into a training set and a test set.
During training, the image segmentation network is trained by adopting a training set in a workpiece segmentation data set, and network parameters are updated based on a cross entropy loss function until the segmentation precision meeting the preset requirement is obtained on a test set, so that a final image segmentation network model is obtained.
Step 500: and inputting the image of the workpiece to be detected into the image segmentation network model to obtain a segmentation characteristic diagram.
Step 600: and thinning a framework of the workpiece area in the segmentation characteristic diagram, and determining the length of the workpiece to be detected according to the pixel parameter after the framework is thinned and the camera calibration parameter of the image of the workpiece to be detected. The invention adopts morphological operationsAnd performing skeleton refinement on a workpiece region in the workpiece binary image until the width of the workpiece is one pixel point, then calculating the number D of the pixel points in the length direction of the workpiece, and obtaining the length D of the workpiece according to a formula D = z x D, wherein z is a scale factor obtained by calibrating a camera. Regarding the camera calibration, since the optical axis of the industrial camera is always perpendicular to the plane of the workpiece in the practical application process, and the relative position of the camera and the plane is kept fixed, the camera calibration is performed by adopting a scale factor method to determine the corresponding relation between the pixels and the actual size of the workpiece. The scale factor z is calculated as:
wherein M is the distance from the industrial camera to the plane of the workpiece,fis the focal length of the lens of the camera, and d is the number of pixels of the workpiece length on the image.
The feature splicing path in the image segmentation network can introduce more fine-grained information into the feature map in the decoding path, so that the resolution difference between the high-level feature and the low-level feature map is made up, and the segmentation accuracy of the network on the workpiece target is effectively improved. Therefore, the method can automatically separate the workpiece target from the image, quickly and accurately output the length measurement result of the workpiece, has the advantages of high precision, high speed, non-contact, easy deployment and the like, can obviously improve the detection efficiency of a production line, and ensures the product quality.
Corresponding to the above method, the present invention further provides a workpiece length measuring system, and fig. 4 is a schematic structural diagram of the workpiece length measuring system of the present invention. As shown in fig. 4, the workpiece length measuring system of the present invention comprises:
an encoding path constructing module 401, configured to construct an encoding path of an image segmentation network; the coding path sequentially comprises a first coding network, a second coding network, a third coding network and a fourth coding network; the first coding network, the second coding network and the third coding network respectively comprise two convolution layers and a pooling layer, and the pooling layer is positioned behind the two convolution layers; the fourth coding network comprises two convolutional layers.
A feature stitching path constructing module 402, configured to construct a feature stitching path of the image segmentation network; the characteristic splicing path sequentially comprises an ECA module and two down-sampling modules; the ECA module receives a feature map output by a first coding network, the ECA module comprises an average pooling layer, a convolution layer, an activation layer and an element-level multiplication layer, the activation layer adopts a sigmoid activation function to activate feature vectors input into the activation layer, and the element-level multiplication layer is used for multiplying the feature map output by the first coding network and the feature vectors output by the activation layer to obtain a cross-channel interactive feature map; the two downsampling modules have the same structure and comprise two convolution layers and a pooling layer, and the pooling layer is positioned between the two convolution layers.
A decoding path constructing module 403, configured to construct a decoding path of the image segmentation network, so as to complete construction of the image segmentation network; the decoding path comprises in sequence: the device comprises a first decoding network, a second decoding network, a third decoding network, a channel conversion layer and a pixel conversion layer; the first decoding network, the second decoding network and the third decoding network have the same structure and respectively comprise an upsampling layer, a splicing layer and two convolution layers; the channel conversion layer is a convolution layer, and the pixel conversion layer outputs a binary image.
And the training module 404 is configured to train the constructed image segmentation network to obtain an image segmentation network model.
And the input module 405 is configured to input the image of the workpiece to be detected into the image segmentation network model to obtain a segmentation feature map.
And the length determining module 406 of the workpiece to be detected is used for performing skeleton refinement on the workpiece region in the segmentation characteristic diagram, and determining the length of the workpiece to be detected according to the pixel parameters after skeleton refinement and the camera calibration parameters of the image of the workpiece to be detected.
As a specific embodiment, the feature splicing path constructing module 402 in the workpiece length measuring system of the present invention specifically includes:
the ECA module construction unit is used for constructing an ECA module; the average pooling layer of the ECA module is used for carrying out global average pooling operation on the feature map output by the first coding network, and the convolution layer of the ECA module is used for carrying out 1-dimensional convolution operation with convolution kernel of 5 on the feature vector output by the average pooling layer.
A downsampling module construction unit for constructing a first downsampling module and a second downsampling module; the first downsampling module is used for downsampling the cross-channel interactive feature map output by the ECA module, and the second downsampling module is used for downsampling the feature map output by the first downsampling module; convolution kernels of two convolution layers in the first downsampling module and the second downsampling module are both 1 x 1, and the pooling layers in the first downsampling module and the second downsampling module are used for performing maximum pooling operation.
As a specific embodiment, the module 406 for determining the length of the workpiece to be measured in the workpiece length measuring system of the present invention specifically includes:
and the skeleton thinning unit is used for thinning the skeleton of the workpiece region in the segmentation characteristic diagram until the width of the workpiece in the segmentation characteristic diagram is one pixel point.
And the pixel number determining unit is used for determining the number of pixels in the length direction of the workpiece after the framework is refined.
A length determination unit for determining the length of the workpiece to be measured by using the formula D = z × D; wherein D is the length of the workpiece to be measured; z is a camera calibration parameter and represents a scale factor; d is the number of pixel points in the length direction of the workpiece after the skeleton is thinned.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.
The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.
Claims (6)
1. A method of measuring the length of a workpiece, comprising:
constructing a coding path of an image segmentation network; the coding path sequentially comprises a first coding network, a second coding network, a third coding network and a fourth coding network; the first coding network, the second coding network and the third coding network each comprise two convolutional layers and a pooling layer, and the pooling layer is located behind the two convolutional layers; the fourth encoding network comprises two convolutional layers;
constructing a characteristic splicing path of the image segmentation network; the characteristic splicing path sequentially comprises an ECA module and two down-sampling modules; the ECA module receives the feature map output by the first coding network, the ECA module comprises an average pooling layer, a convolution layer, an activation layer and an element-level multiplication layer, the activation layer activates the feature vector input into the activation layer by adopting a sigmoid activation function, and the element-level multiplication layer is used for multiplying the feature map output by the first coding network and the feature vector output by the activation layer to obtain a cross-channel interactive feature map; the two downsampling modules have the same structure and comprise two convolution layers and a pooling layer, and the pooling layer is positioned between the two convolution layers;
the constructing of the feature splicing path of the image segmentation network specifically includes:
constructing the ECA module; the average pooling layer of the ECA module is used for performing global average pooling operation on the feature map output by the first coding network, and the convolution layer of the ECA module is used for performing 1-dimensional convolution operation with convolution kernel of 5 on the feature vector output by the average pooling layer;
constructing a first downsampling module and a second downsampling module; the first downsampling module is used for downsampling the cross-channel interactive feature map output by the ECA module, and the second downsampling module is used for downsampling the feature map output by the first downsampling module; convolution kernels of two convolution layers in the first downsampling module and the second downsampling module are both 1 x 1, and pooling layers in the first downsampling module and the second downsampling module are used for performing maximum pooling operation;
constructing a decoding path of the image segmentation network to complete the construction of the image segmentation network; the decoding path sequentially comprises: the device comprises a first decoding network, a second decoding network, a third decoding network, a channel conversion layer and a pixel conversion layer; the first decoding network, the second decoding network and the third decoding network have the same structure and respectively comprise an upsampling layer, a splicing layer and two convolutional layers; the channel conversion layer is a convolution layer, and the pixel conversion layer outputs a binary image;
the convolution kernels of the two convolution layers of the first decoding network, the second decoding network and the third decoding network are all 1 x 1;
the up-sampling layer of the first decoding network is used for performing up-sampling operation on the feature map output by the fourth encoding network to obtain a first up-sampling feature map; the splicing layer of the first decoding network is used for splicing the first up-sampling feature map and the feature map output by the feature splicing path to obtain a first spliced feature map; two convolution layers of the first decoding network are used for carrying out continuous convolution operation twice on the first splicing feature map;
the up-sampling layer of the second decoding network is used for performing up-sampling operation on the feature map output by the first decoding network to obtain a second up-sampling feature map; the splicing layer of the second decoding network is used for splicing the second up-sampling feature map and the feature map output by the first down-sampling module to obtain a second spliced feature map; two convolution layers of the second decoding network are used for carrying out continuous convolution operation twice on the second splicing feature map;
the up-sampling layer of the third decoding network is used for performing up-sampling operation on the feature map output by the second decoding network to obtain a third up-sampling feature map; the splicing layer of the third decoding network is used for splicing the third up-sampling feature map and the cross-channel interactive feature map output by the ECA module to obtain a third spliced feature map; two convolution layers of the third decoding network are used for carrying out continuous convolution operation twice on the third splicing feature map;
training the constructed image segmentation network to obtain an image segmentation network model;
inputting the image of the workpiece to be detected into the image segmentation network model to obtain a segmentation feature map;
and thinning a framework of the workpiece region in the segmentation characteristic diagram, and determining the length of the workpiece to be detected according to the pixel parameters after framework thinning and the camera calibration parameters of the image of the workpiece to be detected.
2. The method of claim 1, wherein the convolution kernels of the convolution layers in the first, second, third and fourth coding networks are all 1 x 1; pooling layers in the first, second, and third coding networks are all used for maximum pooling operations.
3. The method for measuring the length of the workpiece according to claim 1, wherein the skeleton refinement of the workpiece region in the segmentation feature map is performed, and the length of the workpiece to be measured is determined according to the pixel parameters after the skeleton refinement and the camera calibration parameters of the image of the workpiece to be measured, specifically comprising:
performing skeleton refinement on a workpiece region in the segmentation characteristic diagram until the width of the workpiece in the segmentation characteristic diagram is a pixel point;
determining the number of pixel points in the length direction of the workpiece after skeleton refinement;
determining the length of the workpiece to be measured by using a formula D = z × D; wherein D is the length of the workpiece to be measured; z is a camera calibration parameter and represents a scale factor; d is the number of pixel points in the length direction of the workpiece after the skeleton is thinned.
4. A workpiece length measurement system, comprising:
the encoding path construction module is used for constructing an encoding path of the image segmentation network; the coding path sequentially comprises a first coding network, a second coding network, a third coding network and a fourth coding network; the first coding network, the second coding network and the third coding network each comprise two convolutional layers and a pooling layer, and the pooling layer is located behind the two convolutional layers; the fourth encoding network comprises two convolutional layers;
the characteristic splicing path construction module is used for constructing a characteristic splicing path of the image segmentation network; the characteristic splicing path sequentially comprises an ECA module and two down-sampling modules; the ECA module receives the feature map output by the first coding network, the ECA module comprises an average pooling layer, a convolution layer, an activation layer and an element-level multiplication layer, the activation layer activates the feature vector input into the activation layer by adopting a sigmoid activation function, and the element-level multiplication layer is used for multiplying the feature map output by the first coding network and the feature vector output by the activation layer to obtain a cross-channel interactive feature map; the two downsampling modules have the same structure and comprise two convolution layers and a pooling layer, and the pooling layer is positioned between the two convolution layers;
the feature splicing path building module specifically includes:
an ECA module construction unit for constructing the ECA module; the average pooling layer of the ECA module is used for performing global average pooling operation on the feature map output by the first coding network, and the convolution layer of the ECA module is used for performing 1-dimensional convolution operation with convolution kernel of 5 on the feature vector output by the average pooling layer;
a downsampling module construction unit for constructing a first downsampling module and a second downsampling module; the first downsampling module is used for downsampling the cross-channel interactive feature map output by the ECA module, and the second downsampling module is used for downsampling the feature map output by the first downsampling module; convolution kernels of two convolution layers in the first downsampling module and the second downsampling module are both 1 x 1, and pooling layers in the first downsampling module and the second downsampling module are used for performing maximum pooling operation;
the decoding path construction module is used for constructing a decoding path of the image segmentation network to complete construction of the image segmentation network; the decoding path sequentially comprises: the device comprises a first decoding network, a second decoding network, a third decoding network, a channel conversion layer and a pixel conversion layer; the first decoding network, the second decoding network and the third decoding network have the same structure and respectively comprise an upsampling layer, a splicing layer and two convolutional layers; the channel conversion layer is a convolution layer, and the pixel conversion layer outputs a binary image;
the convolution kernels of the two convolution layers of the first decoding network, the second decoding network and the third decoding network are all 1 x 1;
the up-sampling layer of the first decoding network is used for performing up-sampling operation on the feature map output by the fourth coding network to obtain a first up-sampling feature map; the splicing layer of the first decoding network is used for splicing the first up-sampling feature map and the feature map output by the feature splicing path to obtain a first spliced feature map; two convolution layers of the first decoding network are used for carrying out continuous convolution operation twice on the first splicing feature map;
the up-sampling layer of the second decoding network is used for performing up-sampling operation on the feature map output by the first decoding network to obtain a second up-sampling feature map; the splicing layer of the second decoding network is used for splicing the second up-sampling feature map and the feature map output by the first down-sampling module to obtain a second spliced feature map; two convolution layers of the second decoding network are used for carrying out continuous convolution operation twice on the second splicing feature map;
the up-sampling layer of the third decoding network is used for performing up-sampling operation on the feature map output by the second decoding network to obtain a third up-sampling feature map; the splicing layer of the third decoding network is used for splicing the third up-sampling feature map and the cross-channel interactive feature map output by the ECA module to obtain a third spliced feature map; two convolution layers of the third decoding network are used for carrying out continuous convolution operation twice on the third splicing feature map;
the training module is used for training the constructed image segmentation network to obtain an image segmentation network model;
the input module is used for inputting the image of the workpiece to be detected into the image segmentation network model to obtain a segmentation characteristic diagram;
and the length determining module of the workpiece to be detected is used for carrying out skeleton refinement on the workpiece area in the segmentation characteristic diagram and determining the length of the workpiece to be detected according to the pixel parameters after skeleton refinement and the camera calibration parameters of the image of the workpiece to be detected.
5. The workpiece length measurement system of claim 4, wherein convolution kernels of convolution layers in the first, second, third and fourth coding networks are all 1 x 1; pooling layers in the first, second, and third coding networks are all used for maximum pooling operations.
6. The workpiece length measuring system of claim 4, wherein the workpiece length to be measured determining module specifically comprises:
the skeleton thinning unit is used for thinning the skeleton of the workpiece region in the segmentation characteristic diagram until the width of the workpiece in the segmentation characteristic diagram is a pixel point;
the number of pixel points determining unit is used for determining the number of pixel points in the length direction of the workpiece after the skeleton is refined;
a length determination unit for determining the length of the workpiece to be measured by using the formula D = z × D; wherein D is the length of the workpiece to be measured; z is a camera calibration parameter and represents a scale factor; d is the number of pixel points in the length direction of the workpiece after the skeleton is thinned.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210168746.8A CN114241203B (en) | 2022-02-24 | 2022-02-24 | Workpiece length measuring method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210168746.8A CN114241203B (en) | 2022-02-24 | 2022-02-24 | Workpiece length measuring method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114241203A CN114241203A (en) | 2022-03-25 |
| CN114241203B true CN114241203B (en) | 2022-05-13 |
Family
ID=80747846
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210168746.8A Active CN114241203B (en) | 2022-02-24 | 2022-02-24 | Workpiece length measuring method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114241203B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113658147A (en) * | 2021-08-23 | 2021-11-16 | 宁波棱镜空间智能科技有限公司 | Workpiece size measuring device and method based on deep learning |
| CN113947609A (en) * | 2021-10-12 | 2022-01-18 | 中南林业科技大学 | Deep learning network structure and multi-label aortic dissection CT image segmentation method |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11270447B2 (en) * | 2020-02-10 | 2022-03-08 | Hong Kong Applied Science And Technology Institute Company Limited | Method for image segmentation using CNN |
| CN113327254A (en) * | 2021-05-27 | 2021-08-31 | 北京深睿博联科技有限责任公司 | Image segmentation method and system based on U-type network |
| CN113240683B (en) * | 2021-06-08 | 2022-09-20 | 北京航空航天大学 | Attention mechanism-based lightweight semantic segmentation model construction method |
-
2022
- 2022-02-24 CN CN202210168746.8A patent/CN114241203B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113658147A (en) * | 2021-08-23 | 2021-11-16 | 宁波棱镜空间智能科技有限公司 | Workpiece size measuring device and method based on deep learning |
| CN113947609A (en) * | 2021-10-12 | 2022-01-18 | 中南林业科技大学 | Deep learning network structure and multi-label aortic dissection CT image segmentation method |
Non-Patent Citations (2)
| Title |
|---|
| ECAU-Net: Efficient channel attention U-Net for fetal ultrasound cerebellum segmentation;Xin Shu et al.;《Biomedical Signal Processing and Control》;20220211;全文 * |
| 基于通道注意力机制和U-net的医学图像分割方法;王枫 等;《计算机时代》;20210514;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114241203A (en) | 2022-03-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109829891B (en) | Magnetic shoe surface defect detection method based on dense generation of antagonistic neural network | |
| CN111768388B (en) | Product surface defect detection method and system based on positive sample reference | |
| CN111223088A (en) | Casting surface defect identification method based on deep convolutional neural network | |
| CN111950453A (en) | Optional-shape text recognition method based on selective attention mechanism | |
| CN111861880B (en) | Image super-fusion method based on regional information enhancement and block self-attention | |
| CN112861729A (en) | Real-time depth completion method based on pseudo-depth map guidance | |
| CN110827312B (en) | Learning method based on cooperative visual attention neural network | |
| CN112464912B (en) | Robot end face detection method based on YOLO-RGGNet | |
| CN110598698B (en) | Natural scene text detection method and system based on adaptive regional suggestion network | |
| CN109657538B (en) | Scene segmentation method and system based on context information guidance | |
| CN110310305B (en) | Target tracking method and device based on BSSD detection and Kalman filtering | |
| CN116778288A (en) | Multi-mode fusion target detection system and method | |
| CN111368637B (en) | Transfer robot target identification method based on multi-mask convolutional neural network | |
| CN112529931A (en) | Foreground segmentation method and system | |
| CN115797808A (en) | Unmanned aerial vehicle inspection defect image identification method, system, device and medium | |
| CN116503760A (en) | Unmanned aerial vehicle cruising detection method based on self-adaptive edge feature semantic segmentation | |
| CN114037684B (en) | Defect detection method based on yolov and attention mechanism model | |
| CN109919215B (en) | Target detection method for improving characteristic pyramid network based on clustering algorithm | |
| CN115049945A (en) | Method and device for extracting lodging area of wheat based on unmanned aerial vehicle image | |
| CN112070181B (en) | Image stream-based cooperative detection method and device and storage medium | |
| CN111881914B (en) | License plate character segmentation method and system based on self-learning threshold | |
| CN114241203B (en) | Workpiece length measuring method and system | |
| CN116862883A (en) | Concrete slump detection method based on image semantic segmentation | |
| CN111881746A (en) | Face feature point positioning method and system based on information fusion | |
| CN114419078B (en) | Surface defect region segmentation method and device based on convolutional neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |