WO2021024499A1 - Reinforcing bar determination device and reinforcing bar determination method - Google Patents

Reinforcing bar determination device and reinforcing bar determination method Download PDF

Info

Publication number
WO2021024499A1
WO2021024499A1 PCT/JP2019/031535 JP2019031535W WO2021024499A1 WO 2021024499 A1 WO2021024499 A1 WO 2021024499A1 JP 2019031535 W JP2019031535 W JP 2019031535W WO 2021024499 A1 WO2021024499 A1 WO 2021024499A1
Authority
WO
WIPO (PCT)
Prior art keywords
reinforcing bar
image
sample
rebar
similarity
Prior art date
Application number
PCT/JP2019/031535
Other languages
French (fr)
Japanese (ja)
Inventor
古橋　幸人
健二 猪瀬
進 深道
Original Assignee
鹿島建設株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 鹿島建設株式会社 filed Critical 鹿島建設株式会社
Priority to PCT/JP2019/031535 priority Critical patent/WO2021024499A1/en
Priority to JP2021537556A priority patent/JP7438220B2/en
Publication of WO2021024499A1 publication Critical patent/WO2021024499A1/en

Links

Images

Classifications

    • E—FIXED CONSTRUCTIONS
    • E04—BUILDING
    • E04G—SCAFFOLDING; FORMS; SHUTTERING; BUILDING IMPLEMENTS OR AIDS, OR THEIR USE; HANDLING BUILDING MATERIALS ON THE SITE; REPAIRING, BREAKING-UP OR OTHER WORK ON EXISTING BUILDINGS
    • E04G21/00—Preparing, conveying, or working-up building materials or building elements in situ; Other devices or measures for constructional work
    • E04G21/12—Mounting of reinforcing inserts; Prestressing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Definitions

  • the present invention relates to a reinforcing bar determination device and a reinforcing bar determination method.
  • construction inspections of structures such as buildings and civil engineering structures are carried out.
  • the bar arrangement is inspected according to the bar arrangement diagram before the concrete is placed.
  • the diameter and number of used reinforcing bars, the spacing between the reinforcing bars, and the like are specified, and an inspection is performed to see if the reinforcing bars are installed according to the reinforcing bar diagram.
  • the diameter of the reinforcing bar is an important parameter for determining the structural strength, and accurate measurement is required in the bar arrangement inspection. Therefore, it is desired to provide a technique capable of identifying information on the reinforcing bar with high accuracy from the image of the reinforcing bar arrangement.
  • the reinforcing bar determination device of one aspect of the present invention associates at least one of the sample reinforcing bar image showing the reinforcing bar and the feature information extracted from the sample reinforcing bar image with the type information indicating the type of the reinforcing bar shown in the sample reinforcing bar image.
  • a storage unit that stores rebar information including information, an input unit that accepts input of a rebar image showing the rebar to be judged, and an evaluation unit that evaluates the similarity between the sample rebar image and the rebar image based on the rebar information. Based on the evaluation result of the degree of similarity, when the reinforcing bar image and the sample reinforcing bar image are similar to each other satisfying a predetermined condition, the sample reinforcing bar image and the corresponding type information are output.
  • the JIS Japanese Standards Association
  • the JIS Japanese Standards Association
  • the node spacing of the reinforcing bars varies depending on the lot, or even if the same reinforcing bar is photographed, the node spacing of the reinforcing bars detected from the image may differ depending on the imaging conditions.
  • the JIS Japanese Standards Association
  • the JIS Japanese Standards Association
  • the JIS Japanese Standards Association
  • the reinforcing bars of the node spacing may differ depending on the manufacturer.
  • there are various types of deformed reinforcing bars for example, the reinforcing bars of bamboo knots and the reinforcing bars of threaded knots may have different diameters even if the knot spacing is the same.
  • the information in which the sample reinforcing bar image is associated with the type information indicating the type of the reinforcing bar shown in the sample reinforcing bar image is stored in the storage device in advance. Then, when an image showing the reinforcing bar of the target to be determined is input in a bar arrangement inspection or the like, a reinforcing bar image is created by extracting the area where the reinforcing bar to be judged is reflected from the image, and the reinforcing bar image and the sample reinforcing bar are created. Evaluate the similarity with the image.
  • the type information associated with the sample rebar image is obtained as the rebar image. Output as the type of reinforcing bar shown in.
  • a sample rebar image similar to the rebar image to be determined is specified, and the type of the rebar reflected in the rebar image to be determined is determined accordingly. Therefore, various feature information that can be extracted from the image can be used for evaluation of the degree of similarity, and it is possible to determine the type of reinforcing bar that appears in the reinforcing bar image to be determined with high accuracy.
  • embodiments will be described in more detail.
  • FIG. 1 is a diagram illustrating photography of a rebar that has already been constructed according to the embodiment.
  • the user photographs the reinforcing bar 101 to be inspected at the construction site by using the imaging device 102.
  • the photographing device 102 may be, for example, a monocular camera or a stereo camera.
  • FIG. 2 is a diagram illustrating the reinforcing bar determination system 200 according to the embodiment.
  • the reinforcing bar determination system 200 includes, for example, an imaging device 102 and a reinforcing bar determination device 201.
  • the photographing device 102 and the reinforcing bar determination device 201 may be connected via, for example, the network 205.
  • the network 205 is, for example, a LAN (Local Area Network) and the Internet.
  • the photographing device 102 and the reinforcing bar determination device 201 may be connected by short-range wireless communication or by wire, or the photographing device 102 and the reinforcing bar determination device 201 are integrated as an integrated device. May be good.
  • the reinforcing bar determination device 201 is, for example, a computer having a calculation function, for example, a personal computer (PC), a notebook PC, and a tablet terminal.
  • the reinforcing bar determination device 201 may, for example, acquire an image of the reinforcing bar arranged by the photographing device 102, analyze the reinforcing bar arranged in the image, and collect information on the reinforcing bar used for the reinforcing bar arrangement.
  • FIG. 3 is a diagram illustrating a block configuration of the reinforcing bar determination device 201 according to the embodiment.
  • the reinforcing bar determination device 201 includes, for example, a control unit 301, a storage unit 302, a display unit 303, and a communication unit 304.
  • the control unit 301 controls, for example, each unit of the reinforcing bar determination device 201.
  • the control unit 301 operates as an input unit 311, an evaluation unit 312, an output unit 313, and a determination unit 314.
  • the storage unit 302 may store, for example, an image taken by the photographing device 102, reinforcing bar information described later, a sample reinforcing bar image, feature information extracted from the sample reinforcing bar image, and the like.
  • the display unit 303 is, for example, a display device such as a liquid crystal display.
  • the display unit 303 displays information on the display screen according to the instruction of the control unit 301.
  • the communication unit 304 communicates with the photographing device 102 according to the instruction of the control unit 301, for example. Details of each of these units and details of the information stored in the storage unit 302 will be described later.
  • FIG. 4 is a diagram for explaining the reinforcing bar determination process according to the embodiment.
  • the reinforcing bar information 401 is stored in the storage unit 302 of the reinforcing bar determination device 201.
  • the type of the reinforcing bar such as D41 and D25 indicating the reinforcing bar diameter as the type information and the sample reinforcing bar image obtained by photographing the reinforcing bar of that type are associated with each of a plurality of different types of reinforcing bars. Is registered.
  • FIG. 4 shows an image 403 showing the reinforcing bars obtained by taking a picture of the reinforcing bars already constructed by the user, for example, during a bar arrangement inspection. Then, the control unit 301 extracts, for example, an image area in which the reinforcing bar of the type to be determined is reflected from the image 403 to create the reinforcing bar image 404. Then, the control unit 301 evaluates the degree of similarity between the reinforcing bar image 404 and each sample reinforcing bar image registered in the reinforcing bar information 401.
  • the control unit 301 identifies a sample rebar image similar to the rebar image 404 to be determined based on the evaluation result of the similarity, and determines the type information associated with the sample rebar image for the rebar to be determined. Output as the type of. For example, in FIG. 4, the reinforcing bar of D41 is evaluated as the reinforcing bar having the highest degree of similarity, and it can be specified that the diameter of the reinforcing bar to be determined is D41. Since the similarity between images can be evaluated by using various feature information extracted from the images, the similarity can be evaluated by using the feature information with high evaluation accuracy, or a plurality of feature information can be evaluated. It is possible to improve the accuracy of determining the type by using it to evaluate the degree of similarity. Hereinafter, some examples of evaluation of similarity will be described.
  • evaluation example 1 information related to the node spacing and width of the reinforcing bar is collected from the image.
  • FIG. 5 is a diagram illustrating the collection of information regarding the reinforcing bar from the image according to the embodiment.
  • the control unit 301 may, for example, Fourier transform the pixel value of the image in the longitudinal direction of the reinforcing bar to acquire information on the frequency of the maximum peak excluding the DC component. It is presumed that the frequency of the maximum peak excluding the DC component among the frequency components of the pixel value in the longitudinal direction of the reinforcing bar corresponds to the node spacing.
  • control unit 301 specifies, for example, the frequency of the maximum peak excluding the DC component in the frequency spectrum Fourier transformed in the longitudinal direction of the reinforcing bar.
  • the frequency of the maximum peak may be specified, for example, as the frequency of the maximum intensity of the maximum intensity in the frequency spectrum.
  • the pixel value may be a value related to a pixel of an image, and is, for example, a luminance value and a color value.
  • the control unit 301 detects the edge of the reinforcing bar extending substantially parallel to the longitudinal direction of the reinforcing bar shown in the image.
  • the control unit 301 applies edge detection processing to the image, searches for an edge substantially parallel to the longitudinal direction of the reinforcing bar in a direction orthogonal to the longitudinal direction of the reinforcing bar, and determines the edge of the reinforcing bar. It may be detected. Edge detection can be performed using various existing methods such as a method using a Sobel filter or the like. Then, the control unit 301 may specify the distance between the edges at both ends substantially parallel to the longitudinal direction of the detected reinforcing bar as the width of the reinforcing bar reflected in the image.
  • the control unit 301 compares the information on the frequency of the specified maximum peak and the information on the distance between the edges between the rebar image to be determined and the sample rebar image, and evaluates the similarity between the two images. To do.
  • the control unit 301 uses the sum of the magnitude of the difference in the frequencies of the maximum peaks of the two images and the difference in the distance between the edges of the two images (for example, weighted addition) as an index for evaluating the similarity. May be used as. That is, in this case, the control unit 301 may evaluate that the smaller the total value, the higher the degree of similarity between the sample reinforcing bar image and the reinforcing bar image to be determined.
  • the frequency of the maximum peak extracted from the image and the distance between the edges are used as feature information, and the rebar image to be determined is similar to the sample rebar image.
  • the frequency information of a plurality of peaks is acquired from the peaks obtained by Fourier transforming the pixel values of the image in the longitudinal direction of the reinforcing bar and excluding the DC component of the frequency spectrum.
  • FIG. 5B shows an example of collecting information on a plurality of frequencies as feature information from the frequency spectrum according to the embodiment.
  • the control unit 301 may collect frequencies of peaks other than the maximum peak and use them for evaluation of similarity.
  • By performing the comparison using the information of a plurality of frequencies it is possible to compare the frequency components according to, for example, a minute difference in the shape of the nodes for each manufacturer or lot of the same manufacturer other than the node spacing. As a result, the degree of similarity between the rebar image to be determined and the sample rebar image can be evaluated more accurately.
  • a plurality of frequency components to be compared can be determined by various methods. For example, a predetermined number of peaks having the highest peak intensity may be extracted from a plurality of frequency components of the frequency spectrum obtained by Fourier transforming the pixel values of the image in the longitudinal direction of the reinforcing bar, or the low frequency side. A predetermined number of peaks having an intensity equal to or higher than a predetermined threshold may be extracted from the above. Further, the control unit 301 may compare peaks having the same peak extraction order, such as comparing peaks between the rebar image to be determined and the sample rebar image in order from the low frequency side. In another embodiment, comparisons may be performed between peaks with the closest frequencies.
  • control unit 301 may, for example, integrate the magnitude of the difference in the frequencies of the compared peaks and use the total value as an index for evaluating the similarity. That is, in this case, the control unit 301 may evaluate that the smaller the total value, the higher the degree of similarity between the sample reinforcing bar image and the reinforcing bar image to be determined.
  • the similarity may be evaluated by comparing the waveforms obtained by Fourier transforming the pixel values of the images in the longitudinal direction of the reinforcing bar between the reinforcing bar image to be determined and the sample reinforcing bar image.
  • the product moment correlation coefficient can be used to evaluate the similarity of waveforms. For example, suppose there are n sets of data (x 1 , y 1 ), (x 2 , y 2 ), ..., (x n , y n ) with x and y as a set. In this case, the correlation between the paired x and y can be calculated using the following formula of Pearson's product moment correlation coefficient r xy .
  • x and y with an overline represent the average value of x and y, respectively.
  • s x is the standard deviation of x
  • s y is the standard deviation of y
  • s xy is the covariance of x and y.
  • the intensity corresponding to the frequency of the waveform of the frequency spectrum obtained by Fourier transforming the pixel value in the longitudinal direction of the reinforcing bar reflected in the reinforcing bar image to be determined is used as a parameter expressed by y in the above equation.
  • the correlation coefficient r xy can be obtained by inputting the intensities at the same frequency as a set in the frequency spectra obtained from the two images into the above equation, and the obtained r xy is of similarity. It can be used as an index for evaluation.
  • FIG. 5C is a diagram illustrating a frequency spectrum of pixel values in the width direction.
  • a plurality of pixel values in the width direction are acquired at different positions in the longitudinal direction of the reinforcing bar in the image, and they are combined and used to generate a frequency spectrum in the width direction.
  • the composition can be executed by various methods.
  • the composition may be executed by joining the pixel values in the width direction at a plurality of positions into one, or by averaging the pixel values. It may be executed.
  • embodiments may collect features of pixel values in the width direction at one position.
  • the control unit 301 evaluates the similarity between the reinforcing bar image to be determined and the sample reinforcing bar image by using the frequency spectrum of the pixel value in the longitudinal direction of the reinforcing bar and the frequency spectrum of the pixel value in the width direction. Good.
  • the similarity of the frequency spectrum for example, as described above, the sum total value or the average value of the frequency differences of the corresponding peaks in the frequency spectrum acquired from the two images of the rebar image to be determined and the sample rebar image. Can be executed by using as an index.
  • the similarity may be evaluated using the product moment correlation coefficient with the intensity of the frequency spectrum with respect to the frequency as an input.
  • the frequency spectrum of the pixel value in the longitudinal direction and the frequency spectrum of the pixel value in the width direction may be used for one evaluation of similarity in one example.
  • a predetermined number of sample reinforcing bar images at the top are specified by evaluation of similarity using the frequency spectrum of pixel values in the longitudinal direction.
  • the similarity is evaluated individually, such as identifying the most similar sample rebar image by the similarity evaluation using the frequency spectrum of the pixel value in the width direction, and the sample rebar that is similar in multiple stages.
  • the image may be specified.
  • the waveform of the pixel value in the longitudinal direction of the reinforcing bar shown in the image may be used as it is for evaluating the similarity between the reinforcing bar image to be determined and the sample reinforcing bar image.
  • the waveform obtained from the rebar image to be determined and the sample rebar image are obtained. After aligning with the obtained waveform, the similarity may be evaluated using the waveform.
  • the evaluation of the similarity may be performed, for example, by acquiring the peak period in the waveform of the pixel value and performing comparison, or by using the product moment correlation coefficient with the pixel value with respect to the axial coordinates of the waveform as an input. May be good.
  • the degree of similarity may be evaluated by matching the local feature amount between the rebar image to be determined and the sample rebar image.
  • the local feature amount for example, various local feature amounts such as SIFT (Scale-Invariant Feature Transform) and SURF (Speeded-Up Robust Features) can be used.
  • Phase information may be collected from the results of the Fourier transform and used for evaluation of similarity. For example, it is conceivable to acquire phase information from the result of performing a two-dimensional Fourier transform on the rebar image to be determined and the sample rebar image to generate a phase image.
  • the phase image information such as the contour of the reinforcing bar is emphasized. For example, in a luminance image, the bright area of the reinforcing bar may change depending on whether the reinforcing bar is exposed to light from above or below.
  • the outer shape is emphasized in the phase image, it is not easily affected by the irradiation angle of light on the reinforcing bar at the time of photographing.
  • the evaluation of the degree of similarity between the phase image obtained from the rebar image to be determined and the phase image obtained from the sample rebar image may be performed by matching using a local feature amount.
  • phase-limited correlation method or a rotation-invariant phase-limited correlation method may be executed to evaluate the degree of similarity between the rebar image to be determined and the sample rebar image.
  • the similarity is evaluated by using the result of Fourier transform, frequency spectrum, frequency, distance between edges, waveform, phase information, local feature amount, etc. as feature information. doing. Since all of these feature information are information obtained from the image of the reinforcing bar, the similarity of the image of the reinforcing bar can be evaluated by evaluating the similarity using these feature information.
  • the evaluation of the similarity according to the embodiment is not limited to the above, and the similarity may be evaluated by using the above methods in combination or by another method.
  • the type of the reinforcing bar shown in the reinforcing bar image to be determined may be specified by using the trained model obtained by performing machine learning such as deep learning.
  • FIG. 6 is a diagram illustrating the generation of the trained model according to the embodiment. For example, a plurality of information associated with a sample reinforcing bar image showing a reinforcing bar and type information indicating the type of the reinforcing bar shown in the sample reinforcing bar image is used as learning teacher data (FIG. 6A).
  • a trained model is generated by learning the weighting of the neural network so that the type information of the reinforcing bars reflected in the sample reinforcing bar image is output from the output layer for the sample reinforcing bar image input to the input layer. It may be done (Fig. 6 (b)).
  • the control unit 301 can receive output of the reinforcing bar type information corresponding to the reinforcing bar image from the trained model (FIG. 6C). ).
  • learning may be executed using a Siamese network.
  • the Siamese network is a network suitable for calculating, for example, the distance between two images as inputs and whether the images are similar or dissimilar.
  • the machine learning method is not limited to deep learning, and a support vector machine (SVM) or the like may be used.
  • the trained model for example, when a reinforcing bar image is input to the trained model at the stage of use and the type of the reinforcing bar reflected in the reinforcing bar image is determined, the training bar image and the type information of the determination result are taught. Further learning may be performed as data.
  • the degree of similarity between the rebar image to be judged and the sample rebar image can be evaluated by using various methods.
  • a user who shoots a reinforcing bar to be judged at a construction inspection site uses shooting conditions such as a direction, a distance, and a focal length to shoot the reinforcing bar to be judged to be used when shooting a sample reinforcing bar image.
  • Shooting may be performed so as to match the conditions within a predetermined error range.
  • the control unit 301 is, for example, relative to the photographing device and the reinforcing bar reflected in the reinforcing bar image based on the information regarding the relative position and orientation of the photographing device in which the image of the reinforcing bar is captured and the reinforcing bar captured in the reinforcing bar image.
  • the image of the reinforcing bar may be normalized so that the proper position and orientation have a predetermined relationship.
  • information regarding the relative position and orientation of the photographing device that captured the image of the reinforcing bar and the reinforcing bar that appears in the reinforcing bar image may be stored in the storage unit 302.
  • image normalization first, the directing of the shooting direction will be described.
  • the control unit 301 may execute a process of converting the direction of the reinforcing bar in the image obtained by photographing the reinforcing bar used for the evaluation of the similarity to a predetermined direction.
  • the control unit 301 may change the orientation of the reinforcing bars in the image so that the reinforcing bars in the image face the optical axis of the photographing device.
  • facing each other may be, for example, a relationship of a position and orientation in which the normal direction of the plane including the reinforcing bar and the photographing direction (for example, the optical axis) of the photographing apparatus are substantially parallel.
  • the control unit 301 may execute a process of converting the direction of the reinforcing bar to be determined to be reflected in the image so that the direction of the reinforcing bar when the image is taken from the position facing the image is changed.
  • the control unit 301 may, for example, determine the normal direction of the plane including the plurality of determination target reinforcing bars and the photographing direction of the photographing device. The process of converting the image to the opposite direction may be executed so that and are substantially parallel to each other.
  • FIG. 7 is a diagram showing an exemplary face-to-face conversion.
  • FIG. 7A exemplifies a state in which a plane including a plurality of reinforcing bars to be determined is tilted with respect to the photographing direction of the photographing device 102, and the reinforcing bars are photographed in an oblique direction in the image.
  • FIG. 7B shows the state after the face-to-face conversion, so that the photographing direction of the photographing apparatus 102 is substantially parallel to the normal direction of the plane including the reinforcing bar group to be determined. The plane containing the reinforcing bars is rotated.
  • the face-to-face conversion can be executed, for example, as follows. For example, in an image taken at an angle with respect to a plane including a plurality of reinforcing bars 101 as shown in FIG. 7A, the control unit 301 designates the four corners of a rectangle formed by the reinforcing bars assembled in a grid pattern. It may be accepted from the user. Then, the control unit 301 determines the homography matrix so as to face the rectangular planes indicated by the designated four corner points, and executes the face-to-face transformation of the entire image using the determined homography matrix. You can.
  • the control unit 301 can generate three-dimensional data corresponding to the pixels by, for example, performing stereo matching between the left viewpoint image and the right viewpoint image of the stereo image. Then, the control unit 301 uses the three-dimensional data of the plane so that the normal vector of the plane including the reinforcing bar group to be determined represented by the obtained three-dimensional data and the shooting direction of the photographing device 102 are substantially parallel to each other. Identify the matrix that rotates.
  • control unit 301 identifies a homography matrix that transforms the left-viewpoint image or the right-viewpoint image of the stereo image in the opposite direction based on the obtained matrix information, and uses the specified homography matrix.
  • a face-to-face image may be generated.
  • control unit 301 may perform normalization to rotate the image so that the orientation of the reinforcing bar image to be determined and the sample reinforcing bar image in the longitudinal direction of the image are substantially parallel.
  • FIG. 8 is a diagram illustrating the rotation of the image in which the reinforcing bar is captured.
  • FIG. 8 illustrates a case where the direction of the reinforcing bar shown in the image in the longitudinal direction is rotated so as to match the vertical direction.
  • the longitudinal direction of the reinforcing bar in the sample reinforcing bar image and the reinforcing bar image to be determined can be obtained.
  • the longitudinal direction of the reflected reinforcing bar can be made substantially parallel. Thereby, the evaluation accuracy of the similarity can be improved.
  • the image size for example, it is desirable that the predetermined lengths on the reinforcing bars in the two images for evaluating the similarity are normalized so as to represent the same length of the reinforcing bars in the real space.
  • the image size can be normalized as follows, for example.
  • the control unit 301 can acquire the three-dimensional data corresponding to the pixels by stereo matching. Then, the control unit 301 can specify how much the predetermined length on the reinforcing bar in the image corresponds to the actual size in the three-dimensional data of the reinforcing bar. Therefore, for example, using these information, the control unit 301 normalizes the image size so that the predetermined length on the reinforcing bar in the image corresponds to the predetermined actual size length on the reinforcing bar in the real space. can do.
  • the image size may be normalized so that the predetermined length on the reinforcing bar shown in the image corresponds to the predetermined actual size length on the reinforcing bar in the real space.
  • the position between the photographing device and the subject used for normalization processing such as three-dimensional data, the focal length of the photographing device 102 used for photographing the image, and the information on the distance from the photographing device 102 to the subject.
  • Information about the posture may be stored in the storage unit 302.
  • the control unit 301 may, for example, have the user input the information.
  • the control unit 301 may acquire three-dimensional data and information on the shooting distance based on the data obtained by scanning the reinforcing bar with the laser scanner.
  • the control unit 301 may set a predetermined scale or the like in the vicinity of the reinforcing bar of the type to be determined to take an image, and acquire the dimensional information in the real space from the image.
  • the accuracy of evaluation of similarity can be improved by appropriately normalizing the reinforcing bar image used for evaluation of similarity.
  • the image normalization may be executed at various timings.
  • the control unit 301 may execute the normalization of the sample reinforcing bar image at the time of registering the sample reinforcing bar image in the reinforcing bar information 401, or at the time of evaluating the similarity with the image of the reinforcing bar to be determined. It may be executed at both timings.
  • the reinforcing bar image 404 is created by extracting the image area in which the reinforcing bar of the type to be determined is reflected from the image 403.
  • the creation of the reinforcing bar image 404 may be executed, for example, by having the user select an image area in which the reinforcing bar to be determined is captured, or in another embodiment, the control unit 301 automatically captures the image area in which the reinforcing bar to be determined is captured. May be detected and executed.
  • control unit 301 automatically executes the above-mentioned face-to-face conversion, normalization of the longitudinal direction of the reinforcing bar, and normalization of the image size even if it is executed in response to the input of information from the user. You may. Then, for example, the technique described in Patent Document 1 described above or the technique described in International Publication No. 2018/180442 may be used for these normalization processes.
  • FIG. 9 is a diagram illustrating the reinforcing bar information 900 according to the first embodiment.
  • the reinforcing bar information 900 is an example of the above-mentioned reinforcing bar information 401.
  • the reinforcing bar information 900 for example, a record in which the type information and the sample reinforcing bar image information are associated with each other is registered.
  • the type information includes information on the manufacturer, diameter, and section type.
  • the manufacturer is information for identifying the manufacturer of the reinforcing bar shown in the sample reinforcing bar image of the record.
  • the diameter is, for example, information indicating the diameter of the reinforcing bar shown in the sample reinforcing bar image of the record.
  • the type of knot is, for example, information indicating the type of the knot of the reinforcing bar shown in the sample reinforcing bar image of the record, and in one example, the information of the bamboo knot or the screw knot may be registered.
  • the sample rebar image information for example, information indicating the image data of the sample rebar image or the storage location of the sample rebar image in the storage unit 302 may be registered.
  • the control unit 301 can acquire information on the sample reinforcing bar image used for evaluating the similarity with the reinforcing bar image to be determined.
  • FIG. 10 is a diagram illustrating an operation flow of the type determination process according to the first embodiment.
  • the control unit 301 of the reinforcing bar determination device 201 may, for example, accept the input of the reinforcing bar image of the type of determination target, and when the input is input, start the operation flow of FIG.
  • the rebar image to be determined may be input to the control unit 301 from the photographing device 102, or may be input to the control unit 301 by reading the rebar image designated by the user from the storage unit 302. ..
  • step 1001 (hereinafter, step is referred to as "S" and is referred to as S1001)
  • the control unit 301 of the reinforcing bar determination device 201 refers to the reinforcing bar information 900 and reads out a plurality of sample reinforcing bar images.
  • the control unit 301 may read out a sample reinforcing bar image corresponding to all the records registered in the reinforcing bar information 900.
  • control unit 301 evaluates the degree of similarity between the input reinforcing bar image to be determined and the read-out sample reinforcing bar images.
  • the evaluation of the similarity of the images can be performed by using various methods, and the control unit 301 evaluates the similarity by using, for example, the methods of the above-mentioned evaluation examples 1 to 7 or in combination. You can.
  • the control unit 301 identifies a similar sample rebar image that satisfies a predetermined condition from the plurality of sample rebar images read out based on the evaluation result of the similarity.
  • the predetermined condition may be, in one example, being evaluated as being the most similar among the read sample rebar images based on the evaluation result of the similarity. Further, in another embodiment, for example, when the higher the index used for the evaluation of similarity indicates that it is similar, the predetermined condition is that the index used for evaluating the similarity is equal to or higher than a predetermined threshold value. It may be.
  • the predetermined condition is that the index used for evaluating the similarity is equal to or less than a predetermined threshold value.
  • the indexes used for the similar evaluation are, for example, the total value of the differences described in the evaluation example 1, the total value obtained by integrating the magnitudes of the differences described in the evaluation example 2, the product moment correlation coefficient, and the matching of the local feature amount.
  • the control unit 301 specifies the type information associated with the sample reinforcing bar image that satisfies the predetermined conditions and is similar to the reinforcing bar information 401. Then, in S1005, the control unit 301 outputs the specified type information, and this operation flow ends. It should be noted that, for example, it is assumed that there are a plurality of sample rebar images that are similar to the rebar image to be determined by satisfying predetermined conditions. In this case, the control unit 301 may output all the type information of the reinforcing bars corresponding to each of the plurality of sample reinforcing bar images, or output the type information of the reinforcing bar having the smallest diameter among them. May be good. By outputting the type information of the reinforcing bar with the smallest diameter size, even if the determined reinforcing bar type is incorrect, the minimum strength of the structure calculated using that type can be estimated. ..
  • Ribs and knots are formed on the deformed reinforcing bars, and their shapes vary.
  • the reinforcing bar when the reinforcing bar is rotated around the axis in the longitudinal direction of the reinforcing bar and observed from a fixed position orthogonal to the axis in the longitudinal direction, the reinforcing bar has a different shape depending on the rotation angle, as shown in FIG. There is.
  • knots are shown in the entire reinforcing bar, but in FIG. 11 (b), knots are not formed in a part in the axial direction.
  • the information of the reinforcing bar images taken from a plurality of angles for one type is registered in the sample reinforcing bar image information of the reinforcing bar information 1200.
  • a plurality of sample reinforcing bar images are taken by rotating the reinforcing bar from a position facing the reinforcing bar with the longitudinal direction of the reinforcing bar as a rotation axis.
  • information on a plurality of reinforcing bar images obtained by photographing the reinforcing bars from various angles may be registered in the sample reinforcing bar image information of the record identified by one type information.
  • control unit 301 evaluates the similarity between the rebar image to be determined and each of the plurality of sample rebar images corresponding to one record in the rebar information 1200. Then, in S1003, the control unit 301 may specify a record of a sample reinforcing bar image that satisfies a predetermined condition and is similar to the reinforcing bar image based on the evaluation results of the plurality of similarities. In one example, the control unit 301 sets the value of the index evaluated to be the most similar among the indexes of the similarity with each of the plurality of sample rebar images evaluated in S1002 to the sample rebar image of the record type. It may be used as an index of similarity with.
  • control unit 301 may accept the input of the plurality of rebar images to be determined, and may evaluate the similarity between each of the plurality of rebar images and the sample rebar image in S1002. Then, in S1003, the control unit 301 may specify a sample reinforcing bar image that is similar to the plurality of reinforcing bar images by satisfying predetermined conditions, based on the evaluation results of the plurality of similarities.
  • control unit 301 sets the value of the index evaluated to be the most similar among the indexes of the degree of similarity between each of the plurality of reinforcing bar images evaluated in S1002 and the sample reinforcing bar image as the type of the record. It may be used as an index of similarity with the sample reinforcing bar image.
  • the plurality of rebar images to be input may be rebar images taken from different angles as described above, and in another example, a plurality of rebar images taken at substantially the same position may be used. There may be. In this case as well, for example, the appearance of the influence of camera shake caused by the movement of the photographer during shooting may differ depending on the image, and different images are taken due to fluctuations in the light of the illumination. obtain. Therefore, the accuracy of determining the type of reinforcing bar can be improved.
  • the plurality of input reinforcing bar images may be, for example, a plurality of images taken by continuous shooting, moving image shooting, or the like.
  • the sample rebar image information is registered in the sample rebar image information of the rebar information 900 of the storage unit 302, and the sample rebar image is stored in the storage unit 302.
  • An example of the case where is used is explained.
  • the embodiment is not limited to this.
  • feature information extracted from the sample rebar image used for evaluation of similarity may be registered instead of the information of the sample rebar image.
  • FIG. 13 is a diagram illustrating the reinforcing bar information 1300 according to the third embodiment.
  • the sample reinforcing bar image information of the reinforcing bar information 1300 stores a frequency spectrum obtained by Fourier transforming the pixel values in the longitudinal direction of the reinforcing bars reflected in the sample reinforcing bar image.
  • the control unit 301 reads the frequency spectrum from the sample reinforcing bar image information of the reinforcing bar information 1300, and the similarity with the frequency spectrum obtained from the reinforcing bar image of the determination target input in S1002. May be evaluated.
  • the control unit 301 reads the frequency spectrum from the sample reinforcing bar image information of the reinforcing bar information 1300, and the similarity with the frequency spectrum obtained from the reinforcing bar image of the determination target input in S1002. May be evaluated.
  • the feature information registered in the sample reinforcing bar image information of the reinforcing bar information 1300 is not limited to the frequency spectrum, and is, for example, other information used for the evaluation of the similarity in the above-mentioned evaluation examples 1 to 7. May be registered.
  • the feature information registered in the sample reinforcing bar image information is the frequency spectrum obtained from the result of Fourier transform of the sample reinforcing bar image, the frequency of the peak of the frequency spectrum, and the phase information extracted from the result of Fourier transform.
  • the local feature amount extracted from the sample reinforcing bar image or the phase image thereof, and at least one information such as the distance between the edges at both ends substantially parallel to the axis of the reinforcing bar may be registered.
  • the type determination using the image may be executed in multiple stages. For example, as illustrated in FIG. 14, the control unit 301 first determines the sorting type such as whether it is a bamboo knot or a screw knot. Subsequently, when it is determined that the reinforcing bar is a bamboo section, in the second stage, the control unit 301 executes the operation flow of FIG. 10 with reference to the reinforcing bar information 1401 in which only the reinforcing bars belonging to the bamboo section are registered, and further. The type may be determined.
  • the sorting type such as whether it is a bamboo knot or a screw knot.
  • control unit 301 refers to the reinforcing bar information 1402 in which only the reinforcing bar belonging to the threaded node is registered, and operates in FIG. The flow may be executed to perform further type determination.
  • the type can be determined using an evaluation method suitable for determining each type.
  • the above-mentioned plurality of evaluation methods are often used. It is also possible to use them in combination in stages to determine the final type.
  • it is effective for determining the shape of nodes, such as extracting local features for determining screw nodes and bamboo nodes, or using a learning model created to determine these two types by machine learning. It is conceivable to first identify whether it is a screw knot or a bamboo knot by using a simple evaluation method. Then, in the second stage, it is conceivable to determine the type of the reinforcing bar by performing the determination using the frequency spectrum of the pixel value in the longitudinal direction of the reinforcing bar, which is effective for determining the node spacing, for example.
  • various types of reinforcing bars can be used as the sorting type for sorting in advance before the final classification of the reinforcing bar type.
  • the manufacturer's section formed in the reinforcing bar node A process of evaluating a logo or the like and sorting by manufacturer may be executed.
  • sorting by different types of sorting types may be combined and sorting may be performed a plurality of times in multiple stages.
  • the output destination for outputting the type information may be the display screen of the display unit 303 in one example, and in another example, a form for recording the inspection result of the bar arrangement inspection or the like. Inspection information 1500 may be available.
  • FIG. 15 is a diagram illustrating inspection information 1500 such as a form for recording the inspection result of the bar arrangement inspection according to the fifth embodiment.
  • inspection information 1500 for example, information corresponding to a design drawing relating to construction in a certain area at a construction site may be registered, and includes, for example, construction identification information 1511 and inspection target information 1512.
  • the construction identification information 1511 includes, for example, information indicating the location of construction and the outline of construction.
  • the construction identification information 1511 includes information such as a number, a construction name, a construction type, a structure number, and a member name.
  • the number is, for example, information for identification given to the inspection information 1500.
  • the construction name may be, for example, the name of the construction.
  • the work type is, for example, information indicating the location of the work, and for example, the work type in FIG. 15,: Hashidai skeleton work, indicates that the work is the work of the base portion that supports the pier.
  • the structure number is, for example, information for designating a structure to be constructed within the work type. For example, the structure number: A01 shown in FIG.
  • the member name is, for example, information indicating an area to be constructed in the structure.
  • the member name: footings shown in FIG. 15 indicates that the footings work is the foundation part of the pier skeleton work.
  • the inspection target information 1512 for example, a record for associating the inspection target with the information of the inspection item is registered.
  • the inspection target is, for example, information for designating an inspection target (for example, a reinforcing bar) of a bar arrangement inspection performed in the construction shown in the inspection information 1500.
  • the inspection target of the inspection target information 1512 includes the information of the position 1, the position 2, and the reinforcing bar number.
  • the position 1 and the position 2 may be, for example, information indicating an area where an inspection target is arranged in a construction design drawing.
  • the position 2 may be, for example, information that specifies a more detailed area within the position 1.
  • the reinforcing bar number is a number assigned to a reinforcing bar or a group of reinforcing bars used in the area specified by the position information.
  • the inspection item is, for example, information indicating an inspection item to be executed for the inspection target of the inspection target information 1512.
  • the inspection item of the inspection target information 1512 includes the inspection item 1 and the inspection item 2.
  • the inspection item may include, for example, a design value and a measured value.
  • the design value is a value on the design drawing for the inspection item.
  • the measurement result measured by the bar arrangement inspection is registered in the measured value.
  • the inspection items may include, for example, the diameter, position, average spacing of arrangements, cover thickness, and number of reinforcing bars to be inspected.
  • the control unit 301 may output the information of the type determined in the process of S1005 described above to the measured value of the inspection information 1500.
  • the control unit 301 acquires information on the diameter of the reinforcing bar from the reinforcing bar information 900 based on the type of the reinforcing bar image to be determined, and sets the acquired diameter in the measured value of the inspection item as shown in the frame 1501 of FIG. Information may be registered. Further, as shown in the frame 1502 of FIG. 15, the control unit 301 may register the identification information for identifying the reinforcing bar image to be determined used for specifying the measured value in the inspection information 1500.
  • the identification information may be information that identifies feature information such as a frequency spectrum extracted from the reinforcing bar image to be determined.
  • the identification information may be a path or a file name used to access the rebar image to be determined or the feature information extracted from the rebar image to be determined.
  • the control unit 301 identifies the reinforcing bar image to be determined used in the inspection or the feature information extracted from the reinforcing bar image in association with the inspection result registered in the inspection item.
  • the information is recorded in the inspection information 1500.
  • the control unit 301 later described FIG. It is possible to re-execute the operation flow and recalculate the measured value. As a result, it is possible to verify whether or not the inspection results have been tampered with.
  • the frequency spectrum obtained by Fourier transform from the image of the reinforcing bar to be determined may include low-frequency noise caused by a paint or the like attached as a marker on the surface of the reinforcing bar.
  • FIG. 16 is a diagram for explaining noise in the frequency spectrum caused by the paint on the surface of the reinforcing bar image.
  • FIG. 16A shows a state in which the paint 1601 is applied to the reinforcing bar, and the noise 1602 caused by the paint 1601 appears in the frequency spectrum thereof.
  • control unit 301 may extract the frequency component of the frequency range to be evaluated from the frequency spectrum and evaluate the similarity in the extracted range. For example, as shown in FIG. 16, by evaluating the similarity in the frequency range to be evaluated, the influence of the noise 1602 can be eliminated, and the similarity is evaluated low due to the noise 1602. Can be suppressed.
  • the frequency range to be evaluated can be determined based on the position of the peak of the frequency spectrum obtained from the sample reinforcing bar image.
  • the control unit 301 may set a predetermined frequency range based on the position of the maximum peak frequency excluding the DC component in the sample reinforcing bar image used for the evaluation of the similarity as the frequency range to be evaluated.
  • FIG. 17 is a diagram illustrating the type determination of the reinforcing bar according to the seventh embodiment.
  • the control unit 301 first accepts the designation of a part of the image area from the image 403, and extracts the designated area as a sample reinforcing bar image ((1) of FIG. 17).
  • the control unit 301 stores the sample reinforcing bar image of the extracted region in the storage unit 302 in association with the type information ((2) in FIG. 17).
  • the control unit 301 may receive input of type information indicating the type of the sample rebar image from the user and associate the received type information with the sample rebar image.
  • type information indicating the type of the sample rebar image from the user
  • D25 indicating the diameter of the reinforcing bar is associated with the sample reinforcing bar image as the type information.
  • the control unit 301 evaluates the similarity with the sample reinforcing bar image by using the image area in which the other reinforcing bars in the image 403 are captured as the reinforcing bar image to be determined. Then, based on the evaluation result of the similarity, the control unit 301 determines that the type of the rebar image to be determined is the same when the sample rebar image and the rebar image to be determined are similar to each other by satisfying a predetermined condition.
  • the type information of the sample reinforcing bar image may be output ((3) in FIG. 17).
  • the predetermined condition used here for determining that the types of reinforcing bars are the same is, for example, when the index used for evaluating the similarity indicates that the higher the value, the more similar the index is.
  • the index used for evaluation may be equal to or higher than the threshold value.
  • the predetermined condition is that the index used for the evaluation of the similarity is equal to or less than the threshold value. It's okay.
  • the threshold value used for the determination is set based on an empirical rule or the like so that it can be determined whether or not the reinforcing bar shown in the sample reinforcing bar image and the reinforcing bar shown in the judgment target reinforcing bar image are of the same type. Good.
  • the type of the reinforcing bar can be determined under the condition that the imaging conditions of the reinforcing bar image to be determined for the type and the sample reinforcing bar image are close to each other, so that the evaluation accuracy of the similarity can be improved. it can. As an example, it is possible to suppress the influence of disturbances such as lighting conditions on the evaluation of similarity.
  • the sample reinforcing bar image can be obtained on the spot from the image 403 in which the reinforcing bar to be determined is captured at the time of type determination. It can be generated and the type can be determined.
  • multiple images 403 may be taken with the angle of view shifted and used for bar arrangement inspection and the like.
  • a sample reinforcing bar image may be extracted from a part of one image 403, and the extracted sample reinforcing bar image may be used for determining the type of the reinforcing bar to be reflected in the other image 403.
  • the similarity can be evaluated under conditions close to the shooting conditions such as lighting conditions, the evaluation accuracy of the similarity can be improved. Then, by improving the evaluation accuracy of the similarity, it is possible to improve the determination accuracy of the type of reinforcing bar.
  • the control unit 301 may execute an algorithm for detecting the reinforcing bars from the image 403 to detect a plurality of reinforcing bars appearing in the image 403. Then, the control unit 301 accepts the selection of the reinforcing bar by the user from the plurality of detected reinforcing bars, and associates the reinforcing bar selected by the user with the type information input by the user and uses it as a sample reinforcing bar image. You may.
  • a plurality of locations may be selected from the captured images for one type of reinforcing bar and used as a sample reinforcing bar image.
  • the control unit 301 may extract a plurality of regions in which the reinforcing bars are reflected from the captured image 403 for one type of reinforcing bar and register them as a plurality of sample reinforcing bar images.
  • deformed reinforcing bars may have different shapes depending on the orientation even if they are of the same type, and such reinforcing bars may be installed with ribs and knots in different orientations during construction. ..
  • the plurality of sample reinforcing bar images may be extracted from, for example, a plurality of images 403 taken with the angle of view shifted with respect to the reinforcing bar arrangement installed in a certain area.
  • the embodiments are not limited to this.
  • the above-mentioned operation flow is an example, and the embodiment is not limited thereto.
  • the operation flow may be executed by changing the order of processing, may include additional processing, or may omit some processing.
  • the embodiment is not limited to this, and for example, in another embodiment, the image 403 is described.
  • the evaluation of the similarity with the sample reinforcing bar image may be performed on the subject.
  • the similarity can be evaluated by handling the rotation about the normal direction of the image plane. Therefore, for example, after performing the above-mentioned face-to-face processing on the image 403, the control unit 301 rotates the similarity between the image 403 and the sample reinforcing bar image without generating the reinforcing bar image 404 to be determined. It may be evaluated by the limited correlation method. In this case, the control unit 301 can evaluate the similarity as well as specify the position of the reinforcing bar evaluated to have a high degree of similarity to the sample reinforcing bar image on the image 403.
  • an example of determining the diameter information is described as an example of the type, but the embodiment is not limited to this.
  • the input reinforcing bar image and the sample reinforcing bar image can be linked. It can be obtained by evaluating the degree of similarity.
  • the reinforcing bar determination device 201 describes an example in which the reinforcing bar information such as the reinforcing bar information 900, 1200, 1300, 1401, 1402 is stored, but the embodiment is limited to this. It's not something.
  • the reinforcing bar information may be stored in a database server different from the reinforcing bar determination device 201. In this case, the reinforcing bar determination device 201 may access the database server and acquire information from the reinforcing bar information.
  • control unit 301 operates as the input unit 311. Further, for example, in S1002 of FIG. 10, the control unit 301 operates as the evaluation unit 312. For example, in S1003 to S1005 of FIG. 10, the control unit 301 operates as the output unit 313. For example, in the sorting process of FIG. 14, the control unit 301 operates as the determination unit 314.
  • FIG. 18 is a diagram illustrating a hardware configuration of a computer 1800 for realizing the reinforcing bar determination device 201 according to the embodiment.
  • the hardware configuration of FIG. 18 includes, for example, a processor 1801, a memory 1802, a storage device 1803, a communication interface 1804, an external interface 1805, a display device 1806, and an input device 1807.
  • the processor 1801 may be communicably connected to the memory 1802, the storage device 1803, the communication interface 1804, the external interface 1805, the display device 1806, and the input device 1807 via, for example, a bus.
  • the processor 1801 may be, for example, a single processor, a multiprocessor, and a multicore.
  • the processor 1801 provides a part or all of the functions of the control unit 301 described above by executing, for example, a program describing the procedure of the operation flow described above using the memory 1802.
  • the processor 1801 operates as an input unit 311, an evaluation unit 312, an output unit 313, and a determination unit 314 by reading a program stored in the storage device 1803 into the memory 1802 and executing the program.
  • the memory 1802 is, for example, a semiconductor memory, and may include a RAM area and a ROM area.
  • the storage device 1803 is, for example, a semiconductor memory such as a hard disk or a flash memory, or an external storage device.
  • RAM is an abbreviation for Random Access Memory.
  • ROM is an abbreviation for Read Only Memory.
  • the storage unit 302 described above may include, for example, a memory 1802 and a storage device 1803.
  • the communication interface 1804 is, for example, a communication device that connects to a network according to the instruction of the processor 1801 and transmits / receives data.
  • the external interface 1805 may be, for example, an interface with an external device.
  • the reinforcing bar determination device 201 may be connected to the photographing device 102 via the external interface 1805.
  • the reinforcing bar determination device 201 may include a communication interface 1804 such as a Wi-Fi (registered trademark) communication device and a Bluetooth (registered trademark) communication device, via a network or at a short distance. It may be connected to the photographing device 102 by wireless communication.
  • the communication interface 1804 and the external interface 1805 are examples of the above-mentioned communication unit 304.
  • the display device 1806 is a device having a display function such as a liquid crystal display.
  • the display device 1806 is an example of the display unit 303 described above.
  • the input device 1807 is a device that receives input from a user such as a keyboard and a touch panel.
  • Each program according to the above-described embodiment may be provided to the reinforcing bar determination device 201 in the following embodiments, for example. (1) It is pre-installed in the storage unit 302. (2) Provided from a server such as a program server.
  • control unit 301 may be implemented as hardware by a dedicated circuit such as FPGA and SoC.
  • FPGA is an abbreviation for Field Programmable Gate Array.
  • SoC is an abbreviation for System-on-a-chip.
  • the control unit 301 may be a circuit that outputs type information to the input image according to the evaluation of the similarity according to the embodiment.
  • the above-mentioned input unit 311, evaluation unit 312, output unit 313, and determination unit 314 may be individually mounted as circuits such as an input circuit, an evaluation circuit, an output circuit, and a determination circuit. Further, all or a part of them may be implemented as an integrated circuit.

Landscapes

  • Engineering & Computer Science (AREA)
  • Architecture (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Civil Engineering (AREA)
  • Structural Engineering (AREA)
  • Image Analysis (AREA)

Abstract

A reinforcing bar determination device according to one embodiment comprises: a storage unit for storing reinforcing bar information that includes information in which at least one of a sample reinforcing bar image showing a reinforcing bar and characteristic information extracted from the sample reinforcing bar image is associated with classification information representing the classification of the reinforcing bar shown in the sample reinforcing bar image; an input unit that receives a reinforcing bar image showing the reinforcing bar subject to determination; an evaluation unit that evaluates the similarities between the sample reinforcing bar image and the reinforcing bar image on the basis of the reinforcing bar information; and an output unit that outputs the classification information associated with the sample reinforcing bar image if the reinforcing bar image and the sample reinforcing bar image satisfy prescribed conditions and are similar based on the results of the similarities evaluation.

Description

鉄筋判定装置および鉄筋判定方法Reinforcing bar judgment device and rebar judgment method
　本発明は、鉄筋判定装置および鉄筋判定方法に関する。 The present invention relates to a reinforcing bar determination device and a reinforcing bar determination method.
　建築土木現場では、建築物や土木構造物等の構造物の施工検査が行われている。例えば、鉄筋コンクリート造の建築物等の工事では、コンクリートを打設する前に、配筋図に従って配筋の検査が行われる。配筋検査では、例えば、使用された鉄筋の径および本数、並びに鉄筋間の間隔などを特定し、配筋図に従って鉄筋が設置されているかなどの検査が行われる。 At the construction site, construction inspections of structures such as buildings and civil engineering structures are carried out. For example, in the construction of a reinforced concrete building or the like, the bar arrangement is inspected according to the bar arrangement diagram before the concrete is placed. In the bar arrangement inspection, for example, the diameter and number of used reinforcing bars, the spacing between the reinforcing bars, and the like are specified, and an inspection is performed to see if the reinforcing bars are installed according to the reinforcing bar diagram.
　また、配筋検査を支援するために、配筋を撮影した画像から工事で使用された鉄筋に関する情報を収集する技術が開発されている。例えば、検査対象とする鉄筋の径を特定することに関連する技術が知られている（例えば、特許文献１）。 In addition, in order to support the bar arrangement inspection, a technology has been developed to collect information on the reinforcing bars used in the construction from the images of the bar arrangement. For example, a technique related to specifying the diameter of a reinforcing bar to be inspected is known (for example, Patent Document 1).
特開２０１５－００１１４６号公報JP 2015-001146
　しかしながら、配筋検査で収集される鉄筋の情報には、高い精度が求められることがある。例えば、構造物の強度を検査するうえで、鉄筋の径は、構造強度を決める重要なパラメータであり、配筋検査において正確な計測が求められる。そのため、配筋を撮影した画像から鉄筋に関する情報を高い精度で特定することのできる技術の提供が望まれている。 However, high accuracy may be required for the reinforcing bar information collected by the bar arrangement inspection. For example, in inspecting the strength of a structure, the diameter of the reinforcing bar is an important parameter for determining the structural strength, and accurate measurement is required in the bar arrangement inspection. Therefore, it is desired to provide a technique capable of identifying information on the reinforcing bar with high accuracy from the image of the reinforcing bar arrangement.
　１つの側面では、本発明は、鉄筋の写る画像からの鉄筋に関する情報の特定精度を向上させることを目的とする。 On one aspect, it is an object of the present invention to improve the accuracy of identifying information about reinforcing bars from an image showing reinforcing bars.
　本発明の一つの態様の鉄筋判定装置は、鉄筋が写るサンプル鉄筋画像およびサンプル鉄筋画像から抽出された特徴情報の少なくとも一方と、サンプル鉄筋画像に写る鉄筋の種別を示す種別情報とを対応づけた情報を含む鉄筋情報を記憶する記憶部と、判定対象の鉄筋が写る鉄筋画像の入力を受け付ける入力部と、鉄筋情報に基づいて、サンプル鉄筋画像と鉄筋画像との類似度を評価する評価部と、類似度の評価結果に基づいて、鉄筋画像とサンプル鉄筋画像とが所定の条件を満たして類似している場合に、サンプル鉄筋画像と対応する種別情報を出力する出力部と、を備える。 The reinforcing bar determination device of one aspect of the present invention associates at least one of the sample reinforcing bar image showing the reinforcing bar and the feature information extracted from the sample reinforcing bar image with the type information indicating the type of the reinforcing bar shown in the sample reinforcing bar image. A storage unit that stores rebar information including information, an input unit that accepts input of a rebar image showing the rebar to be judged, and an evaluation unit that evaluates the similarity between the sample rebar image and the rebar image based on the rebar information. Based on the evaluation result of the degree of similarity, when the reinforcing bar image and the sample reinforcing bar image are similar to each other satisfying a predetermined condition, the sample reinforcing bar image and the corresponding type information are output.
　一つの態様によれば、鉄筋の写る画像からの鉄筋に関する情報の特定精度を向上させることができる。 According to one aspect, it is possible to improve the accuracy of identifying the information about the reinforcing bar from the image showing the reinforcing bar.
実施形態に係る施工済みの鉄筋の撮影を例示する図である。It is a figure which illustrates the photograph | photograph of the already constructed reinforcing bar which concerns on embodiment. 実施形態に係る鉄筋判定システムを例示する図である。It is a figure which illustrates the reinforcing bar determination system which concerns on embodiment. 実施形態に係る鉄筋判定装置のブロック構成を例示する図である。It is a figure which illustrates the block structure of the reinforcing bar determination device which concerns on embodiment. 実施形態に係る鉄筋判定処理を説明する図である。It is a figure explaining the reinforcing bar determination process which concerns on embodiment. 実施形態に係る画像からの鉄筋に関する情報の収集を例示する図である。It is a figure which illustrates the collection of the information about the reinforcing bar from the image which concerns on embodiment. 実施形態に係る学習済みモデルの生成を例示する図である。It is a figure which illustrates the generation of the trained model which concerns on embodiment. 例示的な正対変換を示す図である。It is a figure which shows the exemplary face-to-face transformation. 鉄筋の写る画像の回転を例示する図である。It is a figure which illustrates the rotation of the image which shows a reinforcing bar. 第１の実施形態に係る鉄筋情報を例示する図である。It is a figure which illustrates the reinforcing bar information which concerns on 1st Embodiment. 第１の実施形態に係る種別判定処理の動作フローを例示する図である。It is a figure which illustrates the operation flow of the type determination process which concerns on 1st Embodiment. 回転角に応じた鉄筋の異なる形状を例示する図である。It is a figure which illustrates the different shape of the reinforcing bar according to the rotation angle. 第２の実施形態に係る鉄筋情報を例示する図である。It is a figure which illustrates the reinforcing bar information which concerns on 2nd Embodiment. 第３の実施形態に係る鉄筋情報を例示する図である。It is a figure which illustrates the reinforcing bar information which concerns on 3rd Embodiment. 多段階での画像を用いた種別の判定を例示する図である。It is a figure which illustrates the type determination using the image in a multi-step. 第５の実施形態に係る検査情報を例示する図である。It is a figure which illustrates the inspection information which concerns on 5th Embodiment. 鉄筋画像の表面の塗料に起因する周波数スペクトルにおけるノイズを説明する図である。It is a figure explaining the noise in the frequency spectrum caused by the paint on the surface of a reinforcing bar image. 第７の実施形態に係る鉄筋の種別判定を例示する図である。It is a figure which illustrates the type determination of the reinforcing bar which concerns on 7th Embodiment. 実施形態に係る鉄筋判定装置を実現するためのコンピュータのハードウェア構成を例示する図である。It is a figure which illustrates the hardware configuration of the computer for realizing the reinforcing bar determination apparatus which concerns on embodiment.
　以下、図面を参照しながら、本発明のいくつかの実施形態について詳細に説明する。なお、複数の図面において対応する要素には同一の符号を付す。 Hereinafter, some embodiments of the present invention will be described in detail with reference to the drawings. The same reference numerals are given to the corresponding elements in the plurality of drawings.
　上述のように、配筋を撮影した画像から工事で使用された鉄筋に関する情報を収集する技術の開発が進められている。 As mentioned above, the development of technology to collect information on the reinforcing bars used in the construction from the images of the reinforcing bars is underway.
　ここで、ＪＩＳ（日本規格協会）規格では、鉄筋の径に対して鉄筋の節間隔の最大値を定めている。そのため、配筋を撮影した画像から、工事で使用された鉄筋に関する情報を特定する一つの手法として、例えば、径の異なる複数の鉄筋について、鉄筋の径と節間隔とを対応づけた情報を用意する。そして、画像から鉄筋の節間隔を特定し、特定した節間隔と対応づけられた鉄筋の径を、画像に写る鉄筋の径として推定することが考えられる。 Here, the JIS (Japanese Standards Association) standard defines the maximum value of the node spacing of the reinforcing bar with respect to the diameter of the reinforcing bar. Therefore, as one method of identifying information about the reinforcing bars used in the construction from the images of the reinforcing bars, for example, for multiple reinforcing bars with different diameters, we prepared information that associates the diameter of the reinforcing bars with the node spacing. To do. Then, it is conceivable to specify the node spacing of the reinforcing bars from the image and estimate the diameter of the reinforcing bars associated with the specified node spacing as the diameter of the reinforcing bars reflected in the image.
　しかしながら、鉄筋の節間隔は、ロットの違いによってばらつきがあり、あるいは、同じ鉄筋を撮影した画像であっても、撮影条件によって画像から検出される鉄筋の節間隔に差が生じ得る。 However, the node spacing of the reinforcing bars varies depending on the lot, or even if the same reinforcing bar is photographed, the node spacing of the reinforcing bars detected from the image may differ depending on the imaging conditions.
　また、上述のように、ＪＩＳ（日本規格協会）規格で定められているのは、鉄筋の径に対して鉄筋の節間隔の最大値であり、上限を定めているだけであるため、同じ径の鉄筋であっても、メーカが異なれば節間隔が異なっていることがある。また、異形鉄筋には、さまざまな種類があり、例えば、竹節の鉄筋と、ねじ節の鉄筋では、節間隔が同じであっても径が異なることがある。このように鉄筋の節の間隔が類似していても、異なる鉄筋種別であることがあり、節間隔に基づいて鉄筋の情報を正確に取得することが難しいことがある。そのため、配筋を撮影した画像から鉄筋の種別をより正確に特定することのできる技術の提供が望まれている。 In addition, as described above, the JIS (Japanese Standards Association) standard defines the maximum value of the node spacing of the reinforcing bar with respect to the diameter of the reinforcing bar, and only the upper limit is set, so the same diameter. Even with the reinforcing bars of, the node spacing may differ depending on the manufacturer. In addition, there are various types of deformed reinforcing bars. For example, the reinforcing bars of bamboo knots and the reinforcing bars of threaded knots may have different diameters even if the knot spacing is the same. Even if the intervals between the reinforcing bars are similar in this way, the reinforcing bar types may be different, and it may be difficult to accurately obtain information on the reinforcing bars based on the intervals between the reinforcing bars. Therefore, it is desired to provide a technique capable of more accurately specifying the type of reinforcing bar from the image obtained by taking the reinforcing bar arrangement.
　以下で述べる実施形態では、サンプル鉄筋画像と、そのサンプル鉄筋画像に写っている鉄筋の種別を示す種別情報とを対応づけた情報が予め記憶装置に記憶される。そして、配筋検査などにおいて種別を判定する対象の鉄筋が写る画像が入力された場合に、その画像から判定対象の鉄筋の写る領域を抽出した鉄筋画像を作成し、その鉄筋画像と、サンプル鉄筋画像との類似度を評価する。そして、類似度の評価結果に基づいて、判定対象の鉄筋画像とサンプル鉄筋画像とが所定の条件を満たして類似している場合に、サンプル鉄筋画像に対応づけられている種別情報を、鉄筋画像に写る鉄筋の種別として出力する。 In the embodiment described below, the information in which the sample reinforcing bar image is associated with the type information indicating the type of the reinforcing bar shown in the sample reinforcing bar image is stored in the storage device in advance. Then, when an image showing the reinforcing bar of the target to be determined is input in a bar arrangement inspection or the like, a reinforcing bar image is created by extracting the area where the reinforcing bar to be judged is reflected from the image, and the reinforcing bar image and the sample reinforcing bar are created. Evaluate the similarity with the image. Then, based on the evaluation result of the degree of similarity, when the rebar image to be determined and the sample rebar image are similar to each other satisfying a predetermined condition, the type information associated with the sample rebar image is obtained as the rebar image. Output as the type of reinforcing bar shown in.
　このように、実施形態によれば画像の類似度を評価することで、判定対象の鉄筋画像と類似するサンプル鉄筋画像を特定し、それによって判定対象の鉄筋画像に写る鉄筋の種別を判定する。そのため、画像から抽出することが可能なさまざまな特徴情報を、類似度の評価に利用することができ、高い精度で判定対象の鉄筋画像に写る鉄筋の種別を判定することが可能である。以下、実施形態を更に詳細に説明する。 In this way, according to the embodiment, by evaluating the similarity of the images, a sample rebar image similar to the rebar image to be determined is specified, and the type of the rebar reflected in the rebar image to be determined is determined accordingly. Therefore, various feature information that can be extracted from the image can be used for evaluation of the degree of similarity, and it is possible to determine the type of reinforcing bar that appears in the reinforcing bar image to be determined with high accuracy. Hereinafter, embodiments will be described in more detail.
　図１は、実施形態に係る施工済みの鉄筋の撮影を例示する図である。図１に示す様に、ユーザは、工事現場で、検査対象となる鉄筋１０１を、撮影装置１０２を用いて撮影する。撮影装置１０２は、例えば、単眼のカメラであっても、またはステレオカメラであってよい。 FIG. 1 is a diagram illustrating photography of a rebar that has already been constructed according to the embodiment. As shown in FIG. 1, the user photographs the reinforcing bar 101 to be inspected at the construction site by using the imaging device 102. The photographing device 102 may be, for example, a monocular camera or a stereo camera.
　図２は、実施形態に係る鉄筋判定システム２００を例示する図である。鉄筋判定システム２００は、例えば、撮影装置１０２および鉄筋判定装置２０１を含む。撮影装置１０２と鉄筋判定装置２０１は、例えば、ネットワーク２０５を経由して接続されてよい。ネットワーク２０５は、例えば、ＬＡＮ（Local Area Network）およびインターネットである。また、別の実施形態では、撮影装置１０２および鉄筋判定装置２０１は、近距離無線通信または有線で接続されていてよく、或いは、撮影装置１０２および鉄筋判定装置２０１は一体の装置として統合されていてもよい。 FIG. 2 is a diagram illustrating the reinforcing bar determination system 200 according to the embodiment. The reinforcing bar determination system 200 includes, for example, an imaging device 102 and a reinforcing bar determination device 201. The photographing device 102 and the reinforcing bar determination device 201 may be connected via, for example, the network 205. The network 205 is, for example, a LAN (Local Area Network) and the Internet. Further, in another embodiment, the photographing device 102 and the reinforcing bar determination device 201 may be connected by short-range wireless communication or by wire, or the photographing device 102 and the reinforcing bar determination device 201 are integrated as an integrated device. May be good.
　鉄筋判定装置２０１は、例えば、演算機能を備えるコンピュータであり、一例では、パーソナルコンピュータ（ＰＣ）、ノートＰＣ、およびタブレット端末である。鉄筋判定装置２０１は、例えば、撮影装置１０２で撮影された配筋の画像を取得し、画像に写る配筋を解析して、配筋に使用されている鉄筋の情報を収集してよい。 The reinforcing bar determination device 201 is, for example, a computer having a calculation function, for example, a personal computer (PC), a notebook PC, and a tablet terminal. The reinforcing bar determination device 201 may, for example, acquire an image of the reinforcing bar arranged by the photographing device 102, analyze the reinforcing bar arranged in the image, and collect information on the reinforcing bar used for the reinforcing bar arrangement.
　図３は、実施形態に係る鉄筋判定装置２０１のブロック構成を例示する図である。鉄筋判定装置２０１は、例えば、制御部３０１、記憶部３０２、表示部３０３、および通信部３０４を含む。制御部３０１は、例えば、鉄筋判定装置２０１の各部を制御する。例えば、制御部３０１は、入力部３１１、評価部３１２、出力部３１３、および判定部３１４として動作する。記憶部３０２は、例えば、撮影装置１０２で撮影された画像、後述する鉄筋情報、サンプル鉄筋画像、サンプル鉄筋画像から抽出された特徴情報などを記憶していてよい。表示部３０３は、例えば、液晶ディスプレイなどの表示装置である。表示部３０３は、制御部３０１の指示に従って表示画面に情報を表示する。通信部３０４は、例えば、制御部３０１の指示に従って撮影装置１０２と通信する。これらの各部の詳細および記憶部３０２に格納されている情報の詳細については後述する。 FIG. 3 is a diagram illustrating a block configuration of the reinforcing bar determination device 201 according to the embodiment. The reinforcing bar determination device 201 includes, for example, a control unit 301, a storage unit 302, a display unit 303, and a communication unit 304. The control unit 301 controls, for example, each unit of the reinforcing bar determination device 201. For example, the control unit 301 operates as an input unit 311, an evaluation unit 312, an output unit 313, and a determination unit 314. The storage unit 302 may store, for example, an image taken by the photographing device 102, reinforcing bar information described later, a sample reinforcing bar image, feature information extracted from the sample reinforcing bar image, and the like. The display unit 303 is, for example, a display device such as a liquid crystal display. The display unit 303 displays information on the display screen according to the instruction of the control unit 301. The communication unit 304 communicates with the photographing device 102 according to the instruction of the control unit 301, for example. Details of each of these units and details of the information stored in the storage unit 302 will be described later.
　図４は、実施形態に係る鉄筋判定処理を説明する図である。図４に示すように、鉄筋判定装置２０１の記憶部３０２には、鉄筋情報４０１が記憶されている。図４の鉄筋情報４０１の例では、種別情報として鉄筋径を示すＤ４１、Ｄ２５などの鉄筋の種別と、その種別の鉄筋を撮影したサンプル鉄筋画像とが、鉄筋の異なる複数の種別ごとに対応づけて登録されている。 FIG. 4 is a diagram for explaining the reinforcing bar determination process according to the embodiment. As shown in FIG. 4, the reinforcing bar information 401 is stored in the storage unit 302 of the reinforcing bar determination device 201. In the example of the reinforcing bar information 401 of FIG. 4, the type of the reinforcing bar such as D41 and D25 indicating the reinforcing bar diameter as the type information and the sample reinforcing bar image obtained by photographing the reinforcing bar of that type are associated with each of a plurality of different types of reinforcing bars. Is registered.
　また、図４には、例えば、ユーザが配筋検査の際などに施工済みの鉄筋を撮影することで得られる鉄筋を写した画像４０３が示されている。そして、制御部３０１は、例えば、画像４０３から種別の判定対象の鉄筋の写る画像領域を抽出して鉄筋画像４０４を作成する。そして、制御部３０１は、鉄筋画像４０４と、鉄筋情報４０１に登録されているそれぞれのサンプル鉄筋画像との類似度を評価する。続いて、制御部３０１は、類似度の評価結果に基づいて、判定対象の鉄筋画像４０４と類似するサンプル鉄筋画像を特定し、そのサンプル鉄筋画像と対応づけられている種別情報を判定対象の鉄筋の種別として出力する。例えば、図４では、Ｄ４１の鉄筋が最も類似度の高い鉄筋として評価されており、判定対象の鉄筋の径がＤ４１であることが特定できる。そして、画像間の類似度は、画像から抽出した様々な特徴情報を用いて評価することができるため、評価精度の高い特徴情報を用いて類似度を評価したり、または、複数の特徴情報を用いて類似度を評価したりすることで、種別の判定精度を高めることが可能である。以下、類似度の評価についていくつかの例を説明する。 Further, FIG. 4 shows an image 403 showing the reinforcing bars obtained by taking a picture of the reinforcing bars already constructed by the user, for example, during a bar arrangement inspection. Then, the control unit 301 extracts, for example, an image area in which the reinforcing bar of the type to be determined is reflected from the image 403 to create the reinforcing bar image 404. Then, the control unit 301 evaluates the degree of similarity between the reinforcing bar image 404 and each sample reinforcing bar image registered in the reinforcing bar information 401. Subsequently, the control unit 301 identifies a sample rebar image similar to the rebar image 404 to be determined based on the evaluation result of the similarity, and determines the type information associated with the sample rebar image for the rebar to be determined. Output as the type of. For example, in FIG. 4, the reinforcing bar of D41 is evaluated as the reinforcing bar having the highest degree of similarity, and it can be specified that the diameter of the reinforcing bar to be determined is D41. Since the similarity between images can be evaluated by using various feature information extracted from the images, the similarity can be evaluated by using the feature information with high evaluation accuracy, or a plurality of feature information can be evaluated. It is possible to improve the accuracy of determining the type by using it to evaluate the degree of similarity. Hereinafter, some examples of evaluation of similarity will be described.
　[類似度の評価例]
　（評価例１）
　評価例１では、画像から鉄筋の節間隔と幅に関連する情報を収集する。図５は、実施形態に係る画像からの鉄筋に関する情報の収集を例示する図である。図５（ａ）に示すように、制御部３０１は、例えば、画像の画素値を鉄筋の長手方向にフーリエ変換し、直流成分を除いて最大ピークの周波数の情報を取得してよい。鉄筋の長手方向の画素値の周波数成分のうちで、直流成分を除いた最大のピークの周波数は、節間隔と対応していることが推定される。そのため、制御部３０１は、例えば、鉄筋の長手方向にフーリエ変換した周波数スペクトルにおいて直流成分を除いて最大のピークの周波数を特定する。最大のピークの周波数は、一例では、周波数スペクトルにおいて最大強度の極大値の周波数として特定されてよい。なお、画素値は、画像の画素に関する値であってよく、例えば、輝度値および色値である。 [Example of evaluation of similarity]
(Evaluation example 1)
In evaluation example 1, information related to the node spacing and width of the reinforcing bar is collected from the image. FIG. 5 is a diagram illustrating the collection of information regarding the reinforcing bar from the image according to the embodiment. As shown in FIG. 5A, the control unit 301 may, for example, Fourier transform the pixel value of the image in the longitudinal direction of the reinforcing bar to acquire information on the frequency of the maximum peak excluding the DC component. It is presumed that the frequency of the maximum peak excluding the DC component among the frequency components of the pixel value in the longitudinal direction of the reinforcing bar corresponds to the node spacing. Therefore, the control unit 301 specifies, for example, the frequency of the maximum peak excluding the DC component in the frequency spectrum Fourier transformed in the longitudinal direction of the reinforcing bar. The frequency of the maximum peak may be specified, for example, as the frequency of the maximum intensity of the maximum intensity in the frequency spectrum. The pixel value may be a value related to a pixel of an image, and is, for example, a luminance value and a color value.
　また、図５（ａ）に示すように、制御部３０１は、画像に写る鉄筋の長手方向と略平行に延びる鉄筋のエッジを検出する。一例では、制御部３０１は、画像に対してエッジ検出の処理をかけて、鉄筋の長手方向に対して直交する方向に、鉄筋の長手方向と略平行なエッジの探索を行い、鉄筋のエッジを検出してよい。エッジ検出は、例えば、Sobelフィルタなどを用いる手法などの既存の様々な手法を用いて実施することができる。そして、制御部３０１は、検出した鉄筋の長手方向に略平行な両端のエッジ間の距離を、画像に写る鉄筋の幅として特定してよい。 Further, as shown in FIG. 5A, the control unit 301 detects the edge of the reinforcing bar extending substantially parallel to the longitudinal direction of the reinforcing bar shown in the image. In one example, the control unit 301 applies edge detection processing to the image, searches for an edge substantially parallel to the longitudinal direction of the reinforcing bar in a direction orthogonal to the longitudinal direction of the reinforcing bar, and determines the edge of the reinforcing bar. It may be detected. Edge detection can be performed using various existing methods such as a method using a Sobel filter or the like. Then, the control unit 301 may specify the distance between the edges at both ends substantially parallel to the longitudinal direction of the detected reinforcing bar as the width of the reinforcing bar reflected in the image.
　そして、制御部３０１は、特定した最大ピークの周波数の情報と、エッジ間の距離の情報とを判定対象の鉄筋画像と、サンプル鉄筋画像との間で比較し、２つの画像の類似度を評価する。一例では、制御部３０１は、２つの画像の最大ピークの周波数の差分の大きさと、２つの画像のエッジ間の距離の差分との合算値（例えば、重み付き加算）を類似度の評価の指標として用いてよい。即ち、この場合、制御部３０１は、合算値が小さいほど、サンプル鉄筋画像と、判定対象の鉄筋画像との類似度が高いと評価してよい。 Then, the control unit 301 compares the information on the frequency of the specified maximum peak and the information on the distance between the edges between the rebar image to be determined and the sample rebar image, and evaluates the similarity between the two images. To do. In one example, the control unit 301 uses the sum of the magnitude of the difference in the frequencies of the maximum peaks of the two images and the difference in the distance between the edges of the two images (for example, weighted addition) as an index for evaluating the similarity. May be used as. That is, in this case, the control unit 301 may evaluate that the smaller the total value, the higher the degree of similarity between the sample reinforcing bar image and the reinforcing bar image to be determined.
　以上で述べたように、評価例１では、画像から抽出した最大ピークの周波数と、エッジ間の距離との２つを特徴情報として用いて、判定対象の鉄筋画像と、サンプル鉄筋画像との類似度を評価している。そのため、例えば、節間隔が似ている鉄筋であっても、幅方向の情報をもとに区別することができ、より正確に類似度を評価して、判定対象の鉄筋の種別を判定することができる。 As described above, in the evaluation example 1, the frequency of the maximum peak extracted from the image and the distance between the edges are used as feature information, and the rebar image to be determined is similar to the sample rebar image. I am evaluating the degree. Therefore, for example, even if the reinforcing bars have similar node spacings, they can be distinguished based on the information in the width direction, and the similarity is evaluated more accurately to determine the type of the reinforcing bar to be determined. Can be done.
　（評価例２）
　評価例２では、画像の画素値を鉄筋の長手方向にフーリエ変換して得た周波数スペクトルの直流成分を除いたピークのうちで、複数のピークの周波数の情報を取得する。図５（ｂ）は、実施形態に係る周波数スペクトルから複数の周波数の情報を特徴情報として収集する例を示している。図５（ｂ）に示すように、制御部３０１は、最大ピーク以外のピークの周波数も収集して類似度の評価に用いてよい。複数の周波数の情報を用いて比較を行うことで、節間隔以外の例えばメーカや同一メーカのロットごとの節の形状の微細な違いなどに応じた周波数成分の比較を実行することができる。その結果、より正確に判定対象の鉄筋画像と、サンプル鉄筋画像との類似度を評価することができる。 (Evaluation example 2)
In the evaluation example 2, the frequency information of a plurality of peaks is acquired from the peaks obtained by Fourier transforming the pixel values of the image in the longitudinal direction of the reinforcing bar and excluding the DC component of the frequency spectrum. FIG. 5B shows an example of collecting information on a plurality of frequencies as feature information from the frequency spectrum according to the embodiment. As shown in FIG. 5B, the control unit 301 may collect frequencies of peaks other than the maximum peak and use them for evaluation of similarity. By performing the comparison using the information of a plurality of frequencies, it is possible to compare the frequency components according to, for example, a minute difference in the shape of the nodes for each manufacturer or lot of the same manufacturer other than the node spacing. As a result, the degree of similarity between the rebar image to be determined and the sample rebar image can be evaluated more accurately.
　なお、比較対象とする複数の周波数成分は様々な方法で決定することができる。例えば、画像の画素値を鉄筋の長手方向にフーリエ変換して得られる周波数スペクトルの複数の周波数成分のうち、ピーク強度が上位の所定の数のピークを抽出してもよく、または、低周波数側から所定の閾値以上の強度を有する所定数のピークを抽出してもよい。また、制御部３０１は、判定対象の鉄筋画像と、サンプル鉄筋画像との間でのピークの比較を、低周波数側から順に比較するなどピークを抽出した順序が同じピーク同士を比較してよく、別の実施形態では、最も近い周波数のピーク同士で比較を実行してもよい。そして、制御部３０１は、例えば、比較を行ったピークの周波数の差分の大きさを積算して、その合計値を類似度の評価の指標として用いてよい。即ち、この場合、制御部３０１は、合計値が小さいほど、サンプル鉄筋画像と、判定対象の鉄筋画像との類似度が高いと評価してよい。 Note that a plurality of frequency components to be compared can be determined by various methods. For example, a predetermined number of peaks having the highest peak intensity may be extracted from a plurality of frequency components of the frequency spectrum obtained by Fourier transforming the pixel values of the image in the longitudinal direction of the reinforcing bar, or the low frequency side. A predetermined number of peaks having an intensity equal to or higher than a predetermined threshold may be extracted from the above. Further, the control unit 301 may compare peaks having the same peak extraction order, such as comparing peaks between the rebar image to be determined and the sample rebar image in order from the low frequency side. In another embodiment, comparisons may be performed between peaks with the closest frequencies. Then, the control unit 301 may, for example, integrate the magnitude of the difference in the frequencies of the compared peaks and use the total value as an index for evaluating the similarity. That is, in this case, the control unit 301 may evaluate that the smaller the total value, the higher the degree of similarity between the sample reinforcing bar image and the reinforcing bar image to be determined.
　（評価例３）
　鉄筋の長手方向に画像の画素値をフーリエ変換した波形同士を、判定対象の鉄筋画像と、サンプル鉄筋画像との間で比較して類似度を評価してもよい。一例として、波形の類似度の評価に積率相関係数を用いることができる。例えば、ｘおよびｙを組みとするn組のデータ(x1, y1),(x2, y2),…,(xn, yn)があるとする。この場合に、組となったｘおよびｙの間の相関は、以下のピアソンの積率相関係数ｒｘｙの式を用いて計算することができる。
Figure JPOXMLDOC01-appb-M000001
 (Evaluation example 3)
The similarity may be evaluated by comparing the waveforms obtained by Fourier transforming the pixel values of the images in the longitudinal direction of the reinforcing bar between the reinforcing bar image to be determined and the sample reinforcing bar image. As an example, the product moment correlation coefficient can be used to evaluate the similarity of waveforms. For example, suppose there are n sets of data (x 1 , y 1 ), (x 2 , y 2 ), ..., (x n , y n ) with x and y as a set. In this case, the correlation between the paired x and y can be calculated using the following formula of Pearson's product moment correlation coefficient r xy .
Figure JPOXMLDOC01-appb-M000001
　ここで、上記の式において、オーバーライン（上線）が付されたｘおよびｙは、それぞれｘおよびｙの平均値を表す。また、式においてsxはxの標準偏差、syはyの標準偏差、sxyはxとyの共分散を表す。そして、例えば、ある鉄筋種別のサンプル鉄筋画像に写る鉄筋の長手方向の画素値をフーリエ変換して得られた周波数スペクトルの波形の周波数と対応する強度を、上記式のxで表現されるパラメータとする。また、判定対象の鉄筋画像に写る鉄筋の長手方向の画素値をフーリエ変換して得られた周波数スペクトルの波形の周波数と対応する強度を上記式のyで表現されるパラメータとする。この場合に、２つの画像から得られた周波数スペクトルにおいて同じ周波数における強度を組みとして上記の式に入力することで、相関係数ｒｘｙを求めることができ、得られたｒｘｙを類似度の評価の指標として用いることができる。 Here, in the above equation, x and y with an overline (overline) represent the average value of x and y, respectively. In the equation, s x is the standard deviation of x, s y is the standard deviation of y, and s xy is the covariance of x and y. Then, for example, the intensity corresponding to the frequency of the waveform of the frequency spectrum obtained by Fourier transforming the pixel value in the longitudinal direction of the reinforcing bar in the sample reinforcing bar image of a certain reinforcing bar type is set as the parameter expressed by x in the above equation. To do. Further, the intensity corresponding to the frequency of the waveform of the frequency spectrum obtained by Fourier transforming the pixel value in the longitudinal direction of the reinforcing bar reflected in the reinforcing bar image to be determined is used as a parameter expressed by y in the above equation. In this case, the correlation coefficient r xy can be obtained by inputting the intensities at the same frequency as a set in the frequency spectra obtained from the two images into the above equation, and the obtained r xy is of similarity. It can be used as an index for evaluation.
　このように、フーリエ変換した波形同士を比較することで、より多くの情報を用いることができるため、節間隔以外の例えばメーカごとや同一メーカのロットごとの節の形状の微細な違いに応じた周波数成分の比較を実行することができる。その結果、より正確に判定対象の鉄筋画像と、サンプル鉄筋画像との類似度を評価することができる。 In this way, more information can be used by comparing the Fourier-transformed waveforms, so that it corresponds to minute differences in the shape of the nodes other than the node spacing, for example, for each manufacturer or for each lot of the same manufacturer. A comparison of frequency components can be performed. As a result, the degree of similarity between the rebar image to be determined and the sample rebar image can be evaluated more accurately.
　（評価例４）
　画像内に写る鉄筋の長手方向の画素値の周波数の情報に加えて、鉄筋の長手方向に直交する鉄筋の幅方向の画素値の周波数の情報を類似度の評価に用いてもよい。図５（ｃ）は、幅方向の画素値の周波数スペクトルを例示する図である。なお、図５（ｃ）では、画像内の鉄筋の長手方向の異なる位置で複数の幅方向の画素値を取得し、それらを合成して幅方向の周波数スペクトルの生成に用いている。それにより、１つの位置で幅方向の画素値の特徴を収集するよりも鉄筋の幅方向の特徴を多く取得でき、鉄筋の種別の判定精度を向上させることが可能である。なお、合成は、様々な方法で実行することができ、例えば、複数の位置における幅方向の画素値を１つにつなぎ合わせることで実行されてもよく、また、画素値の平均をとることで実行されてもよい。しかしながら、実施形態は、１つの位置で幅方向の画素値の特徴を収集してもよい。 (Evaluation example 4)
In addition to the frequency information of the pixel value in the longitudinal direction of the reinforcing bar shown in the image, the frequency information of the pixel value in the width direction of the reinforcing bar orthogonal to the longitudinal direction of the reinforcing bar may be used for the evaluation of the similarity. FIG. 5C is a diagram illustrating a frequency spectrum of pixel values in the width direction. In FIG. 5C, a plurality of pixel values in the width direction are acquired at different positions in the longitudinal direction of the reinforcing bar in the image, and they are combined and used to generate a frequency spectrum in the width direction. As a result, it is possible to acquire more features in the width direction of the reinforcing bar than to collect features of pixel values in the width direction at one position, and it is possible to improve the determination accuracy of the type of reinforcing bar. The composition can be executed by various methods. For example, the composition may be executed by joining the pixel values in the width direction at a plurality of positions into one, or by averaging the pixel values. It may be executed. However, embodiments may collect features of pixel values in the width direction at one position.
　そして、制御部３０１は、鉄筋の長手方向の画素値の周波数スペクトルと、幅方向の画素値の周波数スペクトルとを用いて、判定対象の鉄筋画像と、サンプル鉄筋画像との類似度を評価してよい。周波数スペクトルの類似度の評価は、例えば、上述のように、判定対象の鉄筋画像とサンプル鉄筋画像との２つの画像から取得された周波数スペクトルにおいて対応するピークの周波数の差分の合算値や平均値を指標として用いることで実行することができる。或いは、周波数スペクトルの周波数に対する強度を入力として積率相関係数を用いて類似度を評価してもよい。 Then, the control unit 301 evaluates the similarity between the reinforcing bar image to be determined and the sample reinforcing bar image by using the frequency spectrum of the pixel value in the longitudinal direction of the reinforcing bar and the frequency spectrum of the pixel value in the width direction. Good. For the evaluation of the similarity of the frequency spectrum, for example, as described above, the sum total value or the average value of the frequency differences of the corresponding peaks in the frequency spectrum acquired from the two images of the rebar image to be determined and the sample rebar image. Can be executed by using as an index. Alternatively, the similarity may be evaluated using the product moment correlation coefficient with the intensity of the frequency spectrum with respect to the frequency as an input.
　また、長手方向の画素値の周波数スペクトルと、幅方向の画素値の周波数スペクトルとは、一例では、１つの類似度の評価に用いられてよい。また、別の例では、まず、長手方向の画素値の周波数スペクトルを用いた類似度の評価で上位の所定の数のサンプル鉄筋画像を特定する。その後、幅方向の画素値の周波数スペクトルを用いた類似度の評価で更に最も似ているサンプル鉄筋画像を特定するというように、個別に類似度の評価を実行し、多段階で類似するサンプル鉄筋画像を特定してもよい。 Further, the frequency spectrum of the pixel value in the longitudinal direction and the frequency spectrum of the pixel value in the width direction may be used for one evaluation of similarity in one example. In another example, first, a predetermined number of sample reinforcing bar images at the top are specified by evaluation of similarity using the frequency spectrum of pixel values in the longitudinal direction. After that, the similarity is evaluated individually, such as identifying the most similar sample rebar image by the similarity evaluation using the frequency spectrum of the pixel value in the width direction, and the sample rebar that is similar in multiple stages. The image may be specified.
　（評価例５）
　画像に写る鉄筋の長手方向の画素値の波形をそのまま、判定対象の鉄筋画像と、サンプル鉄筋画像との類似度の評価に用いてもよい。なお、この場合、画像の上辺または下辺など、軸方向の画素値の一方の端部から最初に検出されるピークの位置で、判定対象の鉄筋画像から得られた波形と、サンプル鉄筋画像から得られた波形とを位置合わせした後で、波形を用いた類似度の評価が行われてよい。類似度の評価は、例えば、画素値の波形におけるピークの周期を取得して比較を行ってもよいし、波形の軸方向の座標に対する画素値を入力として積率相関係数を用いて行ってもよい。 (Evaluation example 5)
The waveform of the pixel value in the longitudinal direction of the reinforcing bar shown in the image may be used as it is for evaluating the similarity between the reinforcing bar image to be determined and the sample reinforcing bar image. In this case, at the position of the peak first detected from one end of the pixel value in the axial direction such as the upper side or the lower side of the image, the waveform obtained from the rebar image to be determined and the sample rebar image are obtained. After aligning with the obtained waveform, the similarity may be evaluated using the waveform. The evaluation of the similarity may be performed, for example, by acquiring the peak period in the waveform of the pixel value and performing comparison, or by using the product moment correlation coefficient with the pixel value with respect to the axial coordinates of the waveform as an input. May be good.
　（評価例６）
　判定対象の鉄筋画像と、サンプル鉄筋画像とで局所特徴量のマッチングを行い、類似度を評価してもよい。局所特徴量としては、例えば、ＳＩＦＴ（Scale-Invariant Feature Transform）、ＳＵＲＦ（Speeded-Up Robust Features）などの種々の局所特徴量を用いることができる。 (Evaluation example 6)
The degree of similarity may be evaluated by matching the local feature amount between the rebar image to be determined and the sample rebar image. As the local feature amount, for example, various local feature amounts such as SIFT (Scale-Invariant Feature Transform) and SURF (Speeded-Up Robust Features) can be used.
　（評価例７）
　フーリエ変換の結果から位相の情報を収集して類似度の評価に用いてもよい。例えば、判定対象の鉄筋画像と、サンプル鉄筋画像とに２次元フーリエ変換を行った結果から位相情報を取得し、位相画像を生成することが考えられる。位相画像では、鉄筋の輪郭の情報などが強調される。例えば、輝度画像では、鉄筋に光が上からあたっているか、下からあたっているかの違いで、画像中で鉄筋の明るく写る領域が変わることがある。しかしながら、位相画像では外形が強調されるため、撮影の際の鉄筋に対する光の照射角などの影響を受けにくい。そのため、位相画像を比較に用いることで類似度の評価精度を高めることが可能である。判定対象の鉄筋画像から得た位相画像と、サンプル鉄筋画像から得た位相画像との類似度の評価は、一例では、局所特徴量を用いたマッチングにより実行されてよい。 (Evaluation example 7)
Phase information may be collected from the results of the Fourier transform and used for evaluation of similarity. For example, it is conceivable to acquire phase information from the result of performing a two-dimensional Fourier transform on the rebar image to be determined and the sample rebar image to generate a phase image. In the phase image, information such as the contour of the reinforcing bar is emphasized. For example, in a luminance image, the bright area of the reinforcing bar may change depending on whether the reinforcing bar is exposed to light from above or below. However, since the outer shape is emphasized in the phase image, it is not easily affected by the irradiation angle of light on the reinforcing bar at the time of photographing. Therefore, it is possible to improve the evaluation accuracy of the similarity by using the phase image for comparison. In one example, the evaluation of the degree of similarity between the phase image obtained from the rebar image to be determined and the phase image obtained from the sample rebar image may be performed by matching using a local feature amount.
　また、位相情報を用いて、例えば、位相限定相関法または回転不変位相限定相関法を実行し、判定対象の鉄筋画像とサンプル鉄筋画像との類似度を評価してもよい。 Further, using the phase information, for example, a phase-limited correlation method or a rotation-invariant phase-limited correlation method may be executed to evaluate the degree of similarity between the rebar image to be determined and the sample rebar image.
　以上で、いくつかの類似度の評価法について例示した。なお、上述の評価例１から評価例７では、例えば、フーリエ変換の結果、周波数スペクトル、周波数、エッジ間の距離、波形、位相の情報、局所特徴量などを特徴情報として用いて類似度を評価している。これらの特徴情報は、いずれも鉄筋の画像から得られた情報であるため、これらの特徴情報を用いた類似度の評価により、鉄筋の画像の類似度を評価することができる。しかしながら、実施形態にかかる類似度の評価は上記に限定されるものではなく、上記の手法を組み合わせて用いても、またはその他の手法で類似度が評価されてもよい。 Above, we have illustrated some evaluation methods of similarity. In the above-mentioned evaluation examples 1 to 7, for example, the similarity is evaluated by using the result of Fourier transform, frequency spectrum, frequency, distance between edges, waveform, phase information, local feature amount, etc. as feature information. doing. Since all of these feature information are information obtained from the image of the reinforcing bar, the similarity of the image of the reinforcing bar can be evaluated by evaluating the similarity using these feature information. However, the evaluation of the similarity according to the embodiment is not limited to the above, and the similarity may be evaluated by using the above methods in combination or by another method.
　例えば、別の実施形態では、深層学習などの機械学習を用いて画像間の類似度を評価し、画像に写る鉄筋の種別を表す種別情報を出力することもできる。以下の評価例８では、機械学習を用いる場合について例示する。 For example, in another embodiment, it is possible to evaluate the similarity between images by using machine learning such as deep learning and output type information indicating the type of reinforcing bar shown in the image. In the following evaluation example 8, the case where machine learning is used will be illustrated.
　（評価例８）
　深層学習などの機械学習を行い得られた学習済みモデルを用いて、判定対象の鉄筋画像に写る鉄筋の種別を特定してもよい。図６は、実施形態に係る学習済みモデルの生成を例示する図である。例えば、鉄筋を写したサンプル鉄筋画像と、そのサンプル鉄筋画像に写る鉄筋の種別を示す種別情報と対応づけた複数の情報を学習の教師データとして用いる（図６（ａ））。そして、例えば、入力層に入力されたサンプル鉄筋画像に対して、そのサンプル鉄筋画像に写る鉄筋の種別情報を出力層から出力するように、ニューラルネットワークの重み付けを学習することで学習済みモデルが生成されてよい（図６（ｂ））。制御部３０１は、得られた学習済みモデルに、判定対象の鉄筋画像を入力することで、学習済みモデルから鉄筋画像と対応する鉄筋の種別情報の出力を受けることができる（図６（ｃ））。また、一例として、シャムネットワーク（Siamese Network）を用いて学習が実行されてもよい。シャムネットワークは、例えば、２つの画像を入力として、その画像どうしが似ているか似ていないかの距離を算出するのに適したネットワークである。また、機械学習手法としては、深層学習に限られるものではなく、サポートベクターマシン(support vector machine:SVM)等を用いてもよい。 (Evaluation example 8)
The type of the reinforcing bar shown in the reinforcing bar image to be determined may be specified by using the trained model obtained by performing machine learning such as deep learning. FIG. 6 is a diagram illustrating the generation of the trained model according to the embodiment. For example, a plurality of information associated with a sample reinforcing bar image showing a reinforcing bar and type information indicating the type of the reinforcing bar shown in the sample reinforcing bar image is used as learning teacher data (FIG. 6A). Then, for example, a trained model is generated by learning the weighting of the neural network so that the type information of the reinforcing bars reflected in the sample reinforcing bar image is output from the output layer for the sample reinforcing bar image input to the input layer. It may be done (Fig. 6 (b)). By inputting the reinforcing bar image to be determined into the obtained trained model, the control unit 301 can receive output of the reinforcing bar type information corresponding to the reinforcing bar image from the trained model (FIG. 6C). ). Further, as an example, learning may be executed using a Siamese network. The Siamese network is a network suitable for calculating, for example, the distance between two images as inputs and whether the images are similar or dissimilar. Further, the machine learning method is not limited to deep learning, and a support vector machine (SVM) or the like may be used.
　なお、学習済みモデルは、例えば、利用の段階で、学習済みモデルに鉄筋画像を入力し、鉄筋画像に写る鉄筋の種別を判定し終わった場合、その鉄筋画像と判定結果の種別情報とを教師データとして、更なる学習が実行されてもよい。 For the trained model, for example, when a reinforcing bar image is input to the trained model at the stage of use and the type of the reinforcing bar reflected in the reinforcing bar image is determined, the training bar image and the type information of the determination result are taught. Further learning may be performed as data.
　例えば、以上で述べたように、判定対象の鉄筋画像とサンプル鉄筋画像との類似度は、様々な手法を用いて評価することができる。 For example, as described above, the degree of similarity between the rebar image to be judged and the sample rebar image can be evaluated by using various methods.
　[鉄筋画像の正規化]
　続いて、鉄筋画像の正規化について説明する。例えば、施工検査の現場で撮影された画像に写る判定対象の鉄筋画像と、サンプル鉄筋画像との類似度の評価は、２つの画像に写る鉄筋の撮影条件ができるだけ揃っていた方が、類似度の評価精度が向上する。例えば、判定対象の鉄筋画像とサンプル鉄筋画像とで、画像に写る鉄筋の向きはおおよそ揃っている方が類似度の比較において好ましい。また、画像に写る鉄筋のサイズも、実際の鉄筋のサイズに対して同じ比率で拡大または縮小されて写っている方が、画像間での鉄筋のサイズの比較が可能となるため、好ましい。 [Normalization of rebar image]
Next, the normalization of the reinforcing bar image will be described. For example, in the evaluation of the similarity between the rebar image to be judged and the sample rebar image shown in the image taken at the construction inspection site, it is better that the shooting conditions of the rebar shown in the two images are as good as possible. Evaluation accuracy is improved. For example, it is preferable that the directions of the reinforcing bars shown in the image of the reinforcing bar image to be determined and the sample reinforcing bar image are approximately the same in the comparison of similarity. Further, it is preferable that the size of the reinforcing bar shown in the image is enlarged or reduced at the same ratio with respect to the actual size of the reinforcing bar because the size of the reinforcing bar can be compared between the images.
　そのため、例えば、施工検査の現場で判定対象の鉄筋を撮影するユーザは、判定対象の鉄筋を撮影する向き、距離、および焦点距離などの撮影条件が、サンプル鉄筋画像の撮影の際に用いた撮影条件と所定の誤差範囲内で一致するように撮影を行ってよい。それにより、判定対象の鉄筋画像とサンプル鉄筋画像との類似度の評価精度を向上させることができる。 Therefore, for example, a user who shoots a reinforcing bar to be judged at a construction inspection site uses shooting conditions such as a direction, a distance, and a focal length to shoot the reinforcing bar to be judged to be used when shooting a sample reinforcing bar image. Shooting may be performed so as to match the conditions within a predetermined error range. Thereby, the evaluation accuracy of the degree of similarity between the rebar image to be determined and the sample rebar image can be improved.
　しかしながら、例えば、撮影方向、撮影距離、および焦点距離などの撮影条件が異なる条件で撮影された画像があったとする。この場合でも、撮影方向を揃えたり、２つの画像に写る鉄筋のサイズが、実際の鉄筋のサイズでの比を反映するように変更したりなど、画像を正規化することが可能である。そのため、制御部３０１は、例えば、鉄筋の写る画像を撮影した撮影装置と、鉄筋画像に写る鉄筋との相対的な位置姿勢に関する情報に基づいて、撮影装置と鉄筋画像に写る鉄筋との相対的な位置姿勢が所定の関係となるように鉄筋の写る画像を正規化してよい。鉄筋の写る画像を撮影した撮影装置と、鉄筋画像に写る鉄筋との相対的な位置姿勢に関する情報は、一例では、記憶部３０２に記憶されていてよい。以下、画像の正規化の例として、まず、撮影方向の正対化について説明する。 However, for example, it is assumed that there are images taken under different shooting conditions such as shooting direction, shooting distance, and focal length. Even in this case, it is possible to normalize the images by aligning the shooting directions and changing the sizes of the reinforcing bars appearing in the two images so as to reflect the ratio of the actual reinforcing bar sizes. Therefore, the control unit 301 is, for example, relative to the photographing device and the reinforcing bar reflected in the reinforcing bar image based on the information regarding the relative position and orientation of the photographing device in which the image of the reinforcing bar is captured and the reinforcing bar captured in the reinforcing bar image. The image of the reinforcing bar may be normalized so that the proper position and orientation have a predetermined relationship. In one example, information regarding the relative position and orientation of the photographing device that captured the image of the reinforcing bar and the reinforcing bar that appears in the reinforcing bar image may be stored in the storage unit 302. Hereinafter, as an example of image normalization, first, the directing of the shooting direction will be described.
　（撮影方向の正対化）
　制御部３０１は、例えば、類似度の評価に用いる鉄筋を撮影して得られた画像内での鉄筋の向きを、所定の向きに変換する処理を実行してよい。一例では、制御部３０１は、画像に写る鉄筋が、撮影装置の光軸に対して正対するように画像内での鉄筋の向きを変更してよい。なお、正対するとは、例えば、鉄筋が含まれる平面の法線方向と、撮影装置の撮影方向（例えば、光軸）とが略平行になる位置姿勢の関係であってよい。施工検査の現場では、撮影環境の制限などによって判定対象の鉄筋に正対した位置から撮影することが難しいことがある。この場合に、制御部３０１は、画像に写る判定対象の鉄筋の向きを、正対する位置から撮影した際の鉄筋の向きとなるように画像変換する処理を実行してよい。なお、画像内に略平行に配置された複数の判定対象の鉄筋ある場合には、制御部３０１は、例えば、複数の判定対象の鉄筋が含まれる平面の法線方向と、撮影装置の撮影方向とが略平行になるように、画像を正対変換する処理を実行してよい。 (Shooting direction facing)
For example, the control unit 301 may execute a process of converting the direction of the reinforcing bar in the image obtained by photographing the reinforcing bar used for the evaluation of the similarity to a predetermined direction. In one example, the control unit 301 may change the orientation of the reinforcing bars in the image so that the reinforcing bars in the image face the optical axis of the photographing device. In addition, facing each other may be, for example, a relationship of a position and orientation in which the normal direction of the plane including the reinforcing bar and the photographing direction (for example, the optical axis) of the photographing apparatus are substantially parallel. At the construction inspection site, it may be difficult to shoot from the position facing the reinforcing bar to be judged due to restrictions on the shooting environment. In this case, the control unit 301 may execute a process of converting the direction of the reinforcing bar to be determined to be reflected in the image so that the direction of the reinforcing bar when the image is taken from the position facing the image is changed. When there are a plurality of determination target reinforcing bars arranged substantially in parallel in the image, the control unit 301 may, for example, determine the normal direction of the plane including the plurality of determination target reinforcing bars and the photographing direction of the photographing device. The process of converting the image to the opposite direction may be executed so that and are substantially parallel to each other.
　図７は、例示的な正対変換を示す図である。図７（ａ）では、撮影装置１０２の撮影方向に対して判定対象の複数の鉄筋を含む平面が傾いている状態が例示されており、画像において鉄筋が斜めの向きで撮影されている。また、図７（ｂ）は、正対変換後の状態を示しており、判定対象の鉄筋群を含む平面の法線方向に対して、撮影装置１０２の撮影方向が略平行になるように、鉄筋群を含む平面が回転されている。 FIG. 7 is a diagram showing an exemplary face-to-face conversion. FIG. 7A exemplifies a state in which a plane including a plurality of reinforcing bars to be determined is tilted with respect to the photographing direction of the photographing device 102, and the reinforcing bars are photographed in an oblique direction in the image. Further, FIG. 7B shows the state after the face-to-face conversion, so that the photographing direction of the photographing apparatus 102 is substantially parallel to the normal direction of the plane including the reinforcing bar group to be determined. The plane containing the reinforcing bars is rotated.
　なお、正対変換は、例えば、以下のように実行することができる。例えば、図７（ａ）に示すように複数の鉄筋１０１を含む平面に対して斜めから撮影された画像において、制御部３０１は、格子状に組まれた鉄筋が形成する矩形の四隅の指定をユーザから受け付けてよい。そして、制御部３０１は、指定された四隅の点で示される矩形の平面を正対させるようにホモグラフィ行列を決定し、決定したホモグラフィ行列を用いて、画像全体の正対変換を実行してよい。 Note that the face-to-face conversion can be executed, for example, as follows. For example, in an image taken at an angle with respect to a plane including a plurality of reinforcing bars 101 as shown in FIG. 7A, the control unit 301 designates the four corners of a rectangle formed by the reinforcing bars assembled in a grid pattern. It may be accepted from the user. Then, the control unit 301 determines the homography matrix so as to face the rectangular planes indicated by the designated four corner points, and executes the face-to-face transformation of the entire image using the determined homography matrix. You can.
　また、例えば、判定対象の複数の鉄筋が写る鉄筋画像がステレオカメラで撮影されているとする。この場合、制御部３０１は、例えば、ステレオ画像の左視点画像と右視点画像とでステレオマッチングを実行することで、画素と対応する３次元データを生成することができる。そして、制御部３０１は、得られた３次元データで表される判定対象の鉄筋群を含む平面の法線ベクトルと、撮影装置１０２の撮影方向とが略平行になるように平面の３次元データを回転させる行列を特定する。そして、制御部３０１は、その求めた行列の情報に基づいて、ステレオ画像の左視点画像または右視点画像を正対した向きに変換するホモグラフィ行列を特定し、特定したホモグラフィ行列を用いて正対画像を生成してもよい。 Also, for example, it is assumed that a reinforcing bar image showing a plurality of reinforcing bars to be judged is taken by a stereo camera. In this case, the control unit 301 can generate three-dimensional data corresponding to the pixels by, for example, performing stereo matching between the left viewpoint image and the right viewpoint image of the stereo image. Then, the control unit 301 uses the three-dimensional data of the plane so that the normal vector of the plane including the reinforcing bar group to be determined represented by the obtained three-dimensional data and the shooting direction of the photographing device 102 are substantially parallel to each other. Identify the matrix that rotates. Then, the control unit 301 identifies a homography matrix that transforms the left-viewpoint image or the right-viewpoint image of the stereo image in the opposite direction based on the obtained matrix information, and uses the specified homography matrix. A face-to-face image may be generated.
　（鉄筋の長手方向の向きの正規化）
　判定対象の鉄筋画像における鉄筋の長手方向の向きと、サンプル鉄筋画像に写る鉄筋の長手方向の向きとは、略平行となる向きで類似度の判定を実行することが好ましい。そのため、制御部３０１は、判定対象の鉄筋画像と、サンプル鉄筋画像とで画像に写る鉄筋の長手方向の向きが略平行になるように、画像を回転させる正規化を行ってもよい。 (Normalization of longitudinal orientation of reinforcing bars)
It is preferable to perform the determination of similarity in the direction in which the longitudinal direction of the reinforcing bar in the reinforcing bar image to be determined and the longitudinal direction of the reinforcing bar shown in the sample reinforcing bar image are substantially parallel to each other. Therefore, the control unit 301 may perform normalization to rotate the image so that the orientation of the reinforcing bar image to be determined and the sample reinforcing bar image in the longitudinal direction of the image are substantially parallel.
　図８は、鉄筋の写る画像の回転を例示する図である。図８では、画像に写る鉄筋の長手方向の向きを、縦方向に合わせるように回転させる場合を例示している。図８に示すように、画像に写る鉄筋の長手方向の向きを、縦方向など所定の向きに揃えるように回転させることで、サンプル鉄筋画像に写る鉄筋の長手方向と、判定対象の鉄筋画像に写る鉄筋の長手方向とを略平行にすることができる。それによって、類似度の評価精度を向上させることができる。 FIG. 8 is a diagram illustrating the rotation of the image in which the reinforcing bar is captured. FIG. 8 illustrates a case where the direction of the reinforcing bar shown in the image in the longitudinal direction is rotated so as to match the vertical direction. As shown in FIG. 8, by rotating the direction of the reinforcing bar in the image in the longitudinal direction so as to be aligned with a predetermined direction such as the vertical direction, the longitudinal direction of the reinforcing bar in the sample reinforcing bar image and the reinforcing bar image to be determined can be obtained. The longitudinal direction of the reflected reinforcing bar can be made substantially parallel. Thereby, the evaluation accuracy of the similarity can be improved.
　（画像サイズの正規化）
　画像サイズについても、例えば、類似度を評価する２つの画像において鉄筋上の所定の長さが、実空間における鉄筋の同じ長さを表すように、正規化されることが望ましい。なお、画像サイズの正規化は、例えば、以下のように実施することができる。 (Normalization of image size)
Regarding the image size, for example, it is desirable that the predetermined lengths on the reinforcing bars in the two images for evaluating the similarity are normalized so as to represent the same length of the reinforcing bars in the real space. The image size can be normalized as follows, for example.
　例えば、画像がステレオカメラで撮影されている場合には、制御部３０１は、ステレオマッチングにより画素と対応する３次元データを取得することができる。そして、制御部３０１は、画像内の鉄筋上での所定の長さが、その鉄筋の３次元データにおける実寸でどのくらいの長さと対応しているのかを特定することができる。そのため、例えば、これらの情報を用いて、制御部３０１は、画像に写る鉄筋上の所定の長さが、実空間における鉄筋上で所定の実寸の長さと対応づくように、画像サイズを正規化することができる。 For example, when the image is taken by a stereo camera, the control unit 301 can acquire the three-dimensional data corresponding to the pixels by stereo matching. Then, the control unit 301 can specify how much the predetermined length on the reinforcing bar in the image corresponds to the actual size in the three-dimensional data of the reinforcing bar. Therefore, for example, using these information, the control unit 301 normalizes the image size so that the predetermined length on the reinforcing bar in the image corresponds to the predetermined actual size length on the reinforcing bar in the real space. can do.
　また、例えば、単眼のカメラで撮影された画像であっても、撮影に用いられた撮影装置１０２の焦点距離と、撮影装置から被写体までの距離とが分かれば、画像上のサイズと対応する実寸を求めることが可能である。そのため、これらの情報を用いて、画像に写る鉄筋上の所定の長さが、実空間における鉄筋上で所定の実寸の長さと対応づくように、画像サイズを正規化してもよい。 Further, for example, even if the image is taken by a monocular camera, if the focal length of the shooting device 102 used for shooting and the distance from the shooting device to the subject are known, the actual size corresponding to the size on the image is known. It is possible to find. Therefore, using this information, the image size may be normalized so that the predetermined length on the reinforcing bar shown in the image corresponds to the predetermined actual size length on the reinforcing bar in the real space.
　そして、以上のように、画像サイズを正規化することで、画像に写る鉄筋のサイズを比較することで、実際の鉄筋のサイズの類似度を評価することが可能になる。 Then, as described above, by normalizing the image size, it is possible to evaluate the similarity of the actual reinforcing bar sizes by comparing the sizes of the reinforcing bars reflected in the image.
　なお、例えば、３次元データや、画像の撮影に用いられた撮影装置１０２の焦点距離および撮影装置１０２から被写体までの距離の情報などの正規化処理に用いられる撮影装置と被写体との間の位置姿勢に関する情報は記憶部３０２に記憶されていてよい。また別の実施形態では、制御部３０１は、例えば、ユーザに入力させてこれらの情報を取得してもよい。或いは、制御部３０１は、レーザスキャナで鉄筋をスキャンしたデータに基づいて３次元データや、撮影距離の情報を取得してもよい。更には、制御部３０１は、所定のスケール等を種別の判定対象の鉄筋の付近に設置して画像の撮影を行い、実空間における寸法の情報を画像から取得してもよい。 Note that, for example, the position between the photographing device and the subject used for normalization processing such as three-dimensional data, the focal length of the photographing device 102 used for photographing the image, and the information on the distance from the photographing device 102 to the subject. Information about the posture may be stored in the storage unit 302. In yet another embodiment, the control unit 301 may, for example, have the user input the information. Alternatively, the control unit 301 may acquire three-dimensional data and information on the shooting distance based on the data obtained by scanning the reinforcing bar with the laser scanner. Further, the control unit 301 may set a predetermined scale or the like in the vicinity of the reinforcing bar of the type to be determined to take an image, and acquire the dimensional information in the real space from the image.
　以上で述べたように、類似度の評価に用いる鉄筋画像に適切な正規化を行うことで、類似度の評価精度を高めることができる。なお、画像の正規化は、さまざまなタイミングで実行されてよい。例えば、制御部３０１は、サンプル鉄筋画像の正規化を、鉄筋情報４０１へのサンプル鉄筋画像の登録時に実行してもよく、或いは、判定対象の鉄筋の画像との類似度の評価の際に実行してもよく、その両方のタイミングで実行してもよい。 As described above, the accuracy of evaluation of similarity can be improved by appropriately normalizing the reinforcing bar image used for evaluation of similarity. The image normalization may be executed at various timings. For example, the control unit 301 may execute the normalization of the sample reinforcing bar image at the time of registering the sample reinforcing bar image in the reinforcing bar information 401, or at the time of evaluating the similarity with the image of the reinforcing bar to be determined. It may be executed at both timings.
　ところで、上述の図４に示す例では、画像４０３から種別の判定対象の鉄筋の写る画像領域を抽出して鉄筋画像４０４を作成している。鉄筋画像４０４の作成は、例えば、ユーザに判定対象の鉄筋の写る画像領域を選択させて実行してもよいし、別の実施形態では、制御部３０１が自動で判定対象の鉄筋の写る画像領域を検出し実行してもよい。また、制御部３０１は、上述の正対変換、鉄筋の長手方向の向きの正規化、および画像サイズの正規化についても、ユーザからの情報の入力を受けて実行しても、自動で実行してもよい。そして、例えば、これらの正規化の処理に、上述の特許文献１に記載される技術や、国際公開第２０１８／１８０４４２号に記載される技術が利用されてもよい。 By the way, in the example shown in FIG. 4 described above, the reinforcing bar image 404 is created by extracting the image area in which the reinforcing bar of the type to be determined is reflected from the image 403. The creation of the reinforcing bar image 404 may be executed, for example, by having the user select an image area in which the reinforcing bar to be determined is captured, or in another embodiment, the control unit 301 automatically captures the image area in which the reinforcing bar to be determined is captured. May be detected and executed. Further, the control unit 301 automatically executes the above-mentioned face-to-face conversion, normalization of the longitudinal direction of the reinforcing bar, and normalization of the image size even if it is executed in response to the input of information from the user. You may. Then, for example, the technique described in Patent Document 1 described above or the technique described in International Publication No. 2018/180442 may be used for these normalization processes.
　（第１の実施形態）
　続いて、図９および図１０を参照して、第１の実施形態を説明する。図９は、第１の実施形態に係る鉄筋情報９００を例示する図である。鉄筋情報９００は、上述の鉄筋情報４０１の一例である。鉄筋情報９００には、例えば、種別情報と、サンプル鉄筋画像情報とが対応づけられたレコードが登録されている。図９の例では、種別情報は、メーカ、径、節の種別の情報を含んでいる。メーカは、レコードのサンプル鉄筋画像に写る鉄筋の製造メーカを識別するための情報である。径は、例えば、レコードのサンプル鉄筋画像に写る鉄筋の径を示す情報である。節の種別は、例えば、レコードのサンプル鉄筋画像に写る鉄筋の節の種別を示す情報であり、一例では、竹節またはネジ節の情報が登録されていてよい。また、サンプル鉄筋画像情報には、例えば、サンプル鉄筋画像の画像データまたは記憶部３０２におけるサンプル鉄筋画像の保存場所などを示す情報が登録されていてよい。制御部３０１は、鉄筋情報９００を参照することで、判定対象の鉄筋画像との類似度の評価に用いるサンプル鉄筋画像の情報を取得することができる。 (First Embodiment)
Subsequently, the first embodiment will be described with reference to FIGS. 9 and 10. FIG. 9 is a diagram illustrating the reinforcing bar information 900 according to the first embodiment. The reinforcing bar information 900 is an example of the above-mentioned reinforcing bar information 401. In the reinforcing bar information 900, for example, a record in which the type information and the sample reinforcing bar image information are associated with each other is registered. In the example of FIG. 9, the type information includes information on the manufacturer, diameter, and section type. The manufacturer is information for identifying the manufacturer of the reinforcing bar shown in the sample reinforcing bar image of the record. The diameter is, for example, information indicating the diameter of the reinforcing bar shown in the sample reinforcing bar image of the record. The type of knot is, for example, information indicating the type of the knot of the reinforcing bar shown in the sample reinforcing bar image of the record, and in one example, the information of the bamboo knot or the screw knot may be registered. Further, in the sample rebar image information, for example, information indicating the image data of the sample rebar image or the storage location of the sample rebar image in the storage unit 302 may be registered. By referring to the reinforcing bar information 900, the control unit 301 can acquire information on the sample reinforcing bar image used for evaluating the similarity with the reinforcing bar image to be determined.
　続いて、第１の実施形態に係る種別判定処理を説明する。図１０は、第１の実施形態に係る種別判定処理の動作フローを例示する図である。鉄筋判定装置２０１の制御部３０１は、例えば、種別の判定対象の鉄筋画像の入力を受け付け、入力されると、図１０の動作フローを開始してよい。なお、判定対象の鉄筋画像は、例えば、撮影装置１０２から制御部３０１に入力されてもよいし、記憶部３０２からユーザにより指定された鉄筋画像を読み出すことで制御部３０１に入力されてもよい。 Subsequently, the type determination process according to the first embodiment will be described. FIG. 10 is a diagram illustrating an operation flow of the type determination process according to the first embodiment. The control unit 301 of the reinforcing bar determination device 201 may, for example, accept the input of the reinforcing bar image of the type of determination target, and when the input is input, start the operation flow of FIG. The rebar image to be determined may be input to the control unit 301 from the photographing device 102, or may be input to the control unit 301 by reading the rebar image designated by the user from the storage unit 302. ..
　ステップ１００１（以降、ステップを“Ｓ”と記載し、例えば、Ｓ１００１と表記する）において鉄筋判定装置２０１の制御部３０１は、鉄筋情報９００を参照し、複数のサンプル鉄筋画像を読み出す。例えば、制御部３０１は、鉄筋情報９００に登録されている全てのレコードと対応するサンプル鉄筋画像を読み出してよい。 In step 1001 (hereinafter, step is referred to as "S" and is referred to as S1001), the control unit 301 of the reinforcing bar determination device 201 refers to the reinforcing bar information 900 and reads out a plurality of sample reinforcing bar images. For example, the control unit 301 may read out a sample reinforcing bar image corresponding to all the records registered in the reinforcing bar information 900.
　Ｓ１００２において制御部３０１は、入力された判定対象の鉄筋画像と、読み出した複数のサンプル鉄筋画像との類似度を評価する。画像の類似度の評価は、様々な手法を用いて実行することができ、制御部３０１は、例えば、上述の評価例１から評価例７の手法を用いて、または組み合わせて類似度を評価してよい。 In S1002, the control unit 301 evaluates the degree of similarity between the input reinforcing bar image to be determined and the read-out sample reinforcing bar images. The evaluation of the similarity of the images can be performed by using various methods, and the control unit 301 evaluates the similarity by using, for example, the methods of the above-mentioned evaluation examples 1 to 7 or in combination. You can.
　Ｓ１００３において制御部３０１は、類似度の評価結果に基づいて、読み出した複数のサンプル鉄筋画像のうちで、所定の条件を満たして類似しているサンプル鉄筋画像を特定する。所定の条件は、一例では、類似度の評価結果に基づいて、読み出したサンプル鉄筋画像のうちで最も類似していると評価されることであってよい。また、別の実施形態では、例えば、類似度の評価に用いた指標が高いほど類似していることを示す場合には、所定の条件は、類似度の評価に用いた指標が所定の閾値以上であることであってよい。一方、例えば、類似度の評価に用いた指標が低いほど類似していることを示す場合には、所定の条件は、類似度の評価に用いた指標が所定の閾値以下であることであってよい。類似の評価に用いた指標は、例えば、評価例１で述べた差分の合算値、評価例２で述べた差分の大きさを積算した合計値、積率相関係数、局所特徴量のマッチングの結果、位相限定相関法および回転不変位相限定相関法の実行結果などを含んでよい。 In S1003, the control unit 301 identifies a similar sample rebar image that satisfies a predetermined condition from the plurality of sample rebar images read out based on the evaluation result of the similarity. The predetermined condition may be, in one example, being evaluated as being the most similar among the read sample rebar images based on the evaluation result of the similarity. Further, in another embodiment, for example, when the higher the index used for the evaluation of similarity indicates that it is similar, the predetermined condition is that the index used for evaluating the similarity is equal to or higher than a predetermined threshold value. It may be. On the other hand, for example, when the lower the index used for the evaluation of similarity indicates that it is similar, the predetermined condition is that the index used for evaluating the similarity is equal to or less than a predetermined threshold value. Good. The indexes used for the similar evaluation are, for example, the total value of the differences described in the evaluation example 1, the total value obtained by integrating the magnitudes of the differences described in the evaluation example 2, the product moment correlation coefficient, and the matching of the local feature amount. As a result, the execution results of the phase-limited correlation method and the rotation-invariant phase-limited correlation method may be included.
　Ｓ１００４において制御部３０１は、所定の条件を満たして類似しているサンプル鉄筋画像と鉄筋情報４０１において対応づけられる種別情報を特定する。そして、Ｓ１００５において制御部３０１は、特定した種別情報を出力して本動作フローは終了する。なお、例えば、判定対象の鉄筋画像と所定の条件を満たして類似しているサンプル鉄筋画像が複数あるとする。この場合、制御部３０１は、その複数のサンプル鉄筋画像のそれぞれと対応する鉄筋の種別情報を全て出力してもよいし、それらのうちで径のサイズが最も小さい鉄筋の種別情報を出力してもよい。径のサイズが最も小さい鉄筋の種別情報を出力することで、例え判定された鉄筋の種別が誤っていたとしても、その種別を用いて計算される構造物が最低限有する強度を見積もることができる。 In S1004, the control unit 301 specifies the type information associated with the sample reinforcing bar image that satisfies the predetermined conditions and is similar to the reinforcing bar information 401. Then, in S1005, the control unit 301 outputs the specified type information, and this operation flow ends. It should be noted that, for example, it is assumed that there are a plurality of sample rebar images that are similar to the rebar image to be determined by satisfying predetermined conditions. In this case, the control unit 301 may output all the type information of the reinforcing bars corresponding to each of the plurality of sample reinforcing bar images, or output the type information of the reinforcing bar having the smallest diameter among them. May be good. By outputting the type information of the reinforcing bar with the smallest diameter size, even if the determined reinforcing bar type is incorrect, the minimum strength of the structure calculated using that type can be estimated. ..
　（第２の実施形態）
　続いて、第２の実施形態について説明する。異形鉄筋にはリブや節などが形成されており、その形状は様々である。例えば、鉄筋の長手方向の軸を回転軸として鉄筋を回転させて、長手方向の軸に直交する定位置から観察すると、図１１に示すように、回転角によって鉄筋が異なる形状をしていることがある。例えば、図１１（ａ）では、鉄筋の全体に節が示されているが、図１１（ｂ）では、軸方向に一部において節が形成されていない。そして、このように、異なる形状をしている回転角で撮影された画像で類似度を評価すると、同じ種別の鉄筋の鉄筋画像同士で類似度を評価しても、類似度が低くなってしまうことがある。その結果、鉄筋の種別の判定精度が低下することがある。 (Second Embodiment)
Subsequently, the second embodiment will be described. Ribs and knots are formed on the deformed reinforcing bars, and their shapes vary. For example, when the reinforcing bar is rotated around the axis in the longitudinal direction of the reinforcing bar and observed from a fixed position orthogonal to the axis in the longitudinal direction, the reinforcing bar has a different shape depending on the rotation angle, as shown in FIG. There is. For example, in FIG. 11 (a), knots are shown in the entire reinforcing bar, but in FIG. 11 (b), knots are not formed in a part in the axial direction. Then, when the similarity is evaluated by the images taken at the rotation angles having different shapes in this way, even if the similarity is evaluated between the reinforcing bar images of the same type of reinforcing bars, the similarity is low. Sometimes. As a result, the accuracy of determining the type of reinforcing bar may decrease.
　そこで、第２の実施形態では、図１２に例示するように、１つの種別に対して、複数の角度から撮影した鉄筋画像の情報が鉄筋情報１２００のサンプル鉄筋画像情報に登録される。例えば、１つの種別の鉄筋に対し、鉄筋に正対した位置から鉄筋の長手方向を回転軸として回転させて複数のサンプル鉄筋画像を撮影する。そして、鉄筋情報１２００には、１つの種別情報で識別されるレコードのサンプル鉄筋画像情報に、鉄筋を様々な角度から撮影した複数の鉄筋画像の情報が登録されてよい。 Therefore, in the second embodiment, as illustrated in FIG. 12, the information of the reinforcing bar images taken from a plurality of angles for one type is registered in the sample reinforcing bar image information of the reinforcing bar information 1200. For example, for one type of reinforcing bar, a plurality of sample reinforcing bar images are taken by rotating the reinforcing bar from a position facing the reinforcing bar with the longitudinal direction of the reinforcing bar as a rotation axis. Then, in the reinforcing bar information 1200, information on a plurality of reinforcing bar images obtained by photographing the reinforcing bars from various angles may be registered in the sample reinforcing bar image information of the record identified by one type information.
　そして、例えば、上述のＳ１００２の処理では、制御部３０１は、判定対象の鉄筋画像に対して、鉄筋情報１２００において１つのレコードと対応する複数のサンプル鉄筋画像のそれぞれとの類似度を評価する。そして、Ｓ１００３において制御部３０１は、その複数の類似度の評価結果に基づいて、鉄筋画像と所定の条件を満たして類似しているサンプル鉄筋画像のレコードを特定してよい。なお、一例では、制御部３０１はＳ１００２において評価した複数のサンプル鉄筋画像のそれぞれとの類似度の指標の中から最も似ていると評価される指標の値を、そのレコードの種別のサンプル鉄筋画像との類似度の指標として用いてよい。 Then, for example, in the above-mentioned processing of S1002, the control unit 301 evaluates the similarity between the rebar image to be determined and each of the plurality of sample rebar images corresponding to one record in the rebar information 1200. Then, in S1003, the control unit 301 may specify a record of a sample reinforcing bar image that satisfies a predetermined condition and is similar to the reinforcing bar image based on the evaluation results of the plurality of similarities. In one example, the control unit 301 sets the value of the index evaluated to be the most similar among the indexes of the similarity with each of the plurality of sample rebar images evaluated in S1002 to the sample rebar image of the record type. It may be used as an index of similarity with.
　例えば、このように、様々な角度から撮影したサンプル鉄筋画像を用いて判定対象の鉄筋画像との類似度を評価し、種別を判定することで、見る角度によって形状の異なる鉄筋であっても類似度を適切に評価することができる。そのため、鉄筋の種別の判定精度を向上させることができる。 For example, by evaluating the degree of similarity with the rebar image to be judged using sample rebar images taken from various angles and determining the type, even rebars having different shapes depending on the viewing angle are similar. The degree can be evaluated appropriately. Therefore, the accuracy of determining the type of reinforcing bar can be improved.
　また、上記ではサンプル鉄筋画像を複数登録する例を述べているが、実施形態はこれに限定されるものではない。別の実施形態では、制御部３０１は、判定対象の複数の鉄筋画像の入力を受け付けてよく、Ｓ１００２において複数の鉄筋画像のそれぞれとサンプル鉄筋画像との類似度を評価してよい。そして、Ｓ１００３において制御部３０１は、その複数の類似度の評価結果に基づいて、複数の鉄筋画像と所定の条件を満たして類似しているサンプル鉄筋画像を特定してよい。一例では、制御部３０１はＳ１００２において評価した複数の鉄筋画像のそれぞれとサンプル鉄筋画像との複数の類似度の指標の中から最も似ていると評価される指標の値を、そのレコードの種別のサンプル鉄筋画像との類似度の指標として用いてよい。 Although the above describes an example of registering a plurality of sample reinforcing bar images, the embodiment is not limited to this. In another embodiment, the control unit 301 may accept the input of the plurality of rebar images to be determined, and may evaluate the similarity between each of the plurality of rebar images and the sample rebar image in S1002. Then, in S1003, the control unit 301 may specify a sample reinforcing bar image that is similar to the plurality of reinforcing bar images by satisfying predetermined conditions, based on the evaluation results of the plurality of similarities. In one example, the control unit 301 sets the value of the index evaluated to be the most similar among the indexes of the degree of similarity between each of the plurality of reinforcing bar images evaluated in S1002 and the sample reinforcing bar image as the type of the record. It may be used as an index of similarity with the sample reinforcing bar image.
　なお、入力される判定対象の複数の鉄筋画像は、上述のように異なる角度から撮影された鉄筋画像であってもよいし、別の例では、ほぼ同じ位置で撮影された複数の鉄筋画像であってもよい。この場合にも、例えば、撮影者の撮影の際の動きに起因する手振れなどの影響の出方が画像によって異なることがあり、また、照明の光の揺らぎなどに起因して異なる画像が撮影され得る。そのため、鉄筋の種別の判定精度を向上させることができる。なお、入力される複数の鉄筋画像は、例えば、連射撮影および動画撮影などにより撮影された複数の画像であってもよい。 The plurality of rebar images to be input may be rebar images taken from different angles as described above, and in another example, a plurality of rebar images taken at substantially the same position may be used. There may be. In this case as well, for example, the appearance of the influence of camera shake caused by the movement of the photographer during shooting may differ depending on the image, and different images are taken due to fluctuations in the light of the illumination. obtain. Therefore, the accuracy of determining the type of reinforcing bar can be improved. The plurality of input reinforcing bar images may be, for example, a plurality of images taken by continuous shooting, moving image shooting, or the like.
　（第３の実施形態）
　上述の実施形態では、例えば、図９に示すように、記憶部３０２の鉄筋情報９００のサンプル鉄筋画像情報には、サンプル鉄筋画像の情報が登録され、記憶部３０２にはサンプル鉄筋画像が保存されている場合の例を説明した。しかしながら、実施形態はこれに限定されるものではない。例えば、鉄筋情報９００のサンプル鉄筋画像情報には、サンプル鉄筋画像の情報の代わりに、類似度の評価に用いるサンプル鉄筋画像から抽出された特徴情報が登録されてもよい。 (Third Embodiment)
In the above-described embodiment, for example, as shown in FIG. 9, the sample rebar image information is registered in the sample rebar image information of the rebar information 900 of the storage unit 302, and the sample rebar image is stored in the storage unit 302. An example of the case where is used is explained. However, the embodiment is not limited to this. For example, in the sample rebar image information of the rebar information 900, feature information extracted from the sample rebar image used for evaluation of similarity may be registered instead of the information of the sample rebar image.
　図１３は、第３の実施形態に係る鉄筋情報１３００を例示する図である。図１３の例では、鉄筋情報１３００のサンプル鉄筋画像情報には、サンプル鉄筋画像に写る鉄筋の長手方向の画素値をフーリエ変換して得られた周波数スペクトルが格納されている。 FIG. 13 is a diagram illustrating the reinforcing bar information 1300 according to the third embodiment. In the example of FIG. 13, the sample reinforcing bar image information of the reinforcing bar information 1300 stores a frequency spectrum obtained by Fourier transforming the pixel values in the longitudinal direction of the reinforcing bars reflected in the sample reinforcing bar image.
　そして、例えば、上述のＳ１００１の処理では、制御部３０１は、鉄筋情報１３００のサンプル鉄筋画像情報から周波数スペクトルを読み出し、Ｓ１００２において入力された判定対象の鉄筋画像から得られた周波数スペクトルとの類似度を評価してよい。サンプル鉄筋画像から予め類似度の評価に用いる特徴情報を取得して鉄筋情報１３００に登録しておくことで、類似度の評価のたびにサンプル鉄筋画像から特徴情報を取得する処理を削減することができる。また、記憶部３０２に、サンプル鉄筋画像を保存する必要がない。 Then, for example, in the above-mentioned processing of S1001, the control unit 301 reads the frequency spectrum from the sample reinforcing bar image information of the reinforcing bar information 1300, and the similarity with the frequency spectrum obtained from the reinforcing bar image of the determination target input in S1002. May be evaluated. By acquiring the feature information used for the evaluation of the similarity from the sample reinforcing bar image in advance and registering it in the reinforcing bar information 1300, it is possible to reduce the process of acquiring the feature information from the sample reinforcing bar image every time the similarity is evaluated. it can. Further, it is not necessary to store the sample reinforcing bar image in the storage unit 302.
　なお、鉄筋情報１３００のサンプル鉄筋画像情報に登録される特徴情報は、周波数スペクトルに限定されるものではなく、例えば、上述の評価例１～評価例７で類似度の評価に用いたその他の情報が登録されてもよい。例えば、サンプル鉄筋画像情報に登録される特徴情報は、サンプル鉄筋画像をフーリエ変換した結果、フーリエ変換の結果から得られた周波数スペクトル、周波数スペクトルのピークの周波数、フーリエ変換の結果から抽出した位相情報、サンプル鉄筋画像またはその位相画像から抽出した局所特徴量、ならびに鉄筋の軸に略平行な両端のエッジ間の距離などの少なくとも１つの情報が登録されもよい。 The feature information registered in the sample reinforcing bar image information of the reinforcing bar information 1300 is not limited to the frequency spectrum, and is, for example, other information used for the evaluation of the similarity in the above-mentioned evaluation examples 1 to 7. May be registered. For example, the feature information registered in the sample reinforcing bar image information is the frequency spectrum obtained from the result of Fourier transform of the sample reinforcing bar image, the frequency of the peak of the frequency spectrum, and the phase information extracted from the result of Fourier transform. , The local feature amount extracted from the sample reinforcing bar image or the phase image thereof, and at least one information such as the distance between the edges at both ends substantially parallel to the axis of the reinforcing bar may be registered.
　（第４の実施形態）
　画像を用いた種別の判定を多段階で実行してもよい。例えば、図１４に例示するように、制御部３０１は、まず竹節であるか、またはネジ節であるかなどの仕分種別を先に判定する。続いて、竹節であると判定された場合には、２段階目で、制御部３０１は、竹節に属する鉄筋のみが登録された鉄筋情報１４０１を参照して図１０の動作フローを実行し、更なる種別判定を行ってよい。また、同様に、例えば、ネジ節であると判定された場合には、２段階目で、制御部３０１は、ネジ節に属する鉄筋のみが登録された鉄筋情報１４０２を参照して図１０の動作フローを実行し、更なる種別判定を行ってよい。 (Fourth Embodiment)
The type determination using the image may be executed in multiple stages. For example, as illustrated in FIG. 14, the control unit 301 first determines the sorting type such as whether it is a bamboo knot or a screw knot. Subsequently, when it is determined that the reinforcing bar is a bamboo section, in the second stage, the control unit 301 executes the operation flow of FIG. 10 with reference to the reinforcing bar information 1401 in which only the reinforcing bars belonging to the bamboo section are registered, and further. The type may be determined. Similarly, for example, when it is determined that the thread is a threaded node, in the second stage, the control unit 301 refers to the reinforcing bar information 1402 in which only the reinforcing bar belonging to the threaded node is registered, and operates in FIG. The flow may be executed to perform further type determination.
　このように、仕分種別を用いて多段階で種別を判定することで、個々の種別の判定に適した評価法を用いて種別を判定することができ、例えば、上述の複数の評価法を多段階で組み合わせて利用して最終的な種別を判定することも可能である。一例として、ネジ節と竹節を判定するための局所特徴量を抽出したり、または機械学習によりこの２つの種別を判定するように作成した学習モデルを用いたりなど、節の形状に関する判定に効果的な評価法を用いて、ネジ節か竹節かをまず特定することが考えられる。そして、２段階目で、節間隔の判定に効果的な例えば、画像に写る鉄筋の長手方向の画素値の周波数スペクトルを用いて判定を行い、鉄筋の種別を判定することが考えられる。 In this way, by determining the type in multiple stages using the sorting type, the type can be determined using an evaluation method suitable for determining each type. For example, the above-mentioned plurality of evaluation methods are often used. It is also possible to use them in combination in stages to determine the final type. As an example, it is effective for determining the shape of nodes, such as extracting local features for determining screw nodes and bamboo nodes, or using a learning model created to determine these two types by machine learning. It is conceivable to first identify whether it is a screw knot or a bamboo knot by using a simple evaluation method. Then, in the second stage, it is conceivable to determine the type of the reinforcing bar by performing the determination using the frequency spectrum of the pixel value in the longitudinal direction of the reinforcing bar, which is effective for determining the node spacing, for example.
　なお、最終的な鉄筋の種別判定の前に事前に仕分けを行うための仕分種別としては、鉄筋の様々な種別を用いることができ、別の例では、鉄筋の節に形成されているメーカのロゴなどを評価してメーカごとに仕分ける処理が実行されてもよい。また、異なる種類の仕分種別による仕分けを組み合わせて、仕分けを多段階で複数回行ってもよい。 In addition, various types of reinforcing bars can be used as the sorting type for sorting in advance before the final classification of the reinforcing bar type. In another example, the manufacturer's section formed in the reinforcing bar node A process of evaluating a logo or the like and sorting by manufacturer may be executed. In addition, sorting by different types of sorting types may be combined and sorting may be performed a plurality of times in multiple stages.
　（第５の実施形態）
　上述の実施形態の図１０のＳ１００５において、種別情報を出力する出力先は、一例では表示部３０３の表示画面であってよいし、別の例では、配筋検査の検査結果を記録する帳票などの検査情報１５００あってもよい。 (Fifth Embodiment)
In S1005 of FIG. 10 of the above-described embodiment, the output destination for outputting the type information may be the display screen of the display unit 303 in one example, and in another example, a form for recording the inspection result of the bar arrangement inspection or the like. Inspection information 1500 may be available.
　図１５は、第５の実施形態に係る配筋検査の検査結果を記録する帳票などの検査情報１５００を例示する図である。検査情報１５００には、例えば、工事現場における或るエリアの工事に関する設計図と対応する情報が登録されていてよく、例えば、工事識別情報１５１１および検査対象情報１５１２を含む。 FIG. 15 is a diagram illustrating inspection information 1500 such as a form for recording the inspection result of the bar arrangement inspection according to the fifth embodiment. In the inspection information 1500, for example, information corresponding to a design drawing relating to construction in a certain area at a construction site may be registered, and includes, for example, construction identification information 1511 and inspection target information 1512.
　工事識別情報１５１１は、例えば、工事の場所および工事の概要を示す情報を含む。図１５の例では、工事識別情報１５１１は、番号、工事名、工事種、構造物番号、部材名などの情報を含む。番号は、例えば、検査情報１５００に付与された識別のための情報である。工事名は、例えば、工事の名称であってよい。工種は、例えば、工事の場所を示す情報であり、例えば、図１５の工種：橋台躯体工は、橋脚を支える土台部分の工事であることを示している。構造物番号は、例えば、工種内の工事の対象となる構造物を指定する情報である。例えば、図１５に示される構造物番号：Ａ０１は、どの橋脚の橋台躯体工であるかを示している。部材名は、例えば、構造物内の工事の対象となる領域を示す情報である。例えば、図１５に示される部材名：フーチングは、橋台躯体工内の基礎部分であるフーチングの工事であることを示している。作業者は、例えば、検査情報１５００の工事識別情報１５１１の情報を参照することで、検査情報１５００がどこの工事についての情報を記載した検査情報１５００であるかを把握することができる。 The construction identification information 1511 includes, for example, information indicating the location of construction and the outline of construction. In the example of FIG. 15, the construction identification information 1511 includes information such as a number, a construction name, a construction type, a structure number, and a member name. The number is, for example, information for identification given to the inspection information 1500. The construction name may be, for example, the name of the construction. The work type is, for example, information indicating the location of the work, and for example, the work type in FIG. 15,: Hashidai skeleton work, indicates that the work is the work of the base portion that supports the pier. The structure number is, for example, information for designating a structure to be constructed within the work type. For example, the structure number: A01 shown in FIG. 15 indicates which pier is the pier skeleton. The member name is, for example, information indicating an area to be constructed in the structure. For example, the member name: footings shown in FIG. 15 indicates that the footings work is the foundation part of the pier skeleton work. By referring to, for example, the information of the construction identification information 1511 of the inspection information 1500, the worker can grasp which construction information 1500 is the inspection information 1500.
　また、検査対象情報１５１２には、例えば、検査対象と、検査項目の情報とを対応づけるレコードが登録されている。検査対象は、例えば、検査情報１５００に示される工事で行われる配筋検査の検査対象（例えば、鉄筋）を指定する情報である。図１５の例では、検査対象情報１５１２の検査対象は、位置１、位置２、および鉄筋番号の情報を含んでいる。位置１および位置２は、例えば、工事の設計図において検査対象が配置されるエリアを示す情報であってよい。なお、位置２は、例えば、位置１内での更なる詳細なエリアを指定する情報であってよい。鉄筋番号は、位置の情報で指定されるエリア内で使用されている鉄筋、または鉄筋のグループに割り振られた番号である。 Further, in the inspection target information 1512, for example, a record for associating the inspection target with the information of the inspection item is registered. The inspection target is, for example, information for designating an inspection target (for example, a reinforcing bar) of a bar arrangement inspection performed in the construction shown in the inspection information 1500. In the example of FIG. 15, the inspection target of the inspection target information 1512 includes the information of the position 1, the position 2, and the reinforcing bar number. The position 1 and the position 2 may be, for example, information indicating an area where an inspection target is arranged in a construction design drawing. The position 2 may be, for example, information that specifies a more detailed area within the position 1. The reinforcing bar number is a number assigned to a reinforcing bar or a group of reinforcing bars used in the area specified by the position information.
　また、検査項目は、例えば、検査対象情報１５１２の検査対象に対して実行する検査の項目を示す情報である。図１５の例では、検査対象情報１５１２の検査項目は、検査項目１および検査項目２を含んでいる。また、検査項目１に示すように、検査項目は、例えば、設計値、および計測値を含んでよい。設計値は、検査項目に対する設計図上での値である。計測値には、例えば、配筋検査で計測された計測結果が登録される。検査項目は、例えば、検査対象の鉄筋の径、位置、配置の平均間隔、かぶり厚さ、および本数などを含んでよい。 Further, the inspection item is, for example, information indicating an inspection item to be executed for the inspection target of the inspection target information 1512. In the example of FIG. 15, the inspection item of the inspection target information 1512 includes the inspection item 1 and the inspection item 2. Further, as shown in the inspection item 1, the inspection item may include, for example, a design value and a measured value. The design value is a value on the design drawing for the inspection item. For example, the measurement result measured by the bar arrangement inspection is registered in the measured value. The inspection items may include, for example, the diameter, position, average spacing of arrangements, cover thickness, and number of reinforcing bars to be inspected.
　そして、第５の実施形態では制御部３０１は、上述のＳ１００５の処理で判定した種別の情報を、検査情報１５００の計測値に出力してよい。例えば、制御部３０１は、判定対象の鉄筋画像の種別に基づいて、鉄筋情報９００から鉄筋の径の情報を取得し、図１５の枠１５０１で示すように検査項目の計測値に、取得した径の情報を登録してよい。また、制御部３０１は、図１５の枠１５０２で示すように、計測値の特定に用いた判定対象の鉄筋画像を識別する識別情報を検査情報１５００に登録してよい。なお、別の実施形態では、識別情報は、判定対象の鉄筋画像から抽出された周波数スペクトルなどの特徴情報を識別する情報であってもよい。識別情報は、一例では、判定対象の鉄筋画像または判定対象の鉄筋画像から抽出された特徴情報にアクセスするために用いるパスやファイル名であってよい。 Then, in the fifth embodiment, the control unit 301 may output the information of the type determined in the process of S1005 described above to the measured value of the inspection information 1500. For example, the control unit 301 acquires information on the diameter of the reinforcing bar from the reinforcing bar information 900 based on the type of the reinforcing bar image to be determined, and sets the acquired diameter in the measured value of the inspection item as shown in the frame 1501 of FIG. Information may be registered. Further, as shown in the frame 1502 of FIG. 15, the control unit 301 may register the identification information for identifying the reinforcing bar image to be determined used for specifying the measured value in the inspection information 1500. In another embodiment, the identification information may be information that identifies feature information such as a frequency spectrum extracted from the reinforcing bar image to be determined. In one example, the identification information may be a path or a file name used to access the rebar image to be determined or the feature information extracted from the rebar image to be determined.
　配筋の検査では、検査結果が何者かによって改ざんされていないかなどの確認などが必要となることがある。そのため、制御部３０１は、例えば、上述のように、検査項目に登録する検査結果と対応づけて、検査に用いた判定対象の鉄筋画像または鉄筋画像から抽出された特徴情報を識別するための識別情報を、検査情報１５００に記録する。このように、検査結果と関連づけて、検査に用いた判定対象の鉄筋画像または鉄筋画像から抽出された特徴情報を識別するための識別情報を記録することで、制御部３０１は、後に図１０の動作フローを再度実行して計測値を求めなおすことが可能となる。その結果、検査結果の改ざんの有無を検証することが可能である。 In the bar arrangement inspection, it may be necessary to confirm whether the inspection result has been tampered with by someone. Therefore, for example, as described above, the control unit 301 identifies the reinforcing bar image to be determined used in the inspection or the feature information extracted from the reinforcing bar image in association with the inspection result registered in the inspection item. The information is recorded in the inspection information 1500. In this way, by recording the identification information for identifying the rebar image of the determination target used for the inspection or the feature information extracted from the rebar image in association with the inspection result, the control unit 301 later described FIG. It is possible to re-execute the operation flow and recalculate the measured value. As a result, it is possible to verify whether or not the inspection results have been tampered with.
　（第６の実施形態）
　上述の実施形態において、例えば、判定対象の鉄筋画像からフーリエ変換により取得した周波数スペクトルに、鉄筋の表面にマーカとしてつけられた塗料などに起因する低周波のノイズが含まれてしまうことがある。 (Sixth Embodiment)
In the above-described embodiment, for example, the frequency spectrum obtained by Fourier transform from the image of the reinforcing bar to be determined may include low-frequency noise caused by a paint or the like attached as a marker on the surface of the reinforcing bar.
　図１６は、鉄筋画像の表面の塗料に起因する周波数スペクトルにおけるノイズを説明する図である。例えば、図１６（ａ）では、鉄筋に塗料１６０１が塗られている状態が示されており、その周波数スペクトルにおいて塗料１６０１に起因するノイズ１６０２が表れている。 FIG. 16 is a diagram for explaining noise in the frequency spectrum caused by the paint on the surface of the reinforcing bar image. For example, FIG. 16A shows a state in which the paint 1601 is applied to the reinforcing bar, and the noise 1602 caused by the paint 1601 appears in the frequency spectrum thereof.
　そして、この様にノイズ１６０２を含む図１６（ａ）の周波数スペクトルと、図１６（ｂ）に示すサンプル鉄筋画像の周波数スペクトルとの類似度を評価しても、ノイズ１６０２に起因して類似度が低く評価されてしまい、種別の判定精度が低下してしまうことがある。 Then, even if the similarity between the frequency spectrum of FIG. 16A including the noise 1602 and the frequency spectrum of the sample reinforcing bar image shown in FIG. 16B is evaluated in this way, the similarity is caused by the noise 1602. Is evaluated low, and the judgment accuracy of the type may decrease.
　そこで、第６の実施形態では制御部３０１は、周波数スペクトルから評価対象の周波数範囲の周波数成分を抽出し、抽出した範囲において類似度の評価を行ってよい。例えば、図１６に示すように、評価対象の周波数範囲において類似度の評価を行うことで、ノイズ１６０２の影響を排除することができ、ノイズ１６０２に起因して類似度を低く評価してしまうことを抑制することができる。 Therefore, in the sixth embodiment, the control unit 301 may extract the frequency component of the frequency range to be evaluated from the frequency spectrum and evaluate the similarity in the extracted range. For example, as shown in FIG. 16, by evaluating the similarity in the frequency range to be evaluated, the influence of the noise 1602 can be eliminated, and the similarity is evaluated low due to the noise 1602. Can be suppressed.
　なお、評価対象の周波数範囲は、サンプル鉄筋画像から得られた周波数スペクトルのピークの位置に基づいて決定することができる。例えば、制御部３０１は、類似度の評価に用いるサンプル鉄筋画像において直流成分を除いた最大ピークの周波数の位置を基準とする所定の周波数範囲を評価対象の周波数範囲として設定してよい。 The frequency range to be evaluated can be determined based on the position of the peak of the frequency spectrum obtained from the sample reinforcing bar image. For example, the control unit 301 may set a predetermined frequency range based on the position of the maximum peak frequency excluding the DC component in the sample reinforcing bar image used for the evaluation of the similarity as the frequency range to be evaluated.
　（第７の実施形態）
　また、上述のように、例えば、種別の判定対象の鉄筋画像と、サンプル鉄筋画像との類似度の評価は、２つの画像に写る鉄筋の撮影条件ができるだけ揃っていた方が、類似度の評価精度が向上する。そのため、第７の実施形態では、種別の判定対象の鉄筋の写る鉄筋画像の一部を抽出し、抽出した領域をサンプル鉄筋画像として用いて鉄筋の種別判定を実行する。 (7th Embodiment)
Further, as described above, for example, in the evaluation of the similarity between the reinforcing bar image to be determined by type and the sample reinforcing bar image, it is better that the imaging conditions of the reinforcing bars shown in the two images are as uniform as possible. Accuracy is improved. Therefore, in the seventh embodiment, a part of the reinforcing bar image showing the reinforcing bar to be determined for the type is extracted, and the extracted area is used as the sample reinforcing bar image to perform the type determination of the reinforcing bar.
　図１７は、第７の実施形態に係る鉄筋の種別判定を例示する図である。図１７に示すように、制御部３０１は、まず画像４０３のうちから、一部の画像領域の指定を受け付け、指定された領域をサンプル鉄筋画像として抽出する（図１７の（１））。 FIG. 17 is a diagram illustrating the type determination of the reinforcing bar according to the seventh embodiment. As shown in FIG. 17, the control unit 301 first accepts the designation of a part of the image area from the image 403, and extracts the designated area as a sample reinforcing bar image ((1) of FIG. 17).
　続いて、制御部３０１は、抽出した領域のサンプル鉄筋画像を、種別情報と対応づけて記憶部３０２に記憶する（図１７の（２））。一例では、制御部３０１は、ユーザからサンプル鉄筋画像の種別を示す種別情報の入力を受け付けて、受け付けた種別情報と、サンプル鉄筋画像とを対応づけてよい。図１７の例では、種別情報として鉄筋の径を示すＤ２５がサンプル鉄筋画像と対応づけられている。 Subsequently, the control unit 301 stores the sample reinforcing bar image of the extracted region in the storage unit 302 in association with the type information ((2) in FIG. 17). In one example, the control unit 301 may receive input of type information indicating the type of the sample rebar image from the user and associate the received type information with the sample rebar image. In the example of FIG. 17, D25 indicating the diameter of the reinforcing bar is associated with the sample reinforcing bar image as the type information.
　続いて、制御部３０１は、画像４０３に写る他の鉄筋の写る画像領域を、判定対象の鉄筋画像として、サンプル鉄筋画像との類似度を評価する。そして、制御部３０１は、類似度の評価結果に基づいて、所定の条件を満たしてサンプル鉄筋画像と、判定対象の鉄筋画像が類似している場合に、その判定対象の鉄筋画像の種別として、サンプル鉄筋画像の種別情報を出力してよい（図１７の（３））。なお、ここで鉄筋の種別が同じと判定するために用いる所定の条件は、例えば、類似度の評価に用いた指標が、値が高いほど類似していることを表す場合には、類似度の評価に用いた指標が閾値以上であることであってよい。また、例えば、類似度の評価に用いた指標が、値が低いほど類似していることを表す場合には、所定の条件は、類似度の評価に用いた指標が閾値以下であることであってよい。判定に用いる閾値は、例えば、サンプル鉄筋画像に写る鉄筋と、判定対象の鉄筋画像に写る鉄筋とが同じ種別の鉄筋であるか否かを判定できるように、経験則などに基づいて設定されてよい。 Subsequently, the control unit 301 evaluates the similarity with the sample reinforcing bar image by using the image area in which the other reinforcing bars in the image 403 are captured as the reinforcing bar image to be determined. Then, based on the evaluation result of the similarity, the control unit 301 determines that the type of the rebar image to be determined is the same when the sample rebar image and the rebar image to be determined are similar to each other by satisfying a predetermined condition. The type information of the sample reinforcing bar image may be output ((3) in FIG. 17). It should be noted that the predetermined condition used here for determining that the types of reinforcing bars are the same is, for example, when the index used for evaluating the similarity indicates that the higher the value, the more similar the index is. The index used for evaluation may be equal to or higher than the threshold value. Further, for example, when the index used for the evaluation of the similarity indicates that the lower the value, the more similar the index is, the predetermined condition is that the index used for the evaluation of the similarity is equal to or less than the threshold value. It's okay. The threshold value used for the determination is set based on an empirical rule or the like so that it can be determined whether or not the reinforcing bar shown in the sample reinforcing bar image and the reinforcing bar shown in the judgment target reinforcing bar image are of the same type. Good.
　以上で述べたように、判定対象の鉄筋の写る画像４０３の一部の領域の鉄筋画像を、サンプル鉄筋画像として用いて鉄筋の種別を判定することが可能である。そして、第７の実施形態では、種別の判定対象の鉄筋画像と、サンプル鉄筋画像との撮影条件が近い条件で鉄筋の種別を判定することができるため、類似度の評価精度を向上させることができる。一例として、照明の条件などの外乱が類似度の評価に与える影響を抑えることが可能である。 As described above, it is possible to determine the type of reinforcing bar by using the reinforcing bar image of a part of the region of the image 403 in which the reinforcing bar to be determined is captured as the sample reinforcing bar image. Then, in the seventh embodiment, the type of the reinforcing bar can be determined under the condition that the imaging conditions of the reinforcing bar image to be determined for the type and the sample reinforcing bar image are close to each other, so that the evaluation accuracy of the similarity can be improved. it can. As an example, it is possible to suppress the influence of disturbances such as lighting conditions on the evaluation of similarity.
　また、以上の第７の実施形態によれば、事前にサンプル鉄筋画像を用意していなくても、種別判定の際に撮影した判定対象の鉄筋の写る画像４０３から、その場でサンプル鉄筋画像を生成して種別を判定することができる。 Further, according to the seventh embodiment described above, even if the sample reinforcing bar image is not prepared in advance, the sample reinforcing bar image can be obtained on the spot from the image 403 in which the reinforcing bar to be determined is captured at the time of type determination. It can be generated and the type can be determined.
　更には、画角をずらして画像４０３を複数枚撮影し、配筋検査などに用いることがある。この場合に、１つの画像４０３の一部からサンプル鉄筋画像を抽出し、抽出したサンプル鉄筋画像を他の画像４０３に写る鉄筋の種別判定に用いてもよい。この場合にも、照明の条件などの撮影条件を近い条件で類似度を評価することができるため、類似度の評価精度を向上させることができる。そして、類似度の評価精度を向上させることで、鉄筋の種別の判定精度を向上させることができる。 Furthermore, multiple images 403 may be taken with the angle of view shifted and used for bar arrangement inspection and the like. In this case, a sample reinforcing bar image may be extracted from a part of one image 403, and the extracted sample reinforcing bar image may be used for determining the type of the reinforcing bar to be reflected in the other image 403. Also in this case, since the similarity can be evaluated under conditions close to the shooting conditions such as lighting conditions, the evaluation accuracy of the similarity can be improved. Then, by improving the evaluation accuracy of the similarity, it is possible to improve the determination accuracy of the type of reinforcing bar.
　なお、図１７の例では、ユーザが画像４０３のうちから、鉄筋の写る領域を指定する例を示しているが、実施形態はこれに限定されるものではない。例えば、別の実施形態では、制御部３０１は、画像４０３から鉄筋を検出するアルゴリズムを実行して画像４０３に写る複数の鉄筋を検出してよい。そして、制御部３０１は、検出された複数の鉄筋のうちからのユーザによる鉄筋の選択を受け付け、ユーザによって選択された鉄筋と、ユーザから入力された種別情報とを対応づけてサンプル鉄筋画像として用いてもよい。 Note that the example of FIG. 17 shows an example in which the user specifies an area in which the reinforcing bar is reflected from the image 403, but the embodiment is not limited to this. For example, in another embodiment, the control unit 301 may execute an algorithm for detecting the reinforcing bars from the image 403 to detect a plurality of reinforcing bars appearing in the image 403. Then, the control unit 301 accepts the selection of the reinforcing bar by the user from the plurality of detected reinforcing bars, and associates the reinforcing bar selected by the user with the type information input by the user and uses it as a sample reinforcing bar image. You may.
　また、別の実施形態では、1つの鉄筋の種別に対して撮影画像から複数箇所を選択してサンプル鉄筋画像として用いてもよい。例えば、制御部３０１は、１つの鉄筋の種別に対して撮影画像４０３のうちから鉄筋の写る領域を複数箇所抽出して、複数のサンプル鉄筋画像として登録してもよい。上述のように、異形鉄筋では同一の種類の鉄筋であっても向きによって形状が異なっていることがあり、そのような鉄筋が施工の際にリブや節が異なる向きで設置されることもある。そのため、撮影画像４０３のうちから鉄筋の写る領域を複数箇所抽出して、リブや節の向きが異なる複数の鉄筋の状態を写したサンプル鉄筋画像を取得することで、リブや節の向きや更に照明の条件などの外乱が類似度の評価に与える影響を抑えることができる。なお、複数のサンプル鉄筋画像は、例えば、或るエリアに設置された配筋に対して画角をずらして撮影された複数の画像４０３から抽出されてもよい。 Further, in another embodiment, a plurality of locations may be selected from the captured images for one type of reinforcing bar and used as a sample reinforcing bar image. For example, the control unit 301 may extract a plurality of regions in which the reinforcing bars are reflected from the captured image 403 for one type of reinforcing bar and register them as a plurality of sample reinforcing bar images. As mentioned above, deformed reinforcing bars may have different shapes depending on the orientation even if they are of the same type, and such reinforcing bars may be installed with ribs and knots in different orientations during construction. .. Therefore, by extracting a plurality of areas where the reinforcing bars are reflected from the captured image 403 and acquiring a sample reinforcing bar image showing the state of the plurality of reinforcing bars having different orientations of the ribs and nodes, the orientation of the ribs and nodes and further It is possible to suppress the influence of disturbances such as lighting conditions on the evaluation of similarity. The plurality of sample reinforcing bar images may be extracted from, for example, a plurality of images 403 taken with the angle of view shifted with respect to the reinforcing bar arrangement installed in a certain area.
　以上において、実施形態を例示したが、実施形態はこれに限定されるものではない。例えば、上述の動作フローは例示であり、実施形態はこれに限定されるものではない。可能な場合には、動作フローは、処理の順番を変更して実行されてもよく、別に更なる処理を含んでもよく、または、一部の処理が省略されてもよい。 Although the embodiments have been illustrated above, the embodiments are not limited to this. For example, the above-mentioned operation flow is an example, and the embodiment is not limited thereto. When possible, the operation flow may be executed by changing the order of processing, may include additional processing, or may omit some processing.
　また、上述の実施形態では、判定対象の鉄筋画像４０４を画像４０３から抽出して処理する例を述べたが、実施形態はこれに限定されるものではなく、例えば、別の実施形態では画像４０３を対象にサンプル鉄筋画像との類似度の評価が実行されてもよい。一例として、回転不変位相限定相関法では、画像面の法線方向を軸とした回転を扱って類似度を評価することができる。そのため、制御部３０１は、例えば、画像４０３に上述の正対化の処理を行った後、判定対象の鉄筋画像４０４を生成せずに、画像４０３とサンプル鉄筋画像との類似度を回転不変位相限定相関法で評価してもよい。この場合、制御部３０１は、類似度の評価とともに、画像４０３上でサンプル鉄筋画像と類似度が高いと評価された鉄筋の位置の特定も行うことができる。 Further, in the above-described embodiment, an example in which the reinforcing bar image 404 to be determined is extracted from the image 403 and processed is described, but the embodiment is not limited to this, and for example, in another embodiment, the image 403 is described. The evaluation of the similarity with the sample reinforcing bar image may be performed on the subject. As an example, in the rotation invariant phase-limited correlation method, the similarity can be evaluated by handling the rotation about the normal direction of the image plane. Therefore, for example, after performing the above-mentioned face-to-face processing on the image 403, the control unit 301 rotates the similarity between the image 403 and the sample reinforcing bar image without generating the reinforcing bar image 404 to be determined. It may be evaluated by the limited correlation method. In this case, the control unit 301 can evaluate the similarity as well as specify the position of the reinforcing bar evaluated to have a high degree of similarity to the sample reinforcing bar image on the image 403.
　また、上述の実施形態では、種別の例として径の情報を判定する例を述べているが、実施形態はこれに限定されるものではない。例えば、鉄筋情報４０１に鉄筋の種別に関する様々な情報（例えば、製造メーカ、節間隔、節の種類など）を登録しておくことで、それらの情報を入力された鉄筋画像とサンプル鉄筋画像との類似度を評価することで取得することができる。 Further, in the above-described embodiment, an example of determining the diameter information is described as an example of the type, but the embodiment is not limited to this. For example, by registering various information related to the type of reinforcing bar (for example, manufacturer, node spacing, node type, etc.) in the reinforcing bar information 401, the input reinforcing bar image and the sample reinforcing bar image can be linked. It can be obtained by evaluating the degree of similarity.
　また、上述の実施形態では、鉄筋判定装置２０１が、例えば、鉄筋情報９００，１２００，１３００，１４０１，１４０２などの鉄筋情報を記憶している例を述べているが、実施形態はこれに限定されるものではない。例えば、別の実施形態では、鉄筋情報は、鉄筋判定装置２０１とは別のデータベースサーバに記憶されていてもよい。この場合、鉄筋判定装置２０１は、データベースサーバにアクセスして鉄筋情報から情報を取得してもよい。 Further, in the above-described embodiment, the reinforcing bar determination device 201 describes an example in which the reinforcing bar information such as the reinforcing bar information 900, 1200, 1300, 1401, 1402 is stored, but the embodiment is limited to this. It's not something. For example, in another embodiment, the reinforcing bar information may be stored in a database server different from the reinforcing bar determination device 201. In this case, the reinforcing bar determination device 201 may access the database server and acquire information from the reinforcing bar information.
　なお、上述の実施形態において、例えば、図１０の動作フローの開始時では、制御部３０１は、入力部３１１として動作する。また、例えば、図１０のＳ１００２では、制御部３０１は、評価部３１２として動作する。例えば、図１０のＳ１００３からＳ１００５では、制御部３０１は、出力部３１３として動作する。例えば、図１４の仕分け処理において、制御部３０１は、判定部３１４として動作する。 In the above-described embodiment, for example, at the start of the operation flow of FIG. 10, the control unit 301 operates as the input unit 311. Further, for example, in S1002 of FIG. 10, the control unit 301 operates as the evaluation unit 312. For example, in S1003 to S1005 of FIG. 10, the control unit 301 operates as the output unit 313. For example, in the sorting process of FIG. 14, the control unit 301 operates as the determination unit 314.
　図１８は、実施形態に係る鉄筋判定装置２０１を実現するためのコンピュータ１８００のハードウェア構成を例示する図である。図１８のハードウェア構成は、例えば、プロセッサ１８０１、メモリ１８０２、記憶装置１８０３、通信インタフェース１８０４、外部インタフェース１８０５、表示装置１８０６、および入力装置１８０７を備える。プロセッサ１８０１は、例えば、バスなどを経由して、メモリ１８０２、記憶装置１８０３、通信インタフェース１８０４、外部インタフェース１８０５、表示装置１８０６、および入力装置１８０７と通信可能に接続されていてよい。 FIG. 18 is a diagram illustrating a hardware configuration of a computer 1800 for realizing the reinforcing bar determination device 201 according to the embodiment. The hardware configuration of FIG. 18 includes, for example, a processor 1801, a memory 1802, a storage device 1803, a communication interface 1804, an external interface 1805, a display device 1806, and an input device 1807. The processor 1801 may be communicably connected to the memory 1802, the storage device 1803, the communication interface 1804, the external interface 1805, the display device 1806, and the input device 1807 via, for example, a bus.
　プロセッサ１８０１は、例えば、シングルプロセッサであっても、マルチプロセッサおよびマルチコアであってもよい。プロセッサ１８０１は、メモリ１８０２を利用して例えば上述の動作フローの手順を記述したプログラムを実行することにより、上述した制御部３０１の一部または全部の機能を提供する。例えば、プロセッサ１８０１は、記憶装置１８０３に格納されているプログラムをメモリ１８０２に読み出して実行することで、入力部３１１、評価部３１２、出力部３１３、および判定部３１４として動作する。 The processor 1801 may be, for example, a single processor, a multiprocessor, and a multicore. The processor 1801 provides a part or all of the functions of the control unit 301 described above by executing, for example, a program describing the procedure of the operation flow described above using the memory 1802. For example, the processor 1801 operates as an input unit 311, an evaluation unit 312, an output unit 313, and a determination unit 314 by reading a program stored in the storage device 1803 into the memory 1802 and executing the program.
　メモリ１８０２は、例えば半導体メモリであり、ＲＡＭ領域およびＲＯＭ領域を含んでいてよい。記憶装置１８０３は、例えばハードディスク、フラッシュメモリ等の半導体メモリ、または外部記憶装置である。なお、ＲＡＭは、Random Access Memoryの略称である。また、ＲＯＭは、Read Only Memoryの略称である。上述の記憶部３０２は、例えば、メモリ１８０２および記憶装置１８０３を含んでよい。 The memory 1802 is, for example, a semiconductor memory, and may include a RAM area and a ROM area. The storage device 1803 is, for example, a semiconductor memory such as a hard disk or a flash memory, or an external storage device. RAM is an abbreviation for Random Access Memory. ROM is an abbreviation for Read Only Memory. The storage unit 302 described above may include, for example, a memory 1802 and a storage device 1803.
　通信インタフェース１８０４は、例えば、プロセッサ１８０１の指示に従ってネットワークに接続し、データを送受信する通信機器である。外部インタフェース１８０５は、例えば、外部装置とのインタフェースであってよい。一実施形態においては、鉄筋判定装置２０１は、外部インタフェース１８０５を経由して撮影装置１０２と接続してよい。なお、別の実施形態では、鉄筋判定装置２０１は、Ｗｉ-Ｆｉ（登録商標）通信機器、およびBluetooth（登録商標）通信機器などの通信インタフェース１８０４を備えてよく、ネットワークを経由してまたは近距離無線通信で撮影装置１０２と接続されていてもよい。通信インタフェース１８０４および外部インタフェース１８０５は、上述の通信部３０４の一例である。 The communication interface 1804 is, for example, a communication device that connects to a network according to the instruction of the processor 1801 and transmits / receives data. The external interface 1805 may be, for example, an interface with an external device. In one embodiment, the reinforcing bar determination device 201 may be connected to the photographing device 102 via the external interface 1805. In another embodiment, the reinforcing bar determination device 201 may include a communication interface 1804 such as a Wi-Fi (registered trademark) communication device and a Bluetooth (registered trademark) communication device, via a network or at a short distance. It may be connected to the photographing device 102 by wireless communication. The communication interface 1804 and the external interface 1805 are examples of the above-mentioned communication unit 304.
　表示装置１８０６は、例えば、液晶ディスプレイなどの表示機能を備える装置である。表示装置１８０６は、上述の表示部３０３の一例である。入力装置１８０７は、例えば、キーボードおよびタッチパネルなどのユーザからの入力を受け付ける装置である。 The display device 1806 is a device having a display function such as a liquid crystal display. The display device 1806 is an example of the display unit 303 described above. The input device 1807 is a device that receives input from a user such as a keyboard and a touch panel.
　上述の実施形態に係る各プログラムは、例えば、以下の形態で鉄筋判定装置２０１に提供されてよい。
（１）記憶部３０２に予めインストールされている。
（２）プログラムサーバなどのサーバから提供される。 Each program according to the above-described embodiment may be provided to the reinforcing bar determination device 201 in the following embodiments, for example.
(1) It is pre-installed in the storage unit 302.
(2) Provided from a server such as a program server.
　また、図１８を参照して述べた鉄筋判定装置２０１を実現するためのハードウェア構成は、例示であり、実施形態はこれに限定されるものではない。例えば、上述の制御部３０１の一部または全部の機能がＦＰＧＡおよびＳｏＣなどの専用の回路によるハードウェアとして実装されてもよい。なお、ＦＰＧＡは、Field Programmable Gate Arrayの略称である。ＳｏＣは、System-on-a-chipの略称である。一例として、上述の制御部３０１は、入力画像に対して、上述の実施形態に係る類似度の評価に従って種別情報を出力する回路であってもよい。また、上述の入力部３１１、評価部３１２、出力部３１３、および判定部３１４は、それぞれ個別に入力回路、評価回路、出力回路、および判定回路などの回路として実装されてもよい。また、これらの全部または一部が統合された回路として実装されてもよい。 Further, the hardware configuration for realizing the reinforcing bar determination device 201 described with reference to FIG. 18 is an example, and the embodiment is not limited to this. For example, some or all the functions of the above-mentioned control unit 301 may be implemented as hardware by a dedicated circuit such as FPGA and SoC. FPGA is an abbreviation for Field Programmable Gate Array. SoC is an abbreviation for System-on-a-chip. As an example, the control unit 301 may be a circuit that outputs type information to the input image according to the evaluation of the similarity according to the embodiment. Further, the above-mentioned input unit 311, evaluation unit 312, output unit 313, and determination unit 314 may be individually mounted as circuits such as an input circuit, an evaluation circuit, an output circuit, and a determination circuit. Further, all or a part of them may be implemented as an integrated circuit.
　以上において、いくつかの実施形態が説明される。しかしながら、実施形態は上記の実施形態に限定されるものではなく、上述の実施形態の各種変形形態および代替形態を包含するものとして理解されるべきである。例えば、各種実施形態は、その趣旨および範囲を逸脱しない範囲で構成要素を変形して具体化できることが理解されよう。また、前述した実施形態に開示されている複数の構成要素を適宜組み合わせることにより、種々の実施形態が実施され得ることが理解されよう。更には、実施形態に示される全構成要素からいくつかの構成要素を削除してまたは置換して、或いは実施形態に示される構成要素にいくつかの構成要素を追加して種々の実施形態が実施され得ることが当業者には理解されよう。 In the above, some embodiments will be described. However, the embodiments are not limited to the above embodiments and should be understood to include various variants and alternatives of the above embodiments. For example, it will be understood that various embodiments can be embodied by modifying the components within a range that does not deviate from the purpose and scope. It will also be appreciated that various embodiments can be implemented by appropriately combining the plurality of components disclosed in the above-described embodiments. Further, various embodiments are implemented by removing or replacing some components from all the components shown in the embodiments, or by adding some components to the components shown in the embodiments. Those skilled in the art will understand that it can be done.
１０１　　：鉄筋
１０２　　：撮影装置
２００　　：鉄筋判定システム
２０１　　：鉄筋判定装置
２０５　　：ネットワーク
３０１　　：制御部
３０２　　：記憶部
３０３　　：表示部
３０４　　：通信部
３１１　　：入力部
３１２　　：評価部
３１３　　：出力部
３１４　　：判定部
１８００　：コンピュータ
１８０１　：プロセッサ
１８０２　：メモリ
１８０３　：記憶装置
１８０４　：通信インタフェース
１８０５　：外部インタフェース
１８０６　：表示装置
１８０７　：入力装置 101: Reinforcing bar 102: Imaging device 200: Reinforcing bar determination system 201: Reinforcing bar determination device 205: Network 301: Control unit 302: Storage unit 303: Display unit 304: Communication unit 311: Input unit 312: Evaluation unit 313: Output unit 314: Judgment unit 1800: Computer 1801: Processor 1802: Memory 1803: Storage device 1804: Communication interface 1805: External interface 1806: Display device 1807: Input device

Claims (21)

  1. 　鉄筋が写るサンプル鉄筋画像および前記サンプル鉄筋画像から抽出された特徴情報の少なくとも一方と、前記サンプル鉄筋画像に写る鉄筋の種別を示す種別情報とを対応づけた情報を含む鉄筋情報を記憶する記憶部と、
    　判定対象の鉄筋が写る鉄筋画像の入力を受け付ける入力部と、
    　前記鉄筋情報に基づいて、前記サンプル鉄筋画像と前記鉄筋画像との類似度を評価する評価部と、
    　前記類似度の評価結果に基づいて、前記鉄筋画像と前記サンプル鉄筋画像とが所定の条件を満たして類似している場合に、前記サンプル鉄筋画像と対応する前記種別情報を出力する出力部と、
    　を備える、鉄筋判定装置。     A storage unit that stores rebar information including information in which at least one of the sample rebar image showing the rebar and the feature information extracted from the sample rebar image is associated with the type information indicating the type of the rebar shown in the sample rebar image. When,
    An input unit that accepts input of a reinforcing bar image showing the reinforcing bar to be judged,
    An evaluation unit that evaluates the similarity between the sample reinforcing bar image and the reinforcing bar image based on the reinforcing bar information, and
    Based on the evaluation result of the degree of similarity, when the reinforcing bar image and the sample reinforcing bar image are similar to each other satisfying a predetermined condition, an output unit that outputs the type information corresponding to the sample reinforcing bar image and an output unit.
    Reinforcing bar determination device.
  2. 　前記評価部は、更に、前記鉄筋画像を撮影した撮影装置と、前記鉄筋画像に写る鉄筋との相対的な位置姿勢に関する情報に基づいて、前記撮影装置と前記鉄筋画像に写る鉄筋との相対的な位置姿勢が所定の関係となるように前記鉄筋画像を正規化した後、前記類似度の評価を実行することを特徴とする請求項１に記載の鉄筋判定装置。 Further, the evaluation unit further relatives the photographing device and the reinforcing bar reflected in the reinforcing bar image based on the information regarding the relative position and orientation of the photographing device that captured the reinforcing bar image and the reinforcing bar reflected in the reinforcing bar image. The reinforcing bar determination device according to claim 1, wherein the reinforcing bar images are normalized so that the positions and orientations have a predetermined relationship, and then the evaluation of the similarity is executed.
  3. 　前記評価部は、更に、前記サンプル鉄筋画像を撮影した撮影装置と、前記サンプル鉄筋画像に写る鉄筋との相対的な位置姿勢に関する情報に基づいて、前記撮影装置と前記サンプル鉄筋画像に写る鉄筋との相対的な位置姿勢が所定の関係となるように前記サンプル鉄筋画像を正規化した後、前記類似度の評価を実行することを特徴とする請求項１に記載の鉄筋判定装置。 Further, the evaluation unit includes the photographing device and the reinforcing bar reflected in the sample reinforcing bar image based on the information regarding the relative position and orientation of the photographing device that captured the sample reinforcing bar image and the reinforcing bar reflected in the sample reinforcing bar image. The rebar determination device according to claim 1, wherein the sample rebar image is normalized so that the relative positions and orientations of the rebars have a predetermined relationship, and then the evaluation of the similarity is executed.
  4. 　前記記憶部は、前記鉄筋情報を、異なる複数の種別ごとに複数記憶しており、
    　前記所定の条件は、前記類似度の評価結果に基づいて、前記鉄筋画像と最も類似していると評価されることである、請求項１に記載の鉄筋判定装置。     The storage unit stores a plurality of the reinforcing bar information for each of a plurality of different types.
    The reinforcing bar determination device according to claim 1, wherein the predetermined condition is evaluated to be most similar to the reinforcing bar image based on the evaluation result of the similarity.
  5. 　前記記憶部は、前記鉄筋情報を、異なる複数の種別ごとに複数記憶しており、
    　前記出力部は、前記鉄筋画像と前記所定の条件を満たして類似しているサンプル鉄筋画像が複数ある場合、前記鉄筋画像と前記所定の条件を満たして類似しているサンプル鉄筋画像のうちで、鉄筋の径が最も小さい種別の前記種別情報を出力することを特徴とする請求項１に記載の鉄筋判定装置。     The storage unit stores a plurality of the reinforcing bar information for each of a plurality of different types.
    When there are a plurality of sample reinforcing bar images that satisfy the predetermined conditions and are similar to the reinforcing bar image, the output unit is among the sample reinforcing bar images that satisfy the predetermined conditions and are similar to the reinforcing bar image. The reinforcing bar determination device according to claim 1, wherein the type information of the type having the smallest diameter of the reinforcing bar is output.
  6. 　前記評価部は、前記サンプル鉄筋画像および前記鉄筋画像を鉄筋の画素値を長手方向にフーリエ変換して得られる周波数スペクトルの波形の類似度を評価することで、前記類似度の評価を実行することを特徴とする請求項１に記載の鉄筋判定装置。 The evaluation unit evaluates the similarity by evaluating the similarity of the sample reinforcing bar image and the waveform of the frequency spectrum obtained by Fourier transforming the pixel value of the reinforcing bar in the longitudinal direction from the reinforcing bar image. The reinforcing bar determination device according to claim 1.
  7. 　前記評価部は、前記周波数スペクトルにおける所定の周波数範囲の波形の類似度を評価する、ことを特徴とする請求項６に記載の鉄筋判定装置。 The reinforcing bar determination device according to claim 6, wherein the evaluation unit evaluates the similarity of waveforms in a predetermined frequency range in the frequency spectrum.
  8. 　前記評価部は、前記サンプル鉄筋画像および前記鉄筋画像の画素値を鉄筋の長手方向に直交する幅方向にフーリエ変換した周波数スペクトルを更に用いて前記類似度の評価を実行する、ことを特徴とする請求項６に記載の鉄筋判定装置。 The evaluation unit is characterized in that the evaluation of the similarity is performed by further using the frequency spectrum obtained by Fourier transforming the sample reinforcing bar image and the pixel values of the reinforcing bar image in the width direction orthogonal to the longitudinal direction of the reinforcing bar. The reinforcing bar determination device according to claim 6.
  9. 　前記評価部は、前記サンプル鉄筋画像および前記鉄筋画像の画素値を長手方向にフーリエ変換した周波数スペクトルに含まれる複数のピークの周波数を用いて前記類似度の評価を実行することを特徴とする請求項１に記載の鉄筋判定装置。 The evaluation unit is characterized in that the evaluation of the similarity is performed using the frequencies of a plurality of peaks included in the frequency spectrum obtained by Fourier transforming the sample reinforcing bar image and the pixel values of the reinforcing bar image in the longitudinal direction. Item 1. The reinforcing bar determination device according to Item 1.
  10. 　前記評価部は、更に、前記サンプル鉄筋画像および前記鉄筋画像において鉄筋の長手方向に略平行な鉄筋の両端のエッジを特定し、前記両端のエッジ間の距離を用いて前記類似度の評価を実行することを特徴とする請求項６から９のいずれか１項に記載の鉄筋判定装置。 The evaluation unit further identifies the edges at both ends of the reinforcing bar substantially parallel to the longitudinal direction of the reinforcing bar in the sample reinforcing bar image and the reinforcing bar image, and evaluates the similarity using the distance between the edges at both ends. The reinforcing bar determination device according to any one of claims 6 to 9, wherein the reinforcing bar determination device is characterized.
  11. 　前記評価部は、前記サンプル鉄筋画像および前記鉄筋画像において鉄筋の画素値をフーリエ変換した結果から抽出した位相の情報を用いて前記類似度の評価を実行することを特徴とする請求項１に記載の鉄筋判定装置。 The first aspect of claim 1, wherein the evaluation unit executes the evaluation of the similarity using the phase information extracted from the sample reinforcing bar image and the result of Fourier transforming the pixel value of the reinforcing bar in the reinforcing bar image. Reinforcing bar judgment device.
  12. 　前記評価部は、前記サンプル鉄筋画像および前記鉄筋画像における鉄筋の長手方向の画素値の波形を用いて前記類似度の評価を実行することを特徴とする請求項１から５のいずれか１項に記載の鉄筋判定装置。 The evaluation unit according to any one of claims 1 to 5, wherein the evaluation unit performs the evaluation of the similarity using the sample reinforcing bar image and the waveform of the pixel value in the longitudinal direction of the reinforcing bar in the reinforcing bar image. The rebar determination device described.
  13. 　前記評価部は、前記サンプル鉄筋画像および前記鉄筋画像のそれぞれから抽出した局所特徴量を用いて前記類似度の評価を実行することを特徴とする請求項１から５のいずれか１項に記載の鉄筋判定装置。 The item according to any one of claims 1 to 5, wherein the evaluation unit performs the evaluation of the similarity using the local feature amounts extracted from each of the sample rebar image and the rebar image. Reinforcing bar judgment device.
  14. 　前記入力部には、更に前記判定対象とする鉄筋が含まれる第２の鉄筋画像が入力され、
    　前記評価部は、前記鉄筋情報に基づいて、前記サンプル鉄筋画像と前記第２の鉄筋画像との第２の類似度の評価を実行し、
    　前記出力部は、前記種別情報の出力において、前記類似度の評価結果と前記第２の類似度の評価結果とに基づいて、前記鉄筋画像および前記第２の鉄筋画像と、前記サンプル鉄筋画像とが所定の条件を満たして類似している場合に、前記サンプル鉄筋画像と対応する前記種別情報を出力する、
    ことを特徴とする請求項１から３のいずれか１項に記載の鉄筋判定装置。     A second reinforcing bar image including the reinforcing bar to be determined is further input to the input unit.
    The evaluation unit evaluates the second similarity between the sample rebar image and the second rebar image based on the rebar information.
    In the output of the type information, the output unit includes the reinforcing bar image, the second reinforcing bar image, and the sample reinforcing bar image based on the evaluation result of the similarity and the evaluation result of the second similarity. Outputs the type information corresponding to the sample reinforcing bar image when is similar to each other satisfying a predetermined condition.
    The reinforcing bar determination device according to any one of claims 1 to 3, wherein the reinforcing bar determination device is characterized.
  15. 　前記鉄筋情報は、更に、前記サンプル鉄筋画像に写る鉄筋を別の角度から撮影した第２のサンプル鉄筋画像および前記第２のサンプル鉄筋画像から抽出された第２の特徴情報のうちの少なくとも一方を、前記サンプル鉄筋画像に写る鉄筋の種別を示す前記種別情報と対応づけて含み、
    　前記評価部は、前記鉄筋情報に基づいて、前記第２のサンプル鉄筋画像と前記鉄筋画像との第３の類似度の評価を実行し、
    　前記出力部は、前記種別情報の出力において、前記類似度の評価結果と前記第３の類似度の評価結果とに基づいて、前記鉄筋画像と前記サンプル鉄筋画像および前記第２のサンプル鉄筋画像とが所定の条件を満たして類似している場合に、前記サンプル鉄筋画像および第２のサンプル鉄筋画像と対応する前記種別情報を出力する、
    ことを特徴とする請求項１から３のいずれか１項に記載の鉄筋判定装置。     The rebar information further includes at least one of a second sample rebar image obtained by photographing the rebar imaged in the sample rebar image from another angle and a second feature information extracted from the second sample rebar image. , Included in association with the type information indicating the type of rebar shown in the sample rebar image.
    Based on the reinforcing bar information, the evaluation unit executes an evaluation of a third degree of similarity between the second sample reinforcing bar image and the reinforcing bar image.
    In the output of the type information, the output unit includes the reinforcing bar image, the sample reinforcing bar image, and the second sample reinforcing bar image based on the evaluation result of the similarity and the evaluation result of the third similarity. Outputs the type information corresponding to the sample rebar image and the second sample rebar image when is similar to each other satisfying a predetermined condition.
    The reinforcing bar determination device according to any one of claims 1 to 3, wherein the reinforcing bar determination device is characterized.
  16. 　前記鉄筋情報の前記サンプル鉄筋画像に写る鉄筋は、複数の仕分種別のうちの第１の仕分種別に属する鉄筋であり、
    　前記入力部から入力された前記鉄筋画像が、前記複数の仕分種別のうちのいずれの仕分種別であるかを判定する判定部を更に含み、
    　前記評価部は、前記判定部で前記鉄筋画像が前記第１の仕分種別と判定された場合に、前記鉄筋情報に基づく類似度の評価を実行する、請求項１に記載の鉄筋判定装置。     The reinforcing bar shown in the sample reinforcing bar image of the reinforcing bar information is a reinforcing bar belonging to the first sorting type among a plurality of sorting types.
    Further including a determination unit for determining which of the plurality of sorting types the reinforcing bar image input from the input unit is.
    The reinforcing bar determination device according to claim 1, wherein the evaluation unit executes an evaluation of similarity based on the reinforcing bar information when the determination unit determines that the reinforcing bar image is the first sorting type.
  17. 　前記サンプル鉄筋画像から抽出された特徴情報は、前記サンプル鉄筋画像をフーリエ変換した周波数スペクトル、前記周波数スペクトルのピークの周波数、前記フーリエ変換の結果から抽出した位相情報、前記サンプル鉄筋画像から抽出した局所特徴量、ならびに前記サンプル鉄筋画像に写る鉄筋の長手方向に略平行な鉄筋のエッジ間の距離のうちの少なくとも１つを含む、請求項１に記載の鉄筋判定装置。 The feature information extracted from the sample rebar image includes the frequency spectrum obtained by Fourier transforming the sample rebar image, the peak frequency of the frequency spectrum, the phase information extracted from the result of the Fourier transform, and the local region extracted from the sample rebar image. The reinforcing bar determination device according to claim 1, further comprising a feature amount and at least one of the distances between the edges of the reinforcing bars substantially parallel to the longitudinal direction of the reinforcing bars shown in the sample reinforcing bar image.
  18. 　前記出力部は、前記鉄筋画像と前記サンプル鉄筋画像とが所定の条件を満たして類似している場合に、前記サンプル鉄筋画像と対応する前記種別情報を、前記鉄筋画像または前記鉄筋画像から抽出した特徴情報と関連付けて記憶部に記憶する、ことを特徴とする、請求項１に記載の鉄筋判定装置。 When the rebar image and the sample rebar image are similar to each other satisfying a predetermined condition, the output unit extracts the type information corresponding to the sample rebar image from the rebar image or the rebar image. The reinforcing bar determination device according to claim 1, wherein the reinforcing bar determination device is stored in a storage unit in association with feature information.
  19. 　学習済みモデルを記憶する記憶部と、
    　判定対象の鉄筋の写る鉄筋画像の入力を受け付ける入力部と、
    　前記学習済みモデルに前記鉄筋画像を入力し、結果を出力する出力部と、
    を含み、
    　前記学習済みモデルは、異なる複数の種別ごとに、鉄筋の写るサンプル鉄筋画像または前記サンプル鉄筋画像から抽出された特徴情報と、前記サンプル鉄筋画像に写る鉄筋の種別を示す種別情報とを対応づけた複数のデータを教師データとして用い学習が行われている、ことを特徴とする鉄筋測定装置。     A storage unit that stores the trained model and
    An input unit that accepts input of a reinforcing bar image showing the reinforcing bar to be judged,
    An output unit that inputs the reinforcing bar image to the trained model and outputs the result,
    Including
    In the trained model, the feature information extracted from the sample reinforcing bar image in which the reinforcing bar is shown or the sample reinforcing bar image is associated with the type information indicating the type of the reinforcing bar shown in the sample reinforcing bar image for each of a plurality of different types. A reinforcing bar measuring device characterized in that learning is performed using a plurality of data as teacher data.
  20. 　判定対象の鉄筋が写る鉄筋画像の入力を受け付け、
    　鉄筋が写るサンプル鉄筋画像および前記サンプル鉄筋画像から抽出された特徴情報の少なくとも一方と、前記サンプル鉄筋画像に写る鉄筋の種別を示す種別情報とを対応づけた情報を含む鉄筋情報に基づいて、前記サンプル鉄筋画像と前記鉄筋画像との類似度を評価し、
    　前記類似度の評価結果に基づいて、前記鉄筋画像と前記サンプル鉄筋画像とが所定の条件を満たして類似している場合に、前記サンプル鉄筋画像と対応する前記種別情報を出力する、
    ことを含む、鉄筋判定方法。     Accepts the input of the reinforcing bar image that shows the reinforcing bar to be judged,
    Based on the reinforcing bar information including information in which at least one of the sample reinforcing bar image showing the reinforcing bar and the feature information extracted from the sample reinforcing bar image is associated with the type information indicating the type of the reinforcing bar shown in the sample reinforcing bar image. Evaluate the similarity between the sample rebar image and the rebar image,
    Based on the evaluation result of the degree of similarity, when the reinforcing bar image and the sample reinforcing bar image are similar to each other satisfying a predetermined condition, the type information corresponding to the sample reinforcing bar image is output.
    Reinforcing bar determination method including that.
  21. 　判定対象の鉄筋が写る鉄筋画像の入力を受け付ける入力部と、
    　鉄筋が写るサンプル鉄筋画像および前記サンプル鉄筋画像から抽出された特徴情報の少なくとも一方と、前記サンプル鉄筋画像に写る鉄筋の種別を示す種別情報とを対応づけた情報を含む鉄筋情報に基づいて、前記サンプル鉄筋画像と前記鉄筋画像との類似度を評価する評価部と、
    　前記類似度の評価結果に基づいて、前記鉄筋画像と前記サンプル鉄筋画像とが所定の条件を満たして類似している場合に、前記サンプル鉄筋画像と対応する前記種別情報を出力する出力部と、
    　を備える、鉄筋判定装置。     An input unit that accepts input of a reinforcing bar image showing the reinforcing bar to be judged,
    Based on the reinforcing bar information including information in which at least one of the sample reinforcing bar image showing the reinforcing bar and the feature information extracted from the sample reinforcing bar image is associated with the type information indicating the type of the reinforcing bar shown in the sample reinforcing bar image. An evaluation unit that evaluates the similarity between the sample rebar image and the rebar image,
    Based on the evaluation result of the degree of similarity, when the reinforcing bar image and the sample reinforcing bar image are similar to each other satisfying a predetermined condition, an output unit that outputs the type information corresponding to the sample reinforcing bar image and an output unit.
    Reinforcing bar determination device.
PCT/JP2019/031535 2019-08-08 2019-08-08 Reinforcing bar determination device and reinforcing bar determination method WO2021024499A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/JP2019/031535 WO2021024499A1 (en) 2019-08-08 2019-08-08 Reinforcing bar determination device and reinforcing bar determination method
JP2021537556A JP7438220B2 (en) 2019-08-08 2019-08-08 Reinforcing bar determination device and reinforcing bar determination method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2019/031535 WO2021024499A1 (en) 2019-08-08 2019-08-08 Reinforcing bar determination device and reinforcing bar determination method

Publications (1)

Publication Number Publication Date
WO2021024499A1 true WO2021024499A1 (en) 2021-02-11

Family

ID=74503389

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/031535 WO2021024499A1 (en) 2019-08-08 2019-08-08 Reinforcing bar determination device and reinforcing bar determination method

Country Status (2)

Country Link
JP (1) JP7438220B2 (en)
WO (1) WO2021024499A1 (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114565540A (en) * 2022-04-30 2022-05-31 深圳市巨力方视觉技术有限公司 Machine vision integrated system for denoising based on multi-path contrast images
JP2022164949A (en) * 2021-04-19 2022-10-31 三菱電機エンジニアリング株式会社 Bar arrangement inspection device, bar arrangement inspection method and program
WO2022254715A1 (en) * 2021-06-04 2022-12-08 日本電信電話株式会社 Deterioration determination device, deterioration determination method, and program
JP7214023B1 (en) 2022-02-07 2023-01-27 三菱電機エンジニアリング株式会社 Bar arrangement inspection device, bar arrangement inspection method and program
JP7283849B1 (en) 2022-10-18 2023-05-30 コムシス情報システム株式会社 A bar arrangement inspection system and a bar arrangement inspection method.
WO2024084673A1 (en) * 2022-10-21 2024-04-25 三菱電機株式会社 Reinforcement inspection device, learning device, reinforcement inspection system, and reinforcement inspection method
JP7494341B1 (en) 2023-02-22 2024-06-03 戸田建設株式会社 Reinforcing bar inspection method and reinforcing bar inspection device

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH04177894A (en) * 1990-11-13 1992-06-25 Matsushita Electric Ind Co Ltd Method of identifying sort of electronic part
JPH0696220A (en) * 1992-09-16 1994-04-08 Fujitsu Ltd Object recognizing system using hierarchical neural network
JP2001292314A (en) * 2000-04-06 2001-10-19 Canon Inc Image processing apparatus and image processing method, and storage medium
JP2015087801A (en) * 2013-10-28 2015-05-07 セイコーエプソン株式会社 Image processing apparatus, robot system, image processing method, and image processing program
JP2016003981A (en) * 2014-06-18 2016-01-12 株式会社大林組 Bar arrangement inspection system, bar arrangement inspection program and bar arrangement inspection method
JP2018173277A (en) * 2017-03-31 2018-11-08 オリンパス株式会社 Information processing equipment, information processing method, and, program

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5867210B2 (en) 2011-07-15 2016-02-24 株式会社大林組 Reinforcing bar standard identification device and reinforcing bar standard information creation device
JP6236199B2 (en) 2012-06-18 2017-11-22 株式会社大林組 Bar arrangement system
WO2017094456A1 (en) 2015-12-02 2017-06-08 富士フイルム株式会社 Object inspection apparatus and inspection method

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH04177894A (en) * 1990-11-13 1992-06-25 Matsushita Electric Ind Co Ltd Method of identifying sort of electronic part
JPH0696220A (en) * 1992-09-16 1994-04-08 Fujitsu Ltd Object recognizing system using hierarchical neural network
JP2001292314A (en) * 2000-04-06 2001-10-19 Canon Inc Image processing apparatus and image processing method, and storage medium
JP2015087801A (en) * 2013-10-28 2015-05-07 セイコーエプソン株式会社 Image processing apparatus, robot system, image processing method, and image processing program
JP2016003981A (en) * 2014-06-18 2016-01-12 株式会社大林組 Bar arrangement inspection system, bar arrangement inspection program and bar arrangement inspection method
JP2018173277A (en) * 2017-03-31 2018-11-08 オリンパス株式会社 Information processing equipment, information processing method, and, program

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2022164949A (en) * 2021-04-19 2022-10-31 三菱電機エンジニアリング株式会社 Bar arrangement inspection device, bar arrangement inspection method and program
JP7186821B2 (en) 2021-04-19 2022-12-09 三菱電機エンジニアリング株式会社 Bar arrangement inspection device, bar arrangement inspection method and program
WO2022254715A1 (en) * 2021-06-04 2022-12-08 日本電信電話株式会社 Deterioration determination device, deterioration determination method, and program
JP7214023B1 (en) 2022-02-07 2023-01-27 三菱電機エンジニアリング株式会社 Bar arrangement inspection device, bar arrangement inspection method and program
JP2023114495A (en) * 2022-02-07 2023-08-18 三菱電機エンジニアリング株式会社 Bar arrangement inspection device, bar arrangement inspection method and program
CN114565540A (en) * 2022-04-30 2022-05-31 深圳市巨力方视觉技术有限公司 Machine vision integrated system for denoising based on multi-path contrast images
JP7283849B1 (en) 2022-10-18 2023-05-30 コムシス情報システム株式会社 A bar arrangement inspection system and a bar arrangement inspection method.
JP2024059225A (en) * 2022-10-18 2024-05-01 コムシス情報システム株式会社 Reinforcement inspection system and reinforcement inspection method.
WO2024084673A1 (en) * 2022-10-21 2024-04-25 三菱電機株式会社 Reinforcement inspection device, learning device, reinforcement inspection system, and reinforcement inspection method
JP7494341B1 (en) 2023-02-22 2024-06-03 戸田建設株式会社 Reinforcing bar inspection method and reinforcing bar inspection device

Also Published As

Publication number Publication date
JP7438220B2 (en) 2024-02-26
JPWO2021024499A1 (en) 2021-02-11

Similar Documents

Publication Publication Date Title
WO2021024499A1 (en) Reinforcing bar determination device and reinforcing bar determination method
Prasanna et al. Automated crack detection on concrete bridges
JP4885584B2 (en) Rangefinder calibration method and apparatus
US8019164B2 (en) Apparatus, method and program product for matching with a template
WO2013020143A1 (en) Image-based crack quantification
JP5684505B2 (en) Image processing apparatus, image processing method, and computer program
US9805443B2 (en) Image processing method, image processing apparatus, program, storage medium, production apparatus, and method of producing assembly
JP6899189B2 (en) Systems and methods for efficiently scoring probes in images with a vision system
CN102834845A (en) Method and arrangement for multi-camera calibration
JP6317725B2 (en) System and method for determining clutter in acquired images
KR20130026741A (en) Method and apparatus for providing automated detection of calibration
JP4193519B2 (en) Object identification method and object identification apparatus
TWI664421B (en) Method,apparatus ,and computer software product for image processing
JP2003216931A (en) Specific pattern recognizing method, specific pattern recognizing program, specific pattern recognizing program storage medium and specific pattern recognizing device
JP5099120B2 (en) Template matching device, camera equipped with template matching device, computer matching program for computer
JP2006323779A (en) Image processing method and device
JP5791361B2 (en) PATTERN IDENTIFICATION DEVICE, PATTERN IDENTIFICATION METHOD, AND PROGRAM
JP2008116207A (en) Apparatus and method for image measurement, and program
JP2006317418A (en) Image measuring device, image measurement method, measurement processing program, and recording medium
WO2019229855A1 (en) Object detection device, object detection system, object detection method, and recording medium having program recorded thereon
JP5076567B2 (en) Template matching device, camera equipped with template matching device, computer matching program for computer
Hołowko et al. Color-based algorithm for automatic merging of multiview 3D point clouds
Petty et al. Improved image analysis methodology for detecting changes in evidence positioning at crime scenes
WO2015089564A1 (en) Thickness estimation for microscopy
JP2023166058A (en) Reinforcement inspection device, reinforcement inspection method and program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19940683

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021537556

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19940683

Country of ref document: EP

Kind code of ref document: A1