WO2021079456A1 - カメラ校正装置、カメラ校正方法、及び非一時的なコンピュータ可読媒体 - Google Patents
カメラ校正装置、カメラ校正方法、及び非一時的なコンピュータ可読媒体 Download PDFInfo
- Publication number
- WO2021079456A1 WO2021079456A1 PCT/JP2019/041681 JP2019041681W WO2021079456A1 WO 2021079456 A1 WO2021079456 A1 WO 2021079456A1 JP 2019041681 W JP2019041681 W JP 2019041681W WO 2021079456 A1 WO2021079456 A1 WO 2021079456A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image system
- vanishing point
- system coordinates
- type part
- image
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims description 21
- 238000004364 calculation method Methods 0.000 claims abstract description 63
- 239000013598 vector Substances 0.000 claims description 98
- 239000011159 matrix material Substances 0.000 claims description 9
- 210000004394 hip joint Anatomy 0.000 claims description 7
- 210000003127 knee Anatomy 0.000 claims description 7
- 210000000323 shoulder joint Anatomy 0.000 claims description 4
- 210000003423 ankle Anatomy 0.000 claims description 3
- 210000004705 lumbosacral region Anatomy 0.000 claims description 2
- 210000000707 wrist Anatomy 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 10
- 238000001514 detection method Methods 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 6
- 238000004891 communication Methods 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 239000004020 conductor Substances 0.000 description 1
- 238000004141 dimensional analysis Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/26—Measuring arrangements characterised by the use of optical techniques for measuring angles or tapers; for testing the alignment of axes
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B15/00—Special procedures for taking photographs; Apparatus therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20036—Morphological image processing
- G06T2207/20044—Skeletonization; Medial axis transform
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Definitions
- This disclosure relates to a camera calibration device, a camera calibration method, and a non-temporary computer-readable medium.
- Optical characteristics are parameters unique to each camera, such as focal length, lens distortion, and optical center coordinates, and are collectively called internal parameters.
- the internal parameters are invariant unless the zoom value is changed or a different lens is replaced.
- the parameters that represent the positional relationship of the cameras refer to the rotation matrix and translation vector, and are called “external parameters”.
- the external parameters are invariant unless the camera is moved with respect to the origin of the world coordinate system (three-dimensional coordinates).
- Non-Patent Document 1 As a method for calculating camera parameters, for example, the Tsai method described in Non-Patent Document 1 and the Zhang method described in Non-Patent Document 2 are widely known. These methods use a calibrated object (for example, a calibrated board), and link the world coordinates (three-dimensional coordinates) of the pattern drawn on the calibrated object with the image coordinates of the pattern observed on the image. Then, calculate the camera parameters.
- a calibrated object for example, a calibrated board
- Non-Patent Documents 1 and 2 are huge in order to cover a wide field of view of a camera installed in a wide space such as outdoors and calibrating a wide area environment. Requires a good calibration object. Therefore, it may be virtually impossible to calibrate, or even if it is possible, it may not be convenient. For example, even if a huge proofing object can be prepared, it is necessary to prevent any object that blocks the proofing object from being reflected during shooting. For example, in the case of a road surveillance camera, traffic restrictions for pedestrians and cars are required.
- An object of the present disclosure is to provide a camera calibration device, a camera calibration method, and a non-temporary computer-readable medium capable of calculating camera parameters by a simpler method.
- the camera calibrator is a type 1 part extracted from a plurality of person images included in one image in which the world coordinate space is photographed by the camera or a plurality of images photographed in time series.
- a first coordinate pair including the first image system coordinates and the second image system coordinates of the second type part, a second including the third image system coordinates of the first type part and the fourth image system coordinates of the second type part.
- the fourth coordinate pair including the eighth image system coordinates of the above is acquired, and the first kind part and the second kind part are distributed in the horizontal direction in the world coordinate space, and the third kind part and the fourth kind The parts are distributed in the vertical direction in the world coordinate space, and the acquisition means and The first vanishing point in the horizontal direction is calculated based on the first coordinate pair and the second coordinate pair, and the second vanishing point in the vertical direction is calculated based on the third coordinate pair and the fourth coordinate pair.
- Vanishing point calculation means for calculating A camera parameter calculation means for calculating the camera parameters of the camera based on the first vanishing point and the second vanishing point. To be equipped.
- the camera calibration method is a type 1 part extracted from a plurality of person images included in one image in which the world coordinate space is photographed by the camera or a plurality of images photographed in time series.
- the fourth coordinate pair including the eighth image system coordinates of the above is acquired, and the first kind part and the second kind part are distributed in the horizontal direction in the world coordinate space, and the third kind part and the fourth kind The parts are distributed vertically in the world coordinate space and The first vanishing point in the horizontal direction is calculated based on the first coordinate pair and the second coordinate pair, and the second vanishing point in the vertical direction is calculated based on the third coordinate pair and the fourth coordinate pair. Is calculated and The camera parameters of the camera are calculated based on the first vanishing point and the second vanishing point.
- the non-temporary computer-readable medium according to the third aspect is extracted from a plurality of portrait images included in one image in which the world coordinate space is photographed by a camera or a plurality of images photographed in time series.
- the first coordinate pair including the first image system coordinates of the first type part and the second image system coordinates of the second type part, the third image system coordinates of the first type part and the fourth image system coordinates of the second type part.
- the second coordinate pair including the third coordinate pair including the fifth image system coordinate of the third type part and the sixth image system coordinate of the fourth type part, and the seventh image system coordinate of the third type part and the above.
- a fourth coordinate pair including the eighth image system coordinates of the fourth type part is acquired, and the first type part and the second type part are distributed in the horizontal direction in the world coordinate space, and the third type part and the third type part and the second type part are distributed.
- the type 4 site is distributed in the vertical direction in the world coordinate space.
- the first vanishing point in the horizontal direction is calculated based on the first coordinate pair and the second coordinate pair, and the second vanishing point in the vertical direction is calculated based on the third coordinate pair and the fourth coordinate pair.
- the camera parameters of the camera are calculated based on the first vanishing point and the second vanishing point.
- a program that causes the camera calibration device to execute the process is stored.
- FIG. 1 is a block diagram showing an example of a camera calibration device according to the first embodiment.
- the camera calibration device 10 of FIG. 1 is a device for calculating (calibrating) camera parameters used at the time of shooting with a single camera (not shown).
- the camera calibration device 10 includes an acquisition unit 11, a vanishing point calculation unit 12, and a camera parameter calculation unit 13.
- the acquisition unit 11 is a "first coordinate pair" extracted from “a plurality of person images” included in one image in which the world coordinate space is photographed by a camera (not shown) or a plurality of images photographed in time series. , "Second coordinate pair", “third coordinate pair", and “fourth coordinate pair”.
- the “first coordinate pair” includes the “first image system coordinates” of the "first type part” and the “second image system coordinates” of the "second type part”.
- the “second coordinate pair” includes the "third image system coordinate” of the "first type part” and the "fourth image system coordinate” of the "second type part”.
- the "type 1 part” and the "type 2 part” are parts of a person that are distributed (arranged) in the horizontal direction in the world coordinate space.
- the "third coordinate pair” includes the "fifth image system coordinate” of the "third type part” and the “sixth image system coordinate” of the "fourth type part”.
- the "fourth coordinate pair” includes the “seventh image system coordinate” of the "third type part” and the “eighth image system coordinate” of the "fourth type part”.
- the "type 3 part” and the “type 4 part” are parts of a person that are distributed (arranged) in the vertical direction in the world coordinate space.
- the particle size of the "site” may be freely defined.
- Each of the "first image system coordinates" to the “eighth image system coordinates” is represented by, for example, three-dimensional coordinates obtained by adding scale indefiniteness to the two-dimensional coordinates defining the image plane, so-called "homogeneous coordinates”. ing.
- the first coordinate pair and the third coordinate pair include the image system coordinates extracted from the "first person image” of one person.
- the second coordinate pair and the fourth coordinate pair are from the "second person image” of another person included in the image including the first person image, or to an image different from the image including the first person image. It is extracted from the "third person image" of the above-mentioned one person included.
- the combination of the first type part and the second type part is the combination of the left shoulder joint and the right shoulder joint, the combination of the left hip joint and the right hip joint, the combination of the left eye and the right eye, the combination of the left ear and the right ear, the left knee and the right. It may be a combination of knees or a combination of left ankle and right ankle.
- the combination of the type 3 part and the type 4 part is a combination of the upper end and the lower end of the spine, a combination of the crown and the lumbar region, a combination of the hip joint and the knee (especially in a stationary person), or a combination of the hip joint and the knee. It may be a combination of elbows and wrists.
- the vanishing point calculation unit 12 calculates the "first vanishing point” in the horizontal direction based on the first coordinate pair and the second coordinate pair acquired by the acquisition unit 11, and the third coordinate acquired by the acquisition unit 11.
- the "second vanishing point” in the vertical direction is calculated based on the pair and the fourth coordinate pair.
- the camera parameter calculation unit 13 calculates the camera parameters of the camera (not shown) based on the first vanishing point and the second vanishing point calculated by the vanishing point calculation unit 12.
- the vanishing point calculation unit 12 in the camera calibration device 10 is in the horizontal direction based on the above-mentioned first coordinate pair and second coordinate pair acquired by the acquisition unit 11.
- the "first vanishing point" of the above is calculated, and the “second vanishing point” in the vertical direction is calculated based on the above-mentioned third coordinate pair and fourth coordinate pair acquired by the acquisition unit 11.
- the camera parameter calculation unit 13 calculates the camera parameters of the camera (not shown) based on the first vanishing point and the second vanishing point calculated by the vanishing point calculation unit 12.
- the camera parameters are calculated using the image system coordinates of a predetermined part of the person, so that the camera parameters can be calculated by a simpler method without the need for a calibration object.
- the second embodiment relates to a more specific embodiment.
- FIG. 2 is a block diagram showing an example of the camera calibration device according to the second embodiment.
- the camera calibration device 10 according to the second embodiment has an acquisition unit 11, a vanishing point calculation unit 12, and a camera parameter calculation unit 13 as in the first embodiment.
- the acquisition unit 11 in the second embodiment acquires the "first coordinate pair", the "second coordinate pair", the "third coordinate pair”, and the "fourth coordinate pair” as in the first embodiment.
- the vanishing point calculation unit 12 in the second embodiment determines the "first vanishing point" in the horizontal direction based on the first coordinate pair and the second coordinate pair acquired by the acquisition unit 11.
- the "second vanishing point” in the vertical direction is calculated based on the third coordinate pair and the fourth coordinate pair acquired by the acquisition unit 11.
- the vanishing point calculation unit 12 in the second embodiment crosses a "first image system vector" whose end point is the first image system coordinates and a "second image system vector” whose end point is the second image system coordinates.
- the product is calculated to calculate the "first linear direction vector” which is the direction vector of the first straight line passing through the first image system coordinates or the second image system coordinates.
- the vanishing point calculation unit 12 calculates the cross product of the "third image system vector” having the third image system coordinates as the end point and the "fourth image system vector” having the fourth image system coordinates as the end point.
- the "second straight line direction vector” which is the direction vector of the second straight line passing through the third image system coordinates or the fourth image system coordinates is calculated.
- the vanishing point calculation unit 12 calculates the cross product of the "fifth image system vector" having the fifth image system coordinates as the end point and the "sixth image system vector” having the sixth image system coordinates as the end point. , The "third straight line direction vector” which is the direction vector of the third straight line passing through the fifth image system coordinates or the sixth image system coordinates is calculated. Further, the vanishing point calculation unit 12 calculates the cross product of the "7th image system vector" having the 7th image system coordinates as the end point and the "8th image system vector” having the 8th image system coordinates as the end point.
- the "fourth straight line direction vector” which is the direction vector of the fourth straight line passing through the seventh image system coordinates or the eighth image system coordinates is calculated.
- the starting point of each of the "first image system vector” to the “eighth image system vector” is the origin of the image system.
- the vanishing point calculation unit 12 calculates the cross product of the first linear direction vector and the second linear direction vector, and calculates the "first vanishing point vector" toward the first vanishing point. Further, the vanishing point calculation unit 12 calculates the cross product of the third linear direction vector and the fourth linear direction vector, and calculates the "second vanishing point vector" toward the second vanishing point.
- the camera parameter calculation unit 13 in the second embodiment has an internal parameter calculation unit 13A and an external parameter calculation unit 13B.
- the internal parameter calculation unit 13A calculates the internal parameters of the camera (corresponding to the camera 20 described later) based on the first vanishing point and the second vanishing point calculated by the vanishing point calculation unit 12.
- the external parameter calculation unit 13B is a camera (to the camera 20 described later) based on the first vanishing point and the second vanishing point calculated by the vanishing point calculation unit 12 and the internal parameters calculated by the internal parameter calculation unit 13A. Correspondence) Calculate the external parameters.
- FIG. 3 is a flowchart showing an example of the processing operation by the camera calibration device in the second embodiment.
- the acquisition unit 11 acquires the "first coordinate pair", the "second coordinate pair”, the "third coordinate pair”, and the "fourth coordinate pair” (step S101).
- the "first coordinate pair”, the "second coordinate pair”, the "third coordinate pair”, and the “fourth coordinate pair” are, for example, two people by the camera 20 installed as shown in FIG.
- the world coordinate space in which the people H1 and H2 exist is extracted from one image taken. People H1 and H2 are heading in the same direction in the shooting environment.
- the type 1 part and the type 2 part are the right shoulder and the left shoulder, respectively.
- the type 3 part and the type 4 part are the upper end portion and the lower end portion (for example, the base of the neck and the coccyx) of the spine, respectively.
- FIG. 5 shows a “first coordinate pair”, a “second coordinate pair”, a “third coordinate pair”, and a “fourth coordinate pair”.
- the image system coordinate m 1 and the image system coordinate m 2 are the "first coordinate pair”
- the image system coordinate m 3 and the image system coordinate m 4 are the "third coordinate pair”.
- the "first coordinate pair” and the "third coordinate pair” correspond to the person H1. That is, the image system coordinates m 1 , the image system coordinates m 2 , the image system coordinates m 3 , and the image system coordinates m 4 are the right shoulder P11, the left shoulder P12, the upper end portion P13, and the lower end portion P14 of the human H1, respectively. It corresponds to.
- the image-based coordinates m 1 'and an image-based coordinates m 2' is "second coordinate pair"
- the image-based coordinate m 3 'and the image-based coordinates m 4' is "fourth coordinate pair”.
- the "second coordinate pair” and the "fourth coordinate pair” correspond to the person H2. That is, the image-based coordinates m 1 ', the image based coordinate m 2', image-based coordinate m 3 ', and the image-based coordinates m 4', respectively, right shoulder P21 of human H2, left shoulder P22, spinal upper portion P23, And the lower end P24.
- 4 and 5 are diagrams for explaining an example of the processing operation by the camera calibration device in the second embodiment.
- the vanishing point calculation unit 12 calculates the horizontal "first vanishing point” based on the first coordinate pair and the second coordinate pair acquired by the acquisition unit 11, and the acquisition unit 11 The "second vanishing point" in the vertical direction is calculated based on the third coordinate pair and the fourth coordinate pair acquired by (step S102).
- the vanishing point calculation unit 12 calculates the cross product of the first image system vector m 1 and the second image system vector m 2, and passes through the image system coordinates m 1 or the image system coordinates m 2.
- the first linear direction vector l 1 which is a linear direction vector, is calculated (see FIG. 5). That is, the first linear direction vector l 1 is orthogonal to both the first image system vector m 1 and the second image system vector m 2.
- the vanishing point calculating section 12 passes through the calculated 'as a fourth image based vector m 2' third image based vector m 1 a cross product of the image-based coordinates m 1 'or the image-based coordinate m 2'
- the second linear direction vector l 2 which is the direction vector of the second straight line, is calculated.
- the vanishing point calculation unit 12 calculates the cross product of the fifth image system vector m 3 and the sixth image system vector m 4, and determines the third straight line passing through the image system coordinates m 3 or the image system coordinates m 4.
- the third linear direction vector l 3 which is a direction vector, is calculated.
- the vanishing point calculating section 12 passes through the calculated 'and the eighth image-based vector m 4' seventh image based vector m 3 a cross product of the image-based coordinate m 3 'or the image-based coordinate m 4'
- the fourth straight line direction vector l 4 which is the direction vector of the fourth straight line, is calculated.
- the vanishing point calculation unit 12 uses the following equation (1) to form the first linear direction vector l 1 , the second linear direction vector l 2 , the third linear direction vector l 3 , and the fourth linear direction vector l 3.
- l 4 is calculated.
- "x" is an operator representing a cross product (outer product) of three-dimensional vectors.
- the vanishing point computing section 12 calculates calculates first a linear direction vector l 1 a cross product of the second linear direction vector l 2, the first vanishing point vector V x toward the first vanishing point V x To do. Moreover, the vanishing point calculator 12 calculates a third linear direction vector l 3 4 calculates the cross product of the linear direction vector l 4, the second vanishing point vector V y toward the second vanishing V y To do.
- the vanishing point calculation unit 12 calculates the first vanishing point vector V x and the second vanishing point vector V y using the following equation (2).
- the internal parameter calculation unit 13A calculates the internal parameters of the camera 20 based on the first vanishing point and the second vanishing point calculated by the vanishing point calculation unit 12 (step S103).
- Equation (3) shows that by projecting a unit vector on the X-axis of the world coordinate system, a vector having a scale different from that of the first vanishing point vector (vanishing point in the horizontal direction) V x can be obtained. There is.
- Eq. (3) shows that by projecting the unit vector on the Y-axis of the world coordinate system, a vector having a scale different from that of the second vanishing point vector (vertical vanishing point) V y can be obtained. ing.
- K is a 3 ⁇ 3 upper triangular matrix representing the internal parameters
- R is a 3 ⁇ 3 rotation matrix representing the external parameters
- t is a three-dimensional translation vector which is an external parameter
- ri represents the i-th column of R.
- Equation (4) shows that one constraint equation can be obtained from the vanishing point in the horizontal direction and the vanishing point in the vertical direction. That is, one of the internal parameters can be estimated using this constraint equation.
- the focal length can be calculated (estimated) by solving the equation (4).
- skew, optical center, or lens distortion may be used as an internal parameter to be estimated.
- the focal length information is embedded in the image, it is possible to assume that the focal length is known and use an internal parameter other than the focal length as the estimation target parameter.
- the external parameter calculation unit 13B is based on the first vanishing point and the second vanishing point calculated by the vanishing point calculation unit 12 and the internal parameters calculated by the internal parameter calculation unit 13A.
- the external parameters of the camera 20 are calculated (step S104).
- the three-dimensional coordinates of each part in the world coordinate system are unknown. Therefore, a world coordinate system with an arbitrary part as the origin may be defined.
- the projective transformation of the image system coordinate m 1 is expressed by the following equation (6). That is, the equation (6) shows that the image system coordinate m 1 can be obtained by projecting the origin in the world coordinate system.
- the external parameter calculation unit 13B calculates the rotation matrix R using the equation (5), and calculates the translation vector using the equation (7).
- the vanishing point calculation unit 12 in the camera calibration device 10 is in the horizontal direction based on the above-mentioned first coordinate pair and second coordinate pair acquired by the acquisition unit 11.
- the "first vanishing point" of the above is calculated, and the “second vanishing point” in the vertical direction is calculated based on the above-mentioned third coordinate pair and fourth coordinate pair acquired by the acquisition unit 11.
- the camera parameter calculation unit 13 calculates the camera parameters of the camera (not shown) based on the first vanishing point and the second vanishing point calculated by the vanishing point calculation unit 12.
- the vanishing point is calculated using the image system coordinates of a predetermined part of the person and the camera parameters are calculated based on the vanishing point, as in the first embodiment, so that a calibrated object is required.
- the camera parameters can be calculated by a simpler method. The reason for this is as follows. That is, when a person walks, it can be assumed that, for example, the line segment corresponding to the spine is distributed in the vertical direction, and the line segment connecting both shoulders is distributed in the horizontal direction. When there are a plurality of pedestrians, the conductors are expected to be directed in almost the same direction for all pedestrians. For example, in places such as corridors and pedestrian zones on roads, people move in one direction. Therefore, even if a plurality of different pedestrians can be observed, it is possible to calculate the vanishing point by using the part information of the pedestrian. Therefore, the camera parameters can be calculated by a simpler method without the need for a calibration object.
- the acquisition unit 11 is extracted (detected) outside the camera calibration device 10, and the "first coordinate pair", "second coordinate pair", and "third".
- the acquisition unit 11 may include the site detection unit 11A.
- the site detection unit 11A acquires one image in which the world coordinate space is taken by the camera 20 or a plurality of images taken in time series, and from the one image or a plurality of person images included in the plurality of images. , "First coordinate pair", “second coordinate pair”, “third coordinate pair", and "fourth coordinate pair” are detected.
- FIG. 6 is a block diagram showing an example of the camera calibration device according to the other embodiment ⁇ 1>.
- the acquisition unit 11 may include the site information reception unit 11B in place of the site detection unit 11A or together with the site detection unit 11A as shown in FIG. 7.
- the part information receiving unit 11B receives the manually input information about the "first coordinate pair", the "second coordinate pair", the "third coordinate pair", and the "fourth coordinate pair".
- FIG. 7 is a block diagram showing another example of the camera calibration device according to the other embodiment ⁇ 1>.
- the vanishing point calculation unit 12 has two coordinate pairs related to parts distributed in the horizontal direction in the world coordinate space, and a vertical direction in the world coordinate space. The description has been made assuming that the "first vanishing point" in the horizontal direction and the "second vanishing point” in the vertical direction are calculated based on the two coordinate pairs related to the parts distributed in the above. Not limited to.
- the vanishing point calculation unit 12 receives three or more coordinate pairs related to the parts distributed in the horizontal direction in the world coordinate space, and based on the three or more coordinate pairs, the vanishing point calculation unit 12 is subjected to the horizontal "first" by the minimum square method. You may calculate "1 vanishing point".
- the vanishing point calculation unit 12 receives three or more coordinate pairs related to the parts distributed in the vertical direction in the world coordinate space, and based on the three or more coordinate pairs, the vertical direction is performed by the minimum square method.
- the "second vanishing point" of may be calculated.
- the vanishing point calculation unit 12 further uses known techniques such as so-called RANSAC (Random Sample Consensus) and the weighted least squares method in order to remove inputs with large outliers and errors to improve the estimation accuracy. You may.
- RANSAC Random Sample Consensus
- FIG. 8 is a diagram showing a hardware configuration example of the camera calibration device.
- the camera calibration device 100 has a processor 101 and a memory 102.
- the processor 101 may be, for example, a microprocessor, an MPU (Micro Processing Unit), or a CPU (Central Processing Unit).
- the processor 101 may include a plurality of processors.
- the memory 102 is composed of a combination of a volatile memory and a non-volatile memory.
- the memory 102 may include storage located away from the processor 101. In this case, the processor 101 may access the memory 102 via an I / O interface (not shown).
- the camera calibration device 10 of the first embodiment and the second embodiment can have the hardware configuration shown in FIG.
- the acquisition unit 11, the vanishing point calculation unit 12, and the camera parameter calculation unit 13 of the camera calibration device 10 of the first embodiment and the second embodiment read and execute a program stored in the memory 102 by the processor 101. It may be realized.
- the program is stored using various types of non-transitory computer readable medium and can be supplied to the camera calibrator 10. Examples of non-transitory computer-readable media include magnetic recording media (eg, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (eg, magneto-optical disks). Further, examples of non-temporary computer-readable media include CD-ROM (Read Only Memory), CD-R, and CD-R / W.
- non-transitory computer-readable media include semiconductor memory.
- the semiconductor memory includes, for example, a mask ROM, a PROM (Programmable ROM), an EPROM (Erasable PROM), a flash ROM, and a RAM (Random Access Memory).
- the program may also be supplied to the camera calibrator 10 by various types of temporary computer readable media. Examples of temporary computer-readable media include electrical, optical, and electromagnetic waves.
- the temporary computer-readable medium can supply the program to the camera calibration device 10 via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
Description
前記第1座標ペア及び前記第2座標ペアに基づいて、前記水平方向の第1消失点を算出し、前記第3座標ペア及び前記第4座標ペアに基づいて、前記鉛直方向の第2消失点を算出する消失点算出手段と、
前記第1消失点及び前記第2消失点に基づいて、前記カメラのカメラパラメータを算出するカメラパラメータ算出手段と、
を具備する。
前記第1座標ペア及び前記第2座標ペアに基づいて、前記水平方向の第1消失点を算出し、前記第3座標ペア及び前記第4座標ペアに基づいて、前記鉛直方向の第2消失点を算出し、
前記第1消失点及び前記第2消失点に基づいて、前記カメラのカメラパラメータを算出する。
前記第1座標ペア及び前記第2座標ペアに基づいて、前記水平方向の第1消失点を算出し、前記第3座標ペア及び前記第4座標ペアに基づいて、前記鉛直方向の第2消失点を算出し、
前記第1消失点及び前記第2消失点に基づいて、前記カメラのカメラパラメータを算出する、
処理を、カメラ校正装置に実行させるプログラムが格納している。
図1は、第1実施形態におけるカメラ校正装置の一例を示すブロック図である。図1のカメラ校正装置10は、単一のカメラ(不図示)による撮影時に用いられるカメラパラメータを算出(校正)するための装置である。図1においてカメラ校正装置10は、取得部11と、消失点算出部12と、カメラパラメータ算出部13とを有している。
第2実施形態は、より具体的な実施形態に関する。
図2は、第2実施形態におけるカメラ校正装置の一例を示すブロック図である。図2において第2実施形態におけるカメラ校正装置10は、第1実施形態と同様に、取得部11と、消失点算出部12と、カメラパラメータ算出部13とを有している。
以上の構成を有するカメラ校正装置による処理動作の一例について説明する。図3は、第2実施形態におけるカメラ校正装置による処理動作の一例を示すフローチャートである。
<1>第1実施形態及び第2実施形態では、取得部11がカメラ校正装置10の外部にて抽出(検出)された、「第1座標ペア」、「第2座標ペア」、「第3座標ペア」、及び「第4座標ペア」を取得することを前提に説明を行ったが、本開示はこれに限定されるものではない。例えば、図6に示すように、カメラ校正装置10にて取得部11は、部位検出部11Aを含んでいてもよい。部位検出部11Aは、カメラ20によって世界座標空間が撮影された1つの画像又は時系列に撮影された複数の画像を取得し、該1つの画像又は該複数の画像に含まれる複数の人物画像から、「第1座標ペア」、「第2座標ペア」、「第3座標ペア」、及び「第4座標ペア」を検出する。図6は、他の実施形態<1>におけるカメラ校正装置の一例を示すブロック図である。
11 取得部
11A 部位検出部
11B 部位情報受付部
12 消失点算出部
13 カメラパラメータ算出部
13A 内部パラメータ算出部
13B 外部パラメータ算出部
20 カメラ
Claims (9)
- カメラによって世界座標空間が撮影された1つの画像又は時系列に撮影された複数の画像に含まれる複数の人物画像から抽出された、第1種部位の第1画像系座標及び第2種部位の第2画像系座標を含む第1座標ペア、前記第1種部位の第3画像系座標及び前記第2種部位の第4画像系座標を含む第2座標ペア、第3種部位の第5画像系座標及び第4種部位の第6画像系座標を含む第3座標ペア、並びに、前記第3種部位の第7画像系座標及び前記第4種部位の第8画像系座標を含む第4座標ペアを取得し、前記第1種部位及び前記第2種部位は、前記世界座標空間において水平方向に分布し、前記第3種部位及び前記第4種部位は、前記世界座標空間において鉛直方向に分布する、取得手段と、
前記第1座標ペア及び前記第2座標ペアに基づいて、前記水平方向の第1消失点を算出し、前記第3座標ペア及び前記第4座標ペアに基づいて、前記鉛直方向の第2消失点を算出する消失点算出手段と、
前記第1消失点及び前記第2消失点に基づいて、前記カメラのカメラパラメータを算出するカメラパラメータ算出手段と、
を具備するカメラ校正装置。 - 前記消失点算出手段は、
前記第1画像系座標を終点とする第1画像系ベクトルと前記第2画像系座標を終点とする第2画像系ベクトルとのクロス積を算出して、前記第1画像系座標又は前記第2画像系座標を通る第1直線の方向ベクトルである第1直線方向ベクトルを算出し、
前記第3画像系座標を終点とする第3画像系ベクトルと前記第4画像系座標を終点とする第4画像系ベクトルとのクロス積を算出して、前記第3画像系座標又は前記第4画像系座標を通る第2直線の方向ベクトルである第2直線方向ベクトルを算出し、
前記第5画像系座標を終点とする第5画像系ベクトルと前記第6画像系座標を終点とする第6画像系ベクトルとのクロス積を算出して、前記第5画像系座標又は前記第6画像系座標を通る第3直線の方向ベクトルである第3直線方向ベクトルを算出し、
前記第7画像系座標を終点とする第7画像系ベクトルと前記第8画像系座標を終点とする第8画像系ベクトルとのクロス積を算出して、前記第7画像系座標又は前記第8画像系座標を通る第4直線の方向ベクトルである第4直線方向ベクトルを算出し、
前記第1直線方向ベクトルと前記第2直線方向ベクトルとのクロス積を算出して、前記第1消失点に向かう第1消失点ベクトルを算出し、
前記第3直線方向ベクトルと前記第4直線方向ベクトルとのクロス積を算出して、前記第2消失点に向かう第2消失点ベクトルを算出する、
請求項1記載のカメラ校正装置。 - 前記カメラパラメータ算出手段は、
前記第1消失点及び前記第2消失点に基づいて、前記カメラの内部パラメータを算出する内部パラメータ算出手段と、
前記第1消失点、前記第2消失点、及び、前記内部パラメータに基づいて、前記カメラの外部パラメータを算出する外部パラメータ算出手段と、
を具備する、
請求項1又は2に記載のカメラ校正装置。 - 前記カメラパラメータ算出手段は、
前記第1消失点及び前記第2消失点に基づいて、前記カメラの内部パラメータを算出する内部パラメータ算出手段と、
前記第1消失点、前記第2消失点、及び、前記内部パラメータに基づいて、前記カメラの外部パラメータを算出する外部パラメータ算出手段と、
を具備し、
前記内部パラメータ算出手段は、下記の式を用いて、前記第1消失点ベクトルと前記第2消失点ベクトルとに基づいて、前記内部パラメータを算出する、
請求項2記載のカメラ校正装置。
- 前記第1種部位及び前記第2種部位の組み合わせは、左肩関節及び右肩関節の組み合わせ、左股関節及び右股関節の組み合わせ、左目及び右目の組み合わせ、左耳及び右耳の組み合わせ、左膝及び右膝の組み合わせ、又は、左足首及び右足首の組み合わせである、
請求項1から4のいずれか1項に記載のカメラ校正装置。 - 前記第3種部位及び前記第4種部位の組み合わせは、背骨の上端部と下端部との組み合わせ、頭頂部と腰部との組み合わせ、股関節と膝との組み合わせ、又は、肘と手首との組み合わせである、
請求項1から5のいずれか1項に記載のカメラ校正装置。 - 前記第1座標ペア及び前記第3座標ペアは、一の人物の第1人物画像から抽出され、
前記第2座標ペア及び前記第4座標ペアは、前記第1人物画像が含まれる画像に含まれる他の人物の第2人物画像から、又は、前記第1人物画像が含まれる画像と異なる画像に含まれる前記一の人物の第3人物画像から、抽出されている、
請求項1から6のいずれか1項に記載のカメラ校正装置。 - カメラによって世界座標空間が撮影された1つの画像又は時系列に撮影された複数の画像に含まれる複数の人物画像から抽出された、第1種部位の第1画像系座標及び第2種部位の第2画像系座標を含む第1座標ペア、前記第1種部位の第3画像系座標及び前記第2種部位の第4画像系座標を含む第2座標ペア、第3種部位の第5画像系座標及び第4種部位の第6画像系座標を含む第3座標ペア、並びに、前記第3種部位の第7画像系座標及び前記第4種部位の第8画像系座標を含む第4座標ペアを取得し、前記第1種部位及び前記第2種部位は、前記世界座標空間において水平方向に分布し、前記第3種部位及び前記第4種部位は、前記世界座標空間において鉛直方向に分布し、
前記第1座標ペア及び前記第2座標ペアに基づいて、前記水平方向の第1消失点を算出し、前記第3座標ペア及び前記第4座標ペアに基づいて、前記鉛直方向の第2消失点を算出し、
前記第1消失点及び前記第2消失点に基づいて、前記カメラのカメラパラメータを算出する、
カメラ校正方法。 - カメラによって世界座標空間が撮影された1つの画像又は時系列に撮影された複数の画像に含まれる複数の人物画像から抽出された、第1種部位の第1画像系座標及び第2種部位の第2画像系座標を含む第1座標ペア、前記第1種部位の第3画像系座標及び前記第2種部位の第4画像系座標を含む第2座標ペア、第3種部位の第5画像系座標及び第4種部位の第6画像系座標を含む第3座標ペア、並びに、前記第3種部位の第7画像系座標及び前記第4種部位の第8画像系座標を含む第4座標ペアを取得し、前記第1種部位及び前記第2種部位は、前記世界座標空間において水平方向に分布し、前記第3種部位及び前記第4種部位は、前記世界座標空間において鉛直方向に分布し、
前記第1座標ペア及び前記第2座標ペアに基づいて、前記水平方向の第1消失点を算出し、前記第3座標ペア及び前記第4座標ペアに基づいて、前記鉛直方向の第2消失点を算出し、
前記第1消失点及び前記第2消失点に基づいて、前記カメラのカメラパラメータを算出する、
処理を、カメラ校正装置に実行させるプログラムが格納された非一時的なコンピュータ可読媒体。
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2021553226A JP7239015B2 (ja) | 2019-10-24 | 2019-10-24 | カメラ校正装置、カメラ校正方法、及びプログラム |
PCT/JP2019/041681 WO2021079456A1 (ja) | 2019-10-24 | 2019-10-24 | カメラ校正装置、カメラ校正方法、及び非一時的なコンピュータ可読媒体 |
US17/768,291 US20240233179A9 (en) | 2019-10-24 | 2019-10-24 | Camera calibration apparatus, camera calibration method, and non-transitory computer readable medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2019/041681 WO2021079456A1 (ja) | 2019-10-24 | 2019-10-24 | カメラ校正装置、カメラ校正方法、及び非一時的なコンピュータ可読媒体 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2021079456A1 true WO2021079456A1 (ja) | 2021-04-29 |
Family
ID=75619949
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2019/041681 WO2021079456A1 (ja) | 2019-10-24 | 2019-10-24 | カメラ校正装置、カメラ校正方法、及び非一時的なコンピュータ可読媒体 |
Country Status (3)
Country | Link |
---|---|
US (1) | US20240233179A9 (ja) |
JP (1) | JP7239015B2 (ja) |
WO (1) | WO2021079456A1 (ja) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2023062994A1 (ja) * | 2021-10-12 | 2023-04-20 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ | 学習装置、学習方法、学習プログラム、カメラパラメータ算出装置、カメラパラメータ算出方法及びカメラパラメータ算出プログラム |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2007233908A (ja) * | 2006-03-03 | 2007-09-13 | Matsushita Electric Ind Co Ltd | オブジェクトサイズ算出装置、それを用いたオブジェクト検索装置およびオブジェクト分類装置ならびにオブジェクトサイズ算出方法 |
JP2010025569A (ja) * | 2008-07-15 | 2010-02-04 | Toa Corp | カメラパラメータ特定装置および方法ならびにプログラム |
US20130322767A1 (en) * | 2012-05-31 | 2013-12-05 | Qualcomm Incorporated | Pose estimation based on peripheral information |
-
2019
- 2019-10-24 JP JP2021553226A patent/JP7239015B2/ja active Active
- 2019-10-24 WO PCT/JP2019/041681 patent/WO2021079456A1/ja active Application Filing
- 2019-10-24 US US17/768,291 patent/US20240233179A9/en active Pending
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2007233908A (ja) * | 2006-03-03 | 2007-09-13 | Matsushita Electric Ind Co Ltd | オブジェクトサイズ算出装置、それを用いたオブジェクト検索装置およびオブジェクト分類装置ならびにオブジェクトサイズ算出方法 |
JP2010025569A (ja) * | 2008-07-15 | 2010-02-04 | Toa Corp | カメラパラメータ特定装置および方法ならびにプログラム |
US20130322767A1 (en) * | 2012-05-31 | 2013-12-05 | Qualcomm Incorporated | Pose estimation based on peripheral information |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2023062994A1 (ja) * | 2021-10-12 | 2023-04-20 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ | 学習装置、学習方法、学習プログラム、カメラパラメータ算出装置、カメラパラメータ算出方法及びカメラパラメータ算出プログラム |
Also Published As
Publication number | Publication date |
---|---|
US20240233179A9 (en) | 2024-07-11 |
JPWO2021079456A1 (ja) | 2021-04-29 |
JP7239015B2 (ja) | 2023-03-14 |
US20240135584A1 (en) | 2024-04-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20190311497A1 (en) | Methods and computer program products for calibrating stereo imaging systems by using a planar mirror | |
WO2017022033A1 (ja) | 画像処理装置、画像処理方法および画像処理プログラム | |
KR20150120066A (ko) | 패턴 프로젝션을 이용한 왜곡 보정 및 정렬 시스템, 이를 이용한 방법 | |
CN113841384B (zh) | 校准装置,用于校准的图表和校准方法 | |
JP7151879B2 (ja) | カメラ校正装置、カメラ校正方法、及びプログラム | |
US8531505B2 (en) | Imaging parameter acquisition apparatus, imaging parameter acquisition method and storage medium | |
JP6641729B2 (ja) | ラインセンサカメラのキャリブレーション装置及び方法 | |
US20220180560A1 (en) | Camera calibration apparatus, camera calibration method, and nontransitory computer readable medium storing program | |
JP6662382B2 (ja) | 情報処理装置および方法、並びにプログラム | |
Liu et al. | Epipolar rectification method for a stereovision system with telecentric cameras | |
WO2021079456A1 (ja) | カメラ校正装置、カメラ校正方法、及び非一時的なコンピュータ可読媒体 | |
JP6557640B2 (ja) | カメラキャリブレーション装置、カメラキャリブレーション方法及びカメラキャリブレーションプログラム | |
CN114663516A (zh) | 基于人类姿态校准多相机***的方法和装置 | |
CN110888957A (zh) | 一种对象定位的方法以及相关装置 | |
Bastanlar | A simplified two-view geometry based external calibration method for omnidirectional and PTZ camera pairs | |
JP2021189946A (ja) | 検出装置、検出方法及び検出プログラム | |
KR101673144B1 (ko) | 부분 선형화 기반의 3차원 영상 정합 방법 | |
Wang et al. | Accurate calibration of multi-perspective cameras from a generalization of the hand-eye constraint | |
CN109902695A (zh) | 一种面向像对直线特征匹配的线特征矫正与提纯方法 | |
JP6315542B2 (ja) | 画像生成装置及び画像生成方法 | |
Schmidt et al. | The calibration of the pan-tilt units for the active stereo head | |
Szalóki et al. | Optimizing camera placement for localization accuracy | |
EP4280629A2 (en) | Wirelessly rechargeable hearing device and charger for same | |
Zheng et al. | An efficient and practical calibration method for roadside camera using two vanishing points | |
Komagata et al. | A geometric method for calibration of internal camera parameters of fish‐eye lenses |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 19950090 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2021553226 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 19950090 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 17768291 Country of ref document: US |