WO2019054636A1 - 차량용 카메라 캘리브레이션 장치 및 그 방법 - Google Patents
차량용 카메라 캘리브레이션 장치 및 그 방법 Download PDFInfo
- Publication number
- WO2019054636A1 WO2019054636A1 PCT/KR2018/009001 KR2018009001W WO2019054636A1 WO 2019054636 A1 WO2019054636 A1 WO 2019054636A1 KR 2018009001 W KR2018009001 W KR 2018009001W WO 2019054636 A1 WO2019054636 A1 WO 2019054636A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- lane
- image
- camera
- new
- correction
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 114
- 239000011295 pitch Substances 0.000 claims description 72
- 238000006243 chemical reaction Methods 0.000 claims description 47
- 230000008569 process Effects 0.000 claims description 45
- 238000009825 accumulation Methods 0.000 claims description 9
- 239000000284 extract Substances 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 5
- 239000007787 solid Substances 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000001514 detection method Methods 0.000 abstract description 5
- 238000003708 edge detection Methods 0.000 description 14
- 238000001914 filtration Methods 0.000 description 6
- 230000001131 transforming effect Effects 0.000 description 4
- 238000009499 grossing Methods 0.000 description 3
- 238000003702 image correction Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 241000282465 Canis Species 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000003628 erosive effect Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000035939 shock Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
- B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
- B60R1/27—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view providing all-round vision, e.g. using omnidirectional cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N17/00—Diagnosis, testing or measuring for television systems or their details
- H04N17/002—Diagnosis, testing or measuring for television systems or their details for television cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/57—Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
- B60R2300/105—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using multiple cameras
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
- B60R2300/303—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing using joined images, e.g. multiple camera images
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
- B60R2300/307—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing virtually distinguishing relevant parts of a scene from the background of the scene
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/40—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the details of the power supply or the coupling to vehicle components
- B60R2300/402—Image calibration
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/60—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
- B60R2300/607—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective from a bird's eye viewpoint
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
- G06T2207/30256—Lane; Road marking
Definitions
- the present invention relates to a vehicle camera calibration apparatus and a method thereof, and more particularly to a vehicle camera calibration apparatus and method for automatically calibrating an external parameter of a camera
- the present invention relates to a vehicle camera calibration apparatus and a method thereof.
- each shot image acquired from a plurality of cameras is inputted together with various information, and real-time spatial information is changed from the information of each shot image stored in the camera to a preset common coordinate system of the vehicle, It is possible to display an appropriate top view image to the driver by matching a plurality of captured images based on the information.
- a camera calibration is required to correct the mounting error of the camera as a preliminary problem.
- a specific reference pattern such as a cherker board is installed on the ground, and a method of acquiring information by using the photographed pattern image after photographing the reference pattern is used.
- the camera information acquiring method using the conventional specific reference pattern has an advantage of obtaining precise camera information since the relative position of a specific marker such as a pattern can be precisely known in advance, There is a problem in that it is troublesome to carry out the work after securing a large space enough to be installed in the portable terminal.
- a related prior art is Korean Patent Laid-Open No. 10-2008-0028531 (published on Apr. 4, 2008).
- the present invention relates to an image sensing apparatus for sensing an image of a driving lane of a vehicle from a plurality of cameras mounted on the vehicle during driving of the vehicle, detecting a lane on the lane from the sensed image, It is an object of the present invention to provide a vehicle camera calibration device and a method thereof, which can quickly and easily perform calibration of a camera in which a physical mounting error has occurred due to vehicle operation, even during traveling of a vehicle.
- a vehicle camera calibration apparatus comprising: a camera module that acquires an image of a lane photographed by a plurality of cameras mounted on a vehicle; Output module; a lane detecting module for detecting lanes from the image input from the input / output module and extracting feature points of the lane; and a lane equations and lanes using the initial camera information and external parameter information in the input image from the input / And a camera correction module for calculating the width and then correcting the image by estimating a new external parameter.
- the input / output module includes a video input / output unit for receiving an image from the camera module or outputting a corrected image to the outside, and a storage unit for receiving and storing the changed external parameter information and the corrected image from the camera correction module can do.
- the lane detecting module includes an image processor for removing noise from an image input from the input / output module, a frame accumulator for accumulating images processed by the image processor in successive frames when the lane is a dotted line, An edge detector for detecting an edge of an image of the image accumulated in the frame accumulator and extracting a feature point, and a lane determining unit for determining a lane of the feature point extracted by the edge detector.
- the camera correction module may include an image conversion unit that extracts feature points of a lane from a received image using initial camera information and external parameter information, and then converts the feature points into a top view image; A parameter estimator for estimating a new external parameter using the equation of the lane calculated by the equation calculator, and a correction unit for correcting the image using the new external parameter estimated by the parameter estimator, Section.
- a method of calibrating a camera for a vehicle comprising the steps of: detecting a lane by receiving an image obtained from a camera module; And calculating a lane width and a lane width from the top view image transformed in the image transforming step; and calculating a new lane width using a lane equation calculated in the equation calculating step, A parameter estimation step of estimating an external parameter, and an image correction step of correcting the image using the new external parameter estimated in the parameter estimation step.
- the lane detecting step may include an image processing step of removing noise from an image input from the input / output module, a step of accumulating the image in the image processed in the image processing step as a continuous frame when the lane is a dotted line, An edge detecting step of detecting an edge and extracting a minutia point from an image of the image accumulated in the frame accumulating step and a lane determining step of determining a lane at the minutiae extracted in the edge detecting step can do.
- the image processing step may be performed by performing an algorithm for smoothing an object boundary in an image to remove noise.
- noise is removed by changing the size of an image in order to prevent an edge of the divided lane from being detected as an edge in the obtained image.
- the frame accumulating step in order to make the dotted lane appear as a solid lane in the obtained image, all the pixel values are compared in the image of the accumulated image, and the largest pixel value is stored in the new image .
- the edge detection step may include detecting an edge using an algorithm, and then extracting the center coordinates of the edge and the edge as feature points.
- the lane-determining step may include detecting an outline of a longest line among the lines indicated by the feature points extracted in the edge detection step and determining the outline as a lane.
- the equation of the lane calculated in the equation calculating step may be expressed by a linear equation or a multidimensional equation.
- the parameter estimating step may include a step of correcting the pitch Rx, yaw Ry and roll Rz of the front camera of the vehicle and the correction information of the pitch Rx, yaw Ry, roll Rz,
- the correction information of the pitch Rx, the yaw Ry and the roll Rz of the front camera of the vehicle and the correction information of the pitch of the rear camera and the pitch of the rear camera are determined, (Rz), yaw (Ry) and roll (Rz), and then returns to the image conversion step.
- the method of calculating the correction information of the pitch Rx between the front camera and the rear camera in the primary correction step may be such that the new pitch Rx is corrected until the following Equation 1 is satisfied, And the process is repeated.
- G l is the slope of the left lane and G r is the slope of the right lane.
- a method for calculating correction information of a yaw (Ry) between the front camera and the rear camera in the first and second correction steps is performed by correcting a new yaw until the following equation (2) is satisfied, And then the process returns to the image conversion step and the process is repeated.
- W l is the width of the left lane and W r is the width of the right lane.
- the method of calculating the correction information of the roll (Rz) of the front camera and the rear camera in the primary correction step may be modified by correcting a new roll (New Rz) until the following equation (3) is satisfied, And the process is repeated.
- G l is the slope of the left lane and G r is the slope of the right lane.
- the method of calculating the correction information of the positional shift (Tz) of the rear camera in the secondary correction step may be such that the new positional shift (New Tz) is corrected until the following equation (4) is satisfied, And the process is repeated again.
- C f is the width between the left lane and the right lane in the front lane and C r is the width between the left lane and the right lane in the rear lane.
- the method of calculating correction information of the pitch (Rx) of the left camera in the third correction step modifies a new pitch (New Rx) until the following equation (5) is satisfied, and then returns to the image conversion step (Rx) of the right camera in the third correction step is calculated by correcting a new pitch (New Rx) until the following equation (6) is satisfied, and then the image transformation step And the process is repeated.
- x 1 is the left x coordinate of the left lane of the front lane
- x 2 is the right x coordinate of the left lane of the front lane
- x 5 is the left x coordinate of the left lane of the back lane
- x 6 is the right x coordinate
- X 3 is the left x coordinate of the right lane of the front lane
- x 4 is the right x coordinate of the right lane of the front lane
- x 7 is the left x coordinate of the right lane of the back lane and x coordinate of the right lane of the right lane of the x 8 back lane to be.
- the method for calculating the correction information of the yaw rate (Ry) of the left camera in the third correction step modifies the new yaw until the following equation (7) is satisfied, and then returns to the image conversion step (Ry) of the right camera in the third correction step is calculated by correcting the new yaw (New Ry) until the following equation (8) is satisfied, And the process is repeated.
- x 11 is the left x coordinate of the left lane of the front lane
- x 22 is the right x coordinate of the left lane of the preceding lane
- x 55 is the left x coordinate of the left lane of the back lane
- x 66 is the right x coordinate of the left lane of the back lane
- X 33 is the left x coordinate of the right lane of the front lane
- x 44 is the right x coordinate of the right lane of the preceding lane
- x 77 is the left x coordinate of the right lane of the rear lane
- x 88 is the right lane of the right lane of the back lane Coordinates.
- the method of calculating the correction information of the roll (Rz) of the left camera in the third correction step may be modified by correcting a new roll (New Rz) until the following equation (9) is satisfied, (Rz) of the right camera in the third correction step is calculated by correcting a new roll (New Rz) until the following equation (10) is satisfied, and then the image transformation step And the process is repeated.
- G l is the slope of the left lane and G r is the slope of the right lane.
- the method of calculating the correction information of the positional movement (Tz) of the left camera in the third correction step may be modified by correcting the new position movement (New Tz) until the following Equation (11) is satisfied, (Tz) of the right camera in the third correction step is calculated by correcting the new position movement (New Tz) until the following equation (12) is satisfied And the process may be returned to the image conversion step.
- x 111 is the left x coordinate of the left lane ahead drive
- x 222 is the right x coordinates of the left lane and the front drive
- x 555 is the left x coordinate is x 666 for the left lane and the right x coordinates of the left lane
- x 999 It is left x coordinate of the left lane and drive back x 10 is the right x-coordinate of the left lane, rear drive
- x 333 is the left x coordinate of the right lane drive forward
- x 444 is the right x-coordinate of the right lane drive forward
- x 777 Is the left x coordinate of the right lane
- x 888 is the right x coordinate of the right lane
- x 11 is the left x coordinate of the right lane
- x 12 is the right x coordinate of the right lane.
- an image of a driving lane of a vehicle is captured from a plurality of cameras mounted on a vehicle, a lane is detected from the captured image, and an external parameter
- the calibration of the camera can be performed quickly and easily without stopping the vehicle.
- the present invention carries out the camera calibration during running of the vehicle, there is no need to perform a secondary operation such as securing a space and installing a calibration pattern after stopping the vehicle for correction of the camera mounted on the vehicle, It is possible to reduce the time and cost required for the correction.
- FIG. 1 is a schematic view of a vehicle camera calibration apparatus according to an embodiment of the present invention.
- FIG. 2 is a flowchart illustrating a method of calibrating a camera for a vehicle according to an embodiment of the present invention.
- FIG. 3 is a flowchart showing detailed steps of the lane detection step shown in FIG. 2.
- FIG. 3 is a flowchart showing detailed steps of the lane detection step shown in FIG. 2.
- FIG. 4 is a flowchart showing the detailed steps of the parameter estimation step shown in FIG.
- Fig. 5 is a photograph showing a case where the pitch Rx of the front and rear cameras is different from the image acquired from the camera module shown in Fig. 1.
- Fig. 5 is a photograph showing a case where the pitch Rx of the front and rear cameras is different from the image acquired from the camera module shown in Fig. 1.
- Fig. 6 is a photograph showing a case where the yaw (Ry) of the front and rear cameras is different from the image acquired from the camera module shown in Fig.
- FIG. 7 is a photograph showing a case where the rolls Rz of the front and rear cameras are broken in the image acquired from the camera module shown in Fig.
- FIG. 8 is a photograph showing a case where the positional movement (Tz) of the rear camera is incorrect in the image acquired from the camera module shown in Fig.
- FIG. 9 is a photograph showing a case where the pitch Rx of the left and right cameras is different from the image acquired from the camera module shown in FIG.
- FIG. 10 is a photograph showing a case where the yaw rate Ry of the left and right cameras is incorrect in the image acquired from the camera module shown in Fig.
- FIG. 11 is a photograph showing a case where the rolls Rz of the left and right cameras are broken in the image acquired from the camera module shown in Fig.
- FIG. 12 is a photograph showing a case where the positional shift (Tz) of the left and right cameras is incorrect in the image acquired from the camera module shown in FIG.
- FIG. 1 is a schematic diagram of a vehicle camera calibration apparatus according to an embodiment of the present invention.
- the camera calibration apparatus for a vehicle includes a camera module 100 for acquiring an image of a lane taken from a plurality of cameras mounted on the vehicle, Output module 200 for detecting a lane from an image input from the input / output module 200 and extracting feature points of a lane, a lane detecting module A camera correction module 300 for calculating a lane equation and a lane width using initial camera information and external parameter information in the image input from the input / output module 200, 400).
- the camera module 100 is a device having a plurality of cameras mounted at arbitrary positions of the vehicle.
- the camera module 100 performs a function of acquiring an image of a lane photographed from the plurality of cameras.
- And four cameras are mounted on the front, rear, left, and right sides of the vehicle. By doing so, the surroundings of the vehicle can be visualized by eliminating the square at the time of photographing the peripheral area of the vehicle.
- this is merely an embodiment, and the number of cameras mounted on the vehicle can be increased or decreased.
- the camera of the camera module 100 may use a wide-angle lens having a larger viewing angle than a normal lens or a fisheye lens having an extra-wide angle of view exceeding 180 degrees. By doing so, it is preferable that all of the vehicle surroundings are photographed when each camera image is matched in order to generate a top view image in the future.
- the camera of the camera module 100 It goes without saying that the present invention can be applied regardless of the angle of view.
- the attitude information of each camera (Ry), a roll (Rz), and an external parameter indicating the degree of movement of the back camera and the left and right cameras at a specific position.
- a straight lane in a straight lane will always appear vertically when viewed from the top view image, and all four lanes visible on the four straight lanes will be present. At this time, if the camera calibration is correct, it will appear as one lane and if the camera calibration is incorrect, it will be seen as several lanes. By using these characteristics, it is possible to estimate several external parameters.
- the camera module 100 and the input / output module 200 may be connected by wire or wirelessly to transmit images photographed by the camera.
- the input / output module 200 is a module for receiving the image obtained from the camera module 100 or outputting the corrected image transmitted from the camera correction module 400 to be described later to an output device capable of recognizing by the user, Unit 210 and a storage unit 220.
- the video input and output unit 210 receives an image photographed by the camera of the camera module 100 and transmits the image to the image processing unit 310 of the lane detection module 300 or the correction unit 440 of the camera correction module 400 When the corrected image is stored in the storage unit 220 to be described later, the image is received and output to an external device such as a monitor that can be recognized by the user. At this time, the video input / output unit 210 may perform an image preprocessing process by an image filter if necessary.
- the storage unit 220 stores the corrected image in the correction unit 440 of the camera correction module 400 as well as the new camera estimated by the parameter estimation unit 430 of the camera correction module 400, Information about external parameters can also be stored.
- the storage unit 220 stores information on internal parameters such as a focal length, a principal point, a distortion coefficient, and the like corresponding to the intrinsic characteristics of a plurality of camera lenses of the camera module 100,
- the first external parameter information of the camera such as the initial attitude information and the position information of a plurality of cameras of the camera.
- the first camera information and the first external parameter information stored in the storage unit 220 are provided to the camera correction module 400 to be described later and are used to estimate new external parameters. May be provided for processing in various devices such as a top view system, a panoramic view system, and the like.
- the lane detecting module 300 can detect a lane from an image input from the input / output module 200 and extract feature points of the lane.
- the lane detecting module 300 includes various algorithms for performing the functions, An image processor 310, a frame accumulator 320, an edge detector 330, and a lane determiner 340.
- the image processing unit 310 includes an image processing algorithm for removing noise from the image input from the image input / output unit 210 of the input / output module 200 and performing image processing for more clearly showing the boundary of the road lane .
- the algorithm that the image processing unit 310 described below uses to remove noise from the image and to perform the image processing is not limited to the algorithm described below as an embodiment of the present invention, It is natural that it can be replaced with any number of algorithms.
- the image processing unit 310 may convert the color image input from the image input / output unit 210 to a monochrome image for noise removal and image processing in the embodiment of the present invention. That is, the image processing unit 310 may use the input color image as it is for the above noise removal and image processing, or may convert the input color image into a monochrome image.
- the image processing unit 310 may apply an algorithm to remove noise from the image as described above. For example, in an embodiment of the present invention, a morphology operation and median filtering may be performed.
- the morphological operation performed in the image processing unit 310 is an erosion operation and an expansion operation as a type of image filter used in a preprocessing process such as noise removal and feature point extraction prior to image separation, that is, full-scale image processing.
- a morphology opening is performed after the morphology closing.
- the closing of the image performed by the image processing unit 310 softens the boundaries of the objects shown in the image, thus smoothing the boundaries and eliminating discontinuous data in the image.
- the size of the object can be preserved while noise is removed by the morphology opening performed after the closing of the image processing unit 310.
- the image processing unit 310 may perform median filtering in addition to sequentially applying a morphology opening after the closing of the morphology in an embodiment of the present invention.
- the image processing unit 310 can remove noise by changing the size of the image after the median filtering is performed.
- the edge detecting part 330 detects the split part as an edge
- the lane determining part 340 The lane recognition rate is lowered so as to prevent the lane recognition rate from dropping by eliminating the noise of the cracked portion of the lane.
- the frame accumulation unit 320 accumulates the images processed by the image processing unit 310, that is, when the lane indicated by the noise-removed image is a dotted line, as consecutive frames, In the case of a straight line, the image processed by the image processing unit 310 passes through the frame accumulation unit 320 and is transmitted to the next edge detection unit 330.
- the frame accumulation unit 320 exemplifies accumulation of four frames in one embodiment of the present invention.
- the number of accumulated frames is only an example, and it can be changed as much as necessary.
- the reason why the frame accumulator 320 accumulates the image frames as described above is to make the dotted lanes shown in the image look like the real lane. This is because it is possible to more easily and accurately determine and detect the lane in the real lane when the lane determining unit 340 determines the lane.
- the accumulation of frames in the frame accumulation unit 320 may be performed by comparing all the pixel values in the images of four consecutive images with no noise removed, that is, four accumulated images, and storing the largest pixel value in the new image Make the dotted lane in the image look like the solid lane.
- the edge detector 330 detects an edge at a lower end portion of an image of the image accumulated in the frame accumulator 320 and extracts a feature point.
- the edge detection unit 330 may define a region excluding the regions except for the regions to be a region of interest (ROI) in order to reduce the amount of computation since the lane in the image of the accumulated image exists only in the lower half portion and the upper portion is sky.
- ROI region of interest
- the edge detection unit 330 detects a boundary between the road color and the lane color so that the edge is strongly displayed through the image processing. Then, the edge detection unit 330 detects the edge of the canine edge The edge is detected using an algorithm.
- the reason why the edge detection unit 330 uses the CannyEdge algorithm is that the error rate and the position of the edge points are accurate.
- the CannyEdge algorithm is exemplified as the edge detection algorithm used in the edge detection unit 330, it is needless to say that other algorithms can be used as many as necessary.
- the edge detection unit 330 then extracts the center coordinates of the edge and the edge as feature points because the edge is not only the lane of the image but also the noise of the vehicle of the image. Therefore, in order to reduce such a wide edge, Coordinates.
- the edge detection unit 330 extracts the center coordinates of the edge and the edge as feature points, the lane of the image is represented by one continuous long line, and the noise is not displayed or displayed short.
- the lane determining unit 340 determines lanes at the feature points extracted by the edge detecting unit 330.
- the lane determining unit 340 may determine the lane of the lane by the lane departing from the edge detecting unit 330. For example, It is possible to detect the outline of the longest line among the appearing lines and judge it as a lane.
- the lane determining unit 340 finds two contours in the images acquired from the front and rear cameras of the camera module 100 and finds one contour in the images acquired from the left and right cameras. This is because the front and rear cameras have two lanes on the left and right sides and one lane on the left and right cameras.
- the lane determining unit 340 determines the outline thus determined as a lane.
- the camera correction module 400 performs a function of correcting the external parameters of the four cameras mounted on the front, rear, left and right sides of the vehicle using the lane detected by the lane detecting module 300, An equation calculating unit 420, a parameter estimating unit 430, and a correcting unit 440.
- the camera correction module 400 calculates the lane equations and lane widths using the initial camera information and the external parameter information in the image received from the input / output module 200, estimates new external parameters using the lane equations and lane widths, and corrects the images .
- the lane on the top view image will be on the same straight line and appear vertically when converted to the top view image.
- the external parameters are not accurate, so that the lane on the top view image does not exist in a straight line. Therefore, by using this point, it is possible to estimate the external parameter of the changed camera.
- the image converting unit 410 extracts the feature points of the lane on the input image using the initial camera information and the external parameter information, and converts the feature points into a top view image.
- the image converting unit 410 is based on a top view image that is vertically displayed when converting a received image into a top view image.
- any other top view image can be used.
- the top view image in which the vehicle is viewed vertically, the top view image may also be constructed so that the vehicle is viewed horizontally. Note that the formula for the slope, It is defined differently.
- a transformed top view image is constructed so that the vehicle is viewed horizontally, the slope becomes perpendicular to the slope of the transformed top view image so that the vehicle is viewed vertically, and the formula for the corresponding slope is also different.
- the top view image described below is based on the conversion of the vehicle so as to be viewed vertically, and the slope and the formula are also based on the converted image.
- this is merely an embodiment of the present invention, and it is of course also possible that the vehicle may be based on a transformed top view image.
- the equation calculating unit 420 calculates the equation of the lane by fitting the minutiae by line fitting in the transformed top view image.
- a least mean square (LMS) algorithm may be used for line fitting.
- LMS least mean square
- the above lane equation can be a linear equation, and it can be a non-linear multidimensional equation.
- the parameter estimating unit 430 estimates a new external parameter using the equation of the lane calculated by the equation calculating unit 420.
- the correction unit 440 corrects the image using the new external parameter estimated by the parameter estimation unit 430 and transmits the corrected image to the storage unit 220.
- FIG. 2 is a flowchart illustrating a method of calibrating a vehicle camera according to an embodiment of the present invention.
- the method of calibrating a camera for a vehicle includes a lane detecting step S510, an image converting step S520, an equation calculating step S530, Step S540 and image correction step S550.
- the lane detecting module 300 detects the lane, And may include an image processing step S511, a frame accumulating step S512, an edge detecting step S513, and a lane determining step S514.
- the image processing step S511 is a step in which the lane detecting module 300 removes noise from the image received through the image processing unit 310 and performs image processing for clearly showing the boundary portion of the lane of the road.
- the image processing unit 310 may convert the color image input from the image input / output unit 210 to a monochrome image for noise removal and image processing in the embodiment of the present invention. That is, the image processing unit 310 may use the input color image as it is for the above noise removal and image processing, or may convert the input color image into a monochrome image.
- an algorithm for smoothing an object boundary in an image may be performed in order to remove noise.
- a morphology operation is performed. You can also perform median filtering to remove additional noise.
- the image processing step S511 may remove the noise by changing the size of the image subjected to the median filtering in order to prevent the edge of the divided lane from being detected as an edge in the obtained image.
- the present invention is not limited to the above-mentioned morphology operation algorithm and median filtering algorithm performed in the image processing step S511, and another algorithm that performs the corresponding function may be used.
- the lane detecting module 300 accumulates the noise-removed images as a continuous frame through the frame accumulating unit 320 And if it is a straight line, the next step is carried out immediately.
- the edge detection step S513 is a step in which the lane detection module 300 detects an edge at the lower end portion of the image of the image accumulated through the edge detection unit 330 and performs extraction of the feature point.
- the center coordinates of the edge and the edge may be extracted as feature points.
- a Canny edge algorithm may be used.
- the lane determining step S514 is a step in which the lane detecting module 300 determines a lane at a feature point extracted by the edge detecting unit 330 through the lane determining unit 340.
- the lane determining unit 340 may determine the lane in the minutiae point as described above. In an embodiment of the present invention, S514, the lane determining unit 340 detects an outline of the longest line among the lines indicated by the feature points extracted from the edge detecting unit 330 and judges it as a lane.
- the image conversion step S520 is a step of extracting feature points of the lane from the input image using the initial camera information and the initial external parameter information stored in the image conversion unit 410 of the camera correction module 400 and converting the feature points into a top view image .
- the lane on the top view image is present on the same straight line and appears vertically when converted to the top view image.
- the camera of the camera module 100 whose position is changed due to physical force
- the lane on the image does not exist in a straight line.
- the image conversion step (S520) converts the inputted image into a top view image so as to estimate the changed external parameters of the camera in the subsequent step.
- the equation calculating step S530 is a step of calculating a lane equation and a lane width from the transformed top view image in the image transforming step 520.
- the equation and the lane width are used to estimate a new external parameter at a later step .
- the equation calculating unit 420 of the camera correcting module 400 line-fitting feature points in the top view image converted by the image transforming unit 410 to calculate a lane equation
- a least mean square (LMS) algorithm may be used for line fitting.
- LMS least mean square
- the equation of the lane calculated in the equation calculating step S530 may be a linear equation in the embodiment of the present invention and may be a non-linear multidimensional equation.
- the calculated equation when the equation of the lane is a linear equation Can be expressed by the following equation (1).
- the x and y values in the equation (1) are determined values obtained from the image, and obtaining a and b from the values is a goal of the equation calculating step, , And b denotes y-intercept of the lane.
- the x and y values which are coordinate points of the previously obtained lanes, we can obtain the lane equations a and b using line fitting.
- the parameter estimating step S540 is a step of estimating a new external parameter using the lane equation and the lane width calculated in the equation calculating step S541, S542), a third correction step S543, and a storing step S544.
- step S510 image conversion step S520, and equation calculation step S530 are common.
- the lanes of the front camera, the rear camera, the left camera, and the right camera are detected in the primary correction step S541, the secondary correction step S542, and the third correction step S543,
- the process of transforming and finding the lane equations and lane widths can be performed separately at each stage.
- the lane detecting step S510, the image converting step S520, and the equation calculating step S530 are represented as one, and in practice, the first correcting step S541, the second correcting step S530, The image conversion step S520 and the equation calculation step S530 are separately performed for each camera in the third correction step S543 and the third correction step S543.
- the primary correction step S541 it is determined whether the correction information of the pitch Rx, yaw Ry and roll Rz of the front camera of the vehicle is correct (S541a1), and the pitch Rx, yaw Ry of the rear camera (Ry) and roll (Rx) of the front camera of the vehicle if the correction information of the roll (Rz) is correct (S541b1) Rz are computed in step S541a2 and the correction information for the pitch Rx, yaw Ry and roll Rz of the rear camera is calculated S541b2. Then, the process returns to the image conversion step S520, And the lane equations and lane widths are obtained, and then the primary correction step S541 is performed again.
- a step S541a1 of judging whether the correction information of the pitch Rx of the front camera of the vehicle is correct or a step S541b1 of judging whether the correction information of the pitch Rx of the rear camera is correct in the primary correcting step S541 is as follows .
- FIG. 5 shows a case where the pitch Rx of the front camera and the pitch Rx of the rear camera are different from each other.
- the pitch Rx of the front and rear cameras is smaller than the ground truth value
- the right image in FIG. 5 is a case where the pitch Rx of the front and rear cameras is larger than the ground truth value which is the actual reference value.
- step S541a2 of calculating the correction information of the pitch Rx of the front camera of the vehicle in the primary correction step S541 and the step S541b2 of calculating the correction information of the pitch Rx of the rear camera are steps Is obtained by estimating the pitch Rx of the new front and rear cameras as shown in Equation (2).
- New Rx pitch of new front and rear camera
- an equation calculation step S530 is performed to generate a top view image again using the pitches of the new front and rear cameras (New Rx) and calculate a lane equation in the top view image.
- a step S541a1 of judging whether the correction information of the pitch Rx of the camera is correct or a step S541b1 of judging whether the correction information of the pitch Rx of the rear camera is correct is repeated.
- FIG. 6 shows a case where the yaw rate Ry of the front and rear cameras of the vehicle is different.
- the yaw rate Ry of the front and rear cameras is larger than the ground truth value
- (Ry) of the front and rear cameras is smaller than the ground truth value which is the actual reference value.
- the widths of the left and right lanes in the lanes shown above and below the vehicle are different from each other.
- New Ry New front and rear camera.
- the step S541a1 for determining whether the correction information of the roll Rz of the front camera of the vehicle is correct in the primary correction step S541 is as follows.
- step S542 the position of the rear camera of the vehicle is moved (step S542) (S542a2) and then returns to the image conversion step (S520).
- the step S542a1 of determining whether the correction information of the rear camera of the vehicle is correct in the secondary correction step S542 is as follows.
- the method S542a2 for calculating the correction information of the positional movement Tz of the rear camera in the secondary correction step S542 is performed after the new position movement New Tz is corrected until the following equations 8 and 9 are satisfied And then returning to the image conversion step and repeating the process.
- New Tz Movement of new front and rear camera
- the third correction step S543 determines whether correction information of the pitches Rx, Ry, Rz, and Tz of the left and right cameras of the vehicle is correct (S543a1, 543b1) If the information is correct, the process proceeds to the next step. If not, the correction information of the pitch Rx, yaw, roll Rz and position shift Tz of the left and right cameras of the vehicle is calculated S543a2 and S543b2, The process returns to the conversion step S520.
- Steps S543a1 and S543b1 for determining whether the correction information of the pitch Rx of the right and left cameras is correct in the third correction step S543 are as follows.
- the pitch Rx of the left and right cameras of the vehicle is different.
- the pitch Rx of the left and right cameras is larger than the ground truth value which is the actual reference value.
- the pitch Rx of the left and right cameras is smaller than the ground truth value which is the actual reference value.
- the method of calculating the correction information of the pitch Rx of the left camera in the third correction step S543 (S543a2) corrects the new pitch New Rx until the following equations 10 and 11 are satisfied, And then returning to the conversion step S520 and repeating the process.
- New Rx pitch of new left camera
- Pre Rx The pitch of the left camera before it is repeated
- x 2 x coordinate of the right lane of the left lane ahead
- the method (S543b2) for calculating the correction information of the pitch Rx of the right camera in the third correction step S543 is a step of correcting the new pitch New Rx until the following equations (12) and (13) are satisfied, And then returning to the image conversion step and repeating the process.
- New Rx pitch of new right camera
- Pre Rx pitch of the right camera before repeating
- Steps S543a1 and S543b1 for determining whether the correction information of the yaw Ry of the left and right cameras are correct in the third correction step S543 are as follows.
- FIG. 10 shows a case where the yaw rate Ry of the left and right cameras of the vehicle is wrong.
- the yaw rate Ry of the left and right cameras is larger than the true ground truth value.
- (Ry) of the left and right cameras is smaller than the ground truth value which is the actual reference value.
- (x 22 -x 11 ) and (x 66 -x 55 ) are the same or (x 44 -x 33 ) and (x 88 -x 77 ) .
- the method (S543a2) of calculating the correction information of the yaw (Ry) of the left camera in the third correction step (S543) corrects the new yaw (New Ry) until the following equations (14) and And then returning to the conversion step S520 and repeating the process.
- New Ry New left camera.
- Pre Ry The left camera before repeating.
- the method (S543b2) for calculating the correction information of the yaw of the right camera in the third correction step S543 corrects the new yaw until the following equations (16) and (17) are satisfied, And then returning to the image conversion step and repeating the process.
- New Ry The new right camera.
- x 88 x coordinate of the right lane of the rear lane
- Steps S543a1 and S543b1 for determining whether the correction information of the roll Rz of the right and left cameras are correct in the third correction step S543 are as follows.
- FIG. 11 shows a case where the roll Rz of the left and right cameras of the vehicle is broken.
- the roll Rz of the left and right cameras is larger than the ground truth value as an actual reference value, And the roll Rz of the left and right cameras is smaller than the true ground truth value.
- 1 / G l and 1 / G r have a large value and 1 / G l and 1 / G r are close to 0 when the roll Rz is correct do.
- the method of calculating the correction information of the roll Rz of the left camera in the third correction step S543 (S543a2) corrects the new roll New Rz until the following equations (18) and (19) are satisfied, And then returning to the conversion step and repeating the process.
- the method (S543b2) of calculating the correction information of the roll Rz of the right camera in the third correction step S543 is a step of correcting the new roll New Rz until the following equations (20) and (21) are satisfied, And then returning to the image conversion step and repeating the process.
- Steps S543a1 and S543b1 for determining whether the correction information of the positional shift Tz of the right and left cameras are correct in the third correction step S543 are as follows.
- FIG. 12 shows a case where the positional shifts Tz of the left and right cameras of the vehicle are different.
- the positional shifts Tz of the left and right cameras are larger than the ground truth value
- the image is a case where the positional shift (Tz) of the left and right cameras is smaller than the ground truth value which is the actual reference value.
- the widths of the front and rear lanes of the vehicle and the widths of the left and right lanes of the vehicle are the same.
- the method of calculating the correction information of the positional movement Tz of the left camera in the third correction step S543 (S543a2) corrects the new position movement (New Tz) until the following equations (22) and And then returning to the image conversion step and repeating the process.
- New Tz Move the position of the new left camera
- x 222 x coordinate of the right lane of the left lane ahead
- the method (S543b2) for calculating the correction information of the positional movement (Tz) of the right camera in the third correction step (S543) corrects the new position movement (New Tz) until the following equations And then returning to the image conversion step and repeating the process.
- New Tz Move the position of the new right camera
- the storing step S544 is a step of repeatedly performing the steps until the correction is correctly performed in the first to third steps S541, S542, and S543, and then storing the finally calculated external parameters.
- the image correction step S550 is a step of correcting the image using the new external parameter estimated in the parameter estimation step S540, and the corrected image is transmitted to the storage unit 220 of the input / output module 200.
- the vehicle camera calibration apparatus and method of the present invention can automatically calibrate external parameters even when the vehicle is running.
- an image of a driving lane of a vehicle is captured from a plurality of cameras mounted on a vehicle, a lane is detected from the captured image, and an external parameter
- the calibration of the camera can be performed quickly and easily without stopping the vehicle.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Signal Processing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Mechanical Engineering (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
- Closed-Circuit Television Systems (AREA)
- Traffic Control Systems (AREA)
Abstract
Description
Claims (21)
- 차량에 장착된 다수개의 카메라로부터 차로를 촬영한 영상을 획득하는 카메라 모듈;상기 카메라 모듈로부터 획득된 영상을 입력받거나 보정된 영상을 출력하는입출력 모듈;상기 입출력 모듈에서 입력 받은 영상으로부터 차선을 검출하고 차선의 특징점을 추출하는 차선검출모듈; 및상기 입출력 모듈에서 입력 받은 영상에서 초기 카메라 정보와 외부 파라미터 정보를 이용하여 차선의 방정식과 차선폭을 계산한 후 새로운 외부 파라미터를 추정하여 영상을 보정하는 카메라보정 모듈;을 포함하는 것을 특징으로 하는 차량용 카메라 캘리브레이션 장치.
- 청구항 1에 있어서,상기 입출력 모듈은,상기 카메라 모듈로부터 영상을 입력 받거나 보정된 영상을 외부로 출력하는 영상입출력부;와,상기 카메라보정 모듈로부터 변경된 외부 파라미터 정보와 보정된 영상을 전달 받아 저장하는 저장부;를 포함하는 것을 특징으로 하는 차량용 카메라 캘리브레이션 장치.
- 청구항 1에 있어서,상기 차선검출모듈은,상기 입출력 모듈로부터 입력 받은 영상에서 노이즈를 제거하는 영상처리부;차선이 점선일 경우에 상기 영상처리부에서 처리된 영상을 연속된 프레임으로 누적하는 프레임누적부;상기 프레임누적부에서 누적된 영상의 이미지에서 엣지를 검출하고 특징점을 추출하는 엣지검출부; 및상기 엣지검출부에서 추출된 특징점에서 차선을 결정하는 차선판단부;를 포함하는 것을 특징으로 하는 차량용 카메라 캘리브레이션 장치.
- 청구항 1에 있어서,상기 카메라보정 모듈은,초기 카메라 정보와 외부 파라미터 정보를 이용하여 입력받은 영상에서 차선의 특징점을 추출한 후 탑뷰 이미지로 변환하는 이미지변환부;상기 이미지변환부에서 변환된 탑뷰 이미지에서 차선의 방정식을 구하는 방정식산출부;상기 방정식산출부에서 산출된 차선의 방정식을 이용하여 새로운 외부 파라미터를 추정하는 파라미터추정부; 및상기 파라미터추정부에서 추정된 새로운 외부 파라미터를 이용하여 영상을 보정하는 보정부;를 포함하는 것을 특징으로 하는 차량용 카메라 캘리브레이션 장치.
- 카메라 모듈로부터 획득된 영상을 입력 받아 차선을 검출하는 차선검출단계;초기 카메라 정보와 외부 파라미터 정보를 이용하여 입력받은 영상에서 차선의 특징점을 추출한 후 탑뷰 이미지로 변환하는 이미지변환 단계;상기 이미지변환 단계에서 변환된 탑뷰 이미지에서 차선의 방정식과 차선폭을 구하는 방정식산출 단계;상기 방정식산출 단계에서 산출된 차선의 방정식과 차선폭을 이용하여 새로운 외부 파라미터를 추정하는 파라미터추정 단계; 및상기 파라미터추정 단계에서 추정된 새로운 외부 파라미터를 이용하여 영상을 보정하는 영상보정 단계;를 포함하는 것을 특징으로 하는 차량용 카메라 캘리브레이션 방법.
- 청구항 5에 있어서,상기 차선검출 단계는,입출력 모듈로부터 입력 받은 영상에서 노이즈를 제거하는 영상처리 단계;영상처리 단계에서 처리된 영상에서 차선이 점선일 때는 연속된 프레임으로 누적하고 직선일 때는 다음 단계를 수행하는 프레임누적 단계;상기 프레임누적 단계부에서 누적된 영상의 이미지에서 엣지를 검출하고 특징점을 추출하는 엣지검출 단계; 및상기 엣지검출 단계에서 추출된 특징점에서 차선을 결정하는 차선판단 단계;를 포함하는 것을 특징으로 하는 차량용 카메라 캘리브레이션 방법.
- 청구항 6에 있어서,상기 영상처리 단계는 노이즈를 제거하기 위해 영상에서의 물체 경계를 부드럽게 하는 알고리즘을 수행하는 것을 특징으로 하는 차량용 카메라 캘리브레이션 방법.
- 청구항 6에 있어서,상기 영상처리 단계는 획득된 영상에서 차선이 갈라진 영역이 엣지로 검출되는 것을 방지하기 위하여 영상의 크기를 변경하여 노이즈를 제거하는 것을 특징으로 하는 차량용 카메라 캘리브레이션 방법.
- 청구항 6에 있어서,상기 프레임누적 단계는 획득된 영상에서 점선 차선을 실선 차선으로 보이도록 하기 위하여 누적된 영상의 이미지에서 모든 픽셀 값을 비교하여 가장 큰 픽셀값을 새로운 이미지에 저장하는 것을 특징으로 하는 차량용 카메라 캘리브레이션 방법.
- 청구항 6에 있어서,상기 엣지검출 단계는 알고리즘을 이용하여 엣지를 검출한 후 상기 엣지와 엣지의 중앙 좌표를 특징점으로 추출하는 것을 특징으로 하는 차량용 카메라 캘리브레이션 방법.
- 청구항 6에 있어서,상기 차선판단 단계는 상기 엣지검출 단계에서 추출된 특징점에 의해 나타나는 선중에서 가장 길이가 긴 윤곽선을 검출하여 차선으로 판단하는 것을 특징으로 하는 차량용 카메라 캘리브레이션 방법.
- 청구항 5에 있어서,상기 방정식산출 단계에서 산출된 차선의 방정식은 직선의 방정식 또는 다차원 방정식으로 표현되는 것을 특징으로 하는 차량용 카메라 캘리브레이션 방법.
- 청구항 5에 있어서,상기 파라미터추정 단계는,차량의 전방 카메라의 피치(Rx), 요(Ry), 롤(Rz)의 보정 정보와, 후방 카메라의 피치(Rx), 요(Ry), 롤(Rz)의 보정 정보가 정확한지 판단한 후, 각각의 보정 정보가 정확하면 다음 단계로 진행하고 정확하지 않으면 차량의 전방 카메라의 피치(Rx), 요(Ry), 롤(Rz)의 보정 정보와 후방 카메라의 피치(Rx), 요(Ry), 롤(Rz)의 보정 정보를 계산한 후 이미지변환 단계로 되돌아가는 1차 보정 단계;차량의 후방 카메라의 위치이동(Tz)의 보정 정보가 정확한지 판단한 후, 상기 보정 정보가 정확하면 다음 단계로 진행하고 정확하지 않으면 차량의 후방 카메라의 위치이동(Tz)의 보정 정보를 계산한 후 이미지변환 단계로 되돌아가는 2차 보정 단계;차량의 좌우측 카메라의 피치(Rx), 요(Ry), 롤(Rz), 위치이동(Tz)의 보정 정보가 정확한지 판단한 후, 각각의 보정 정보가 정확하면 다음 단계로 진행하고 정확하지 않으면 차량의 좌우측 카메라의 피치(Rx), 요(Ry), 롤(Rz), 위치이동(Tz)의 보정 정보를 계산한 후 이미지변환 단계로 되돌아가는 3차 보정 단계; 및상기 1차 내지 3차 단계에서 계산된 외부 파라미터를 저장하는 저장 단계;를 포함하는 것을 특징으로 하는 차량용 카메라 캘리브레이션 방법.
- 청구항 13에 있어서,상기 3차 보정 단계의 좌측카메라의 피치(Rx)의 보정 정보를 계산하는 방법은 하기 수학식 5가 만족될 때까지 새로운 피치(New Rx)를 수정한 후 상기 이미지 변환 단계로 다시 되돌아가 과정을 반복하고,상기 3차 보정 단계의 우측카메라의 피치(Rx)의 보정 정보를 계산하는 방법은 하기 수학식 6이 만족될 때까지 새로운 피치(New Rx)를 수정한 후 상기 이미지 변환 단계로 다시 되돌아가 과정을 반복하는 것을 특징으로 하는 차량용 카메라 캘리브레이션 방법.여기서, x 1은 전방 차로 왼쪽 차선의 왼쪽 x좌표이고 x 2는 전방 차로 왼쪽 차선의 오른쪽 x좌표이고, x 5는 후방 차로 왼쪽 차선의 왼쪽 x좌표이고 x 6는 후방 차로 왼쪽 차선의 오른쪽 x좌표이고, x 3는 전방 차로 오른쪽 차선의 왼쪽 x좌표이고 x 4는 전방 차로 오른쪽 차선의 오른쪽 x좌표이고, x 7는 후방 차로 오른쪽 차선의 왼쪽 x좌표이고 x 8 후방 차로의 오른쪽 차선의 오른쪽 x좌표이다.
- 청구항 13에 있어서,상기 3차 보정 단계의 좌측카메라의 요(Ry)의 보정 정보를 계산하는 방법은 하기 수학식 7이 만족될 때까지 새로운 요(New Ry)를 수정한 후 상기 이미지변환 단계로 다시 되돌아가 과정을 반복하고,상기 3차 보정 단계의 우측카메라의 요(Ry)의 보정 정보를 계산하는 방법은 하기 수학식 8이 만족될 때까지 새로운 요(New Ry)를 수정한 후 상기 이미지변환 단계로 다시 되돌아가 과정을 반복하는 것을 특징으로 하는 차량용 카메라 캘리브레이션 방법.여기서, x 11은 전방 차로 왼쪽 차선의 왼쪽 x좌표이고 x 22는 전방 차로 왼쪽 차선의 오른쪽 x좌표이고, x 55는 후방 차로 왼쪽 차선의 왼쪽 x좌표이고 x 66은 후방 차로 왼쪽 차선의 오른쪽 x좌표이고, x 35은 전방 차로 오른쪽 차선의 왼쪽 x좌표이고 x 44는 전방 차로 오른쪽 차선의 오른쪽 x좌표이고, x 77은 후방 차로 오른쪽 차선의 왼쪽 x좌표이고 x 88은 후방 차로의 오른쪽 차선의 오른쪽 x좌표이다.
- 청구항 13에 있어서,상기 3차 보정 단계의 좌측카메라의 롤(Rz)의 보정 정보를 계산하는 방법은 하기 수학식 9가 만족될 때까지 새로운 롤(New Rz)을 수정한 후 상기 이미지변환 단계로 다시 되돌아가 과정을 반복하고,상기 3차 보정 단계의 우측카메라의 롤(Rz)의 보정 정보를 계산하는 방법은 하기 수학식 10이 만족될 때까지 새로운 롤(New Rz)을 수정한 후 상기 이미지변환 단계로 다시 되돌아가 과정을 반복하는 것을 특징으로 하는 차량용 카메라 캘리브레이션 방법.여기서, G l는 왼쪽 차선의 기울기이며 G r은 오른쪽 차선의 기울기이다.
- 청구항 13에 있어서,상기 3차 보정 단계의 좌측카메라의 위치이동(Tz)의 보정 정보를 계산하는 방법은 하기 수학식 11이 만족될 때까지 새로운 위치이동(New Tz)을 수정한 후 상기 이미지변환 단계로 다시 되돌아가 과정을 반복하고,상기 3차 보정 단계의 우측카메라의 위치이동(Tz)의 보정 정보를 계산하는 방법은 하기 수학식 12가 만족될 때까지 새로운 위치이동(New Tz)을 수정한 후 상기 이미지변환 단계로 다시 되돌아가 과정을 반복하는 것을 특징으로 하는 차량용 카메라 캘리브레이션 방법.여기서, x 111은 전방 차로 왼쪽 차선의 왼쪽 x좌표이고 x 222는 전방 차로 왼쪽 차선의 오른쪽 x좌표이고, x 555는 왼쪽 차선의 왼쪽 x좌표이고 x 666는 왼쪽 차선의 오른쪽 x좌표이고, x 999는 후방 차로 왼쪽 차선의 왼쪽 x좌표이고 x 10는 후방 차로 왼쪽 차선의 오른쪽 x좌표이고, x 333은 전방 차로 오른쪽 차선의 왼쪽 x좌표이고 x 444는 전방 차로 오른쪽 차선의 오른쪽 x좌표이고, x 777은 오른쪽 차선의 왼쪽 x좌표이고 x 888은 오른쪽 차선의 오른쪽 x좌표이고, x 11은 후방 차로 오른쪽 차선의 왼쪽 x좌표이고 x 12는 후방 차로 오른쪽 차선의 오른쪽 x좌표이다.
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE112018000107.5T DE112018000107T5 (de) | 2017-09-13 | 2018-08-08 | Fahrzeugkamerakalibrierungsvorrichtung und -Verfahren |
CN201880002468.0A CN109791700A (zh) | 2017-09-13 | 2018-08-08 | 车辆相机校准装置及其方法 |
US16/315,941 US11398051B2 (en) | 2017-09-13 | 2018-08-08 | Vehicle camera calibration apparatus and method |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR10-2017-0116914 | 2017-09-13 | ||
KR1020170116914A KR101969030B1 (ko) | 2017-09-13 | 2017-09-13 | 차량용 카메라 캘리브레이션 장치 및 그 방법 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2019054636A1 true WO2019054636A1 (ko) | 2019-03-21 |
Family
ID=65722960
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/KR2018/009001 WO2019054636A1 (ko) | 2017-09-13 | 2018-08-08 | 차량용 카메라 캘리브레이션 장치 및 그 방법 |
Country Status (5)
Country | Link |
---|---|
US (1) | US11398051B2 (ko) |
KR (1) | KR101969030B1 (ko) |
CN (1) | CN109791700A (ko) |
DE (1) | DE112018000107T5 (ko) |
WO (1) | WO2019054636A1 (ko) |
Families Citing this family (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11699207B2 (en) * | 2018-08-20 | 2023-07-11 | Waymo Llc | Camera assessment techniques for autonomous vehicles |
DE102019003238B4 (de) * | 2019-05-08 | 2023-04-20 | Mercedes-Benz Group AG | Fahrzeugortung durch Kartenabgleich unter Berücksichtigung eines Straßenprofils |
KR102274026B1 (ko) * | 2019-05-22 | 2021-07-09 | (주)베이다스 | 차량용 카메라 캘리브레이션 장치 및 그 방법 |
KR102254950B1 (ko) | 2019-06-26 | 2021-05-24 | 주식회사 티맥스티베로 | 캘리브레이션 방법 |
CN110796102B (zh) * | 2019-10-31 | 2023-04-14 | 重庆长安汽车股份有限公司 | 一种车辆目标感知***及方法 |
CN111523471B (zh) * | 2020-04-23 | 2023-08-04 | 阿波罗智联(北京)科技有限公司 | 车辆所在车道的确定方法、装置、设备以及存储介质 |
KR20220026422A (ko) * | 2020-08-25 | 2022-03-04 | 삼성전자주식회사 | 카메라 캘리브레이션 장치 및 이의 동작 방법 |
US11908164B2 (en) * | 2021-07-23 | 2024-02-20 | Embark Trucks Inc. | Automatic extrinsic calibration using sensed data as a target |
CN114387448A (zh) * | 2021-12-24 | 2022-04-22 | 鞍钢集团矿业有限公司 | 针对变电站支柱式绝缘子串的边缘轮廓提取方法 |
CN114663529B (zh) * | 2022-03-22 | 2023-08-01 | 阿波罗智能技术(北京)有限公司 | 一种外参确定方法、装置、电子设备及存储介质 |
GB2618127B (en) * | 2022-04-21 | 2024-07-31 | Continental Automotive Romania Srl | Computer-implemented method of determining and adjusting camera pitch angle error of a vehicle camera |
CN114897683A (zh) * | 2022-04-25 | 2022-08-12 | 深圳信路通智能技术有限公司 | 车侧图像的获取方法及其装置、***和计算机设备 |
CN117036505B (zh) * | 2023-08-23 | 2024-03-29 | 长和有盈电子科技(深圳)有限公司 | 车载摄像头在线标定方法及*** |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH11281352A (ja) * | 1998-03-31 | 1999-10-15 | Nissan Motor Co Ltd | 道路形状検出装置 |
KR20140033803A (ko) * | 2012-09-10 | 2014-03-19 | 현대모비스 주식회사 | 카메라 공차 보정 방법 및 장치와 이를 이용한 어라운드 뷰 모니터링 시스템 |
KR20150002039A (ko) * | 2013-06-28 | 2015-01-07 | (주) 세인 | 차량에 설치되어 있는 블랙박스와 같은 영상 단말기를 이용하여 영상에서 차선을 인식 경고, 기록하는 장치의 실시간성 확보를 위한 방법 |
KR20150112656A (ko) * | 2014-03-28 | 2015-10-07 | 주식회사 하나비전테크 | 카메라 캘리브레이션 방법 및 그 장치 |
KR101584907B1 (ko) * | 2014-07-29 | 2016-01-22 | 울산대학교 산학협력단 | 관심 영역을 이용한 차선 인식 방법 및 그 장치 |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20080028531A (ko) | 2006-09-27 | 2008-04-01 | 현대자동차주식회사 | 차량 전방카메라 보정장치 |
KR101163446B1 (ko) * | 2009-03-18 | 2012-07-18 | 기아자동차주식회사 | 가상 차선을 이용하는 차선 이탈 경보 방법 및 그에 따른 시스템 |
US8935057B2 (en) * | 2012-01-17 | 2015-01-13 | LimnTech LLC | Roadway mark data acquisition and analysis apparatus, systems, and methods |
JP5812061B2 (ja) * | 2013-08-22 | 2015-11-11 | 株式会社デンソー | 物標検出装置およびプログラム |
FR3014553A1 (fr) * | 2013-12-11 | 2015-06-12 | Parrot | Procede de calibration angulaire de la position d'une camera video embarquee dans un vehicule automobile |
CN104029680B (zh) * | 2014-01-02 | 2016-12-07 | 上海大学 | 基于单目摄像头的车道偏离预警***及方法 |
CN104129389B (zh) * | 2014-08-06 | 2017-02-01 | 中电海康集团有限公司 | 一种有效判别车辆行驶状态的方法及其装置 |
JP6601336B2 (ja) * | 2016-07-08 | 2019-11-06 | 株式会社デンソー | 区画線認識装置 |
US10706551B2 (en) * | 2017-03-30 | 2020-07-07 | 4DM Inc. | Object motion mapping using panchromatic and multispectral imagery from single pass electro-optical satellite imaging sensors |
-
2017
- 2017-09-13 KR KR1020170116914A patent/KR101969030B1/ko active IP Right Grant
-
2018
- 2018-08-08 US US16/315,941 patent/US11398051B2/en active Active
- 2018-08-08 WO PCT/KR2018/009001 patent/WO2019054636A1/ko active Application Filing
- 2018-08-08 CN CN201880002468.0A patent/CN109791700A/zh active Pending
- 2018-08-08 DE DE112018000107.5T patent/DE112018000107T5/de active Pending
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH11281352A (ja) * | 1998-03-31 | 1999-10-15 | Nissan Motor Co Ltd | 道路形状検出装置 |
KR20140033803A (ko) * | 2012-09-10 | 2014-03-19 | 현대모비스 주식회사 | 카메라 공차 보정 방법 및 장치와 이를 이용한 어라운드 뷰 모니터링 시스템 |
KR20150002039A (ko) * | 2013-06-28 | 2015-01-07 | (주) 세인 | 차량에 설치되어 있는 블랙박스와 같은 영상 단말기를 이용하여 영상에서 차선을 인식 경고, 기록하는 장치의 실시간성 확보를 위한 방법 |
KR20150112656A (ko) * | 2014-03-28 | 2015-10-07 | 주식회사 하나비전테크 | 카메라 캘리브레이션 방법 및 그 장치 |
KR101584907B1 (ko) * | 2014-07-29 | 2016-01-22 | 울산대학교 산학협력단 | 관심 영역을 이용한 차선 인식 방법 및 그 장치 |
Also Published As
Publication number | Publication date |
---|---|
US20210343044A1 (en) | 2021-11-04 |
KR101969030B1 (ko) | 2019-04-16 |
CN109791700A (zh) | 2019-05-21 |
US11398051B2 (en) | 2022-07-26 |
KR20190030242A (ko) | 2019-03-22 |
DE112018000107T5 (de) | 2019-05-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2019054636A1 (ko) | 차량용 카메라 캘리브레이션 장치 및 그 방법 | |
WO2015152692A1 (ko) | 차량 주변 이미지 생성 장치 및 방법 | |
WO2015152691A2 (ko) | 차량 주변 이미지 생성 장치 및 방법 | |
WO2016090559A1 (zh) | 一种图像处理方法、装置及摄像设备 | |
WO2020141657A1 (ko) | 신체 측정 디바이스 및 그 제어 방법 | |
WO2017188706A1 (ko) | 이동 로봇 및 이동 로봇의 제어방법 | |
WO2018062647A1 (ko) | 정규화된 메타데이터 생성 장치, 객체 가려짐 검출 장치 및 그 방법 | |
WO2015102391A1 (ko) | 뎁스 영상 분석을 통한 사용자의 골프 스윙 자세 분석을 위한 영상 생성 방법과 이를 이용한 골프 스윙 자세 분석 방법 및 장치 | |
WO2013009020A4 (ko) | 시청자 얼굴 추적정보 생성방법 및 생성장치, 그 기록매체 및 3차원 디스플레이 장치 | |
WO2012064106A2 (en) | Method and apparatus for video stabilization by compensating for view direction of camera | |
WO2016088960A1 (ko) | 야간 환경에서의 운전자 보조 시스템을 위한 위험 보행자 검출 방법 및 시스템 | |
EP2422526A2 (en) | Method and apparatus for automatic transformation of three-dimensional video | |
WO2016145602A1 (en) | Apparatus and method for focal length adjustment and depth map determination | |
EP2907298A1 (en) | Image processing device and image processing method | |
WO2015088289A1 (ko) | 스테레오 카메라, 이를 구비한 차량 운전 보조 장치, 및 차량 | |
WO2021141338A1 (ko) | 선박 및 항만 모니터링 장치 및 방법 | |
WO2021141339A1 (ko) | 해수면을 고려한 항만 및 선박 모니터링 방법 및 장치 | |
WO2015183050A1 (ko) | 옵티컬 트래킹 시스템 및 옵티컬 트래킹 시스템의 마커부 자세 및 위치 산출방법 | |
WO2020116680A1 (ko) | 차량 번호판 검출 및 인식 시스템 및 방법 | |
WO2018212608A1 (ko) | 구동형 마킹 시스템, 구동형 마킹 장치의 제어방법 및 컴퓨터 판독 가능한 기록매체 | |
WO2016206107A1 (en) | System and method for selecting an operation mode of a mobile platform | |
WO2021040156A1 (ko) | 신체 측정 디바이스 및 그 제어 방법 | |
WO2020004837A1 (ko) | 자동 편광 조절 장치 및 방법 | |
WO2014178610A1 (ko) | 옵티컬 트랙킹 시스템 및 이를 이용한 트랙킹 방법 | |
WO2015199354A1 (ko) | 인접 합산 이진화를 이용한 동공 추출 방법 및 이를 이용한 동공 추출 제어장치 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 18855438 Country of ref document: EP Kind code of ref document: A1 |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 18855438 Country of ref document: EP Kind code of ref document: A1 |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 18855438 Country of ref document: EP Kind code of ref document: A1 |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 22.01.2021) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 18855438 Country of ref document: EP Kind code of ref document: A1 |