WO2005088971A1 - 画像生成装置、画像生成方法、および画像生成プログラム - Google Patents
画像生成装置、画像生成方法、および画像生成プログラム Download PDFInfo
- Publication number
- WO2005088971A1 WO2005088971A1 PCT/JP2005/002977 JP2005002977W WO2005088971A1 WO 2005088971 A1 WO2005088971 A1 WO 2005088971A1 JP 2005002977 W JP2005002977 W JP 2005002977W WO 2005088971 A1 WO2005088971 A1 WO 2005088971A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- blind spot
- imaging
- viewpoint
- data
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 26
- 238000003384 imaging method Methods 0.000 claims abstract description 138
- 238000006243 chemical reaction Methods 0.000 claims description 40
- 238000013507 mapping Methods 0.000 claims description 6
- 238000012544 monitoring process Methods 0.000 abstract description 11
- 238000010586 diagram Methods 0.000 description 35
- 238000005259 measurement Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 9
- 238000012545 processing Methods 0.000 description 7
- 239000013598 vector Substances 0.000 description 4
- 240000004050 Pentaglottis sempervirens Species 0.000 description 3
- 235000004522 Pentaglottis sempervirens Nutrition 0.000 description 3
- 239000003086 colorant Substances 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 241000282472 Canis lupus familiaris Species 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
- B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
- B60R1/27—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view providing all-round vision, e.g. using omnidirectional cameras
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/31—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles providing stereoscopic vision
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/10—Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
- G08G1/166—Anti-collision systems for active traffic, e.g. moving vehicles, pedestrians, bikes
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
- G08G1/167—Driving aids for lane monitoring, lane changing, e.g. blind spot detection
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
- B60R2300/102—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using 360 degree surveillance camera system
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
- B60R2300/105—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using multiple cameras
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
- B60R2300/302—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
- B60R2300/303—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing using joined images, e.g. multiple camera images
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
- B60R2300/304—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing using merged images, e.g. merging camera image with stored images
- B60R2300/305—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing using merged images, e.g. merging camera image with stored images merging camera image with lines or icons
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
- B60R2300/802—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for monitoring and displaying vehicle exterior blind spot views
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
- B60R2300/8033—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for pedestrian protection
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
- B60R2300/8093—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for obstacle warning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/08—Indexing scheme for image data processing or generation, in general involving all processing steps from image acquisition to 3D model generation
Definitions
- Image generation apparatus image generation method, and image generation program
- the present invention provides intuitive information on the movement of an object, such as a vehicle, when the object moves, based on a plurality of images taken by one or more cameras attached to the object, such as a vehicle.
- a monitoring sight device that monitors a monitoring target around a vehicle, a store, or a house, or in a city, etc., uses one or more cameras to photograph the monitoring target, and displays the image on the monitor device.
- a surveillance camera device when there are a plurality of cameras and the number of monitor devices is smaller than the number of cameras, for example, when there is only one monitor device in spite of two cameras, a plurality of captured These camera images were integrated and displayed on a single monitor device, or switched and displayed sequentially.
- the administrator needs to consider the continuity of independently displayed images in order to monitor images from the respective cameras.
- an input image from one or more cameras attached to a vehicle or the like is mapped to a predetermined space model in a three-dimensional space, and the mapped space is mapped. Creates and displays an image as viewed from any virtual viewpoint in the above 3D space with reference to the data.
- a technology relating to a surveillance camera device see, for example, Patent Document 1).
- Patent Document 1 Japanese Patent No. 3286306
- the conventional surveillance camera device as described above has a problem in that it is difficult to determine the relationship between the object on which the camera is mounted, such as a vehicle equipped with an imaging means, and the monitoring target photographed by the camera. there were.
- the present invention has been made in view of the above-mentioned drawbacks of the related art. When displaying an image of a monitoring target around a vehicle, a store, or a house, or in a city, an imaging method of the vehicle or the like is used. It is an object of the present invention to provide an image generation device, an image generation method, and an image generation program capable of intuitively displaying the relationship between a placed object and a captured image of a monitoring target.
- the present invention employs the following configuration in order to solve the above-mentioned problems.
- an image generating apparatus includes one or more image capturing units attached to an image capturing unit-located object for capturing an image, and an image captured by the image capturing unit.
- Space reconstruction means for mapping to a space model; viewpoint transformation means for generating viewpoint transformation image data viewed from an arbitrary virtual viewpoint in a three-dimensional space based on the spatial data mapped by the space reconstruction means;
- a display unit for displaying an image viewed from an arbitrary virtual viewpoint in the three-dimensional space based on the viewpoint conversion image data generated by the viewpoint conversion unit. Viewpoint converted image data, captured image data representing the captured image, the spatial model, and the mapped spatial data.
- movement information calculating means for calculating movement information related to the movement of the imaging device-arranged object, wherein the display means comprises: The moving information calculated by the moving information calculating means is displayed together with the image.
- the movement information may include movement direction information indicating a moving direction, movement trajectory information indicating a predicted movement trajectory, movement speed information indicating a moving speed, It is desirable that the destination information is any one of the destination information related to the direction, place name, landmark, etc. of the destination.
- the image generation apparatus of the present invention provides viewpoint conversion image data generated by the viewpoint conversion means, captured image data representing the captured image, the space model, and the mapped image corresponding to different times. Further comprising a collision possibility calculating means for calculating a possibility of collision between the imaging means arranged object model and the space model based on any of the spatial data obtained by the imaging means, wherein the display means is generated by the viewpoint converting means. Sa In the image of the imaging means arranged object model, which is displayed by being superimposed on the image obtained by the viewpoint conversion image data, the display form of the portion having the possibility of collision calculated by the possibility of collision calculation means is changed. It is desirable to display.
- the display mode is at least one of a color, a border, and a warning icon.
- the image generation device of the present invention further includes a blind spot for a predetermined location of the imaging device arranged object based on an imaging device arranged object model represented by data corresponding to the imaging device arranged object. It is preferable that the apparatus further comprises blind spot calculation means for calculating blind spot information indicating an area, and that the display means displays the blind spot information calculated by the blind spot calculation means together with the image pickup means arranged object model.
- the image generation apparatus of the present invention may be configured such that any one of viewpoint converted image data converted by the viewpoint converting means, captured image data representing the captured image, the spatial model, and the spatial data mapped as described above.
- the image forming apparatus further includes another object blind spot calculation means for calculating other object blind spot information indicating a blind spot area of the other object, wherein the display means is calculated by the other object blind spot calculation means together with the imaging means arranged object model. It is desirable to display the blind spot information on the other object.
- the object arranged with the imaging means or another object is at least one of a vehicle, a pedestrian, a building, a road structure and the like.
- an image generating apparatus includes one or more image capturing units attached to an image capturing unit-arranged object for capturing an image, and image data captured by the image capturing unit.
- the other object recognition means for recognizing another object different from the object arranged with the imaging means, and the blind spot of the other object recognized by the other object recognition means based on other object data indicating data on the predetermined other object.
- Display means for displaying other object blind spot information calculated by the other object blind spot calculation means together with the other object blind spot calculation means for calculating other object blind spot information indicating an area to be obtained, and the imaging means arranged object model; It is characterized by having.
- the object arranged with the imaging means or another object is at least one of a vehicle, a pedestrian, a building, a road structure, and the like.
- the image generation method of the present invention maps the captured image captured by one or more imaging devices attached to the imaging device-located object to a spatial model, Based on the generated spatial data, viewpoint-converted image data viewed from an arbitrary virtual viewpoint in the three-dimensional space is generated, and based on the generated viewpoint-converted image data, an arbitrary virtual image in the three-dimensional space is generated.
- a computer-implemented image generation method for displaying an image viewed from a viewpoint wherein the generated viewpoint-converted image data, the captured image data representing the captured image, the space model, and the mapped space data are provided.
- Moving information related to the movement of the imaging means arranged object is further calculated based on any of Characterized in that together with the image of the stage disposed object model displaying the movement information
- the image generation program provides a computer with a computer for spatially capturing a captured image captured by one or a plurality of imaging devices attached to an imaging means disposed object.
- a step of mapping to a model a step of generating viewpoint-converted image data viewed from an arbitrary virtual viewpoint in a three-dimensional space based on the mapped spatial data, and a step of generating viewpoint-converted image data based on the generated viewpoint-converted image data.
- Movement information related to the movement of the imaging device arranged object based on any of the data, the spatial model, and the mapped spatial data. Further comprising the steps of calculating, the display, and displaying the movement information with the image of the imaging means disposed object model in the viewpoint conversion image.
- the image generation method of the present invention includes the steps of: capturing an image by one or a plurality of imaging devices attached to an imaging device disposed object; Based on the other object, the other object different from the object arranged with the imaging means is recognized, and based on the other object data indicating data on the predetermined other object, the blind spot of the recognized other object is calculated.
- This is a computer-executed image generation method for calculating other object blind spot information indicating a region of interest and displaying the above calculated other object blind spot information together with the above-mentioned imaging means arranged object model.
- an image generating program of the present invention includes a procedure for capturing an image by a computer using one or a plurality of imaging devices attached to an imaging means disposed object; A procedure for recognizing another object different from the image-capturing means arranged object based on the captured image data; and a blind spot of the recognized other object based on other object data indicating data on a predetermined other object.
- An image generation program for executing a procedure for calculating other object blind spot information indicating an area and a procedure for displaying the calculated other object blind spot information together with the imaging means arranged object model.
- FIG. 1 is a block diagram of an image generation device that generates a spatial model by generating a spatial model by a distance measuring device.
- FIG. 2 is a block diagram of an image generation device that generates a spatial model by an imaging device and generates a viewpoint-converted image.
- FIG. 3 is a block diagram of an image generation device for generating a space model by a distance measuring device and displaying movement information in a viewpoint-converted image.
- FIG. 4 is a diagram showing a visual field that can be observed by a driver driving a vehicle.
- FIG. 5 is a diagram showing a display example in which moving direction information is displayed.
- FIG. 6 is a diagram showing a display example in which movement trajectory information is displayed.
- FIG. 7 is a diagram showing a display example in which moving speed information is displayed.
- FIG. 8 is a diagram showing a display example in which destination information is displayed.
- FIG. 9 is a diagram showing a display example in which the display form of an image is displayed differently depending on the possibility of collision between two objects, together with the display of movement information.
- FIG. 10A is a diagram (part 1) for explaining a blind spot.
- FIG. 10B is a diagram (part 2) for explaining a blind spot.
- FIG. 11 is a diagram showing a display example in which blind spot information is displayed.
- FIG. 12 is a view showing a display example in which other object blind spot information is displayed.
- FIG. 13 is a block diagram of an image generation device for generating a space model by an imaging device and displaying movement information in a viewpoint-converted image.
- FIG. 14 is a flowchart showing a flow of an image generation process for displaying movement information, possibility of collision, blind spot information, and blind spot information of another object in a viewpoint-converted image.
- FIG. 15 is a block diagram of an image generating apparatus for displaying blind spot information of another object.
- FIG. 16 is a diagram showing a display example in which other object blind spot information is displayed.
- FIG. 17 is a flowchart showing a flow of an image generation process for displaying other object blind spot information.
- FIG. 18 is a diagram illustrating a relationship between the host vehicle and another vehicle for explaining a calculation example of the possibility of collision.
- FIG. 19 is a diagram showing relative vectors for explaining a calculation example of the possibility of collision.
- an image generation apparatus that generates an image viewed from a virtual viewpoint based on image data captured by a plurality of cameras and displays the image from the virtual viewpoint will be described with reference to FIGS.
- a plurality of cameras are used in the example shown in the figure, it is also possible to sequentially move the installation position of one camera to obtain the same imaging data as when a plurality of cameras are provided.
- these cameras are installed in vehicles, rooms (specific portions thereof, etc.), pedestrians, buildings, road structures, and the like as imaging device arrangement objects. This is the same for the embodiments described below.
- FIG. 1 is a block diagram of an image generating apparatus that generates a spatial model by a distance measuring apparatus and generates a viewpoint-converted image.
- an image generation device 100 includes a distance measurement device 101, a space model generation device 103, a calibration device 105, one or more imaging devices 107, a space reconstruction device 109, a viewpoint conversion device 112, and a display device 114. Is provided.
- Distance measuring apparatus 101 measures the distance to a target object (obstacle) using a distance sensor that measures the distance. For example, if it is mounted on a vehicle, At least the distance to the obstacle around the vehicle is measured using the distance sensor.
- the space model generation device 103 generates a three-dimensional space model 104 based on the distance image data 102 measured by the distance measurement device 101, and stores it in a database (the concept is illustrated as a substance. Similarly).
- the space model 104 is generated based on measurement data obtained by a separate sensor as described above, a predetermined model determined in advance, or generated from a plurality of input images on the spot. And their data are stored in a database.
- the imaging device 107 is, for example, a camera, and is mounted on an imaging device-located object to capture an image and store the captured image data 108 in a database. If the imaging device-arranged object is a vehicle, the imaging device 107 captures an image around the vehicle.
- the spatial reconstruction device 109 maps the captured image data 108 captured by the image capturing device 107 to the spatial model 104 generated by the spatial model generating device 103. Then, data obtained by mapping the photographed image data 108 to the space model 104 is stored as space data 111 in the database.
- the calibration device 105 inputs, for example, parameters such as a mounting position, a mounting angle, a lens distortion correction value, and a lens focal length of the imaging device 107 in order to correct lens distortion due to temperature. Or by calculation. That is, if the imaging device 107 is a camera, camera calibration is performed. Camera calibration refers to the camera position, camera mounting angle, camera lens distortion correction value, camera lens focal length, etc. in the three-dimensional real world for cameras placed in the three-dimensional real world. This is to determine and correct camera parameters representing the above camera characteristics.
- the viewpoint conversion device 112 generates viewpoint conversion image data 113 viewed from an arbitrary virtual viewpoint in a three-dimensional space based on the spatial data 111 mapped by the space reconstruction device 109, Store it in the database.
- the display device 114 displays an image viewed from an arbitrary virtual viewpoint in the three-dimensional space based on the viewpoint converted image data 113 generated by the viewpoint conversion device 112.
- FIG. 2 is a block diagram of an image generation device that generates a spatial model by using an imaging device to generate a viewpoint-converted image.
- the image generation device 200 includes a distance measurement device 201, a space model generation device 103, a calibration device 105, one or more imaging devices 107, a space reconstruction device 109, a viewpoint conversion device 112, and The display device 114 is provided.
- the image generation device 200 differs from the image generation device 100 described with reference to FIG. 1 in that a distance measurement device 201 is provided instead of the distance measurement device 101.
- a distance measurement device 201 is provided instead of the distance measurement device 101.
- the description will focus on the distance measuring apparatus 201, and other components will not be described because they are the same as those described with reference to FIG.
- the distance measuring device 201 measures the distance to the obstacle based on the captured image data 108 captured by the capturing device 107.
- This method generally uses stereo ranging, which searches for corresponding points in images from multiple cameras that have captured the same field of view, calculates the parallax between the images, and calculates the distance based on the principle of triangulation. is there.
- the distance image data 202 can be combined with data obtained by measuring the distance to an obstacle using a distance sensor that measures the distance.
- the space model generation device 103 generates a space model 104 of a three-dimensional space based on the distance image data 202 measured by the distance measurement device 201, and stores the space model 104 in a database.
- FIG. 3 to FIG. 13 when displaying an image from a virtual viewpoint based on a plurality of images taken by one or more cameras attached to an object such as a vehicle.
- an image generation apparatus capable of generating image data for displaying information on the movement of the above object in an intuitive manner when the object moves will be described.
- This image generation device can be applied to the image generation device described with reference to FIG. 1 or FIG.
- FIG. 3 is a block diagram of an image generation device for generating a spatial model by a distance measuring device and displaying movement information in a viewpoint-converted image.
- an image generation device 300 includes a distance measurement device 101, a space model generation device 103, Calibration device 105, one or more imaging devices 107, spatial reconstruction device 109, viewpoint conversion device 112, display device 314, movement information calculation device 315, collision possibility calculation device 316, blind spot calculation device 317, other objects A recognition device 318 and another object blind spot calculation device 319 are provided.
- Image generating apparatus 300 is different from image generating apparatus 100 described with reference to Fig. 1 in that movement information calculating apparatus 315, collision possibility calculating apparatus 316, blind spot calculating apparatus 317, and other object recognizing apparatus 318 are also provided. And a blind spot calculation device 319 for other objects, and a display device 314 in place of the display device 114.
- the following description focuses on the movement information calculation device 315, the possibility of collision calculation device 316, the blind spot calculation device 317, the other object recognition device 318, the other object blind spot calculation device 319, and the display device 314, and other components. Are the same as those described with reference to FIG.
- the movement information calculation device 315 includes the viewpoint conversion image data 113 generated by the viewpoint conversion device 112, the captured image data 108 representing the captured image, the spatial model 104, and the mapped spatial data 111. Based on either of them, the movement information relating to the movement of the object placed on the imaging device is calculated. Examples of the movement information related to the movement of the object arranged with the imaging device include movement direction information indicating a moving direction, movement trajectory information indicating a predicted movement trajectory, movement speed information indicating a moving speed, and predicted movement. There are destination information and the like related to the direction, place name, landmark, and the like.
- the display device 314 displays the movement information calculated by the movement information calculation device 315 together with the image of the imaging device arrangement object model corresponding to the imaging device arrangement object.
- FIG. 4 is a diagram showing a state of a visual field that can be observed by a driver driving a vehicle.
- the driver sees three vehicles, Vehicle A, Vehicle B, and Vehicle C, on the road.
- the vehicle is provided with a distance sensor (ranging device 101) for measuring the distance to an obstacle present around the vehicle, and a plurality of cameras (image pickup device 107) for photographing an image around the vehicle. Have been.
- a distance sensor ranging device 101
- a plurality of cameras image pickup device 107
- the space model generation device 103 generates a space model 104 of a three-dimensional space based on the distance image data 102 measured by the distance sensor, and stores the space model 104 in a database. Then, the camera captures an image around the vehicle and stores the captured image data 108 in the database.
- the spatial reconstruction device 109 maps the captured image data 108 captured by the camera to the spatial model 104 generated by the spatial model generating device 103, and stores the spatial model 111 as spatial data 111 in the database.
- the viewpoint conversion device 112 Based on the spatial data 111 mapped by the spatial reconstruction device 109, the viewpoint conversion device 112 sets the viewpoint conversion image data 113 viewed from the virtual viewpoint, for example, as the virtual viewpoint above the vehicle. Generate and store it in a database.
- the movement information calculation device 315 is configured to convert the viewpoint conversion image data 113 generated by the viewpoint conversion device 112, the captured image data 108 representing the captured image, the spatial model 104, and the mapped spatial data 111. Based on any of them, movement information related to the movement of the object placed on the imaging device, for example, movement direction information indicating a moving direction, movement trajectory information indicating a predicted movement trajectory, movement speed information indicating a moving speed, Calculate destination information relating to the measured azimuth, place name, landmark, etc. of the destination.
- the display device 314 is usually installed in the vehicle.
- the display device 314 is shared with a monitor of a car navigation system, and based on the viewpoint conversion image data 113 generated by the viewpoint conversion device 112,
- the viewpoint conversion image data 113 generated by the viewpoint conversion device 112
- the moving direction information and the like calculated by the movement information calculation device 315 are displayed Displays transfer information.
- FIG. 5 is a diagram showing a display example in which the moving direction information is displayed.
- FIG. 5 the object A is the viewpoint converted image of the vehicle A in FIG. 4, and the object B is the vehicle object C in the same manner. Then, moving direction information indicating the moving direction of the own vehicle is displayed together with the own vehicle corresponding model based on the data stored in the imaging device arranged object model 110.
- FIG. 6 is a diagram showing a display example in which the movement trajectory information is displayed.
- FIG. 6 is also a display example of a viewpoint conversion image, but is an example of a virtual viewpoint different from FIG.
- the display example in Fig. 5 is a bird's-eye view in which the virtual viewpoint is in the sky above the vehicle and looks forward from it
- the display example in Fig. 6 shows that the virtual viewpoint is The figure is above the vehicle and looking directly below.
- the object A is the viewpoint converted image of the vehicle A in FIG. 4, and similarly, the object B is not displayed as the object corresponding to the force vehicle C which is the viewpoint converted image of the vehicle B. Then, movement trajectory information indicating a predicted movement trajectory of the own vehicle is displayed together with the own vehicle corresponding model based on the data stored in the imaging device arranged object model 110.
- FIG. 7 is a diagram showing a display example in which moving speed information is displayed.
- Fig. 7 is a bird's-eye view looking forward from a virtual viewpoint above the host vehicle, as in Fig. 5, and object A is the viewpoint-converted image of vehicle A in Fig. 4; Vehicle B and object C are viewpoint converted images of vehicle C.
- the moving speed information indicating the moving speed of the own vehicle is displayed together with the own vehicle corresponding model based on the data stored in the imaging device arranged object model 110.
- FIG. 8 is a diagram showing a display example in which destination information is displayed.
- Fig. 8 is also an overhead view of looking ahead from a virtual viewpoint above the host vehicle, and object A is the viewpoint converted image of vehicle A in Fig. 4; Vehicle B and object C are viewpoint converted images of vehicle C. Then, the destination information regarding the predicted destination direction, the place name, the landmark, etc. of the own vehicle is stored in the imaging device arranged object model 110 and displayed together with the own vehicle corresponding model based on the data. Being done.
- the movement information shown in Figs. 5 to 8 may be displayed simultaneously.
- the collision possibility calculation device 316 includes viewpoint conversion image data 113 generated by the viewpoint conversion device 112, captured image data 108 representing the captured image, the space model 104, and the Based on one of the mapped spatial data 111, the possibility of collision between the imaging device arranged object model 110 and the spatial model 104 is calculated.
- the possibility of collision is, for example, the displacement of each of two objects. It can be easily obtained from the moving direction and the moving speed.
- the display device 314 displays the image-capturing device arrangement object model 110 that is displayed by being superimposed on the image based on the viewpoint-converted image data 113 generated by the viewpoint-converting device 112.
- the display form of the portion having the possibility of collision calculated by the collision possibility calculation device 316 is displayed differently according to the possibility of collision.
- the presence / absence and form of the warning icon such as the color, the presence or absence of the border, and the thickness, are displayed differently.
- the display device 314 may display the image as an integrated background model when the collision probability calculated by the collision probability calculation device 316 is equal to or less than a predetermined value, or may display the image as an integrated model. You can even display it. Further, the display form may be such that the meaning of display information is recognized by color.
- FIG. 9 is a diagram showing a display example in which the display mode of an image is displayed differently depending on the possibility of collision between two objects, together with the display of the movement information.
- Fig. 9 is also a bird's-eye view of looking ahead from a virtual viewpoint above the own vehicle in the same way as Fig. 5, and object A is the viewpoint converted image of vehicle A in Fig. 4; Vehicle B and object C are viewpoint converted images of vehicle C.
- movement direction information indicating a moving direction
- movement speed information indicating a moving speed
- a place name a landmark
- the own vehicle collides with each of the vehicles A, B and C.
- the display modes of the object A, object B, and object C are displayed differently depending on the possibility. For example, the possibility of collision with the host vehicle is the highest among the three other vehicles, and the object C, which is the viewpoint conversion image of the vehicle C, is displayed in red, and the possibility of collision is higher than that of the vehicle C.
- the object A which is the viewpoint converted image of the vehicle A
- the object B which is the viewpoint converted image of the vehicle B
- a mode in which at least one of the hue, saturation, and lightness of the color related to the display is displayed differently may be adopted.
- the blind spot calculation device 317 is a blind spot that indicates an area that is a blind spot for a predetermined location of the imaging device placement object based on the imaging device placement object model 110 represented by data corresponding to the imaging device placement object. Calculate information. For example, if the image-capturing-device-placed object is a vehicle, an area that is a blind spot for the driver of the vehicle is calculated.
- FIGS. 10A and 10B are diagrams for explaining blind spots.
- the imaging device arranged object is a vehicle
- FIG. 1OA is a diagram of the vehicle viewed from above
- FIG. 10B is a diagram of the vehicle viewed from the side. Then, the driver's blind spot (the blind spot due to pillars and the vehicle body) and the blind spot (the blind spot due to the vehicle body) with respect to the driver as the predetermined place of the vehicle are shaded. .
- the display device 314 displays the blind spot information calculated by the blind spot calculation device 317 together with the imaging device arranged object model 110.
- the blind spot information here indicates a blind spot from the viewpoint of the driver.
- FIG. 11 is a diagram showing a display example in which blind spot information is displayed.
- the object A is the viewpoint transformed image of the vehicle A in FIG. 4, and similarly, the object B is the viewpoint transformed image of the vehicle B, and the object corresponding to the force vehicle C is not displayed. Then, the blind spot area of the own vehicle is displayed as blind spot information together with the own vehicle corresponding model based on the data stored in the imaging device arranged object model 110 and the moving trajectory information which is one of the moving information. I have.
- the other object recognition device 318 stores any one of the viewpoint conversion image data 113 converted by the viewpoint conversion device 112, the captured image data 108 representing the captured image, the space model 104, and the mapped space data 111. On the basis of this, another object different from the above-mentioned object provided with the imaging device is recognized. For example, if the image-capturing device-arranged object is a vehicle, a preceding vehicle, an opposing vehicle, a pedestrian, and the like are recognized.
- the other object blind spot calculation device 319 performs other object blind spot information indicating an area which is recognized by the other object recognition device 318 and becomes a blind spot of the other object based on the other object data indicating data on the predetermined other object. Is calculated. For example, if the image-capturing-device-placed object is a vehicle and another object different from the image-capturing-device-placed object is also a vehicle, the calculated blind spot area is It is a blind spot for the driver of the body vehicle. In addition, area and other object data
- a database in which the imaging device arrangement object model 110 is stored may be used.
- the display device 314 displays the blind spot information on the other object calculated by the other object blind spot calculation device 319 together with the imaging device arranged object model 110.
- FIG. 12 shows the other object blind spot information. It is a figure which shows the example of a display.
- the object A is the viewpoint converted image of the vehicle A in FIG. 4, and similarly, the object B is the viewpoint converted image of the vehicle B.
- the object corresponding to the force vehicle C is not displayed.
- the object A is the other object recognized by the other object recognition device 318, and the area where the blind spot of the object A becomes the blind spot information of the other object is the own vehicle corresponding model and the movement trajectory information which is one of the movement information. Is displayed. As a result, it can be recognized that the own vehicle enters the blind spot of the object A.
- FIG. 13 is a block diagram of an image generation device for generating a space model by an imaging device and displaying movement information in a viewpoint-converted image.
- an image generation device 1300 includes a distance measurement device 201, a space model generation device 103, a calibration device 105, one or more imaging devices 107, a space reconstruction device 109, a viewpoint conversion device 112, and a display device 314. , A movement information calculation device 315, a collision possibility calculation device 316, a blind spot calculation device 317, another object recognition device 318, and another object blind spot calculation device 319.
- the image generating apparatus 1300 differs from the image generating apparatus 300 described with reference to FIG. 3 in that a distance measuring apparatus 201 is provided instead of the distance measuring apparatus 101. Note that the distance measuring device 201 is described with reference to FIG.
- FIG. 14 is a flowchart illustrating a flow of an image generation process for displaying movement information, possibility of collision, blind spot information, and blind spot information of another object in a viewpoint-converted image.
- step S1401 an image around the vehicle is captured using a camera attached to an object such as a vehicle.
- step S 1402 captured image data 108, which is data of the image captured in step S 1401, is mapped to the space model 104 to generate space data 111.
- step S1403 based on the spatial data 111 mapped in step S1402, viewpoint conversion image data 113 viewed from an arbitrary virtual viewpoint in a three-dimensional space is generated.
- step S1404 based on any of the generated viewpoint converted image data 113, the captured image data 108, the spatial model 104, and the mapped spatial data 111, the imaging device Calculate movement information related to the movement of the placed object.
- step S1405 when displaying an image viewed from an arbitrary virtual viewpoint in the three-dimensional space, the movement information calculated in step S1404, for example, the movement direction information indicating the moving direction, is predicted. It displays moving path information indicating the moving path to be moved, moving speed information indicating the moving speed, moving destination information relating to the predicted direction of the moving destination, place names, landmarks, and the like.
- step S1406 it is determined whether or not the possibility of collision between the imaging device arranged object model 110 and the space model 104 is displayed. For example, whether or not to display is determined based on whether or not there is an instruction from a user such as a vehicle driver.
- step S1407 If it is determined in step S1406 that display is to be performed (YES), in step S1407, the generated viewpoint converted image data 113, the captured image data 108, the space model 104, and the Based on one of the mapped spatial data 111, the possibility of collision between the imaging device arranged object model 110 and the spatial model 104 is calculated.
- step S1408 in addition to the display of the movement information in step S1405, the imaging device arrangement object model 110 that is displayed by being superimposed on the image based on the viewpoint converted image data 113 generated by the viewpoint conversion device 112.
- the imaging device arrangement object model 110 that is displayed by being superimposed on the image based on the viewpoint converted image data 113 generated by the viewpoint conversion device 112.
- the image having the possibility of collision calculated by the possibility of collision calculation device 316 in the image of It is displayed differently depending on the performance. For example, different colors, borders, warning icons, etc. are displayed.
- step S1406 After the display of the possibility of collision in step S1408, or when it is determined that the possibility of collision is not displayed in step S1406 (step S1406: NO), the process returns to step S1409, where It is determined whether or not to display blind spot information indicating a blind spot area for a predetermined place. For example, whether or not to display is determined based on whether or not there is an instruction from a user such as a vehicle driver.
- step S1410 based on the imaging device arrangement object model 110 represented by the data corresponding to the imaging device arrangement object, Blind spot information indicating a blind spot area for a predetermined location of the object, for example, a blind spot area for the driver of the vehicle is calculated if the imaging device-located object is a vehicle.
- step S 1411 in addition to the display of the movement information in step S 1405, and possibly the display of the possibility of collision in step S 1408, the blind spot calculation device 317 is included together with the imaging device arrangement object model.
- the blind spot information calculated by the blind spot calculation device 317 is included together with the imaging device arrangement object model.
- step S1412 After displaying the blind spot information in step S1411, or when it is determined in step S1409 that the blind spot information is not to be displayed (step S1409: NO), in step S1412, the other object indicating the blind spot area of the other object It is determined whether to display the blind spot information. For example, whether or not to display is determined based on whether or not there is an instruction from a user such as a vehicle driver.
- step S1413 If it is determined in step S1412 that display is to be performed (YES), in step S1413, the generated viewpoint converted image data 113, the captured image data 108, the spatial model 104, and the mapped spatial data 111 Based on either of them, another object different from the imaging device-arranged object, for example, a preceding vehicle or the like is recognized, and in step S1414, the other object recognition is performed based on the other object data indicating data on the other object.
- Other object blind spot information indicating an area recognized by the device 318 and serving as a blind spot of another object is calculated.
- the imaging device-located object is a vehicle
- the calculated blind spot area is an area that becomes a blind spot for a driver of a vehicle that is another object.
- the area and other object data are obtained by using the imaging device arranged object model 11.
- a database in which 0 is stored can also be used.
- step S1415 in addition to the display of the movement information in step S1405, and in some cases, the possibility of collision in step S1408 or the display of blind spot information in step S1411 are added,
- the blind spot information on the other object calculated by the other object blind spot calculation device 319 is displayed together with the object model.
- FIGS. 15 to 17 based on an image taken by one or more cameras attached to an object such as a vehicle, the blind spot for the object in the image is intuitively obtained.
- An image generation device capable of generating image data for displaying so as to be understood will be described.
- FIG. 15 is a block diagram of an image generating apparatus for displaying blind spot information on another object.
- the image generation device 1500 includes one or a plurality of imaging devices 1501, another object recognition device 1503, another object blind spot calculation device 1505, and a display device 1506.
- the imaging device 1501 is, for example, a camera, captures an image attached to the imaging device-located object, and stores the captured image data 1502 in the database. If the imaging device-arranged object is a vehicle, the imaging device 1501 captures an image around the vehicle.
- the other object recognition device 1503 recognizes another object different from the object arranged with the imaging device, based on the image data captured by the imaging device 1501. For example, if the imaging device-arranged object is a vehicle, a preceding vehicle, an oncoming vehicle, a pedestrian, and the like are recognized.
- the other object blind spot calculating device 1505 calculates the blind spot of the other object recognized by the other object recognizing device 1503 based on the other object data 1504 indicating data on the predetermined other object.
- Other object blind spot information indicating an area is calculated. For example, if the imaging device-arranged object is a vehicle and another object different from the imaging device-arranged object is also a vehicle, the calculated blind spot area is an area that becomes a blind spot for the driver of the other object vehicle. . Also the area As the other object data 1504, it is desirable to use a database in which the imaging device arranged object model 1504 is stored.
- the display device 1506 displays the other object blind spot information calculated by the other object blind spot calculation device 1505 together with the imaging device arranged object model 1504 as another object.
- FIG. 16 is a diagram illustrating a display example in which other object blind spot information is displayed.
- the other object is recognized based on the image captured by the camera, and the other object data 1 is recognized.
- Other object blind spot information calculated based on 504 is displayed. Thus, it can be recognized that the own vehicle enters the blind spot of another object.
- FIG. 17 is a flowchart showing the flow of an image generation process for displaying other object blind spot information.
- step S1701 an image around the vehicle is captured using a camera attached to an object such as a vehicle.
- step S1702 based on the captured image data, another object different from the image-capturing-device-arranged object, for example, a preceding vehicle, an oncoming vehicle, a pedestrian, or the like is recognized.
- another object different from the image-capturing-device-arranged object for example, a preceding vehicle, an oncoming vehicle, a pedestrian, or the like is recognized.
- step S1703 other object blind spot information indicating a blind spot area of the recognized other object, such as a vehicle, is calculated based on other object data 1504 indicating data on a predetermined other object. I do.
- the other object blind spot information calculated by the other object blind spot calculation device 1505 is displayed together with the imaging device arranged object model 1504 as another object. Further, when the own vehicle is in a blind spot of another object, the display mode of the possibility of collision may be further changed when the own vehicle is in a blind spot of another object.
- the vehicle is used as the imaging device-located object, and the images obtained by the imaging devices 107 and 1501 mounted on the vehicle are used.
- the camera parameters are known or calculated, When it is measurable, road-facing structures and stores The same can be applied to a case where an image of a surveillance camera installed in a pavement or the like is included.
- Distance measuring device
- the cameras 101 and 102 are also arranged in the same way as the cameras.
- Distance information can be used.
- the display devices 114, 314, and 1506 that display the viewpoint-converted images and the imaging devices 107 and 1501 move relatively to each other without necessarily having to be installed on the same imaging device-arranged object. If there are obstacles,
- a plurality of image generating devices 100, 200, 300, 1300, and 1500 (a plurality of image generating devices of the same type
- the exchange of data may be performed immediately.
- each of the image generation devices 100 It has a coordinate and posture calculation device that communicates with 200, 300, 1300, and 1500 and calculates its reference coordinates.
- the coordinate / posture calculation device is a device that calculates a position / posture for generating a viewpoint-transformed image.
- This device uses, for example, GPS (Global Positioning System) data of latitude, longitude, and altitude to provide coordinates of a virtual viewpoint.
- GPS Global Positioning System
- the coordinate conversion may be performed by calculating the eye-to-position coordinates of the eye to generate a predetermined viewpoint-converted image. This is equivalent to setting a desired virtual viewpoint within those coordinate systems.
- the observer When the observer is a person, for example, the observer wears an HMD (Head Mounted Display) or the like, can observe the viewpoint-converted image, and observes the observer captured by the camera of the object arranged with the imaging device.
- the position, posture, and direction of the user may be measured.
- the coordinate posture information measured by a GPS or a gyro sensor, a camera device, or a person's line of sight detection device attached to the observer may be used in combination.
- the observer's viewpoint As a virtual viewpoint, it is possible to calculate the movement information, the possibility of collision, the blind spot, and the like for the observer.
- the person can grasp the obstacle for himself on the virtual viewpoint image displayed on the HMD, etc., and threaten himself / herself, for example, suspiciously hiding behind a shadow or hiding behind an observer. It is possible to recognize people, dogs, vehicles, etc., and even more accurately and accurately by using images and spatial models of imaging devices, placed objects, and image generation devices in the vicinity of distant objects.
- the generated multi-viewpoint converted image can be used.
- red, yellow, green, and blue are displayed in the descending order of the risk of collision according to the calculated collision probability. It is acceptable to display these colors differently based on the relative speed.
- a guardrail portion close to the own vehicle in distance is displayed in red, and a guardrail far from the own vehicle (for example, the opposite lane side) is displayed in blue. Since the road surface is an area where the vehicle including the own vehicle travels, the road surface is displayed in blue or green even if the distance to the own vehicle is short.
- the collision probability is calculated based on the relative speed, the distance, and the like, and the vehicle having the highest collision probability at the time of calculation is displayed in red.
- the color of other vehicles is changed according to the change in the possibility of collision, such as low collision probability and the display of vehicles in green.
- FIG. 18 is a diagram illustrating a relationship between the host vehicle and another vehicle for explaining a calculation example of the possibility of collision
- FIG. 19 is a diagram illustrating a relative vector for describing an example of calculation of the possibility of collision. It is a figure.
- the possibility of collision may be determined by increasing the height.
- the area with a high risk of collision due to the difference in hue is changed based on the distance between the vehicle and another object, the relative speed, and the possibility of collision calculated from these points.
- the display of the replaced image is changed.
- the degree of possibility of collision may be indicated by displaying the viewpoint-converted image in a blurred manner. For example, based on distance, relative speed, or the likelihood of a collision, the danger of colliding may be reduced, objects may be displayed with some blur, and those of higher danger may be clearly displayed without blurring. , Making it easier to recognize objects with a high risk of collision.
- a plurality of imaging devices may be used so as to configure a so-called three-lens stereo camera. Or, it may be used to compose a four-lens stereo camera. It is known that the use of a three- or four-lens stereo camera in this way can provide more reliable and stable processing results in 3D reconstruction processing, etc. (Fumiaki Tomita: Published by IPSJ) "Information Processing” Vol. 42, No. 4, "Highly Functional 3D Vision System” etc.). In particular, it is known that three-dimensional reconstruction can be performed in more complex scenes by arranging multiple cameras so as to have a baseline length in two directions. By arranging a plurality of cameras in one base line length direction, it is possible to realize a so-called multi-baseline stereo camera, and more accurate stereo measurement is possible.
- the present invention is applicable not only to a person but also to a moving object such as a vehicle.
- the CPU connected to the bus, the ROM or RAM memory, the input device, the output device, and the external device. It can also be realized by a system including a recording device, a medium driving device, a portable recording medium, and a network connection device. That is, a ROM or RAM memory, an external recording device, or a portable recording medium that records software program codes for realizing the system of each of the embodiments described above is supplied to the image generating device, and the It goes without saying that this can also be achieved by a computer reading and executing the program code.
- the program code itself read from the portable recording medium or the like realizes a new function of the present invention
- the portable recording medium or the like on which the program code is recorded constitutes the present invention. Will do.
- Portable recording media for supplying program codes include, for example, flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, DVD-ROMs, DVD-RAMs, and magnetic tapes.
- Various types of recording media recorded via a network connection device such as a nonvolatile memory card, a ROM card, and electronic mail / personal computer communication can be used.
- each of the above-described embodiments are implemented when the computer executes the program code read into the memory, and the computer operates on the computer based on the instructions of the program code.
- the operating system or the like performs part or all of the actual processing, and the functions of the above-described embodiments are also realized by the processing.
- a program code read from a portable recording medium or a program (data) provided by a program (data) provider A function expansion board inserted into the computer ⁇ a function expansion board connected to the computer After being written to the memory provided in the unit, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing based on the instructions of the program code. Also, the functions of the above-described embodiments can be realized.
- the present invention can take various configurations or shapes without departing from the gist of the present invention, which is not limited to the above-described embodiments.
- the relationship between the object and the taken image is determined. It can be displayed intuitively.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Geometry (AREA)
- Mechanical Engineering (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Computer Graphics (AREA)
- Closed-Circuit Television Systems (AREA)
- Traffic Control Systems (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
- Processing Or Creating Images (AREA)
Abstract
Description
Claims
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/519,333 US20070009137A1 (en) | 2004-03-16 | 2006-09-12 | Image generation apparatus, image generation method and image generation program |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2004073887A JP2005268847A (ja) | 2004-03-16 | 2004-03-16 | 画像生成装置、画像生成方法、および画像生成プログラム |
JP2004-073887 | 2004-03-16 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/519,333 Continuation US20070009137A1 (en) | 2004-03-16 | 2006-09-12 | Image generation apparatus, image generation method and image generation program |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2005088971A1 true WO2005088971A1 (ja) | 2005-09-22 |
Family
ID=34975976
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2005/002977 WO2005088971A1 (ja) | 2004-03-16 | 2005-02-24 | 画像生成装置、画像生成方法、および画像生成プログラム |
Country Status (3)
Country | Link |
---|---|
US (1) | US20070009137A1 (ja) |
JP (1) | JP2005268847A (ja) |
WO (1) | WO2005088971A1 (ja) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2010136263A (ja) * | 2008-12-08 | 2010-06-17 | Brother Ind Ltd | ヘッドマウントディスプレイ |
Families Citing this family (37)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4596978B2 (ja) * | 2005-03-09 | 2010-12-15 | 三洋電機株式会社 | 運転支援システム |
JP4887980B2 (ja) * | 2005-11-09 | 2012-02-29 | 日産自動車株式会社 | 車両用運転操作補助装置および車両用運転操作補助装置を備えた車両 |
JP4719590B2 (ja) * | 2006-03-01 | 2011-07-06 | アルパイン株式会社 | 車載周辺状況提示装置 |
JP2008219063A (ja) * | 2007-02-28 | 2008-09-18 | Sanyo Electric Co Ltd | 車両周辺監視装置及び方法 |
US8190355B2 (en) * | 2007-10-10 | 2012-05-29 | International Business Machines Corporation | Driving assistance and monitoring |
JP5132376B2 (ja) * | 2008-03-19 | 2013-01-30 | アルパイン株式会社 | 音声改善装置および音声改善方法 |
US8170787B2 (en) * | 2008-04-15 | 2012-05-01 | Caterpillar Inc. | Vehicle collision avoidance system |
EP2181880B1 (en) * | 2008-11-04 | 2012-01-04 | Volvo Car Corporation | Electronic arrangement for controlling a signal from an instrument |
FR2942342B1 (fr) * | 2009-02-13 | 2011-03-18 | Peugeot Citroen Automobiles Sa | Dispositif de fourniture d'images representatives de la zone avant situee devant un vehicule et de la position de ce dernier dans la zone situee en aval de cette zone avant, et procede associe |
JP2010258691A (ja) * | 2009-04-23 | 2010-11-11 | Sanyo Electric Co Ltd | 操縦支援装置 |
JP5299101B2 (ja) * | 2009-06-11 | 2013-09-25 | コニカミノルタ株式会社 | 周辺表示装置 |
JP2011080845A (ja) * | 2009-10-06 | 2011-04-21 | Topcon Corp | 3次元データ作成方法及び3次元データ作成装置 |
WO2011042876A1 (en) * | 2009-10-07 | 2011-04-14 | Alon Atsmon | Automatic content analysis method and system |
JP5135380B2 (ja) * | 2010-04-12 | 2013-02-06 | 住友重機械工業株式会社 | 処理対象画像生成装置、処理対象画像生成方法、及び操作支援システム |
JP5444139B2 (ja) * | 2010-06-29 | 2014-03-19 | クラリオン株式会社 | 画像のキャリブレーション方法および装置 |
US20130286163A1 (en) * | 2010-11-08 | 2013-10-31 | X6D Limited | 3d glasses |
EP2705664A2 (en) | 2011-05-03 | 2014-03-12 | Atsmon, Alon | Automatic image content analysis method and system |
JP5799631B2 (ja) * | 2011-07-20 | 2015-10-28 | 日産自動車株式会社 | 車両用画像生成装置及び車両用画像生成方法 |
FR2979311B1 (fr) * | 2011-08-31 | 2014-08-15 | Peugeot Citroen Automobiles Sa | Dispositif de traitement pour associer a un element graphique une couleur en fonction d'un niveau de risque de collision d'un obstacle avec une trajectoire future d'un vehicule, pour un systeme d'aide a la conduite |
DE102011082475A1 (de) * | 2011-09-12 | 2013-03-14 | Robert Bosch Gmbh | Fahrerassistenzsystem zur Unterstützung eines Fahrers in kollisionsrelevanten Situationen |
DE102011083770A1 (de) * | 2011-09-29 | 2013-04-04 | Bayerische Motoren Werke Aktiengesellschaft | Verfahren zur rechnergestützten Verarbeitung des Nahfeldes eines Fahrzeugs |
EP2581268B2 (en) * | 2011-10-13 | 2019-09-11 | Harman Becker Automotive Systems GmbH | Method of controlling an optical output device for displaying a vehicle surround view and vehicle surround view system |
JP5811804B2 (ja) * | 2011-11-24 | 2015-11-11 | トヨタ自動車株式会社 | 車両用周辺監視装置 |
TWI474173B (zh) * | 2012-02-21 | 2015-02-21 | Hon Hai Prec Ind Co Ltd | 行走輔助系統及行走輔助方法 |
US8768583B2 (en) | 2012-03-29 | 2014-07-01 | Harnischfeger Technologies, Inc. | Collision detection and mitigation systems and methods for a shovel |
KR101896715B1 (ko) * | 2012-10-31 | 2018-09-07 | 현대자동차주식회사 | 주변차량 위치 추적 장치 및 방법 |
US9877011B2 (en) | 2013-03-19 | 2018-01-23 | Hitachi Kokusai Electric Inc. | Image display apparatus and image display method |
DE102013107578A1 (de) | 2013-07-17 | 2015-01-22 | Dr. Ing. H.C. F. Porsche Aktiengesellschaft | Verfahren und System zum Simulieren von Kameraeigenschaften |
WO2017038166A1 (ja) * | 2015-08-28 | 2017-03-09 | ソニー株式会社 | 情報処理装置、情報処理方法及びプログラム |
CN108140309B (zh) * | 2015-11-20 | 2020-12-08 | 三菱电机株式会社 | 驾驶辅助装置、驾驶辅助***以及驾驶辅助方法 |
JP6555195B2 (ja) * | 2016-06-13 | 2019-08-07 | 株式会社デンソー | 画像生成装置 |
KR102462502B1 (ko) * | 2016-08-16 | 2022-11-02 | 삼성전자주식회사 | 스테레오 카메라 기반의 자율 주행 방법 및 그 장치 |
US10315649B2 (en) * | 2016-11-29 | 2019-06-11 | Ford Global Technologies, Llc | Multi-sensor probabilistic object detection and automated braking |
JP7157409B2 (ja) * | 2017-08-04 | 2022-10-20 | 日本電気株式会社 | 三次元画像生成装置、三次元画像生成方法、および三次元画像生成プログラム |
DE102017218090A1 (de) * | 2017-10-11 | 2019-04-11 | Robert Bosch Gmbh | Verfahren zur Darstellung von Bildern eines Kamerasystems eines Fahrzeugs |
JP2019129383A (ja) * | 2018-01-23 | 2019-08-01 | アルパイン株式会社 | 映像処理装置 |
FR3098620B1 (fr) * | 2019-07-12 | 2021-06-11 | Psa Automobiles Sa | Procédé de génération d’une représentation visuelle de l’environnement de conduite d’un véhicule |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2000064117A1 (en) * | 1999-04-16 | 2000-10-26 | Telefonaktiebolaget Lm Ericsson (Publ) | Data processing in an access node |
JP2002029346A (ja) * | 2000-07-14 | 2002-01-29 | Hiroyuki Yokozawa | 自動車の運転支援システム及び運転支援プログラムを記録した記録媒体 |
JP2002314991A (ja) * | 2001-02-09 | 2002-10-25 | Matsushita Electric Ind Co Ltd | 画像合成装置 |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1179958B1 (en) * | 1999-04-16 | 2012-08-08 | Panasonic Corporation | Image processing device and monitoring system |
JP2002362270A (ja) * | 2001-06-11 | 2002-12-18 | Matsushita Electric Ind Co Ltd | 運転支援装置 |
-
2004
- 2004-03-16 JP JP2004073887A patent/JP2005268847A/ja active Pending
-
2005
- 2005-02-24 WO PCT/JP2005/002977 patent/WO2005088971A1/ja active Application Filing
-
2006
- 2006-09-12 US US11/519,333 patent/US20070009137A1/en not_active Abandoned
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2000064117A1 (en) * | 1999-04-16 | 2000-10-26 | Telefonaktiebolaget Lm Ericsson (Publ) | Data processing in an access node |
JP2002029346A (ja) * | 2000-07-14 | 2002-01-29 | Hiroyuki Yokozawa | 自動車の運転支援システム及び運転支援プログラムを記録した記録媒体 |
JP2002314991A (ja) * | 2001-02-09 | 2002-10-25 | Matsushita Electric Ind Co Ltd | 画像合成装置 |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2010136263A (ja) * | 2008-12-08 | 2010-06-17 | Brother Ind Ltd | ヘッドマウントディスプレイ |
Also Published As
Publication number | Publication date |
---|---|
JP2005268847A (ja) | 2005-09-29 |
US20070009137A1 (en) | 2007-01-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2005088971A1 (ja) | 画像生成装置、画像生成方法、および画像生成プログラム | |
CN109360245B (zh) | 无人驾驶车辆多相机***的外参数标定方法 | |
EP3008708B1 (en) | Vision augmented navigation | |
JP5208203B2 (ja) | 死角表示装置 | |
JP4696248B2 (ja) | 移動体ナビゲート情報表示方法および移動体ナビゲート情報表示装置 | |
JP5157067B2 (ja) | 自動走行用マップ作成装置、及び自動走行装置。 | |
US7321386B2 (en) | Robust stereo-driven video-based surveillance | |
Li et al. | Easy calibration of a blind-spot-free fisheye camera system using a scene of a parking space | |
CN110174093A (zh) | 定位方法、装置、设备和计算机可读存储介质 | |
KR102200299B1 (ko) | 3d-vr 멀티센서 시스템 기반의 도로 시설물 관리 솔루션을 구현하는 시스템 및 그 방법 | |
JP2006250917A (ja) | 高精度cv演算装置と、この高精度cv演算装置を備えたcv方式三次元地図生成装置及びcv方式航法装置 | |
JP2004198212A (ja) | 移動体周辺監視装置 | |
KR102167835B1 (ko) | 영상 처리 방법 및 장치 | |
JP2003044996A (ja) | 障害物検出装置 | |
JP2011170599A (ja) | 屋外構造物計測装置及び屋外構造物計測方法 | |
JP2020126612A (ja) | スマートフォンを利用する歩行者を保護するための先端歩行者補助システムを提供する方法及び装置 | |
KR20200043252A (ko) | 차량의 부감 영상 생성 시스템 및 그 방법 | |
JP7372970B2 (ja) | 情報処理システム、センサシステム、情報処理方法及びプログラム | |
JP2000293693A (ja) | 障害物検出方法および装置 | |
CN108195359B (zh) | 空间数据的采集方法及*** | |
KR100550430B1 (ko) | 3차원 정보를 이용한 이동체의 주행 안내장치 및 방법 | |
JP2017182564A (ja) | 位置合わせ装置、位置合わせ方法及び位置合わせ用コンピュータプログラム | |
JP2005258792A (ja) | 画像生成装置、画像生成方法、および画像生成プログラム | |
Trusheim et al. | Cooperative image orientation considering dynamic objects | |
Kloeker et al. | Comparison of Camera-Equipped Drones and Infrastructure Sensors for Creating Trajectory Datasets of Road Users. |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A1 Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SM SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW |
|
AL | Designated countries for regional patents |
Kind code of ref document: A1 Designated state(s): BW GH GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LT LU MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
WWE | Wipo information: entry into national phase |
Ref document number: 11519333 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWW | Wipo information: withdrawn in national office |
Country of ref document: DE |
|
WWP | Wipo information: published in national office |
Ref document number: 11519333 Country of ref document: US |
|
122 | Ep: pct application non-entry in european phase |