WO2016157277A1 - Method and device for generating travelling environment abstract image - Google Patents

Method and device for generating travelling environment abstract image Download PDF

Info

Publication number
WO2016157277A1
WO2016157277A1 PCT/JP2015/059543 JP2015059543W WO2016157277A1 WO 2016157277 A1 WO2016157277 A1 WO 2016157277A1 JP 2015059543 W JP2015059543 W JP 2015059543W WO 2016157277 A1 WO2016157277 A1 WO 2016157277A1
Authority
WO
WIPO (PCT)
Prior art keywords
scene
abstract
environment
information
driving
Prior art date
Application number
PCT/JP2015/059543
Other languages
French (fr)
Japanese (ja)
Inventor
永井　徹
奥出　真理子
新　吉高
雅幸 竹村
Original Assignee
株式会社日立製作所
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社日立製作所 filed Critical 株式会社日立製作所
Priority to PCT/JP2015/059543 priority Critical patent/WO2016157277A1/en
Publication of WO2016157277A1 publication Critical patent/WO2016157277A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems

Definitions

  • the present invention aims to reduce the number of verification and evaluation work for the in-vehicle camera image processing logic for advanced driving support systems such as pedestrian detection and lane departure warning.
  • the present invention also relates to a traveling environment abstracted video creation method and apparatus that reduces the number of scene creation man-hours by using video from an in-vehicle camera as a video scene required when appropriately evaluating the driving characteristics of a driver using a driving simulator.
  • the video scene of the driving simulator had to be created manually from scratch based on the scenario created from scratch.
  • Abstraction scene generation means for generating an abstract scene based on the driving environment information from the onboard camera image, driving condition change means for automatically generating abstract scenes with different driving environment conditions, and the generated abstraction Abstract scene storage means for storing scenes in a reusable format and scene condition matrix generation means for confirming the collection status of the entire scene necessary for image recognition logic evaluation are provided.
  • the present invention provides an in-vehicle camera video input means for storing in-vehicle camera images in an image memory and a driving environment information input means for inputting in-vehicle driving environment information in a driving environment abstract video creating apparatus.
  • An abstract scene generating means for generating an abstract scene based on the in-vehicle camera video in the image memory and the driving environment information, a driving condition changing means for generating abstract scenes having different driving environment conditions, and generation
  • An abstract scene storage means for storing the abstract scenes is provided.
  • the present invention is characterized in that in the driving environment abstract video creation device, there is provided a scene condition matrix generation means capable of confirming the collection status of the entire scene required for image recognition logic evaluation.
  • the present invention is characterized in that, in the driving environment abstract video creation device, the abstract scene generation means generates an abstract scene using camera parameter information of an in-vehicle camera.
  • the present invention is the travel environment abstract video creation device, wherein the abstract scene generation means classifies information on road surfaces, vehicles, pedestrians, and features based on in-vehicle camera video information, and the travel environment information In other words, an abstract scene is generated in association with.
  • the present invention is the driving environment abstract video creation device, wherein the abstract scene generation means generates an abstract scene in which the driving environment conditions are changed based on the abstract scene stored in the abstract scene storage means. It is characterized by generating.
  • the present invention is the driving environment abstract video creation device, wherein the abstract scene storage means stores the abstract scene in association with the driving environment information obtained when the abstract scene is generated, A desired abstract scene is searched using the traveling environment information as a search condition.
  • the scene condition matrix generation unit can check the driving environment condition of the evaluation scene and the presence / absence of the evaluation scene, which are necessary for the image recognition logic evaluation of the in-vehicle camera in advance. Represent in tabular form and search the abstract scene storage means using the driving environment condition as a key. If there is an abstract scene that does not correspond even if it is searched, the abstract scene will be based on the driving environment condition of the abstract scene that does not apply. And the scene condition matrix information is updated.
  • the present invention is the driving environment abstract video creation device, wherein the scene condition matrix generating means generates the scene condition matrix based on the attribute information of the driver when the driving characteristics of the driver are evaluated using a driving simulator. Created and searched from the abstract scene storage means using the driving environment condition as a key. If there is an abstract scene that does not correspond even if the search is found, an abstract scene is created based on the driving environment condition of the abstract scene that does not apply.
  • the feature is to generate and update the scene condition matrix information.
  • the present invention is the driving environment abstract video creation device, wherein the scene condition matrix generating means generates the scene condition matrix based on the attribute information of the driver when the driving characteristics of the driver are evaluated using a driving simulator. Created and searched from the abstract scene storage means using the driving environment condition as a key, and when there is an abstract scene corresponding to the search, information in the scene condition matrix is abstracted corresponding to the selected matrix information It is characterized by displaying a scene.
  • the video of an in-vehicle camera is stored in an image memory
  • the driving environment information of the vehicle is input
  • the in-vehicle in the image memory Generating an abstract scene based on the camera image and the driving environment information; changing driving conditions for generating abstract scenes having different driving environment conditions; and storing the generated abstract scene in a storage means. It is characterized by.
  • the present invention is characterized in that, in the driving environment abstract video creation method, a scene condition matrix capable of confirming the collection status of the entire scene necessary for image recognition logic evaluation is generated.
  • the present invention is characterized in that the abstract scene is generated by using the camera parameter information of the in-vehicle camera in the driving environment abstract video creation method.
  • the abstract scene is classified into information on a road surface, a vehicle, a pedestrian, and a feature based on the in-vehicle camera video information, and is associated with the travel environment information. It is characterized by generating.
  • the present invention is characterized in that, in the driving environment abstract video creation method, an abstract scene in which the driving environment condition is changed is generated based on the accumulated abstract scene.
  • the present invention relates to a travel environment abstract video creation method, wherein the abstract scene and the travel environment information obtained when creating the abstract scene are stored in association with each other, and the travel environment information is used as a search condition. It is characterized by searching for abstract scenes.
  • the present invention is a travel environment abstract video creation method, which is represented in a tabular form in which a travel environment condition of an evaluation scene necessary for image recognition logic evaluation of an in-vehicle camera and a presence / absence of an evaluation scene can be confirmed in advance. If there is an abstract scene that does not correspond even if it is searched, the abstract condition is generated and the scene condition matrix information is updated based on the driving environment condition of the abstract scene that does not correspond Is.
  • the present invention provides a driving environment abstracted image creation method in which, when generating the scene condition matrix information, when evaluating the driving characteristics of the driver using a driving simulator, the scene is based on the driver attribute information. Create a condition matrix, search for the driving environment condition from the abstract scene storage means as a key, and if there is an abstract scene that does not correspond even if searched, based on the driving environment condition of the abstract scene not applicable, An abstract scene is generated and the scene condition matrix information is updated.
  • the present invention is a driving environment abstract video creation method, wherein scene condition matrix information is generated based on driver attribute information when evaluating driving characteristics of a driver using a driving simulator. Then, when the abstraction scene is searched from the abstraction scene storage means as a key and there is a corresponding abstraction scene at the time of the search, the information in the scene condition matrix is abstracted corresponding to the selected matrix information It is characterized by displaying a scene.
  • an abstract scene by CG is created based on the evaluation video that is actually taken and collected during the running of the evaluation video necessary for the verification work of the in-vehicle camera logic, and the number of lanes, the width, etc. It is possible to automatically generate evaluation videos with changed environmental conditions and store them in the video abstract scene storage database. As a result, it is possible to reduce the number of verification evaluation work steps for the in-vehicle camera image processing logic for the advanced driving support system.
  • evaluation video scenes in the database can be displayed or confirmed by classifying them into collected evaluation videos and uncollected evaluation videos according to the scene condition matrix for recognition logic. Can be taken out efficiently. Further, it is possible to prevent mistakes such as collecting double evaluation images under the same driving conditions. .
  • FIG. 1 is a diagram showing the overall system configuration of the present invention.
  • the traveling environment abstracted image creating apparatus used in the present invention includes an in-vehicle camera 11 mounted on a vehicle for the purpose of assisting safe driving of the vehicle, and a white line while the vehicle is traveling based on a traveling image obtained from the in-vehicle camera.
  • Road alignment information number of lanes, width, intersection shape, curve, weaving, merge, etc.
  • weather information time zone, weather, western sun, backlight, etc.
  • moving body information crossing pedestrians
  • Oncoming vehicles, etc. sensor information
  • sensor information travel location on map, travel speed, presence or absence of sudden braking, etc.
  • camera information installation position on vehicle body, lens used, imaging device, etc.
  • Driving environment information 10 abstract scene generation means 13 for generating an abstract scene based on in-vehicle camera video information and driving environment information, and reuse the generated abstract scene in association with the driving environment information It is possible to define the conditions of the evaluation scene necessary for the evaluation of the abstract scene storage means 14 to be stored in a possible format and the image processing recognition logic of the in-vehicle camera,
  • Fig. 2, Fig. 3, and Fig. 4 show the overall processing flow of the travel environment abstract video creation device.
  • Figure 2 shows the procedure for abstract scene storage processing.
  • in-vehicle camera video input processing the video of the in-vehicle camera is temporarily stored in the image memory in the controller 12 (20).
  • driving environment information input process road environment information including road linear information, weather information, moving body information, and vehicle dynamics information obtained from vehicle sensors included in the video information of the in-vehicle camera is manually or automatically (21).
  • abstract scene generation process an abstract scene is created based on the video information of the in-vehicle camera and the traveling environment information (22).
  • the generated abstract scene is stored in association with the travel environment information used when generating the abstract scene (23).
  • FIG. 3 shows the procedure of the abstraction scene generation process for evaluation.
  • a scene condition matrix for defining requirements required for evaluation of image recognition logic for in-vehicle cameras to be developed.
  • an abstract scene satisfying the requirement is searched from the abstract scene storage unit 14 (31).
  • the abstract scene is searched until all the requirements in the scene condition matrix are satisfied (32). If there is an abstract scene that cannot be found, the driving environment condition is changed to change the abstract scene.
  • automatic generation is attempted (34), if it is determined that the automatically generated abstract scene cannot be used as an evaluation scene, the in-vehicle camera video is newly collected in actual vehicle driving, or is mounted on the vehicle from another driving vehicle. An option for collecting camera images is also conceivable (35).
  • FIG. 4 shows the procedure of the abstract scene generation process.
  • a driving environment condition necessary for evaluation is set (40).
  • a confirmation moving image is created by the two-dimensional animation display function of the traffic flow simulator (41).
  • the camera conditions of the vehicle-mounted camera to be used are set (42), and a three-dimensional animation moving image is generated by a three-dimensional traffic flow simulator equipped with a camera model (43).
  • an abstract scene suitable for the driving environment condition can be obtained.
  • FIG. 5 is a diagram showing an example of abstract scene generation.
  • the image information 50 of the actual vehicle-mounted camera that is the basis is used to generate the abstract scene.
  • video information a video obtained by cutting out a near-miss scene recorded in a drive recorder is also targeted.
  • a pop-up image 51 of a pedestrian that causes a near-miss is included.
  • it is necessary to manually check the collected video such as background (features) 52 included in the video, pedestrians and oncoming vehicles (moving objects) 53, road environment information such as the road surface 54, and the like.
  • PreScan http://www.tass-safe.jp/prescan/index.htm
  • An abstract scene 55 is created by simulation software such as http://www.forum8.co.jp/).
  • FIG. 6 is a diagram showing an example of distance information utilization abstract scene generation.
  • a method of creating an abstract scene using distance information measured by a stereo camera, for example, obtaining information such as the number of lanes, the width 60, the distance to pedestrians and oncoming vehicles, and whether overtaking is prohibited 61 Can do.
  • the camera model is calculated using mathematical expressions that can simulate the camera installation position, lens, and image sensor, and the camera model is based on the calculation results.
  • a converted CG image 62 is generated.
  • FIG. 7 is a diagram showing an example of creating a travel environment condition change abstract scene.
  • the daytime driving scene is changed to the night scene 70, and is created.
  • parts such as road surfaces, buildings, vehicles, and pedestrians are already registered as parts, so when creating a night scene, select the objects that make up the sky part.
  • the night scene 70 can be created by repainting the night sky color texture.
  • an abstract scene can be created by selecting a road surface object and pasting a texture relating to the lane for increasing or decreasing the number of lanes and width.
  • FIG. 8 is a diagram showing an example of information stored in the abstract scene accumulation DB.
  • the created abstract scene is stored in the DB so that a desired scene can be searched in a reusable format using the driving environment information as a keyword.
  • the storage information in the DB is basically composed of an abstract scene 84, travel environment information 80, driving situation information 81, sensor information (own vehicle) 82, and camera parameters 83.
  • driving simulator When using the driving simulator to evaluate the driving characteristics of the driver, information including driving status information such as the age and sex of the driver as the subject is also stored in association with each other.
  • FIG. 9 is a diagram showing an example of the scene condition matrix.
  • a scene condition matrix is defined in which scene conditions necessary for logic evaluation are arranged.
  • the horizontal axis 90 represents the weather conditions such as night, rainy weather, western sun, backlight, etc.
  • the vertical axis 91 defined the conditions related to the driving path such as width, number of lanes, intersections, and curves.
  • FIG. 10 is a diagram showing an example of a scene condition matrix for recognition logic.
  • a scene condition matrix is prepared for different in-vehicle camera recognition logic.
  • a scene condition matrix corresponding to each driving support system such as a scene condition matrix 100 for a pedestrian detection system and a scene condition matrix 101 for a white line departure warning system is defined.
  • FIG. 11 is a diagram showing an example of information content of a scene condition matrix for recognition logic.
  • images collected by the in-vehicle camera, driving conditions for example, road width, number of lanes, presence / absence of intersections, curves, etc.
  • weather conditions shooting date / time, weather, western day
  • management in a table format, for example, it is easy to determine under which shooting conditions the evaluation video is collected or not. The collection status can be visualized.
  • each scene in the matrix corresponds to an actual abstract scene, you can check what the actual scene is when you select a scene item.
  • item scene 1 in the matrix (scene in which the number of lanes is increased in the daytime) is selected, it can be confirmed that the scene 110 is obtained by increasing the number of lanes from one lane to two lanes.
  • the scene 2 is the night scene 111 and the scene 5 is the backlight scene 112 having a wider width.
  • Scene 6 is a scene 113 in which the width is widened in the daytime.
  • the curve scene 115 at night is not collected as an image.
  • FIG. 12 is a diagram showing an example of changing the driving environment information condition.
  • the conditions of the driving environment conditions are changed by changing the conditions such as the number of lanes 120, the lane width 121, and the presence / absence of signalized intersection 122. Changes can be made.
  • FIG. 13 is a diagram illustrating an example of removing private information from the travel environment information. Since the video scene collected by the in-vehicle camera includes license plate information 132 and 134 of the traveling vehicles 130 and 131 and personal information that can identify the pedestrians, an abstract scene is CG created from the collected video. At this time, it is necessary to remove the private information 133, 135, or to remove the private information in advance based on the collected video.
  • FIG. 14 is a diagram showing an example in which the traveling environment information collection status is displayed on the map.
  • the video abstract scenes 140, 141, and 142 stored in the video abstract scene storage database of FIG. 8 are collected and created can be displayed on the map.
  • Such position information is required for the on-vehicle camera logic evaluation.
  • FIG. 15 is a diagram illustrating an example in which the travel environment information is used for a driving simulator.
  • driving vehicle information such as at least the age, gender, and driving history of the driving simulator subject is set (150), and a scene that matches the set conditions is searched from the video abstraction scene storage database of FIG. (151)
  • the searched scene is set in the driving simulator, and the driving characteristics are diagnosed (152).
  • the diagnosis result when there is a scene that is not good, the scene is searched from the video abstract scene storage database (153), and the driving characteristics are repeatedly diagnosed by the driving simulator. If there is no problem in the diagnosis result, the driving vehicle information of the subject and the usage history information of the usage scene are recorded as driving characteristic information and used for the driver's safe driving support (154).

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Automation & Control Theory (AREA)
  • Traffic Control Systems (AREA)
  • Image Processing (AREA)

Abstract

Conventionally, in evaluation of image processing logics for advanced driving assistance systems using vehicle-mounted cameras, it is necessary to actually drive vehicles to manually photograph a large number of evaluation images under different travelling environment conditions by using human wave tactics, and evaluate the images. Thus, verification man-hours are high. To reduce the man-hours is a significant problem. The present invention comprises: an abstract scene generating means that generates abstract scenes on the basis of travelling environment information from images taken by a vehicle-mounted camera; a travelling condition changing means that internally performs automatic generation of abstract scenes for different travelling environment conditions; an abstract scene accumulating means that stores the generated abstract scenes in a format that allows reuse of the generated abstract scenes; and a scene condition matrix generating means that can check the entire collection status of scenes required for evaluation of image recognition logics, so that evaluation scenes required for evaluation of image recognition logics for a vehicle-mounted camera can be collected efficiently. Further, since the abstract scenes can be used as input images in a driving simulator, scenes in which a driver cannot drive well can be efficiently generated, for example. According to the present invention, since abstract scenes of different travelling environment conditions can be automatically generated on the basis of images taken by a vehicle-mounted camera, the man-hours can be reduced for developing image recognition logics for a vehicle-mounted camera or for creating input scenes for a driving simulator.

Description

走行環境抽象化映像作成方法及び装置Driving environment abstraction video creation method and apparatus
　本発明は、歩行者検知や車線逸脱警報など先進運転支援システム向け車載カメラ画像処理ロジックの検証評価作業工数低減を目的とする。また、ドライビングシミュレータによりドライバの運転特性を適正に評価する際に必要となる映像シーンを車載カメラの映像を利用することでシーン作成工数の低減を図る走行環境抽象化映像作成方法及び装置に関する。 The present invention aims to reduce the number of verification and evaluation work for the in-vehicle camera image processing logic for advanced driving support systems such as pedestrian detection and lane departure warning. The present invention also relates to a traveling environment abstracted video creation method and apparatus that reduces the number of scene creation man-hours by using video from an in-vehicle camera as a video scene required when appropriately evaluating the driving characteristics of a driver using a driving simulator.
　従来車載カメラを用いた先進運転支援システム向け画像処理ロジックの評価は、実車走行により異なる走行環境条件において人手により大量の評価映像を人海戦術的に撮影し、評価していた。 Conventionally, evaluation of image processing logic for advanced driving support systems using in-vehicle cameras has been performed by manually shooting a large number of evaluation images under human driving tactics under different driving environment conditions.
　また、ドライビングシミュレータの映像シーンは、基本的に作成したシナリオをもとにゼロからシミュレータへの入力シーンを人手により作成しなければならなかった。 Also, the video scene of the driving simulator had to be created manually from scratch based on the scenario created from scratch.
　実車走行により異なる走行環境条件において人手により大量の評価映像を人海戦術的に撮影し、評価しなければならないため、検証作業工数低減が重要な課題となる。また、ドライビングシミュレータにより高齢者や運転に不慣れなドライバに対し、危険予知訓練を目的とした運転訓練をする場合、個々のドライバに適した訓練シーンを用意する必要があるが、現状映像シーンの作成は人手により作成するため一般的なシーンでしか訓練できず、必ずしもドライバが苦手なシーンにより訓練できるとは限らないという問題がある。 Since a large number of evaluation images must be manually photographed and evaluated manually under different driving environment conditions depending on actual vehicle travel, reducing the number of verification work is an important issue. In addition, when driving training for the purpose of risk prediction training for elderly people and drivers who are unfamiliar with driving using a driving simulator, it is necessary to prepare a training scene suitable for each driver. However, since it is created manually, it can be trained only in a general scene, and it is not always possible to train in a scene that the driver is not good at.
　車載カメラの映像から走行環境情報をもとに抽象化シーンを生成する抽象化シーン生成手段と、走行環境条件の異なる抽象化シーンを内部で自動生成する走行条件変更手段と、生成された抽象化シーンを再利用可能な形式で格納する抽象化シーン蓄積手段と、画像認識ロジック評価に必要となるシーン全体の収集状況を確認できるシーン条件マトリクス生成手段を設ける。 Abstraction scene generation means for generating an abstract scene based on the driving environment information from the onboard camera image, driving condition change means for automatically generating abstract scenes with different driving environment conditions, and the generated abstraction Abstract scene storage means for storing scenes in a reusable format and scene condition matrix generation means for confirming the collection status of the entire scene necessary for image recognition logic evaluation are provided.
　このような処理により、走行環境の異なる評価条件において収集できていない抽象化シーンについても、内部で自動生成することができるため車載カメラ用画像認識ロジックの評価作業工数低減ができる。同様にドライビングシミュレータの特に、ドライバが苦手とする運転シーンで作成されていないシーンについても、走行環境条件をもとに内部で自動生成することができるため、入力シーン作成の工数低減ができる。 By such processing, abstract scenes that cannot be collected under different evaluation conditions in the driving environment can be automatically generated internally, so that it is possible to reduce the number of evaluation work steps for image recognition logic for in-vehicle cameras. Similarly, especially in the driving simulator, a scene that is not created in the driving scene that the driver is not good at can be automatically generated internally based on the driving environment condition, so that the man-hour for creating the input scene can be reduced.
　上記課題を達成するために、本発明は走行環境抽象化映像作成装置において、車載カメラの映像を画像メモリに蓄積する車載カメラ映像入力手段と、車両の走行環境情報を入力する走行環境情報入力手段と、前記画像メモリ内の車載カメラ映像と該走行環境情報をもとに抽象化シーンを生成する抽象化シーン生成手段と、走行環境条件の異なる抽象化シーンを生成する走行条件変更手段と、生成された抽象化シーンを格納する抽象化シーン蓄積手段を備えたことを特徴とするものである。 In order to achieve the above object, the present invention provides an in-vehicle camera video input means for storing in-vehicle camera images in an image memory and a driving environment information input means for inputting in-vehicle driving environment information in a driving environment abstract video creating apparatus. An abstract scene generating means for generating an abstract scene based on the in-vehicle camera video in the image memory and the driving environment information, a driving condition changing means for generating abstract scenes having different driving environment conditions, and generation An abstract scene storage means for storing the abstract scenes is provided.
　更に、本発明は走行環境抽象化映像作成装置において、画像認識ロジック評価に必要となるシーン全体の収集状況を確認できるシーン条件マトリクス生成手段を備えたことを特徴とするものである。 Furthermore, the present invention is characterized in that in the driving environment abstract video creation device, there is provided a scene condition matrix generation means capable of confirming the collection status of the entire scene required for image recognition logic evaluation.
　更に、本発明は走行環境抽象化映像作成装置において、前記抽象化シーン生成手段は、車載カメラのカメラパラメタ情報を用いて抽象化シーンを生成することを特徴とするものである。 Furthermore, the present invention is characterized in that, in the driving environment abstract video creation device, the abstract scene generation means generates an abstract scene using camera parameter information of an in-vehicle camera.
　更に、本発明は走行環境抽象化映像作成装置において、前記抽象化シーン生成手段は、車載カメラ映像情報をもとに路面や、車両、歩行者、地物に関する情報に分類し、前記走行環境情報と関連付けて抽象化シーンを生成することを特徴とするものである。 Furthermore, the present invention is the travel environment abstract video creation device, wherein the abstract scene generation means classifies information on road surfaces, vehicles, pedestrians, and features based on in-vehicle camera video information, and the travel environment information In other words, an abstract scene is generated in association with.
　更に、本発明は走行環境抽象化映像作成装置において、前記抽象化シーン生成手段は、抽象化シーン蓄積手段に蓄積された抽象化シーンをもとに、前記走行環境条件を変更した抽象化シーンを生成することを特徴とするものである。 Further, the present invention is the driving environment abstract video creation device, wherein the abstract scene generation means generates an abstract scene in which the driving environment conditions are changed based on the abstract scene stored in the abstract scene storage means. It is characterized by generating.
　更に、本発明は走行環境抽象化映像作成装置において、前記抽象化シーン蓄積手段は、前記抽象化シーンと、前記抽象化シーンを作成する際に得られる走行環境情報とを関連付けて蓄積し、前記走行環境情報を検索条件として所望の抽象化シーンを検索することを特徴とするものである。 Furthermore, the present invention is the driving environment abstract video creation device, wherein the abstract scene storage means stores the abstract scene in association with the driving environment information obtained when the abstract scene is generated, A desired abstract scene is searched using the traveling environment information as a search condition.
　更に、本発明は走行環境抽象化映像作成装置において、前記シーン条件マトリクス生成手段は、予め車載カメラの画像認識ロジック評価に必要となる評価シーンの走行環境条件と、評価シーンの有無が確認可能な表形式で表し、抽象化シーン蓄積手段から走行環境条件をキーに検索し、検索しても該当しない抽象化シーンがある場合、該当しない抽象化シーンの走行環境条件をもとに、抽象化シーンを生成しシーン条件マトリクス情報を更新すること特徴とするものである。 Furthermore, in the driving environment abstract video creation apparatus according to the present invention, the scene condition matrix generation unit can check the driving environment condition of the evaluation scene and the presence / absence of the evaluation scene, which are necessary for the image recognition logic evaluation of the in-vehicle camera in advance. Represent in tabular form and search the abstract scene storage means using the driving environment condition as a key. If there is an abstract scene that does not correspond even if it is searched, the abstract scene will be based on the driving environment condition of the abstract scene that does not apply. And the scene condition matrix information is updated.
　更に、本発明は走行環境抽象化映像作成装置において、前記シーン条件マトリクス生成手段は、ドライバの運転特性の評価をドライビングシミュレータを用いて実施する場合、ドライバの属性情報をもとにシーン条件マトリクスを作成し、前記抽象化シーン蓄積手段から走行環境条件をキーに検索し、検索しても該当しない抽象化シーンがある場合、該当しない抽象化シーンの走行環境条件をもとに、抽象化シーンを生成しシーン条件マトリクス情報を更新すること特徴とするものである。 Further, the present invention is the driving environment abstract video creation device, wherein the scene condition matrix generating means generates the scene condition matrix based on the attribute information of the driver when the driving characteristics of the driver are evaluated using a driving simulator. Created and searched from the abstract scene storage means using the driving environment condition as a key. If there is an abstract scene that does not correspond even if the search is found, an abstract scene is created based on the driving environment condition of the abstract scene that does not apply. The feature is to generate and update the scene condition matrix information.
　更に、本発明は走行環境抽象化映像作成装置において、前記シーン条件マトリクス生成手段は、ドライバの運転特性の評価をドライビングシミュレータを用いて実施する場合、ドライバの属性情報をもとにシーン条件マトリクスを作成し、前記抽象化シーン蓄積手段から走行環境条件をキーに検索し、検索した際に該当する抽象化シーンがある場合、シーン条件マトリクス内の情報を選択された該マトリクス情報に対応する抽象化シーンを表示すること特徴とするものである。 Further, the present invention is the driving environment abstract video creation device, wherein the scene condition matrix generating means generates the scene condition matrix based on the attribute information of the driver when the driving characteristics of the driver are evaluated using a driving simulator. Created and searched from the abstract scene storage means using the driving environment condition as a key, and when there is an abstract scene corresponding to the search, information in the scene condition matrix is abstracted corresponding to the selected matrix information It is characterized by displaying a scene.
　また、上記課題を達成するために、本発明は走行環境抽象化映像作成方法において、車載カメラの映像を画像メモリに蓄積すること、車両の走行環境情報を入力すること、前記画像メモリ内の車載カメラ映像と該走行環境情報をもとに抽象化シーンを生成すること、走行環境条件の異なる抽象化シーンを生成する走行条件を変更すること、生成された抽象化シーンを記憶手段に格納することを特徴とするものである。 In order to achieve the above object, according to the present invention, in a running environment abstracted video creation method, the video of an in-vehicle camera is stored in an image memory, the driving environment information of the vehicle is input, the in-vehicle in the image memory Generating an abstract scene based on the camera image and the driving environment information; changing driving conditions for generating abstract scenes having different driving environment conditions; and storing the generated abstract scene in a storage means. It is characterized by.
　更に、本発明は走行環境抽象化映像作成方法において、画像認識ロジック評価に必要となるシーン全体の収集状況を確認できるシーン条件マトリクスを生成することを特徴とするものである。 Furthermore, the present invention is characterized in that, in the driving environment abstract video creation method, a scene condition matrix capable of confirming the collection status of the entire scene necessary for image recognition logic evaluation is generated.
　更に、本発明は走行環境抽象化映像作成方法において、前記車載カメラのカメラパラメタ情報を用いて前記抽象化シーンを生成することを特徴とするものである。 Furthermore, the present invention is characterized in that the abstract scene is generated by using the camera parameter information of the in-vehicle camera in the driving environment abstract video creation method.
　更に、本発明は走行環境抽象化映像作成方法において、前記車載カメラ映像情報をもとに路面や、車両、歩行者、地物に関する情報に分類し、前記走行環境情報と関連付けて抽象化シーンを生成することを特徴とするものである。 Further, according to the present invention, in the traveling environment abstract video creation method, the abstract scene is classified into information on a road surface, a vehicle, a pedestrian, and a feature based on the in-vehicle camera video information, and is associated with the travel environment information. It is characterized by generating.
　更に、本発明は走行環境抽象化映像作成方法において、蓄積された前記抽象化シーンをもとに、前記走行環境条件を変更した抽象化シーンを生成することを特徴とするものである。 Further, the present invention is characterized in that, in the driving environment abstract video creation method, an abstract scene in which the driving environment condition is changed is generated based on the accumulated abstract scene.
　更に、本発明は走行環境抽象化映像作成方法において、前記抽象化シーンと、前記抽象化シーンを作成する際に得られる走行環境情報とを関連付けて蓄積し、前記走行環境情報を検索条件として所望の抽象化シーンを検索することを特徴とするものである。 Furthermore, the present invention relates to a travel environment abstract video creation method, wherein the abstract scene and the travel environment information obtained when creating the abstract scene are stored in association with each other, and the travel environment information is used as a search condition. It is characterized by searching for abstract scenes.
　更に、本発明は走行環境抽象化映像作成方法において、予め車載カメラの画像認識ロジック評価に必要となる評価シーンの走行環境条件と、評価シーンの有無が確認可能な表形式で表し、走行環境条件をキーに検索し、検索しても該当しない抽象化シーンがある場合、該当しない抽象化シーンの走行環境条件をもとに、抽象化シーンを生成しシーン条件マトリクス情報を更新すること特徴とするものである。 Furthermore, the present invention is a travel environment abstract video creation method, which is represented in a tabular form in which a travel environment condition of an evaluation scene necessary for image recognition logic evaluation of an in-vehicle camera and a presence / absence of an evaluation scene can be confirmed in advance. If there is an abstract scene that does not correspond even if it is searched, the abstract condition is generated and the scene condition matrix information is updated based on the driving environment condition of the abstract scene that does not correspond Is.
　更に、本発明は走行環境抽象化映像作成方法において、前記シーン条件マトリクス情報を生成する際に、ドライバの運転特性の評価をドライビングシミュレータを用いて実施する場合、ドライバの属性情報をもとにシーン条件マトリクスを作成し、前記抽象化シーンの記憶手段から走行環境条件をキーに検索し、検索しても該当しない抽象化シーンがある場合、該当しない抽象化シーンの走行環境条件をもとに、抽象化シーンを生成しシーン条件マトリクス情報を更新すること特徴とするものである。 Further, the present invention provides a driving environment abstracted image creation method in which, when generating the scene condition matrix information, when evaluating the driving characteristics of the driver using a driving simulator, the scene is based on the driver attribute information. Create a condition matrix, search for the driving environment condition from the abstract scene storage means as a key, and if there is an abstract scene that does not correspond even if searched, based on the driving environment condition of the abstract scene not applicable, An abstract scene is generated and the scene condition matrix information is updated.
　更に、本発明は走行環境抽象化映像作成方法において、シーン条件マトリクス情報は、ドライバの運転特性の評価をドライビングシミュレータを用いて実施する場合、ドライバの属性情報をもとにシーン条件マトリクス情報を作成し、前記抽象化シーンの記憶手段から走行環境条件をキーに検索し、検索した際に該当する抽象化シーンがある場合、シーン条件マトリクス内の情報を選択された該マトリクス情報に対応する抽象化シーンを表示すること特徴とするものである。 Furthermore, the present invention is a driving environment abstract video creation method, wherein scene condition matrix information is generated based on driver attribute information when evaluating driving characteristics of a driver using a driving simulator. Then, when the abstraction scene is searched from the abstraction scene storage means as a key and there is a corresponding abstraction scene at the time of the search, the information in the scene condition matrix is abstracted corresponding to the selected matrix information It is characterized by displaying a scene.
　本発明では、車載カメラロジックの検証作業に必要となる評価映像を、実際に走行中に撮影し収集した評価映像をもとに、ＣＧによる抽象化シーンを作成し、車線数や幅員などの走行環境条件を変更した評価映像も自動生成し、映像抽象化シーン蓄積データベースに蓄えることが実現できる。これにより、先進運転支援システム向け車載カメラ画像処理ロジックの検証評価作業工数の低減を実現できる。 In the present invention, an abstract scene by CG is created based on the evaluation video that is actually taken and collected during the running of the evaluation video necessary for the verification work of the in-vehicle camera logic, and the number of lanes, the width, etc. It is possible to automatically generate evaluation videos with changed environmental conditions and store them in the video abstract scene storage database. As a result, it is possible to reduce the number of verification evaluation work steps for the in-vehicle camera image processing logic for the advanced driving support system.
　さらに、データベース内の評価映像シーンは、認識ロジック用シーン条件マトリクスにより、収集済みの評価映像と未収集の評価映像とに分類して表示または、確認ができるため車載カメラロジックの評価に必要なシーンが効率よく取り出すことができる。また、同じ走行条件の評価映像を二重に収集するなどのミスを未然に防ぐことが実現できる。。 In addition, the evaluation video scenes in the database can be displayed or confirmed by classifying them into collected evaluation videos and uncollected evaluation videos according to the scene condition matrix for recognition logic. Can be taken out efficiently. Further, it is possible to prevent mistakes such as collecting double evaluation images under the same driving conditions. .
本発明の全体システム構成を示す図である。It is a figure which shows the whole system configuration | structure of this invention. 抽象化シーン蓄積処理フローを示す図である。It is a figure which shows the abstract scene accumulation | storage process flow. 評価用抽象化シーン生成処理フローを示す図である。It is a figure which shows the abstract scene production | generation process for evaluation. 抽象化シーン生成処理フローを示す図である。It is a figure which shows the abstraction scene production | generation processing flow. 抽象化シーン生成の例を示す図である。It is a figure which shows the example of an abstract scene production | generation. 抽象化シーン生成の例を示す図である。It is a figure which shows the example of an abstract scene production | generation. 走行環境条件変更抽象化シーン作成の例を示す図である。It is a figure which shows the example of driving environment condition change abstraction scene creation. 抽象化シーン蓄積ＤＢの格納情報の例を示す図である。It is a figure which shows the example of the storage information of abstract scene accumulation | storage DB. 認識ロジック用シーン条件マトリクスの例を示す図である。It is a figure which shows the example of the scene condition matrix for recognition logic. 認識ロジック用シーン条件マトリクスの例を示す図である。It is a figure which shows the example of the scene condition matrix for recognition logic. 認識ロジック用シーン条件マトリクスの情報内容の例を示す図である。It is a figure which shows the example of the information content of the scene condition matrix for recognition logic. 走行環境情報条件の変更例を示す図である。It is a figure which shows the example of a change of driving environment information conditions. 走行環境情報からプライベート情報を除去する例を示す図である。It is a figure which shows the example which removes private information from driving environment information. マップ上で走行環境情報の収集状況を表示した例を示す図である。It is a figure which shows the example which displayed the collection condition of traveling environment information on the map. 走行環境情報をドライビングシミュレータに利用する例を示す図である。It is a figure which shows the example which utilizes driving environment information for a driving simulator.
　本発明の実施例を以下図面を用いて説明する。図１は本発明の全体システム構成を示す図である。 Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing the overall system configuration of the present invention.
　本発明で使用する走行環境抽象化映像作成装置は、車両走行の安全運転支援を目的として車両に搭載された車載カメラ１１と、車載カメラから得られる走行映像をもとに車両が走行中に白線逸脱した際に警告するシステムや、車線変更時に変更先の車線後方から車両が急接近した際警告する後方車接近警報システムなど、先進安全運転支援システムを実現する画像処理認識ロジックを搭載したコントローラ１２と、車両走行中の道路線形情報（車線数、幅員、交差点形状、カーブ、織り込み、合流など）と、気象情報（時間帯、天候、西日、逆光など）と、移動体情報（横断歩行者、対向車両など）と、センサ情報（地図上の走行位置、走行速度、急ブレーキの有無など）と、カメラ情報（車体への取付け位置、使用レンズ、撮像素子など）からなる走行環境情報１０と、車載カメラの映像情報と走行環境情報をもとに抽象化シーンを生成する抽象化シーン生成手段１３と、生成された抽象化シーンを走行環境情報と関連付けて再利用可能な形式で蓄積する抽象化シーン蓄積手段１４と、車載カメラの画像処理認識ロジックの評価に必要となる評価シーンの条件を定義し、評価条件に適合した評価シーンの有無を確認することができるシーン条件マトリクス生成手段１５を備えている。
 以下、本発明の実施の形態を図面を参照して説明する。 The traveling environment abstracted image creating apparatus used in the present invention includes an in-vehicle camera 11 mounted on a vehicle for the purpose of assisting safe driving of the vehicle, and a white line while the vehicle is traveling based on a traveling image obtained from the in-vehicle camera. A controller 12 equipped with an image processing recognition logic that realizes an advanced safe driving support system, such as a system that warns when deviating, or a rear vehicle approach warning system that warns when a vehicle suddenly approaches from the rear of the lane to be changed when changing lanes Road alignment information (number of lanes, width, intersection shape, curve, weaving, merge, etc.), weather information (time zone, weather, western sun, backlight, etc.) and moving body information (crossing pedestrians) , Oncoming vehicles, etc.), sensor information (travel location on map, travel speed, presence or absence of sudden braking, etc.), camera information (installation position on vehicle body, lens used, imaging device, etc.) Driving environment information 10, abstract scene generation means 13 for generating an abstract scene based on in-vehicle camera video information and driving environment information, and reuse the generated abstract scene in association with the driving environment information It is possible to define the conditions of the evaluation scene necessary for the evaluation of the abstract scene storage means 14 to be stored in a possible format and the image processing recognition logic of the in-vehicle camera, and confirm whether there is an evaluation scene that matches the evaluation condition. A scene condition matrix generation means 15 is provided.
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
　図２、図３、図４に走行環境抽象化映像作成装置の全体処理の流れを示す。 Fig. 2, Fig. 3, and Fig. 4 show the overall processing flow of the travel environment abstract video creation device.
　図2は抽象化シーン蓄積処理の手順を示す。 Figure 2 shows the procedure for abstract scene storage processing.
　抽象化シーンの作成からシーンの蓄積処理を行うには、まず、車載カメラ映像入力処理では車載カメラの映像をコントローラ１２内の画像メモリに一旦格納する（２０）。次に、走行環境情報入力処理では車載カメラの映像情報に含まれる道路線形情報や、気象情報、移動体情報、更には、車両センサから得られる車両の動態情報からなる走行環境情報を人手あるいは自動で取得する（２１）。さらに、抽象化シーン生成処理では車載カメラの映像情報と走行環境情報をもとに抽象化シーンを作成する（２２）。最後に、抽象化シーン蓄積処理では、生成された抽象化シーンを抽象化シーン生成時に使用した走行環境情報と関連付けて蓄積する（２３）。 In order to perform scene accumulation processing from creation of an abstract scene, first, in-vehicle camera video input processing, the video of the in-vehicle camera is temporarily stored in the image memory in the controller 12 (20). Next, in the driving environment information input process, road environment information including road linear information, weather information, moving body information, and vehicle dynamics information obtained from vehicle sensors included in the video information of the in-vehicle camera is manually or automatically (21). Further, in the abstract scene generation process, an abstract scene is created based on the video information of the in-vehicle camera and the traveling environment information (22). Finally, in the abstract scene storage process, the generated abstract scene is stored in association with the travel environment information used when generating the abstract scene (23).
　次に、図3は評価用抽象化シーン生成処理の手順を示す。
蓄積された抽象化シーンから画像処理認識ロジックの評価に必要な抽象化シーンを検索するには、まず、今回開発対象の車載カメラ用画像認識ロジック評価に必要となる要件定義のためのシーン条件マトリクスを入力する（３０）。次に、抽象化シーン蓄積部１４より要件を満足する抽象化シーンを検索する（３１）。ここで、シーン条件マトリクス内の全ての要件が満足するまで抽象化シーンの検索を行うが（３２）、見つからない抽象化シーンがある場合には走行環境条件を変更して該当する抽象化シーンの自動生成を試みるが（３４）、自動生成した抽象化シーンが評価シーンとして利用できないと判断した場合には、車載カメラ映像を実車走行で新規で収集するかまたは、他の走行中の車両から車載カメラ映像の収集を行う選択肢も考えられる（３５）。 Next, FIG. 3 shows the procedure of the abstraction scene generation process for evaluation.
To search for abstract scenes necessary for evaluating image processing recognition logic from the stored abstract scenes, first, a scene condition matrix for defining requirements required for evaluation of image recognition logic for in-vehicle cameras to be developed. (30). Next, an abstract scene satisfying the requirement is searched from the abstract scene storage unit 14 (31). Here, the abstract scene is searched until all the requirements in the scene condition matrix are satisfied (32). If there is an abstract scene that cannot be found, the driving environment condition is changed to change the abstract scene. Although automatic generation is attempted (34), if it is determined that the automatically generated abstract scene cannot be used as an evaluation scene, the in-vehicle camera video is newly collected in actual vehicle driving, or is mounted on the vehicle from another driving vehicle. An option for collecting camera images is also conceivable (35).
　次に、図4は抽象化シーン生成処理の手順を示す。
抽象化シーン作成の流れとしては、まず、評価に必要な走行環境条件を設定する（４０）。次に与えられた走行環境条件をもとに、交通流シミュレータの２次元のアニメーション表示機能により確認用の動画を作成する（４１）。さらに、使用予定の車載カメラのカメラ条件を設定し(４２)、カメラモデルを実装した３次元交通流シミュレータにより、３次元アニメーション動画を生成する（４３）。この結果、走行環境条件に適合した抽象化シーンを得ることができる。 Next, FIG. 4 shows the procedure of the abstract scene generation process.
As a flow of creating an abstract scene, first, a driving environment condition necessary for evaluation is set (40). Next, based on the given driving environment conditions, a confirmation moving image is created by the two-dimensional animation display function of the traffic flow simulator (41). Furthermore, the camera conditions of the vehicle-mounted camera to be used are set (42), and a three-dimensional animation moving image is generated by a three-dimensional traffic flow simulator equipped with a camera model (43). As a result, an abstract scene suitable for the driving environment condition can be obtained.
　図５は、抽象化シーン生成の例を示す図である。 FIG. 5 is a diagram showing an example of abstract scene generation.
　抽象化シーンの生成にはもとになる実際の車載カメラの映像情報５０を利用する。映像情報としては、ドライブレコーダに記録されたヒヤリハットシーンを切り出した映像も対象となる。この場合、ヒヤリハットの要因となる歩行者の飛び出し画像５１が含まれる。抽象化シーン作成には、収集された映像を確認しながら人手により、例えば映像中に含まれる背景(地物)５２、歩行者や対向車両(移動体)５３、路面５４などの走行環境情報の構成要素に解析して分解した後に、解析した情報をもとにカメラモデルを実装したPreScan（http://www.tass-safe.jp/prescan/index.htm）や、UC-win/Road(http://www.forum8.co.jp/)などのシミュレーションソフトにより抽象化シーン５５を作成する。 The image information 50 of the actual vehicle-mounted camera that is the basis is used to generate the abstract scene. As video information, a video obtained by cutting out a near-miss scene recorded in a drive recorder is also targeted. In this case, a pop-up image 51 of a pedestrian that causes a near-miss is included. For creating an abstract scene, it is necessary to manually check the collected video, such as background (features) 52 included in the video, pedestrians and oncoming vehicles (moving objects) 53, road environment information such as the road surface 54, and the like. After analyzing and disassembling the components, PreScan (http://www.tass-safe.jp/prescan/index.htm) that implements the camera model based on the analyzed information and UC-win / Road ( An abstract scene 55 is created by simulation software such as http://www.forum8.co.jp/).
　図６は、距離情報利用抽象化シーン生成の例を示す図である。抽象化シーンを作成する方法として、ステレオカメラにより計測される距離情報を用いて例えば、車線数や幅員６０、歩行者や対向車までの距離、さらには追越し禁止の有無６１などの情報を得ることができる。抽象化シーン作成では、前述のシーン解析により得られる走行環境情報を用いて、カメラの設置位置やレンズ及び撮像素子について模擬できる数式で表現したカメラモデルにより計算し、計算結果をもとにカメラモデル変換を行ったＣＧ映像６２を生成する。 FIG. 6 is a diagram showing an example of distance information utilization abstract scene generation. As a method of creating an abstract scene, using distance information measured by a stereo camera, for example, obtaining information such as the number of lanes, the width 60, the distance to pedestrians and oncoming vehicles, and whether overtaking is prohibited 61 Can do. In creating an abstract scene, using the driving environment information obtained from the scene analysis described above, the camera model is calculated using mathematical expressions that can simulate the camera installation position, lens, and image sensor, and the camera model is based on the calculation results. A converted CG image 62 is generated.
　図７は、走行環境条件変更抽象化シーン作成の例を示す図である。オリジナルの抽象化シーン５５をもとに走行環境情報変更の一例として昼間の走行シーンを夜間シーン７０に変更して作成したものである。ここで、オリジナルの走行シーンを作成する際に既に路面や建物、車両、歩行者などオブジェクトとして部品登録されているので夜間のシーンを作成する場合には、空の部分を構成するオブジェクトを選択し、夜間の空の色のテクスチャに塗り替えることで夜間シーン７０を作成することができる。他の走行環境条件についても同様に車線数や幅員の増減についても、路面のオブジェクトを選択し車線に関するテクスチャを貼り付けることで抽象化シーンを作成することができる。ここで、車線数や幅員が少ない状態から増やす場合には、予め路面オブジェクトを構成する３Ｄポリゴンモデルの形状をポリゴンモデル編集ソフトを使って必要に応じて作成しておく必要がある。その後、作成した３Ｄポリゴンモデル上に路面テクスチャを貼り付けるという手順になる。 FIG. 7 is a diagram showing an example of creating a travel environment condition change abstract scene. As an example of changing the driving environment information based on the original abstract scene 55, the daytime driving scene is changed to the night scene 70, and is created. Here, when creating an original driving scene, parts such as road surfaces, buildings, vehicles, and pedestrians are already registered as parts, so when creating a night scene, select the objects that make up the sky part. The night scene 70 can be created by repainting the night sky color texture. Similarly, with regard to other driving environment conditions, an abstract scene can be created by selecting a road surface object and pasting a texture relating to the lane for increasing or decreasing the number of lanes and width. Here, when the number of lanes or the width is increased from a small state, it is necessary to create the shape of the 3D polygon model constituting the road surface object in advance using the polygon model editing software as necessary. Thereafter, the road surface texture is pasted on the created 3D polygon model.
　図８は、抽象化シーン蓄積ＤＢの格納情報の例を示す図である。作成した抽象化シーンは走行環境情報をキーワードとして、所望のシーンが再利用可能な形式で検索できるようにＤＢ内に格納される。ＤＢ内の格納情報は、抽象化シーン８４と、走行環境情報８０と、運転状況情報８１と、センサ情報(自車)８２と、カメラパラメタ８３で基本的に構成されるが、抽象化シーンを用いてドライビングシミュレータによりドライバの運転特性を評価する際には、被験者となるドライバの年齢や性別などの運転状況情報も含めた情報についても関連付けて格納する。 FIG. 8 is a diagram showing an example of information stored in the abstract scene accumulation DB. The created abstract scene is stored in the DB so that a desired scene can be searched in a reusable format using the driving environment information as a keyword. The storage information in the DB is basically composed of an abstract scene 84, travel environment information 80, driving situation information 81, sensor information (own vehicle) 82, and camera parameters 83. When using the driving simulator to evaluate the driving characteristics of the driver, information including driving status information such as the age and sex of the driver as the subject is also stored in association with each other.
　図９は、シーン条件マトリクスの例を示す図である。一般に車載カメラの画像認識ロジックの評価を行うには、最初にロジック評価に必要となるシーンの条件を整理したシーン条件マトリクスを定義する。シーン条件の定義の一例としては、例えば横軸９０に夜間、雨天、西日、逆光などの気象条件を表記し、縦軸９１に幅員、車線数、交差点、カーブなど走行路に関する条件を定義し、マトリクス内の横軸と縦軸の条件がクロスしたセルに該当する抽象化シーンがある場合には、シーン名称を記載し、記載された抽象化シーンを対応づける。また、該当シーンが存在しない場合には存在しないことをセル内に記載する。 FIG. 9 is a diagram showing an example of the scene condition matrix. In general, in order to evaluate the image recognition logic of an in-vehicle camera, first, a scene condition matrix is defined in which scene conditions necessary for logic evaluation are arranged. As an example of the definition of the scene condition, for example, the horizontal axis 90 represents the weather conditions such as night, rainy weather, western sun, backlight, etc., and the vertical axis 91 defined the conditions related to the driving path such as width, number of lanes, intersections, and curves. When there is an abstract scene corresponding to a cell in which the conditions of the horizontal axis and the vertical axis in the matrix cross, the scene name is described, and the described abstract scene is associated. If the corresponding scene does not exist, it is described in the cell that it does not exist.
　図１０は、認識ロジック用シーン条件マトリクスの例を示す図である。シーン条件マトリクスは、異なる車載カメラ用認識ロジック向けにシーン条件マトリクスを用意する。例えば、歩行者検知システム用のシーン条件マトリクス１００や白線逸脱警告システム用シーン条件マトリクス１０１など、運転支援システム毎に夫々対応するシーン条件マトリクスを定義する。 FIG. 10 is a diagram showing an example of a scene condition matrix for recognition logic. As the scene condition matrix, a scene condition matrix is prepared for different in-vehicle camera recognition logic. For example, a scene condition matrix corresponding to each driving support system such as a scene condition matrix 100 for a pedestrian detection system and a scene condition matrix 101 for a white line departure warning system is defined.
　図１１は、認識ロジック用シーン条件マトリクスの情報内容の例を示す図である。図に示す認識ロジック用シーン条件マトリックスでは、車載カメラで収集した映像と、走行条件（例えば、道路幅員、車線数、交差点の有無、カーブなど）や、気象条件（撮影日時や、天気、西日や逆光の有無など）などの、撮影条件と関連付けてテーブル形式で管理することで、例えば、どの撮影条件での評価映像が収集できているのか、またはいないのかを容易に判断できるため、評価映像の収集状況の可視化ができる。 FIG. 11 is a diagram showing an example of information content of a scene condition matrix for recognition logic. In the scene condition matrix for recognition logic shown in the figure, images collected by the in-vehicle camera, driving conditions (for example, road width, number of lanes, presence / absence of intersections, curves, etc.), weather conditions (shooting date / time, weather, western day) (E.g., the presence or absence of backlight), etc., and management in a table format, for example, it is easy to determine under which shooting conditions the evaluation video is collected or not. The collection status can be visualized.
　　また、マトリックス内の各シーンと、実際の抽象化シーンが対応しているため、シーン項目を選択すると実際のシーンがどんなものか確認できる。マトリクス内の項目シーン１（昼間で車線数を増やしているシーン）を選択すると、車線数を１車線化から２車線化に増やしたシーン１１０であることが確認できる。同様にシーン２が夜間シーンで１１１あり、シーン５は幅員を拡幅した逆光シーン１１２であることも分かる。また、シーン６は昼間で幅員を拡幅したのみのシーン１１３である。さらに、夜間でカーブのシーン１１５は映像として収集できていないことも確認できる。車載カメラロジックの映像評価には、このマトリックスを用いることで、所望の評価映像を容易に取り出すことができる。また、収集できていない映像も把握できるため、同じ条件で２重に評価映像を収集するなどの誤りも未然に防ぐことができる。 Also, since each scene in the matrix corresponds to an actual abstract scene, you can check what the actual scene is when you select a scene item. When item scene 1 in the matrix (scene in which the number of lanes is increased in the daytime) is selected, it can be confirmed that the scene 110 is obtained by increasing the number of lanes from one lane to two lanes. Similarly, it can be seen that the scene 2 is the night scene 111 and the scene 5 is the backlight scene 112 having a wider width. Scene 6 is a scene 113 in which the width is widened in the daytime. Further, it can be confirmed that the curve scene 115 at night is not collected as an image. By using this matrix for video evaluation of the in-vehicle camera logic, a desired evaluation video can be easily extracted. In addition, since the video that has not been collected can be grasped, it is possible to prevent an error such as collecting the evaluation video twice under the same conditions.
　図１２は、走行環境情報条件の変更例を示す図である。走行環境情報の条件変更方法としては、例えば交通流シミュレータのネットワークモデルの編集機能を用いて、車線数１２０や、車線幅員１２１、さらには信号交差点の有無１２２等の条件変更を行い走行環境条件の変更を行うことができる。 FIG. 12 is a diagram showing an example of changing the driving environment information condition. As a method for changing the conditions of the driving environment information, for example, using the editing function of the network model of the traffic flow simulator, the conditions of the driving environment conditions are changed by changing the conditions such as the number of lanes 120, the lane width 121, and the presence / absence of signalized intersection 122. Changes can be made.
　図１３は、走行環境情報からプライベート情報を除去する例を示す図である。車載カメラで収集した映像シーンには、走行車両１３０，１３１のナンバープレート情報１３２，１３４や、歩行者が特定できるような個人情報などが含まれるため、収集した映像から抽象化シーンをＣＧ作成する際にプライベート情報を除去１３３，１３５したり、収集した映像をもとにプライベート情報を予め除去することが必要となる。 FIG. 13 is a diagram illustrating an example of removing private information from the travel environment information. Since the video scene collected by the in-vehicle camera includes license plate information 132 and 134 of the traveling vehicles 130 and 131 and personal information that can identify the pedestrians, an abstract scene is CG created from the collected video. At this time, it is necessary to remove the private information 133, 135, or to remove the private information in advance based on the collected video.
　図１４は、マップ上で走行環境情報の収集状況を表示した例を示す図である。図８の映像抽象化シーン蓄積データベース内に格納されている各映像抽象化シーン１４０，１４１，１４２が、どこで収集し作成されたのかをマップ上で表示することができる。このような位置情報が、車載カメラロジック評価の際に必要となる。 FIG. 14 is a diagram showing an example in which the traveling environment information collection status is displayed on the map. Where the video abstract scenes 140, 141, and 142 stored in the video abstract scene storage database of FIG. 8 are collected and created can be displayed on the map. Such position information is required for the on-vehicle camera logic evaluation.
　図１５は、走行環境情報をドライビングシミュレータに利用する例を示す図である。まず、ドライビングシミュレータの被験者の、少なくともドライバーの年齢や性別、運転歴などの運転車情報を設定し（１５０）、設定された条件に合ったシーンを、図８の映像抽象化シーン蓄積データベースから検索し（１５１）、検索されたシーンをドライビングシミュレータに設定し、運転特性の診断を行う（１５２）。診断結果にもとづき、さらに苦手なシーンがある場合に当該シーンを映像抽象化シーン蓄積データベース内から検索し（１５３）、ドライビングシミュレータにより運転特性の診断を繰り返し行う。診断結果に問題がなかった場合、被験者の運転車情報と、利用シーンの使用履歴情報を、運転特性の情報として記録しドライバーの安全運転支援に利用する（１５４）。 FIG. 15 is a diagram illustrating an example in which the travel environment information is used for a driving simulator. First, driving vehicle information such as at least the age, gender, and driving history of the driving simulator subject is set (150), and a scene that matches the set conditions is searched from the video abstraction scene storage database of FIG. (151) The searched scene is set in the driving simulator, and the driving characteristics are diagnosed (152). Based on the diagnosis result, when there is a scene that is not good, the scene is searched from the video abstract scene storage database (153), and the driving characteristics are repeatedly diagnosed by the driving simulator. If there is no problem in the diagnosis result, the driving vehicle information of the subject and the usage history information of the usage scene are recorded as driving characteristic information and used for the driver's safe driving support (154).

Claims (18)

  1. 車載カメラの映像を画像メモリに蓄積する車載カメラ映像入力手段と、
    車両の走行環境情報を入力する走行環境情報入力手段と、
    前記画像メモリ内の車載カメラ映像と該走行環境情報をもとに抽象化シーンを生成する抽象化シーン生成手段と、
    走行環境条件の異なる抽象化シーンを生成する走行条件変更手段と、
    生成された抽象化シーンを格納する抽象化シーン蓄積手段を備えたことを特徴とする走行環境抽象化映像作成装置。     In-vehicle camera video input means for storing in-vehicle camera video in an image memory;
    Driving environment information input means for inputting vehicle driving environment information;
    Abstract scene generation means for generating an abstract scene based on the in-vehicle camera video in the image memory and the driving environment information;
    Driving condition changing means for generating abstract scenes having different driving environment conditions;
    A travel environment abstract video creating apparatus comprising abstract scene storage means for storing a generated abstract scene.
  2. 請求項１の走行環境抽象化映像作成装置において、
    画像認識ロジック評価に必要となるシーン全体の収集状況を確認できるシーン条件マトリクス生成手段を備えたことを特徴とする走行環境抽象化映像作成装置。     In the traveling environment abstract image creation device according to claim 1,
    A travel environment abstract video creation apparatus comprising scene condition matrix generation means capable of confirming the collection status of an entire scene required for image recognition logic evaluation.
  3. 請求項１の走行環境抽象化映像作成装置において、
    前記抽象化シーン生成手段は、車載カメラのカメラパラメタ情報を用いて抽象化シーンを生成することを特徴とする走行環境抽象化映像作成装置。     In the traveling environment abstract image creation device according to claim 1,
    The abstraction scene generation unit generates an abstraction scene using camera parameter information of a vehicle-mounted camera.
  4. 請求項１の走行環境抽象化映像作成装置において、
    前記抽象化シーン生成手段は、車載カメラ映像情報をもとに路面や、車両、歩行者、地物に関する情報に分類し、前記走行環境情報と関連付けて抽象化シーンを生成することを特徴とする走行環境抽象化映像作成装置。     In the traveling environment abstract image creation device according to claim 1,
    The abstract scene generation means classifies information on road surfaces, vehicles, pedestrians, and features based on in-vehicle camera video information, and generates an abstract scene in association with the travel environment information. Driving environment abstraction video creation device.
  5. 請求項１の走行環境抽象化映像作成装置において、
    前記抽象化シーン生成手段は、抽象化シーン蓄積手段に蓄積された抽象化シーンをもとに、前記走行環境条件を変更した抽象化シーンを生成することを特徴とする走行環境抽象化映像作成装置。     In the traveling environment abstract image creation device according to claim 1,
    The abstraction scene generating means generates an abstraction scene in which the driving environment condition is changed based on the abstraction scene stored in the abstraction scene storage means. .
  6. 請求項１の走行環境抽象化映像作成装置において、
    前記抽象化シーン蓄積手段は、前記抽象化シーンと、前記抽象化シーンを作成する際に得られる走行環境情報とを関連付けて蓄積し、前記走行環境情報を検索条件として所望の抽象化シーンを検索することを特徴とする走行環境抽象化映像作成装置。     In the traveling environment abstract image creation device according to claim 1,
    The abstract scene storage means stores the abstract scene in association with the driving environment information obtained when the abstract scene is created, and searches for a desired abstract scene using the driving environment information as a search condition. A travel environment abstraction video creation device characterized by:
  7. 請求項2の走行環境抽象化映像作成装置において、
    前記シーン条件マトリクス生成手段は、予め車載カメラの画像認識ロジック評価に必要となる評価シーンの走行環境条件と、評価シーンの有無が確認可能な表形式で表し、抽象化シーン蓄積手段から走行環境条件をキーに検索し、検索しても該当しない抽象化シーンがある場合、該当しない抽象化シーンの走行環境条件をもとに、抽象化シーンを生成しシーン条件マトリクス情報を更新すること特徴とする走行環境抽象化映像作成装置。     In the traveling environment abstract video creation device of claim 2,
    The scene condition matrix generating means is expressed in a tabular form that can confirm whether or not there is an evaluation scene in advance and an evaluation scene driving environment condition necessary for image recognition logic evaluation of the in-vehicle camera. If there is an abstract scene that does not correspond even if it is searched, the abstract condition is generated and the scene condition matrix information is updated based on the driving environment condition of the abstract scene that does not correspond Driving environment abstraction video creation device.
  8. 請求項7の走行環境抽象化映像作成装置において、
    前記シーン条件マトリクス生成手段は、ドライバの運転特性の評価をドライビングシミュレータを用いて実施する場合、ドライバの属性情報をもとにシーン条件マトリクスを作成し、
    前記抽象化シーン蓄積手段から走行環境条件をキーに検索し、検索しても該当しない抽象化シーンがある場合、該当しない抽象化シーンの走行環境条件をもとに、抽象化シーンを生成しシーン条件マトリクス情報を更新すること特徴とする走行環境抽象化映像作成装置。     In the traveling environment abstract video creation device of claim 7,
    The scene condition matrix generating means creates a scene condition matrix based on the attribute information of the driver when evaluating the driving characteristics of the driver using a driving simulator,
    A search is made from the abstract scene storage means using the driving environment condition as a key, and if there is an abstract scene that does not correspond even if the search is performed, an abstract scene is generated based on the driving environment condition of the abstract scene that does not correspond. A travel environment abstract image creating apparatus characterized by updating condition matrix information.
  9. 請求項7の走行環境抽象化映像作成装置において、
    前記シーン条件マトリクス生成手段は、ドライバの運転特性の評価をドライビングシミュレータを用いて実施する場合、ドライバの属性情報をもとにシーン条件マトリクスを作成し、前記抽象化シーン蓄積手段から走行環境条件をキーに検索し、検索した際に該当する抽象化シーンがある場合、シーン条件マトリクス内の情報を選択された該マトリクス情報に対応する抽象化シーンを表示すること特徴とするものである。     In the traveling environment abstract video creation device of claim 7,
    The scene condition matrix generating means creates a scene condition matrix based on the attribute information of the driver and evaluates the driving environment conditions from the abstract scene storage means when evaluating the driving characteristics of the driver using a driving simulator. When the key is searched and there is an abstract scene corresponding to the search, the abstract scene corresponding to the selected matrix information is displayed as information in the scene condition matrix.
  10. 車載カメラの映像を画像メモリに蓄積すること、
    車両の走行環境情報を入力すること、
    前記画像メモリ内の車載カメラ映像と該走行環境情報をもとに抽象化シーンを生成すること、
    走行環境条件の異なる抽象化シーンを生成する走行条件を変更すること、
    生成された抽象化シーンを記憶手段に格納することを特徴とする走行環境抽象化映像作成方法。     Storing video from in-vehicle cameras in image memory;
    Entering vehicle driving environment information,
    Generating an abstract scene based on the in-vehicle camera video in the image memory and the driving environment information;
    Changing driving conditions that generate abstract scenes with different driving environment conditions;
    A method for creating an abstract video for a driving environment, wherein the generated abstract scene is stored in a storage means.
  11. 請求項１０の走行環境抽象化映像作成方法において、
    画像認識ロジック評価に必要となるシーン全体の収集状況を確認できるシーン条件マトリクスを生成することを特徴とする走行環境抽象化映像作成方法。     The traveling environment abstract image creating method according to claim 10,
    A running environment abstracted video creation method, characterized in that a scene condition matrix is generated that can confirm the collection status of an entire scene required for image recognition logic evaluation.
  12. 請求項１０の走行環境抽象化映像作成方法において、
    前記車載カメラのカメラパラメタ情報を用いて前記抽象化シーンを生成することを特徴とする走行環境抽象化映像作成方法。     The traveling environment abstract image creating method according to claim 10,
    A method of creating an abstract video for a driving environment, wherein the abstract scene is generated using camera parameter information of the in-vehicle camera.
  13. 請求項１０の走行環境抽象化映像作成方法において、
    前記車載カメラ映像情報をもとに路面や、車両、歩行者、地物に関する情報に分類し、前記走行環境情報と関連付けて抽象化シーンを生成することを特徴とする走行環境抽象化映像作成方法。     The traveling environment abstract image creating method according to claim 10,
    A travel environment abstracted video creation method that classifies information on road surfaces, vehicles, pedestrians, and features based on the in-vehicle camera video information and generates an abstract scene in association with the travel environment information .
  14. 請求項１０の走行環境抽象化映像作成方法において、
    蓄積された前記抽象化シーンをもとに、前記走行環境条件を変更した抽象化シーンを生成することを特徴とする走行環境抽象化映像作成方法。     The traveling environment abstract image creating method according to claim 10,
    A travel environment abstracted video creation method, wherein an abstract scene in which the travel environment condition is changed is generated based on the stored abstract scene.
  15. 請求項１０の走行環境抽象化映像作成方法において、
    前記抽象化シーンと、前記抽象化シーンを作成する際に得られる走行環境情報とを関連付けて蓄積し、前記走行環境情報を検索条件として所望の抽象化シーンを検索することを特徴とする走行環境抽象化映像作成方法。     The traveling environment abstract image creating method according to claim 10,
    The abstraction scene and the travel environment information obtained when creating the abstraction scene are stored in association with each other, and a desired abstract scene is searched using the travel environment information as a search condition. Abstract video creation method.
  16. 請求項１１の走行環境抽象化映像作成方法において、
    予め車載カメラの画像認識ロジック評価に必要となる評価シーンの走行環境条件と、評価シーンの有無が確認可能な表形式で表し、走行環境条件をキーに検索し、検索しても該当しない抽象化シーンがある場合、該当しない抽象化シーンの走行環境条件をもとに、抽象化シーンを生成しシーン条件マトリクス情報を更新すること特徴とする走行環境抽象化映像作成方法。     In the traveling environment abstract image | video creation method of Claim 11,
    Expressed in a tabular format that can be used to check the presence of an evaluation scene and the evaluation environment's driving environment conditions required for evaluating the image recognition logic of an on-vehicle camera in advance. A running environment abstracted video creation method characterized by generating an abstract scene and updating scene condition matrix information based on a running environment condition of an abstract scene that does not correspond when there is a scene.
  17. 請求項１６の走行環境抽象化映像作成方法において、
    前記シーン条件マトリクス情報を生成する際に、ドライバの運転特性の評価をドライビングシミュレータを用いて実施する場合、ドライバの属性情報をもとにシーン条件マトリクスを作成し、
    前記抽象化シーンの記憶手段から走行環境条件をキーに検索し、検索しても該当しない抽象化シーンがある場合、該当しない抽象化シーンの走行環境条件をもとに、抽象化シーンを生成しシーン条件マトリクス情報を更新すること特徴とする走行環境抽象化映像作成方法。     The traveling environment abstract image creating method according to claim 16,
    When generating the scene condition matrix information, if the driving characteristics of the driver are evaluated using a driving simulator, a scene condition matrix is created based on the driver attribute information,
    Searching from the abstract scene storage means using the driving environment condition as a key, and if there is an abstract scene that is not applicable even if the search is performed, an abstract scene is generated based on the driving environment condition of the abstract scene that does not apply. A running environment abstracted image creation method characterized by updating scene condition matrix information.
  18. 請求項１６の走行環境抽象化映像作成方法において、
    シーン条件マトリクス情報は、ドライバの運転特性の評価をドライビングシミュレータを用いて実施する場合、ドライバの属性情報をもとにシーン条件マトリクス情報を作成し、
    前記抽象化シーンの記憶手段から走行環境条件をキーに検索し、検索した際に該当する抽象化シーンがある場合、シーン条件マトリクス内の情報を選択された該マトリクス情報に対応する抽象化シーンを表示すること特徴とする走行環境抽象化映像作成方法。     The traveling environment abstract image creating method according to claim 16,
    Scene condition matrix information creates scene condition matrix information based on driver attribute information when evaluating driving characteristics of a driver using a driving simulator.
    The abstract scene corresponding to the selected matrix information is selected from the information in the scene condition matrix when the abstract scene corresponding to the search is found from the storage means of the abstract scene. A method of creating an abstract image of a driving environment characterized by displaying.
PCT/JP2015/059543 2015-03-27 2015-03-27 Method and device for generating travelling environment abstract image WO2016157277A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2015/059543 WO2016157277A1 (en) 2015-03-27 2015-03-27 Method and device for generating travelling environment abstract image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2015/059543 WO2016157277A1 (en) 2015-03-27 2015-03-27 Method and device for generating travelling environment abstract image

Publications (1)

Publication Number Publication Date
WO2016157277A1 true WO2016157277A1 (en) 2016-10-06

Family

ID=57004146

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2015/059543 WO2016157277A1 (en) 2015-03-27 2015-03-27 Method and device for generating travelling environment abstract image

Country Status (1)

Country Link
WO (1) WO2016157277A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108961396A (en) * 2018-07-03 2018-12-07 百度在线网络技术（北京）有限公司 Generation method, device and the terminal device of three-dimensional scenic
WO2020026461A1 (en) * 2018-08-03 2020-02-06 日本電気株式会社 Information processing device, information processing method, and information processing program
WO2021049062A1 (en) * 2019-09-10 2021-03-18 株式会社日立製作所 Recognition model distribution system and recognition model updating method

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08235491A (en) * 1995-02-27 1996-09-13 Toyota Motor Corp Recorder and analyzer for running state of vehicle
JP2004265396A (en) * 2003-02-13 2004-09-24 Vingo:Kk Image forming system and image forming method
JP2006107521A (en) * 2005-10-17 2006-04-20 Denso Corp Mobile communication equipment
JP2007292825A (en) * 2006-04-21 2007-11-08 Mitsubishi Precision Co Ltd Simulated view generating device
JP2009140439A (en) * 2007-12-10 2009-06-25 Toyota Motor Corp Device for diagnosing driver's driving capability and driving support device for adjusting content of driving support according to the driver's driving capability
JP2010175329A (en) * 2009-01-28 2010-08-12 Mitsubishi Electric Corp On-vehicle information device
JP2012048529A (en) * 2010-08-27 2012-03-08 Mitsubishi Precision Co Ltd Method for generating external appearance display image of planimetric feature and device therefor

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08235491A (en) * 1995-02-27 1996-09-13 Toyota Motor Corp Recorder and analyzer for running state of vehicle
JP2004265396A (en) * 2003-02-13 2004-09-24 Vingo:Kk Image forming system and image forming method
JP2006107521A (en) * 2005-10-17 2006-04-20 Denso Corp Mobile communication equipment
JP2007292825A (en) * 2006-04-21 2007-11-08 Mitsubishi Precision Co Ltd Simulated view generating device
JP2009140439A (en) * 2007-12-10 2009-06-25 Toyota Motor Corp Device for diagnosing driver's driving capability and driving support device for adjusting content of driving support according to the driver's driving capability
JP2010175329A (en) * 2009-01-28 2010-08-12 Mitsubishi Electric Corp On-vehicle information device
JP2012048529A (en) * 2010-08-27 2012-03-08 Mitsubishi Precision Co Ltd Method for generating external appearance display image of planimetric feature and device therefor

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108961396A (en) * 2018-07-03 2018-12-07 百度在线网络技术（北京）有限公司 Generation method, device and the terminal device of three-dimensional scenic
WO2020026461A1 (en) * 2018-08-03 2020-02-06 日本電気株式会社 Information processing device, information processing method, and information processing program
JPWO2020026461A1 (en) * 2018-08-03 2021-08-02 日本電気株式会社 Information processing equipment, information processing methods and information processing programs
JP7081669B2 (en) 2018-08-03 2022-06-07 日本電気株式会社 Information processing equipment, information processing methods and information processing programs
US11475773B2 (en) 2018-08-03 2022-10-18 Nec Corporation Alert of occurrence of pre-dangerous state of vehicle
WO2021049062A1 (en) * 2019-09-10 2021-03-18 株式会社日立製作所 Recognition model distribution system and recognition model updating method
JP2021043622A (en) * 2019-09-10 2021-03-18 株式会社日立製作所 Recognition model distribution system and recognition model updating method
JP7414434B2 (en) 2019-09-10 2024-01-16 株式会社日立製作所 Recognition model distribution system and recognition model update method

Similar Documents

Publication Publication Date Title
CN107031656B (en) Virtual sensor data generation for wheel immobilizer detection
JP6599986B2 (en) Hyperclass expansion and regularization deep learning for fine-grained image classification
DE112019000279T5 (en) CONTROLLING AUTONOMOUS VEHICLES USING SAFE ARRIVAL TIMES
DE112019001605T5 (en) TRAINING, TESTING AND VERIFYING AUTONOMOUS MACHINERY USING SIMULATED ENVIRONMENTS
DE102020110458A1 (en) VEHICLE PATH FORECAST
JP7183390B2 (en) Camera evaluation technology for autonomous vehicles
DE102018218895A1 (en) A system and method for providing an infrastructure-based security alert associated with at least one lane
JP2017535873A (en) Continuous occlusion model for street scene recognition
US11919545B2 (en) Scenario identification for validation and training of machine learning based models for autonomous vehicles
JP7486564B2 (en) Enhanced navigation guidance by landmarks under difficult driving conditions
DE102020100685A1 (en) PREDICTION OF TEMPORARY INFORMATION IN AUTONOMOUS MACHINE APPLICATIONS
CN105138525B (en) Traffic video processing unit and method and retrieval device and method
DE112018004953T5 (en) INFORMATION PROCESSING DEVICE, INFORMATION PROCESSING PROCESS, PROGRAM AND MOVING BODY
CN112289037B (en) Motor vehicle illegal parking detection method and system based on high visual angle under complex environment
DE102022102189A1 (en) Multimodal segmentation mesh for improved semantic labeling in map generation
DE102021132082A1 (en) END-TO-END SYSTEM TRAINING USING UNITED IMAGES
DE102021131489A1 (en) SEMANTIC ANNOTATION OF SENSOR DATA USING UNRELIABLE MAP ANNOTATION INPUTS
WO2016157277A1 (en) Method and device for generating travelling environment abstract image
DE102022100413A1 (en) OBJECT DETECTION USING RADAR AND LIDAR COMBINATION
CN116457800A (en) Architecture for map change detection in an autonomous vehicle
US20220092313A1 (en) Method for deep neural network functional module deduplication
US20220172606A1 (en) Systems and Methods for Extracting Data From Autonomous Vehicles
DE102022102187A1 (en) HIGH-FIDELITY DATA-DRIVEN MULTIMODAL SIMULATION
CN117242460A (en) Computerized detection of unsafe driving scenarios
DE102021133740A1 (en) LEARNING TO IDENTIFY SAFETY CRITICAL SCENARIOS FOR AN AUTONOMOUS VEHICLE

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15887433

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15887433

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP