CN113297878A - Road intersection identification method and device, computer equipment and storage medium - Google Patents

Road intersection identification method and device, computer equipment and storage medium Download PDF

Info

Publication number
CN113297878A
CN113297878A CN202010110375.9A CN202010110375A CN113297878A CN 113297878 A CN113297878 A CN 113297878A CN 202010110375 A CN202010110375 A CN 202010110375A CN 113297878 A CN113297878 A CN 113297878A
Authority
CN
China
Prior art keywords
street view
intersection
view image
boundary
identification
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010110375.9A
Other languages
Chinese (zh)
Other versions
CN113297878B (en
Inventor
夏德国
张刘辉
杨建忠
曹雪卉
姜海林
李崎玮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN202010110375.9A priority Critical patent/CN113297878B/en
Publication of CN113297878A publication Critical patent/CN113297878A/en
Application granted granted Critical
Publication of CN113297878B publication Critical patent/CN113297878B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/52Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/54Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/25Determination of region of interest [ROI] or a volume of interest [VOI]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/26Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/56Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/588Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Traffic Control Systems (AREA)
  • Image Analysis (AREA)

Abstract

The application provides a road intersection identification method, a road intersection identification device, computer equipment and a storage medium, and relates to the technical field of image processing, wherein the method comprises the following steps: the method comprises the steps of obtaining street view images, identifying the street view images by adopting a semantic segmentation model to determine a vehicle driving area from the street view images, identifying whether the street view images contain the intersection or not according to the boundary bending degree of the vehicle driving area so as to accurately identify the intersection and the peripheral information of the intersection, and determining the position of the intersection according to the GPS information of each image.

Description

Road intersection identification method and device, computer equipment and storage medium
Technical Field
The application relates to the technical field of artificial intelligence, in particular to the technical field of image processing, and specifically relates to a road intersection identification method, a road intersection identification device, computer equipment and a storage medium.
Background
With the increasingly complex urban roads and the rapid popularization of smart phones, mobile phone navigation software has become a necessary tool for users to go out. Wherein the map data is the basis and core of map navigation. With the acceleration of urban road construction and the stricter requirements of users on the accuracy of map data, it becomes more important to acquire urban road information more accurately and quickly.
Roads in the map data take intersections as traction points, and the trafficability of the roads around the intersections is described. Description of road intersections is an important component of map data and road navigation. Therefore, in the process of collecting road information, the detection of road intersections is very important.
The following ways are commonly used in the related art to detect intersections: (1) the data that combines laser range finder, laser radar and monocular camera to gather detects, however, equipment such as laser range finder is expensive, and detecting system is complicated, and the cost is higher, can not detect far away intersection moreover. (2) The vehicle GPS gathered information is used for detecting the road intersection, but the road intersection detection mode needs the detected road to acquire a large amount of GPS data, and the road with sparse GPS cannot accurately detect the information around the road intersection and the intersection.
Disclosure of Invention
The present application is directed to solving, at least to some extent, one of the technical problems in the related art.
Therefore, a first object of the present application is to provide a method for identifying a road intersection, in which a plurality of frames of street view images collected sequentially are subjected to semantic segmentation and identification to determine a vehicle driving area, and whether the street view images include the intersection is identified based on the degree of curvature of a boundary of the vehicle driving area, so that the road intersection and the peripheral information of the intersection can be accurately identified, and meanwhile, the position of the road intersection can be determined according to the GPS information of each image.
A second object of the present application is to provide a road intersection identification device.
A third object of the present application is to propose a computer device.
A fourth object of the present application is to propose a non-transitory computer-readable storage medium.
In order to achieve the above object, an embodiment of a first aspect of the present application provides a method for identifying a road intersection, including:
obtaining a street view image;
identifying the street view image by adopting a semantic segmentation model so as to determine a vehicle driving area from the street view image;
and identifying whether the street view image contains a road intersection or not according to the boundary bending degree of the vehicle driving area.
In order to achieve the above object, an embodiment of a second aspect of the present application provides a road intersection identification device, including:
the acquisition module is used for acquiring street view images;
the segmentation module is used for identifying the street view image by adopting a semantic segmentation model so as to determine a vehicle driving area from the street view image;
and the identification module is used for identifying whether the street view image contains a road intersection or not according to the boundary bending degree of the vehicle driving area.
To achieve the above object, an embodiment of a third aspect of the present application provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the processor implements the intersection identification method according to the first aspect.
To achieve the above object, a fourth aspect of the present application provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method for identifying a road intersection according to the first aspect is implemented.
The technical scheme provided by the embodiment of the application can have the following beneficial effects:
the method comprises the steps of obtaining street view images, identifying the street view images by adopting a semantic segmentation model to determine a vehicle driving area from the street view images, identifying whether the street view images contain the intersection or not according to the boundary bending degree of the vehicle driving area so as to accurately identify the intersection and the peripheral information of the intersection, and determining the position of the intersection according to the GPS information of each image.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 is a schematic flow chart of a road intersection identification method according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a plurality of consecutive frames of street view images provided herein;
fig. 3 is a schematic diagram of a street view image subjected to semantic segmentation according to an embodiment of the present disclosure;
fig. 4 is a schematic flowchart of another method for identifying a road intersection according to an embodiment of the present application;
FIG. 5 is a schematic diagram illustrating a boundary curvature determination provided in an embodiment of the present application;
FIG. 6 is a schematic illustration of another determination of boundary curvature provided by an embodiment of the present application;
fig. 7 is a schematic flow chart of another method for identifying a road intersection according to the present application;
FIG. 8 is a schematic diagram of the street view image screening according to the degree of boundary curvature provided by the present application;
fig. 9 is a schematic diagram of an intersection location traffic object identification result provided by the present application;
fig. 10 is a schematic structural diagram of a road intersection recognition device according to an embodiment of the present application; and
fig. 11 is a block diagram of an electronic device of a method for identifying a road intersection according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining the present application and should not be construed as limiting the present application.
A method, an apparatus, a computer device, and a storage medium for identifying a road intersection according to an embodiment of the present application are described below with reference to the accompanying drawings.
Fig. 1 is a schematic flow chart of a road intersection identification method provided in the embodiment of the present application.
As shown in fig. 1, the method comprises the steps of:
step 101, obtaining a street view image.
The street view image is a multi-frame image which is collected sequentially.
Specifically, the street view image is acquired by a camera provided on a road information acquisition vehicle driven by a driver, and the road in the street view image contains various information such as a road intersection, a zebra crossing, a traffic light, an electronic eye, and the like.
Fig. 2 is a schematic diagram of a plurality of consecutive frames of street view images provided by the present application, and fig. 2 shows a continuous track chart of street view images collected during vehicle-mounted driving, where 6 street view images are collected and numbered as 1,2,3, 4, 5, and 6 according to the collection sequence. In the actual acquisition process, the number of the continuously acquired multiple frames of images may be set according to specific situations, which is not limited in this embodiment.
And step 102, identifying the street view image by adopting a semantic segmentation model so as to determine a vehicle driving area from the street view image.
As a possible implementation manner, the collected street view image is input into a trained semantic segmentation model, the semantic segmentation model performs semantic segmentation on each street view image by taking pixels as units, the pixels representing the same category are combined, and finally, the output image after the semantic segmentation can clearly identify buildings, vehicles, fences on roads, road edges, driving roads and the like contained in the street view image so as to determine the driving area of the vehicles in the street view image.
Fig. 3 is a schematic diagram of street view images subjected to semantic segmentation according to the embodiment of the present application, fig. 3 is a diagram obtained by performing semantic segmentation on continuous multi-frame street view images corresponding to fig. 2, as shown in fig. 3, the street view image numbered 5 indicates a plurality of classifications identified by semantic segmentation in the street view image, including sky, buildings, lane lines, zebra crossings, ground arrows and driving roads, different grays in fig. 3 represent different classifications obtained by semantic segmentation, for clarity, the street view labeled 5 in fig. 3 is taken as an example, and the arrows specifically indicate the different classes into which the segmentation is performed, where each arrow indicates a class resulting from the semantic segmentation, however, in practical applications, further classes can be identified, which are only schematically listed here and do not constitute a limitation to the results obtained by semantic segmentation in this embodiment.
And 103, identifying whether the street view image contains a road intersection or not according to the boundary bending degree of the vehicle driving area.
Specifically, for each frame of street view image, an identification feature is extracted, wherein the identification feature comprises the boundary bending degree of a vehicle driving area in the corresponding street view image and the difference information of the boundary bending degree of the vehicle driving area and the adjacent street view image, the extracted identification feature is input into a classification model, and as the classification model learns the mapping relation between the identification feature and a road intersection identification result, whether the corresponding street view image contains a road intersection or not can be identified according to the classification model.
Furthermore, the collection vehicle collects continuous street view images of roads and also has corresponding geographic position information, wherein the geographic position information is GPS information. Each street view image has corresponding GPS information, so that after the street view image containing the intersection is identified and obtained, the GPS position of the street view image containing the intersection is the GPS position of the intersection, the intersection can be determined, meanwhile, the position information of the intersection can be determined, the determined intersection and the position information can be added into a geographic information database, and the geographic information database can be updated.
According to the method for identifying the road intersection, the street view image is obtained through the camera, the street view image is identified through the semantic segmentation model, the vehicle driving area is determined from the street view image, whether the street view image contains the road intersection or not is identified according to the boundary bending degree of the vehicle driving area, so that the peripheral information of the road intersection and the intersection can be accurately identified, meanwhile, the position of the road intersection can be determined according to the GPS information of each image, and the cost of identifying the road intersection is reduced due to the fact that expensive laser distance meters, laser radars and other devices are not adopted.
Based on the previous embodiment, this embodiment provides a possible implementation manner of the road intersection identification method, which explains how to determine the boundary bending degree at each boundary point of the driving area after determining the driving area of the vehicle in the street view image through the semantic segmentation model. Fig. 4 is a schematic flowchart of another method for identifying a road intersection according to an embodiment of the present application.
As shown in fig. 4, after step 102, the following steps may be further included:
in step 401, the degree of curvature of the boundary of the vehicle driving region is determined.
In the present embodiment, the following two possible implementations are provided for determining the degree of bending at the boundary point.
As a possible implementation manner, for each boundary point in the vehicle driving area, an identification frame using the corresponding boundary point as an angular point is determined, and the boundary bending degree at the corresponding boundary point is determined according to the number of pixel points belonging to the vehicle driving area in the identification frame. This is because the greater the degree of curvature of the boundary of the vehicle travel region, the greater the number of pixels of the travel region included in the recognition frame having the corresponding boundary point as the corner point. After the boundary pixels of all driving areas in the image can be found after the boundary pixels are identified through a semantic segmentation model, then, an M-N identification frame is drawn for each boundary pixel, M and N are natural numbers larger than 1, further, the number of the pixels of the driving areas contained in each identification frame is counted, the boundary bending degree of the corresponding boundary is indicated by the number of the pixels, wherein the more the number of the pixels of the driving areas contained in the corresponding identification frame is, the larger the boundary bending degree of the corresponding boundary point is. For example, as shown in fig. 5, the number of pixels in the driving area included in the recognition frame in the map identified as 4 is smaller than the number of pixels in the driving area included in the recognition frame corresponding to the boundary point in the street view image identified as 5, that is, the degree of curvature of the boundary corresponding to the recognition frame in the map identified as 4 is smaller than the degree of curvature of the boundary corresponding to the recognition frame in the map identified as 5.
As another possible implementation manner, a plurality of boundary points in the vehicle driving area are grouped, wherein each group includes a fixed number of boundary points which are adjacently arranged, the boundary points in the same group are fitted to obtain a fitted straight line, and the boundary bending degree of each boundary point in each group is determined according to an included angle between the corresponding fitted straight line and the adjacent fitted straight line for each group of boundary points. For example, as shown in fig. 6, a gray portion of the ground, corresponding to the vehicle driving area, groups a plurality of boundary points in the vehicle driving area, for example, 20 boundary points are grouped into one group, and the group is divided into a plurality of groups in the vertical direction, for example, from top to bottom, in the horizontal direction, for example, from left to right, and the boundary points in the same group are fitted to obtain a corresponding fitting straight line, for example, a plurality of fitting straight lines indicated by white arrows corresponding to letters a-d in the right boundary in fig. 6. When the fitting straight line c and the fitting straight line d are reached forwards, the direction angles indicated by the fitting straight line c and the fitting straight line d are different greatly, that is, the included angle between the fitting straight line c and the fitting straight line d is large, so that the boundary bending degree at each boundary point in the group c and the group d is large.
The recognition frame of the boundary points in fig. 5 and the fitting straight line of each group in fig. 6 are only schematic representations, and do not limit the present embodiment.
In the method for identifying the intersection, the boundary bending degree at the boundary point in the vehicle driving area is identified and obtained by semantically segmenting the continuous street view images, the boundary bending degree at each boundary point can be determined, and the higher the boundary bending degree is, the higher the possibility of the intersection is, so that the boundary with the boundary bending degree greater than the threshold value is determined as the intersection contained in the street view images, the intersection identification based on the bending degree at the boundary is realized, and the identification accuracy is improved.
Based on the foregoing embodiments, this embodiment provides a possible implementation manner of a road intersection identification method, which explains that, in order to improve the accuracy of road intersection identification, according to the boundary curvature degree of consecutive multi-frame street view images, traffic objects present around an intersection, and difference information between adjacent street view images, the accuracy of identifying whether a street view image includes a road intersection is improved through interaction among multiple identification features. Fig. 7 is a schematic flow chart of another method for identifying a road intersection provided by the present application.
As shown in fig. 7, step 103 may comprise the following sub-steps:
and step 1031, screening the multiple frames of street view images collected in sequence to reserve partial street view images with the maximum value of the boundary bending degree larger than a set threshold value.
In the embodiment, the bending degree of each boundary point in each frame of street view image is determined, in the embodiment, an image in which the maximum value of the bending degree of the boundary point of each frame of street view image is greater than a set threshold is screened, and as the greater the curvature of the boundary is, the greater the possibility that the image contains a road intersection is, a part of street view images in which the maximum value of the bending degree of the boundary is greater than the set threshold can be used as candidate street view images possibly containing the road intersection, so that the calculation amount of identifying the road intersection by using a classification model in the subsequent steps is reduced, and the efficiency and the accuracy of identifying the road intersection are improved.
It should be understood that the set threshold of the degree of curvature is small, so as to realize recall of the street view image possibly containing the intersection as the preliminary identification of whether the street view image contains the intersection. And in the subsequent step, whether the street view image contains the accurate identification of the road intersection or not can be further accurately identified according to the determined identification characteristics.
As shown in fig. 8, the degree of curvature of the boundary point is determined by the number of pixels in the driving area included in the boundary point identification frame. In the street view images numbered 1,2,3 and 6, the boundaries of the driving area are smooth, that is, the maximum value of the curvature of the boundary of each boundary point is not greater than a set threshold, and in the street view images numbered 4 and 5, the boundary of the driving area is obviously curved, that is, the maximum value of the curvature of the boundary of each boundary point is greater than the set threshold, so that the street view images numbered 4 and 5 are screened out to be used as street view images possibly containing the road intersection, and further identification is performed through subsequent steps, so that whether the screened street view images really contain the road intersection is accurately determined, and the efficiency of identifying the road intersection is improved.
Step 1032, extracting identification features for each frame of street view image.
Specifically, for each frame of screened street view image, the intersection position is determined from the vehicle driving area of the corresponding street view image, wherein the intersection position is a section of the vehicle driving area of the corresponding street view image, in which the degree of boundary bending is greater than the set threshold value, because a section with a greater degree of boundary bending is more likely to be the area where the intersection is located, and therefore, a section of the vehicle driving area, in which the degree of boundary bending is greater than the set threshold value, can be used as the intersection position.
Furthermore, according to the semantic segmentation model, traffic objects present around the position of each intersection are determined, where the traffic objects include traffic markings and traffic lane dividing facilities, and may further include traffic lights, electronic eyes, and the like, where the traffic markings are, for example, zebra stripes, stop bars, and the like, and the traffic lane dividing facilities are, for example, lane lines, sign markings, and the like, as shown in fig. 9, taking the street view image marked as fig. 5 in fig. 9 as an example, after passing through the semantic segmentation model, it is recognized that the left side of the intersection includes the zebra stripes, and the left lower side includes the stop lines and the sign line-ground arrows. This is because at the non-intersection position, the surroundings of the driving area generally present impassable objects such as fences, road edges, buildings, etc., while at the intersection position, the surroundings of the determined intersection position generally present zebra stripes, stop lines, marking lines, etc., so that the traffic objects present at the surroundings of the determined intersection position are taken as a kind of identification feature, which can improve the accuracy of the intersection identification, and at the same time, the correspondence between the identified intersection and the traffic objects present at the surroundings is also determined, increasing the information contained in the identified intersection. And regarding each frame of street view image, taking the traffic objects and the boundary bending degree of the vehicle driving area, the difference information between the traffic objects in the adjacent street view image and the difference information between the boundary bending degree in the adjacent street view image as the identification characteristics of the corresponding street view image.
And 1033, inputting the extracted identification features into a classification model to identify whether the corresponding street view image contains the intersection.
Specifically, the extracted identification features are input into the classification model, and the classification model learns the mapping relationship between the identification features and the intersection identification result, so that the street view image containing the intersection can be accurately identified, and meanwhile, the identification features also contain the traffic objects presented around the intersection, so that after the street view image containing the intersection is identified, the traffic objects contained in the street view image containing the intersection can be obtained, for example, the street view image corresponding to fig. 5 is an image containing the intersection, the left side of the intersection is identified to contain the zebra crossing, and the left lower side of the intersection contains the zebra crossing and the marking line (ground passing arrow), thereby realizing not only accurately identifying the street intersection contained in the street view image, but also obtaining the traffic objects contained around the street intersection, the richness of the navigation map information created subsequently is improved.
Further to improve the accuracy of the classification model identification, optionally, the identification feature further includes a road intersection probability identified by the convolutional neural network according to the input street view image, where the convolutional neural network has learned a mapping relationship between the identification feature of the street view image and the road intersection probability.
Further, for each street view image, based on the identification features, a classification model obtained through training is adopted to predict and score whether the corresponding street view image contains the intersection, namely, whether the street view image contains the intersection is identified, wherein a calculation formula adopted when the classification model identifies is as follows:
Figure BDA0002389008110000101
Si=f(Ri,Di,Vi,Ti) Wherein C is an identification frame which takes all boundary points of the positions of the road intersections in the vehicle driving area in the corresponding street view determined according to the semantic segmentation model as corner points, and R is an identification frame which takes all boundary points as corner pointsiIntersections around the identified frames corresponding to each respective boundary pointThrough object, and degree of boundary curvature, DiThe difference information of the traffic objects in the adjacent street view images and the difference information of the boundary bending degree in the adjacent street view images. ViAnd representing the intersection probability output by the convolutional neural network. T isiThe attribute features of the intersection can be extracted from an existing database, and if the attribute features of the intersection do not exist in the database, the attribute features can be set as default values. f is a rank function, is a classification model obtained through supervised training, such as a Gradient Boosting Decision Tree (GBDT) model, and improves the accuracy of classification model identification by adding identification features.
In the intersection recognition method of the present embodiment, after determining the vehicle travel area in the street view image, screening the multi-frame street view images collected in sequence to reserve partial street view images with the maximum value of the boundary bending degree larger than a set threshold value, realizing the preliminary screening of the street view images containing the road intersection, further, extracting identification characteristics from the screened images possibly containing the intersection, and determining the boundary bending degree of the traffic objects and the vehicle driving area presented around the intersection contained in the identification characteristics, and the difference information with the traffic object in the adjacent street view image and the difference information with the boundary bending degree in the adjacent street view image are input into the classification model, and accurately identifying street view images containing the road intersection by using the classification model, wherein the identification characteristics also comprise traffic objects appearing around the road intersection.
Based on the above embodiment, as a possible implementation manner, the identification feature may further include one or more combinations of the area of the vehicle driving region, the number of lanes included in the vehicle driving region, and the road grade of the road segment where the vehicle driving region is located, that is, the identification feature is added, so that the identification features include not only the traffic objects present around the intersection, the degree of curvature of the boundary of the vehicle travel area included in the above embodiment, and the difference information with the traffic object in the adjacent street view image and the difference information with the boundary bending degree in the adjacent street view image, and one or more combinations of the area of the vehicle driving area, the number of lanes contained in the vehicle driving area and the road grade of the road section where the vehicle driving area is located, by adding the identification features input into the classification model, the accuracy of road intersection identification can be improved.
In order to realize the embodiment, the application also provides a road intersection identification device.
Fig. 10 is a schematic structural diagram of a road intersection recognition device according to an embodiment of the present application.
As shown in fig. 10, the apparatus includes: an acquisition module 91, a segmentation module 92 and an identification module 93.
The acquiring module 91 is configured to acquire a street view image.
And the segmentation module 92 is used for identifying the street view image by adopting a semantic segmentation model so as to determine the vehicle driving area from the street view image.
The identifying module 93 is configured to identify whether the street view image includes a road intersection according to the degree of curvature of the boundary of the driving area of the vehicle.
Further, in a possible implementation manner of the embodiment of the present application, as a possible implementation manner, the apparatus further includes: and determining a module.
As a first possible implementation manner, the determining module is configured to determine, for each boundary point in the vehicle driving area, an identification frame using the corresponding boundary point as an angular point; and determining the boundary bending degree at the corresponding boundary point according to the number of the pixel points belonging to the vehicle driving area in the identification frame.
As a second possible implementation manner, the determining module is further configured to group a plurality of boundary points in the vehicle driving area, where each group includes a fixed number of boundary points arranged adjacently; fitting boundary points in the same group to obtain a fitted straight line; and for each group of boundary points, determining the boundary bending degree of each boundary point in the group according to the included angle between the corresponding fitting straight line and the adjacent fitting straight line.
As a possible implementation manner, the street view image is a plurality of frames collected sequentially, and the identifying module 93 includes:
the extraction unit is used for extracting identification characteristics aiming at each frame of street view image; wherein the identification feature includes a degree of curvature of a boundary of a driving area of the vehicle in the corresponding street view image and includes difference information from a degree of curvature of a boundary of an adjacent street view image.
And the identification unit is used for inputting the extracted identification features into the classification model so as to identify whether the corresponding street view image contains the intersection, wherein the classification model learns the mapping relation between the identification features and the intersection identification result.
As a possible implementation manner, the identification module 93 further includes:
and the screening unit is used for screening the multi-frame street view images collected in sequence so as to reserve partial street view images with the maximum value of the boundary bending degree larger than a set threshold value.
As a possible implementation manner, the extracting unit is specifically configured to:
for each frame of street view image, determining the position of a road intersection from the vehicle driving area of the corresponding street view image; the method comprises the steps that road intersection positions are sections with the boundary bending degree larger than a set threshold value in a vehicle driving area of a corresponding street view image, traffic objects presented around each road intersection position are determined according to a semantic segmentation model, and for each frame of street view image, the traffic objects presented around the road intersection, the boundary bending degree of the vehicle driving area, difference information between the traffic objects in adjacent street view images and the boundary bending degree in the adjacent street view images are used as identification features of the corresponding street view images.
As one possible implementation, the traffic object includes traffic markings and traffic lane markings.
As a possible implementation, the identification feature further includes one or more combinations of an area of a vehicle travel area, the number of lanes included in the vehicle travel area, and a road grade of a road segment in which the vehicle travel area is located.
As a possible implementation manner, the identification feature further includes a road intersection probability identified by the convolutional neural network according to the input street view image, wherein the convolutional neural network has learned a mapping relationship between the identification feature of the street view image and the road intersection probability.
It should be noted that the explanation of the embodiment of the method for identifying a road intersection is also applicable to the apparatus for identifying a road intersection of the embodiment, and the principle is the same, and is not repeated here.
In the road intersection recognition device provided by the embodiment of the application, the street view image is obtained through the camera, the street view image is recognized by adopting the semantic segmentation model, the vehicle driving area is determined from the street view image, whether the street view image contains the road intersection is recognized according to the boundary bending degree of the vehicle driving area, so that the peripheral information of the road intersection and the intersection can be accurately recognized, meanwhile, the position of the road intersection can be determined according to the GPS information of each image, and the cost of road intersection recognition is reduced because expensive laser distance meters, laser radars and other equipment are not adopted.
In order to implement the foregoing embodiments, the present application provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and running on the processor, and when the processor executes the computer program, the processor implements the intersection identification method according to the foregoing method embodiments.
In order to implement the above embodiments, the present application provides a non-transitory computer-readable storage medium, on which a computer program is stored, which when executed by a processor, implements the intersection identification method as described in the foregoing method embodiments.
According to an embodiment of the present application, an electronic device and a readable storage medium are also provided.
As shown in fig. 11, the embodiment of the present application is a block diagram of an electronic device according to a method for identifying a road intersection. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.
As shown in fig. 11, the electronic apparatus includes: one or more processors 1001, memory 1002, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input/output apparatus (such as a display device coupled to the interface). In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). Fig. 11 illustrates an example of one processor 1001.
The memory 1002 is a non-transitory computer readable storage medium provided herein. Wherein the memory stores instructions executable by at least one processor to cause the at least one processor to perform the method of intersection identification provided herein. A non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to perform the intersection identification method provided by the present application.
The memory 1002, as a non-transitory computer-readable storage medium, may be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions/modules (e.g., the acquisition module 91, the segmentation module 92, and the identification module 93 shown in fig. 10) corresponding to the intersection identification method in the embodiment of the present application. The processor 1001 executes various functional applications of the server and data processing, i.e., implements the intersection identification method lane in the above-described method embodiment, by running non-transitory software programs, instructions, and modules stored in the memory 1002.
The memory 1002 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the electronic device for the intersection recognition method, and the like. Further, the memory 1002 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 1002 may optionally include memory located remotely from the processor 1001, which may be connected to the electronic device of the intersection identification method via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device of the intersection identification method may further include: an input device 1003 and an output device 1004. The processor 1001, the memory 1002, the input device 1003, and the output device 1004 may be connected by a bus or other means, and fig. 11 illustrates an example of connection by a bus.
The input device 1003 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic equipment of the intersection recognition method, such as an input device such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointer, one or more mouse buttons, a track ball, a joystick, or the like. The output devices 1004 may include a display device, auxiliary lighting devices (e.g., LEDs), and tactile feedback devices (e.g., vibrating motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device can be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
According to the technical scheme of the embodiment of the application, the street view image is obtained, the street view image is identified by adopting the semantic segmentation model, the vehicle driving area is determined from the street view image, whether the street view image contains the road intersection or not is identified according to the boundary bending degree of the vehicle driving area, so that the peripheral information of the road intersection and the intersection can be accurately identified, meanwhile, the position of the road intersection can be determined according to the GPS information of each image, and the cost of identifying the road intersection is reduced because expensive laser distance meters, laser radars and other equipment are not adopted.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and the present invention is not limited thereto as long as the desired results of the technical solutions disclosed in the present application can be achieved.
The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims (20)

1. A method of identifying a road intersection, the method comprising:
obtaining a street view image;
identifying the street view image by adopting a semantic segmentation model so as to determine a vehicle driving area from the street view image;
and identifying whether the street view image contains a road intersection or not according to the boundary bending degree of the vehicle driving area.
2. The method for identifying a road intersection as claimed in claim 1, wherein the street view image is a plurality of frames collected sequentially; the identifying whether the street view image contains a road intersection or not according to the boundary bending degree of the vehicle driving area comprises the following steps:
extracting identification features aiming at each frame of the street view image; wherein the identification feature includes a boundary curvature degree of the vehicle driving region in the corresponding street view image and includes difference information from a boundary curvature degree of an adjacent street view image;
inputting the extracted identification features into a classification model to identify whether the corresponding street view image contains a road intersection; and the classification model learns the mapping relation between the identification features and the road intersection identification results.
3. The method for identifying a road intersection according to claim 2, wherein before extracting identification features for each frame of the street view image, the method further comprises:
and screening the multiple frames of street view images collected sequentially to reserve partial street view images with the maximum value of the boundary bending degree larger than a set threshold value.
4. The method for identifying a road intersection according to claim 2, wherein the extracting identification features for each frame of the street view image comprises:
for each frame of street view image, determining the position of a road intersection from the vehicle driving area of the corresponding street view image; the intersection position is a section of the vehicle driving area of the corresponding street view image, wherein the boundary bending degree of the vehicle driving area is greater than the set threshold value;
determining traffic objects appearing around the position of each road intersection according to the semantic segmentation model;
and regarding each frame of street view image, taking the traffic objects presented around the intersection, the boundary bending degree of the vehicle driving area, the difference information between the traffic objects in the adjacent street view image and the difference information between the boundary bending degree in the adjacent street view image as the identification characteristics of the corresponding street view image.
5. The method of identifying a pathway intersection of claim 4, wherein the traffic objects include traffic markings and traffic lane markings.
6. The intersection identification method according to claim 2, characterized in that the identification features further include one or more combinations of an area of the vehicle travel area, the number of lanes contained in the vehicle travel area, and a road grade of a road section on which the vehicle travel area is located.
7. The method of identifying intersections according to claim 2 wherein the identifying characteristics further include intersection probabilities identified by the convolutional neural network from the input streetscape images;
and the convolutional neural network learns the mapping relation between the identification features of the street view image and the intersection probability.
8. The method for identifying a road intersection according to any one of claims 1-7, wherein the identifying the street view image by adopting a semantic segmentation model to determine a vehicle driving area from the street view image comprises the following steps:
determining an identification frame with the corresponding boundary point as an angular point for each boundary point in the vehicle driving area;
and determining the boundary bending degree at the corresponding boundary point according to the number of the pixel points belonging to the vehicle driving area in the identification frame.
9. The method for identifying a road intersection according to any one of claims 1-7, wherein the identifying the street view image by adopting a semantic segmentation model to determine a vehicle driving area from the street view image comprises the following steps:
grouping a plurality of boundary points in the vehicle driving area, wherein each group comprises a fixed number of boundary points which are adjacently arranged;
fitting boundary points in the same group to obtain a fitted straight line;
and for each group of boundary points, determining the boundary bending degree of each boundary point in the group according to the included angle between the corresponding fitting straight line and the adjacent fitting straight line.
10. A road intersection identification device, the device comprising:
the acquisition module is used for acquiring street view images;
the segmentation module is used for identifying the street view image by adopting a semantic segmentation model so as to determine a vehicle driving area from the street view image;
and the identification module is used for identifying whether the street view image contains a road intersection or not according to the boundary bending degree of the vehicle driving area.
11. The intersection identification device of claim 10, wherein the street view images are a plurality of frames collected sequentially; the identification module comprises:
the extraction unit is used for extracting identification characteristics aiming at each frame of the street view image; wherein the identification feature includes a boundary curvature degree of the vehicle driving region in the corresponding street view image and includes difference information from a boundary curvature degree of an adjacent street view image;
the identification unit is used for inputting the extracted identification features into a classification model so as to identify whether the corresponding street view image contains a road intersection or not; and the classification model learns the mapping relation between the identification features and the road intersection identification results.
12. The intersection identification device of claim 11, wherein the identification module further comprises:
and the screening unit is used for screening the multi-frame street view images collected in sequence so as to reserve partial street view images with the maximum value of the boundary bending degree larger than a set threshold value.
13. The intersection recognition device of claim 11, wherein the extraction unit is specifically configured to:
for each frame of street view image, determining the position of a road intersection from the vehicle driving area of the corresponding street view image; the intersection position is a section of the vehicle driving area of the corresponding street view image, wherein the boundary bending degree of the vehicle driving area is greater than the set threshold value;
determining traffic objects appearing around the position of each road intersection according to the semantic segmentation model;
and regarding each frame of street view image, taking the traffic objects presented around the intersection, the boundary bending degree of the vehicle driving area, the difference information between the traffic objects in the adjacent street view image and the difference information between the boundary bending degree in the adjacent street view image as the identification characteristics of the corresponding street view image.
14. The intersection identification device of claim 13, wherein the traffic objects comprise traffic markings and traffic lane markings.
15. The intersection identification device of claim 11, wherein the identification features further comprise one or more combinations of an area of the vehicle travel area, a number of lanes contained in the vehicle travel area, and a road grade of a road segment on which the vehicle travel area is located.
16. The intersection identification device of claim 11, wherein the identification features further comprise intersection probabilities identified by the convolutional neural network from the input streetscape image;
and the convolutional neural network learns the mapping relation between the identification features of the street view image and the intersection probability.
17. An intersection identification device according to any one of claims 10 to 16, further comprising:
the determining module is used for determining an identification frame with the corresponding boundary point as an angular point for each boundary point in the vehicle driving area; and determining the boundary bending degree at the corresponding boundary point according to the number of the pixel points belonging to the vehicle driving area in the identification frame.
18. The intersection identification device of any of claims 10-16, wherein the determining module is further configured to:
grouping a plurality of boundary points in the vehicle driving area, wherein each group comprises a fixed number of boundary points which are adjacently arranged; fitting boundary points in the same group to obtain a fitted straight line; and for each group of boundary points, determining the boundary bending degree of each boundary point in the group according to the included angle between the corresponding fitting straight line and the adjacent fitting straight line.
19. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor when executing the program implementing a method of identifying a pathway intersection as claimed in any one of claims 1 to 9.
20. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the program, when executed by a processor, implements a method of identifying a pathway intersection as claimed in any one of claims 1 to 9.
CN202010110375.9A 2020-02-21 2020-02-21 Road intersection identification method, device, computer equipment and storage medium Active CN113297878B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010110375.9A CN113297878B (en) 2020-02-21 2020-02-21 Road intersection identification method, device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010110375.9A CN113297878B (en) 2020-02-21 2020-02-21 Road intersection identification method, device, computer equipment and storage medium

Publications (2)

Publication Number Publication Date
CN113297878A true CN113297878A (en) 2021-08-24
CN113297878B CN113297878B (en) 2023-12-08

Family

ID=77317671

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010110375.9A Active CN113297878B (en) 2020-02-21 2020-02-21 Road intersection identification method, device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113297878B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113865610A (en) * 2021-09-30 2021-12-31 北京百度网讯科技有限公司 Method, apparatus, device, medium and product for generating navigation information
WO2023179030A1 (en) * 2022-03-24 2023-09-28 商汤集团有限公司 Road boundary detection method and apparatus, and electronic device, storage medium and computer program product

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102908154A (en) * 2012-11-07 2013-02-06 北京工业大学 Method for detecting start reaction time of driver based on machine vision
CN103162691A (en) * 2011-12-13 2013-06-19 爱信艾达株式会社 Display system, display method and display program supporting a lane change in a vehicle
CN103954277A (en) * 2014-04-30 2014-07-30 百度在线网络技术(北京)有限公司 Method and device for detecting positions of interest points
CN104422458A (en) * 2013-08-30 2015-03-18 深圳富泰宏精密工业有限公司 Vehicle navigation system and vehicle navigation method
US20150153735A1 (en) * 2013-12-04 2015-06-04 Mobileye Vision Technologies Ltd. Systems and methods for detecting and responding to traffic laterally encroaching upon a vehicle
CN109147331A (en) * 2018-10-11 2019-01-04 青岛大学 A kind of congestion in road condition detection method based on computer vision
CN109582993A (en) * 2018-06-20 2019-04-05 长安大学 Urban transportation scene image understands and multi-angle of view gunz optimization method
CN110170148A (en) * 2019-05-13 2019-08-27 苏州米龙信息科技有限公司 A kind of intermittent training system and method with amusement interactive function

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103162691A (en) * 2011-12-13 2013-06-19 爱信艾达株式会社 Display system, display method and display program supporting a lane change in a vehicle
CN102908154A (en) * 2012-11-07 2013-02-06 北京工业大学 Method for detecting start reaction time of driver based on machine vision
CN104422458A (en) * 2013-08-30 2015-03-18 深圳富泰宏精密工业有限公司 Vehicle navigation system and vehicle navigation method
US20150153735A1 (en) * 2013-12-04 2015-06-04 Mobileye Vision Technologies Ltd. Systems and methods for detecting and responding to traffic laterally encroaching upon a vehicle
CN103954277A (en) * 2014-04-30 2014-07-30 百度在线网络技术(北京)有限公司 Method and device for detecting positions of interest points
CN109582993A (en) * 2018-06-20 2019-04-05 长安大学 Urban transportation scene image understands and multi-angle of view gunz optimization method
CN109147331A (en) * 2018-10-11 2019-01-04 青岛大学 A kind of congestion in road condition detection method based on computer vision
CN110170148A (en) * 2019-05-13 2019-08-27 苏州米龙信息科技有限公司 A kind of intermittent training system and method with amusement interactive function

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
李观石 等: "基于MMS街景的导航数据采集方法研究", 《现代测绘》, vol. 36, no. 01, pages 44 - 46 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113865610A (en) * 2021-09-30 2021-12-31 北京百度网讯科技有限公司 Method, apparatus, device, medium and product for generating navigation information
WO2023179030A1 (en) * 2022-03-24 2023-09-28 商汤集团有限公司 Road boundary detection method and apparatus, and electronic device, storage medium and computer program product

Also Published As

Publication number Publication date
CN113297878B (en) 2023-12-08

Similar Documents

Publication Publication Date Title
CN112560609B (en) Road condition estimation method, method for establishing road condition estimation model and corresponding device
CN111709328B (en) Vehicle tracking method and device and electronic equipment
CN111695488B (en) Method, device, equipment and storage medium for identifying interest surface
CN111797187A (en) Map data updating method and device, electronic equipment and storage medium
US20180124319A1 (en) Method and apparatus for real-time traffic information provision
US20210209395A1 (en) Method, electronic device, and storage medium for recognizing license plate
CN111275983B (en) Vehicle tracking method, device, electronic equipment and computer-readable storage medium
CN111611955B (en) Method, device, equipment and storage medium for identifying passable construction road
CN112507949A (en) Target tracking method and device, road side equipment and cloud control platform
CN111767831B (en) Method, apparatus, device and storage medium for processing image
CN113283272B (en) Real-time image information prompting method and device for road congestion and electronic equipment
CN111950537A (en) Zebra crossing information acquisition method, map updating method, device and system
CN113091757B (en) Map generation method and device
CN114037966A (en) High-precision map feature extraction method, device, medium and electronic equipment
CN112334906A (en) Example segmentation inferred from machine learning model output
CN114443794A (en) Data processing and map updating method, device, equipment and storage medium
CN113297878B (en) Road intersection identification method, device, computer equipment and storage medium
CN111597986B (en) Method, apparatus, device and storage medium for generating information
CN111931683A (en) Image recognition method, image recognition device and computer-readable storage medium
CN111950345A (en) Camera identification method and device, electronic equipment and storage medium
CN115661522A (en) Vehicle guiding method, system, equipment and medium based on visual semantic vector
CN113011298B (en) Truncated object sample generation, target detection method, road side equipment and cloud control platform
CN111442775B (en) Road identification method and device, electronic equipment and readable storage medium
CN113361303B (en) Temporary traffic sign board identification method, device and equipment
CN111325811B (en) Lane line data processing method and processing device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant