CN117078799B - Special parking space synthesis method and device based on BEV image - Google Patents
Special parking space synthesis method and device based on BEV image Download PDFInfo
- Publication number
- CN117078799B CN117078799B CN202310951572.7A CN202310951572A CN117078799B CN 117078799 B CN117078799 B CN 117078799B CN 202310951572 A CN202310951572 A CN 202310951572A CN 117078799 B CN117078799 B CN 117078799B
- Authority
- CN
- China
- Prior art keywords
- parking space
- special
- special mark
- bev
- bev image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001308 synthesis method Methods 0.000 title claims description 9
- 230000002194 synthesizing effect Effects 0.000 claims abstract description 36
- 238000000034 method Methods 0.000 claims abstract description 26
- 230000015572 biosynthetic process Effects 0.000 claims abstract description 20
- 238000003786 synthesis reaction Methods 0.000 claims abstract description 20
- 238000012216 screening Methods 0.000 claims abstract description 15
- 238000004422 calculation algorithm Methods 0.000 claims description 12
- 239000002131 composite material Substances 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 230000009466 transformation Effects 0.000 claims description 9
- 238000001514 detection method Methods 0.000 abstract description 7
- 238000012549 training Methods 0.000 abstract description 6
- 240000004282 Grewia occidentalis Species 0.000 description 15
- 238000002372 labelling Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 230000004927 fusion Effects 0.000 description 4
- 239000003550 marker Substances 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 235000004522 Pentaglottis sempervirens Nutrition 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000000049 pigment Substances 0.000 description 2
- 241001300198 Caperonia palustris Species 0.000 description 1
- 235000000384 Veronica chamaedrys Nutrition 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/60—Editing figures and text; Combining figures or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
- G06V10/225—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition based on a marking or identifier characterising the area
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V20/586—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of parking space
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
- G06T2207/30264—Parking
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a special parking space synthesizing method and device based on BEV images, wherein the method comprises the following steps: acquiring BEV images and 2D special marks; identifying a parking space and a state of the parking space in the BEV image, and screening a target parking space for synthesizing with the 2D special mark according to the state of the parking space; and calculating the coordinate position of the target parking space for placing the 2D special mark according to the type of the 2D special mark, and fusing the 2D special mark and the BEV image into a target BEV image according to the coordinate position of the 2D special mark. The invention realizes the synthesis of the special parking space in the aerial view, can meet the detection requirements of the special parking space in different scenes, and solves the model training problem in the data shortage to a great extent.
Description
Technical Field
The invention relates to the field of software development, in particular to a special parking space synthesis method and device based on BEV images.
Background
With the continuous progress of artificial intelligence technology, image-based automatic driving technology is also rapidly developed, wherein an automatic parking system is vital, a parking space detection method is layered along with the automatic parking system, and the effect of a parking space detection algorithm is largely determined by the quality of training data. At present, the mode of acquiring training data, namely the data with marked aerial view (BEV Bird Eye View), is mainly vehicle acquisition, but for some special parking spaces, such as special parking spaces for disabled people or forbidden parking spaces, the acquisition difficulty is extremely high due to the fact that the number of the special parking spaces is relatively rare and the distribution is unpredictable, and if the type of parking spaces are required to be detected independently in the parking space detection process, more relevant data are required to be acquired by adopting a data synthesis method.
Disclosure of Invention
Aiming at the technical problems, the invention provides a special parking space synthesizing method and device based on BEV images, which can realize the synthesis of special parking spaces based on BEV images.
In a first aspect of the invention, a method for synthesizing a special parking space based on BEV images is provided, comprising:
acquiring BEV images and 2D special marks;
identifying a parking space and a state of the parking space in the BEV image, and screening a target parking space for synthesizing with the 2D special mark according to the state of the parking space;
And calculating the coordinate position of the target parking space for placing the 2D special mark according to the type of the 2D special mark, and fusing the 2D special mark and the BEV image into a target BEV image according to the coordinate position of the 2D special mark.
In an optional embodiment, the identifying the parking space and the state of the parking space in the BEV image, and screening the target parking space for synthesizing with the 2D special mark according to the state of the parking space includes:
identifying four corner coordinates of a parking space in the BEV image;
Determining the state of the parking space according to whether the parking space is occupied or not;
and selecting the unoccupied parking space as a target parking space for synthesizing the 2D special mark.
In an alternative embodiment, the identifying the parking space and the status of the parking space in the BEV image further includes:
The shape of the parking space and the linearity of the parking space are identified, the shape of the parking space comprises parallel parking spaces, vertical parking spaces and diagonal parking spaces, and the linearity of the parking space comprises a closed parking space line, a semi-closed parking space line, an open parking space line, two-angle parking space lines and four-angle parking space lines;
And identifying whether the parking space is locked or not, and if the vehicle is parked, determining that the parking space is occupied if the parking space is locked or parked.
In an optional embodiment, the calculating the coordinate position of the target parking space for placing the 2D special mark according to the type of the 2D special mark includes:
determining prior knowledge of the position of the 2D special mark in the parking space according to the type of the 2D special mark;
performing binarization processing on the 2D special mark, and determining a center point of the 2D special mark;
and determining the coordinate position for placing the 2D special mark in the target parking space according to the central point of the 2D special mark and the coordinates of four corner points of the parking space.
In an alternative embodiment, the fusing the 2D special marker and the BEV image into the target BEV image according to the coordinate position of the 2D special marker includes:
The 2D special mark is adjusted in size to adapt to a synthesized template, and the color of the 2D special mark is consistent with the color adjustment of the parking space line of the parking space;
and converting the center point of the 2D special mark into the synthetic template, and projecting the synthetic template into the BEV image through a homography transformation algorithm according to the corresponding relation between four vertex coordinates of the synthetic template and four corner coordinates of the target parking space.
In an alternative embodiment, the fusing the 2D special marker and the BEV image into the target BEV image according to the coordinate position of the 2D special marker includes:
Converting the 2D special mark into a preset synthesis template through a homography transformation algorithm;
Fusing the composite template with the BEV image to a target BEV image; the size of the synthesized template is the same as that of the target parking space.
In an alternative embodiment, the identifying the parking space and the status of the parking space in the BEV image includes:
and identifying the color of the parking space line of the target parking space, and processing the background color of the 2D special mark into the color consistent with the color of the parking space line of the target parking space.
In a second aspect of the invention, there is provided a special parking space synthesizing device based on BEV images, comprising:
The acquisition module is used for acquiring the BEV image and the 2D special mark;
The identifying and screening module is used for identifying the parking space and the state of the parking space in the BEV image, and screening out a target parking space for synthesizing with the 2D special mark according to the state of the parking space;
And the synthesis module is used for calculating the coordinate position of the target parking space for placing the 2D special mark according to the type of the 2D special mark, and fusing the 2D special mark and the BEV image into a target BEV image according to the coordinate position of the 2D special mark.
In a third aspect of the present invention, there is provided an electronic apparatus comprising:
At least one processor; and at least one memory communicatively coupled to the processor, wherein: the memory stores program instructions executable by the processor, the processor invoking the program instructions to perform the method according to the first aspect of the embodiments of the invention.
In a fourth aspect of the invention, a computer-readable storage medium is provided, on which a computer program is stored which, when run by a computer, performs the method according to the first aspect of the embodiment of the invention.
According to the invention, the parking space and the state of the parking space in the BEV image are identified, the target parking space for synthesizing with the 2D special mark is screened according to the state of the parking space, then the coordinate position of the target parking space for placing the 2D special mark is calculated according to the type of the 2D special mark, the 2D special mark and the BEV image are fused into the target BEV image according to the coordinate position of the 2D special mark, the synthesis of the special parking space in a bird's eye view is realized, the detection requirement of the special parking space under different scenes can be met, and the model training problem when data shortage is solved to a great extent.
Drawings
FIG. 1 is a flow chart of a method for synthesizing a special parking space based on BEV images in an embodiment of the invention.
FIG. 2 is a schematic diagram of another method for synthesizing a special parking space based on BEV images in accordance with an embodiment of the present invention.
FIG. 3 is a schematic diagram of another method for synthesizing a special parking space based on BEV images in accordance with an embodiment of the present invention.
Fig. 4 is a schematic diagram of a method for synthesizing a special parking space based on BEV images in an embodiment of the present invention.
Fig. 5 is a scene comparison diagram of three synthetic 2D special markers in an embodiment of the invention.
FIG. 6 is a schematic block diagram of a special parking space synthesizing device based on BEV images in an embodiment of the invention.
Fig. 7 is a schematic structural view of an electronic device according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It is also to be understood that the terminology used in the description of the present disclosure is for the purpose of describing particular embodiments only, and is not intended to be limiting of the disclosure. As used in the specification and claims of this disclosure, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should be further understood that the term "and/or" as used in the present disclosure and claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
For normal parking space, certain special parking space quantity is few, seeks the degree of difficulty of gathering very big, even can gather, also under certain fixed scene, makes its proportion that appears in the training set very low, and data distribution is uneven, if need to detect this type parking space, the degree of difficulty is very big, and the effect is very poor. Therefore, more relevant data are required to be acquired by adopting a data synthesis mode, and because the synthesized data can be theoretically generated in any scene, the distribution of corresponding data can be increased while the data acquisition pressure is greatly reduced by adopting the synthesized data, and the detection effect of a model is better improved. And the synthesized data has the corresponding label result, so that the cost of part of manual labeling can be saved. Therefore, the invention aims to construct a data synthesis system for synthesizing a special parking space by firstly detecting the parking space in the BEV image and then adding a 2D special mark.
The invention provides a new BEV image special parking space synthesizing method. Firstly, relevant BEV images are imported from cloud data lakes, parking spaces in the images are detected by calling a parking space pre-labeling model deployed by the cloud, parking spaces needing to be synthesized are selected from the parking spaces, and positions needing to be placed with marks are calculated according to corner coordinates detected by the parking spaces. And then carrying out data enhancement on the mark template to be synthesized, and finally projecting the enhanced template to a corresponding position to be fused with the original BEV image.
Specifically, fig. 1 is a schematic flow chart of a special parking space synthesizing method based on BEV images in an embodiment of the invention. Referring to fig. 1, the present invention provides a special parking space synthesizing method based on BEV images, comprising the following steps:
step 100: BEV images and 2D special markers are acquired.
The BEV image can be obtained from a cloud data lake, and the BEV image comprises parking spaces, wherein the parking spaces can be parking spaces in an original scene or parking spaces synthesized by the BEV image in the later period. The 2D special mark comprises a special parking space mark for the disabled, a special parking space for units, a private parking space, a school bus parking space, a civil air defense parking space and the like. The 2D special mark may be obtained from TT100K (traffic sign dataset).
Step 200: and identifying the parking space and the state of the parking space in the BEV image, and screening out a target parking space for synthesizing with the 2D special mark according to the state of the parking space.
In this step, a parking space pre-labeling model (trained based on neural networks) may be used to identify the parking space in the BEV image. The parking space pre-labeling model can detect four corner coordinates of each parking space and a space line of each parking space in the BEV image, and can output the space shape and the space linearity of each parking space. Of course, the occupation information of each parking space can also be obtained, so that the available parking space can be screened out.
In some embodiments, the shape of the parking space and the linearity of the parking space are identified, the shape of the parking space comprises parallel parking spaces, perpendicular parking spaces and inclined parking spaces, and the linearity of the parking space comprises a closed parking space line, a semi-closed parking space line, an open parking space line, two-angle parking space lines and four-angle parking space lines.
Because the 2D special signs may appear randomly in the parking scenes, not every parking scene has a 2D special sign on the parking spot. Therefore, 2D special marks are not needed to be synthesized on each parking space, the occupied parking space can be screened, the unoccupied parking space is taken as a target parking space, and the target parking space can be randomly synthesized with the 2D special marks.
Step 300: and calculating the coordinate position of the target parking space for placing the 2D special mark according to the type of the 2D special mark, and fusing the 2D special mark and the BEV image into a target BEV image according to the coordinate position of the 2D special mark.
Considering that the types of the 2D special marks are different, the sizes of the 2D special marks are also different, and the placement positions of the 2D special marks are also different, so that the coordinate positions of the marks can be calculated by combining the types of the 2D special marks with priori knowledge. In this step, the 2D special mark may be directly attached to the parking space in the BEV image according to the coordinate position, or may be fused with the BEV image through a composite template.
Further, in the step 200, the identifying the parking space and the state of the parking space in the BEV image, and screening the target parking space for synthesizing with the 2D special mark according to the state of the parking space, as shown in fig. 2, includes the following steps:
Step 210: and identifying the parking space in the BEV image and four corner coordinates of the parking space.
The calculation data for determining the 2D special mark synthesis position is identified by using the parking space pre-marking model, and the region range of the parking space can be determined according to the four corner coordinates of the parking space. In some embodiments, the color of the parking space line may also be obtained through the four corner coordinates, so that the color is assigned to the 2D special mark synthesis, so that the 2D special mark synthesis is similar to or consistent with the color of the parking space.
Step 220: and determining the state of the parking space according to whether the parking space is occupied or not.
After the parking space in the BEV image is identified by using the parking space pre-labeling model, whether the parking space is locked or not and whether a vehicle is parked or not can be identified, if the parking space is locked or parked, the fact that the parking space is occupied is determined, and therefore the state of the parking space is determined.
Whether the parking space is locked or not can be judged by identifying whether the parking space is locked or not in the parking space area; or whether the parking space is provided with an obstacle is identified, if so, the parking space is judged to be provided with the obstacle, and the parking space is represented to be occupied by parking. In addition, the parking space is locked, and the general 2D special mark can not be used together with the ground lock, and still is judged to be in an occupied state.
Step 230: and selecting the unoccupied parking space as a target parking space for synthesizing the 2D special mark.
The unoccupied parking spaces in the BEV image are determined through the steps, and then the parking spaces are determined to be target parking spaces, so that the special parking spaces can be synthesized with the 2D special mark.
When the target parking space and the 2D special mark are synthesized, the coordinate position of the 2D special mark placed on the target parking space needs to be determined. In some embodiments of the invention, a 2D special sign is converted into a parallelogram-shaped composite template similar to the size of the parking space. And then, carrying out homography transformation calculation on the composite template and the parking space, and projecting the composite template into the whole BEV image. The method comprises the following steps:
when the coordinate position of the target parking space for placing the 2D special mark is calculated, the method is implemented by the following steps, as shown in fig. 3 and 4:
Step 310: and determining prior knowledge of the position of the 2D special mark in the parking space according to the type of the 2D special mark.
For example, a parking space mark special for the disabled is generally located at the center of the left or right direction of a parking space, and is located at a front quarter position in the front-rear direction, and coordinates of the position can be calculated by using coordinates of four corner points to serve as a target position. The four corner coordinates of each parking space of the BEV image can be obtained through the output of a parking space pre-labeling model.
Step 320: and carrying out binarization processing on the 2D special mark, and determining the center point of the 2D special mark.
Step 330: and determining the coordinate position for placing the 2D special mark in the target parking space according to the central point of the 2D special mark and the coordinates of four corner points of the parking space.
The 2D special mark and the BEV image are synthesized by using a Poisson fusion algorithm, and the fusion center point coordinate of the Poisson fusion algorithm is determined first; the coordinates of the fusion center point are the geometric center of the 2D special mark. And carrying out binarization processing on the 2D special mark to obtain a binarization template, wherein most 2D special marks are in irregular shapes, the coordinates of the farthest points in the up-down, left-right directions of the 2D special mark in the binarization template need to be obtained, and the average value of the last four coordinate points is the coordinates of the center point.
In some embodiments, the 2D special mark is resized to adapt to a composite template, the color of the 2D special mark is consistent with the color adjustment of the parking space line of the parking space, and the positioning and calculation are completed by adopting a flow as shown in fig. 4, which is exemplified as a special parking space mark for the disabled.
Illustratively, in some embodiments, the 2D special markers are converted into a preset composite template by a homography transformation algorithm; fusing the composite template with the BEV image to a target BEV image; the size of the synthesized template is the same as that of the target parking space.
Specifically, according to the conversion of the center point of the 2D special mark into the synthesis template, the synthesis template is a parallelogram template with the same size as the parking space to be synthesized in the BEV image, based on the center point of the 2D special mark and the placement position of the 2D special mark at approximately one quarter of the parallelogram, the attachment position of the 2D special mark in the synthesis template can be determined, and the 2D special mark is directly attached to the synthesis template.
And then, according to the corresponding relation between the four vertex coordinates of the synthetic template and the four corner coordinates of the target parking space, projecting the synthetic template to the corresponding parking space in the BEV image through a homography transformation algorithm.
Furthermore, in the identifying of the parking space pre-labeling model, the parking space line color of the target parking space can be identified when the parking space and the state of the parking space in the BEV image are identified, and then the background color of the 2D special mark can be processed to be consistent with the parking space line color of the target parking space when the binarization processing is performed on the 2D special mark; and the background pigment of the 2D special mark is adjusted to be consistent with the pigment of the parking space line of the target parking space.
Referring to fig. 5, fig. 5 is a schematic diagram of a BEV image synthesis 2D special mark of three complementary scenes. The upper graph in the graph a is a BEV original image, and the lower graph is a BEV image of a single-side synthesized 2D special mark in a parallel parking space; the upper graph in the b graph is a BEV original image, and the lower graph is a BEV image of a double-sided synthetic 2D special mark in a parallel parking space; c, the upper graph in the graph is a BEV original image, and the lower graph is a BEV image for synthesizing a 2D special mark in the inclined four-corner line parking spaces;
From the above, the invention identifies the parking space and the state of the parking space in the BEV image, screens out the target parking space for synthesizing with the 2D special mark according to the state of the parking space, calculates the coordinate position of the target parking space for placing the 2D special mark according to the type of the 2D special mark, fuses the 2D special mark and the BEV image into the target BEV image according to the coordinate position of the 2D special mark, realizes the synthesis of the special parking space in the bird's eye view, can meet the detection requirement of the special parking space under different scenes, and greatly solves the model training problem when the data is in shortage.
Referring to fig. 6, the present invention further provides a special parking space synthesizing device based on BEV images, including:
an acquisition module 61 for acquiring BEV images and 2D special marks;
The identifying and screening module 62 is configured to identify a parking space in the BEV image and a state of the parking space, and screen a target parking space for synthesizing with the 2D special mark according to the state of the parking space;
And a synthesis module 63, configured to calculate a coordinate position of the target parking space for placing the 2D special mark according to the type of the 2D special mark, and fuse the 2D special mark and the BEV image into a target BEV image according to the coordinate position of the 2D special mark.
The identifying and screening module 62 is specifically configured to identify a parking space in the BEV image and four corner coordinates of the parking space; determining the state of the parking space according to whether the parking space is occupied or not; and selecting the unoccupied parking space as a target parking space for synthesizing the 2D special mark. The method comprises the steps of identifying the shape of a parking space and the linearity of the parking space, wherein the shape of the parking space comprises parallel parking spaces, vertical parking spaces and inclined parking spaces, and the linearity of the parking space comprises a closed parking space line, a semi-closed parking space line, an opening parking space line, two-angle parking space lines and four-angle parking space lines; and identifying whether the parking space is locked or not, and if the vehicle is parked, determining that the parking space is occupied if the parking space is locked or parked.
In some embodiments, the identification screening module 62 is further configured to identify a stall line color of the target parking stall, and process the background color of the 2D special mark to be consistent with the stall line color of the target parking stall.
The above-mentioned synthesis module 63 is specifically configured to determine a priori knowledge of the location of the 2D special sign in the parking space according to the type of the 2D special sign; performing binarization processing on the 2D special mark, and determining a center point of the 2D special mark; and determining the coordinate position for placing the 2D special mark in the target parking space according to the central point of the 2D special mark and the coordinates of four corner points of the parking space.
Illustratively, the 2D special mark is sized to accommodate a composite template, and the color of the 2D special mark is adjusted to be consistent with the parking space line color of the parking space. The 2D special mark is converted into a preset synthetic template through a homography transformation algorithm; fusing the composite template with the BEV image to a target BEV image; the size of the synthesized template is the same as that of the target parking space. And converting the center point of the 2D special mark into the synthetic template, and projecting the synthetic template into the BEV image through a homography transformation algorithm according to the corresponding relation between four vertex coordinates of the synthetic template and four corner coordinates of the target parking space.
As shown in fig. 7, the present invention further provides an electronic device, including:
At least one processor; and at least one memory communicatively coupled to the processor, wherein: the memory stores program instructions executable by the processor, which are invoked by the processor to perform the special park combination method based on BEV images described above.
The invention also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program realizes the special parking space synthesizing method based on the BEV image when being executed by a processor.
It is understood that the computer-readable storage medium may include: any entity or device capable of carrying a computer program, a recording medium, a USB flash disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a software distribution medium, and so forth. The computer program comprises computer program code. The computer program code may be in the form of source code, object code, executable files, or in some intermediate form, among others. The computer readable storage medium may include: any entity or device capable of carrying computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a software distribution medium, and so forth.
In some embodiments of the present invention, the BEV image-based special parking space synthesizing device may include a controller, which is a single chip microcomputer chip, integrated with a processor, a memory, a communication module, and the like. The processor may refer to a processor comprised by the controller. The Processor may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (DIGITAL SIGNAL Processor, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), off-the-shelf Programmable gate array (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and additional implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, in computer software, or in a combination of the two, and that the elements and steps of the examples have been generally described in terms of function in the foregoing description to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (9)
1. The special parking space synthesizing method based on the BEV image is characterized by comprising the following steps of:
acquiring BEV images and 2D special marks;
identifying a parking space and a state of the parking space in the BEV image, and screening a target parking space for synthesizing with the 2D special mark according to the state of the parking space;
Calculating the coordinate position of the target parking space for placing the 2D special mark according to the type of the 2D special mark, and fusing the 2D special mark and the BEV image into a target BEV image according to the coordinate position of the 2D special mark;
The identifying of the parking space and the state of the parking space in the BEV image, and the screening of the target parking space for synthesizing with the 2D special mark according to the state of the parking space, comprises the following steps:
identifying four corner coordinates of a parking space in the BEV image; determining the state of the parking space according to whether the parking space is occupied or not; and selecting the unoccupied parking space as a target parking space for synthesizing the 2D special mark.
2. The BEV image-based special parking spot synthesis method according to claim 1, wherein the identifying the parking spot and the state of the parking spot in the BEV image further comprises:
Identifying the shape of the parking space and the linearity of the parking space, wherein the shape of the parking space comprises parallel parking spaces, vertical parking spaces or inclined train spaces, and the linearity of the parking space comprises a closed parking space line, a semi-closed parking space line, an open parking space line, a two-angle parking space line or a four-angle parking space line;
And identifying whether the parking space is locked or not, and if the vehicle is parked, determining that the parking space is occupied if the parking space is locked or parked.
3. The BEV image-based special parking space synthesis method according to claim 1, wherein the calculating the coordinate position of the target parking space for placing the 2D special mark according to the type of the 2D special mark comprises:
determining prior knowledge of the position of the 2D special mark in the parking space according to the type of the 2D special mark;
performing binarization processing on the 2D special mark, and determining a center point of the 2D special mark;
and determining the coordinate position for placing the 2D special mark in the target parking space according to the central point of the 2D special mark and the coordinates of four corner points of the parking space.
4. The BEV image-based special parking space synthesis method according to claim 1, wherein the fusing the 2D special mark with the BEV image according to the coordinate position of the 2D special mark into a target BEV image comprises:
The 2D special mark is adjusted in size to adapt to a synthesized template, and the color of the 2D special mark is consistent with the color adjustment of the parking space line of the parking space;
and converting the center point of the 2D special mark into the synthetic template, and projecting the synthetic template into the BEV image through a homography transformation algorithm according to the corresponding relation between four vertex coordinates of the synthetic template and four corner coordinates of the target parking space.
5. The BEV image-based special parking space synthesis method according to claim 1, wherein the fusing the 2D special mark with the BEV image according to the coordinate position of the 2D special mark into a target BEV image comprises:
Converting the 2D special mark into a preset synthesis template through a homography transformation algorithm;
Fusing the composite template with the BEV image to a target BEV image; the size of the synthesized template is the same as that of the target parking space.
6. The BEV image-based special parking spot synthesis method according to claim 1, wherein the identifying a parking spot and a state of a parking spot in the BEV image comprises:
and identifying the color of the parking space line of the target parking space, and processing the background color of the 2D special mark into the color consistent with the color of the parking space line of the target parking space.
7. A BEV image-based special parking spot synthesizing apparatus, comprising:
The acquisition module is used for acquiring the BEV image and the 2D special mark;
The identifying and screening module is used for identifying the parking space and the state of the parking space in the BEV image, and screening out a target parking space for synthesizing with the 2D special mark according to the state of the parking space;
The synthesis module is used for calculating the coordinate position of the target parking space for placing the 2D special mark according to the type of the 2D special mark, and fusing the 2D special mark and the BEV image into a target BEV image according to the coordinate position of the 2D special mark;
The identifying and screening module is also used for identifying the parking space in the BEV image and four corner coordinates of the parking space; determining the state of the parking space according to whether the parking space is occupied or not; and selecting the unoccupied parking space as a target parking space for synthesizing the 2D special mark.
8. An electronic device, comprising:
At least one processor; and at least one memory communicatively coupled to the processor, wherein: the memory stores program instructions executable by the processor, the processor invoking the program instructions to perform the method of any of claims 1-6.
9. A computer-readable storage medium, on which a computer program is stored, which, when being run by a computer, performs the method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310951572.7A CN117078799B (en) | 2023-07-31 | 2023-07-31 | Special parking space synthesis method and device based on BEV image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310951572.7A CN117078799B (en) | 2023-07-31 | 2023-07-31 | Special parking space synthesis method and device based on BEV image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117078799A CN117078799A (en) | 2023-11-17 |
| CN117078799B true CN117078799B (en) | 2024-05-03 |
Family
ID=88716199
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310951572.7A Active CN117078799B (en) | 2023-07-31 | 2023-07-31 | Special parking space synthesis method and device based on BEV image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117078799B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009205191A (en) * | 2008-02-26 | 2009-09-10 | Hitachi Ltd | Parking space recognition system |
| CN106080774A (en) * | 2016-07-13 | 2016-11-09 | 重庆长安汽车股份有限公司 | Guiding parking system and method |
| CN110097776A (en) * | 2018-01-30 | 2019-08-06 | 杭州海康威视数字技术股份有限公司 | A kind of method for detecting parking stalls, monitor camera and monitor terminal |
| CN112896151A (en) * | 2021-02-25 | 2021-06-04 | 中国第一汽车股份有限公司 | Parking mode determination method and device, electronic equipment and storage medium |
| CN113724527A (en) * | 2021-08-29 | 2021-11-30 | 丽台(上海)信息科技有限公司 | Parking space management method |
| CN115762218A (en) * | 2022-10-10 | 2023-03-07 | 中汽创智科技有限公司 | Parking space map generation method, device, equipment and storage medium |
| CN116486406A (en) * | 2023-04-24 | 2023-07-25 | 零束科技有限公司 | Parking space marking method, parking method, device, electronic equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4970516B2 (en) * | 2009-09-30 | 2012-07-11 | 日立オートモティブシステムズ株式会社 | Surrounding confirmation support device |
-
2023
- 2023-07-31 CN CN202310951572.7A patent/CN117078799B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009205191A (en) * | 2008-02-26 | 2009-09-10 | Hitachi Ltd | Parking space recognition system |
| CN106080774A (en) * | 2016-07-13 | 2016-11-09 | 重庆长安汽车股份有限公司 | Guiding parking system and method |
| CN110097776A (en) * | 2018-01-30 | 2019-08-06 | 杭州海康威视数字技术股份有限公司 | A kind of method for detecting parking stalls, monitor camera and monitor terminal |
| CN112896151A (en) * | 2021-02-25 | 2021-06-04 | 中国第一汽车股份有限公司 | Parking mode determination method and device, electronic equipment and storage medium |
| CN113724527A (en) * | 2021-08-29 | 2021-11-30 | 丽台(上海)信息科技有限公司 | Parking space management method |
| CN115762218A (en) * | 2022-10-10 | 2023-03-07 | 中汽创智科技有限公司 | Parking space map generation method, device, equipment and storage medium |
| CN116486406A (en) * | 2023-04-24 | 2023-07-25 | 零束科技有限公司 | Parking space marking method, parking method, device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117078799A (en) | 2023-11-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110619750B (en) | Intelligent aerial photography identification method and system for illegal parking vehicle | |
| JP5115792B2 (en) | Image processing apparatus and method, and program | |
| EP2916540B1 (en) | Image processing system and image processing method | |
| CN106446949B (en) | A kind of vehicle model recognition methods and device | |
| CN101950350B (en) | Clear path detection using a hierachical approach | |
| US10643091B2 (en) | Automatic feature point detection for calibration of multi-camera systems | |
| JP4970516B2 (en) | Surrounding confirmation support device | |
| JP4725391B2 (en) | Visibility measuring device for vehicle and driving support device | |
| CN107845104A (en) | A kind of method, associated processing system, passing vehicle detecting system and vehicle for detecting passing vehicle | |
| CN108082037A (en) | Brake lamp detects | |
| KR102089343B1 (en) | Around view monitoring system and calibration method for around view cameras | |
| JP2007234019A (en) | Vehicle image area specifying device and method for it | |
| CN110386065A (en) | Monitoring method, device, computer equipment and the storage medium of vehicle blind zone | |
| CN105303157B (en) | Extend the algorithm of the detection range for the detection of AVM stop line | |
| CN102419820A (en) | Method for rapidly detecting car logo in videos and images | |
| WO2012014627A1 (en) | Vehicle periphery monitoring device | |
| CN110276343A (en) | The method of the segmentation and annotation of image | |
| JP2009230233A (en) | Stripe pattern detection system, stripe pattern detection method, and program for stripe pattern detection | |
| JP2009199284A (en) | Road object recognition method | |
| US20200137322A1 (en) | Image processing apparatus | |
| JP2009025910A (en) | Obstacle detection device, obstacle detection system, and obstacle detection method | |
| JP2011254128A (en) | Plane view generating device and plane view generating method | |
| WO2020020445A1 (en) | A method and a system for processing images to obtain foggy images | |
| CN111723805B (en) | Method and related device for identifying foreground region of signal lamp | |
| CN117078799B (en) | Special parking space synthesis method and device based on BEV image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |