CN112455337A

CN112455337A - Image processing method, computer equipment and vehicle

Info

Publication number: CN112455337A
Application number: CN202011351536.XA
Authority: CN
Inventors: 胡泽钢; 倪绪能; 陈兴文
Original assignee: Beijing Overlooking Technology Co ltd
Current assignee: Beijing Yankan Intelligent Technology Co.,Ltd.
Priority date: 2020-11-26
Filing date: 2020-11-26
Publication date: 2021-03-09

Abstract

The invention provides an image processing method applied to a vehicle, wherein the vehicle is provided with a display screen, and the image processing method comprises the following steps: acquiring a plurality of paths of images by utilizing a plurality of fisheye cameras which are arranged on a vehicle and have different visual angles; carrying out distortion correction on the multi-path images by using a preset algorithm to obtain multi-path corrected images; splicing the multiple paths of corrected images to obtain a panoramic image; processing the image generated by the front fisheye camera to form an automobile data recorder image; acquiring an instruction input by a user; and outputting the panoramic image and the automobile data recorder image according to the instruction. The invention also provides computer equipment and a vehicle applying the image processing method. The method applied to the image processing method in the vehicle can realize 360-degree panoramic view and the functions of the automobile data recorder at the same time by only using the panoramic camera, so that the hardware cost is saved.

Description

Image processing method, computer equipment and vehicle

Technical Field

The present invention relates to the field of image processing, and in particular, to an image processing method, a computer device, and a vehicle.

Background

With the vigorous development of electric automobiles and automobile image industries, the application scenes of vehicle-mounted cameras are more and more, and the application scene requirements are more and more complex, such as DVR (digital video recorder), 360-degree look around, lane detection, automatic parking and the like. Due to the complex application scene, the number of vehicle-mounted cameras mounted on a vehicle is increased, and the cost is also increased.

The existing solution of 360-degree panoramic view and automobile data recorder is that images acquired by 4 fisheye cameras are spliced to realize the 360-degree panoramic view function, and the images acquired by 1 front camera are added to realize the automobile data recorder function. The disadvantage of this scheme is that the cost is too high and the reusability of hardware is low.

Therefore, it is an urgent problem to provide an image processing algorithm that can realize looking around and driving recording by algorithm optimization with as little hardware as possible.

Disclosure of Invention

The invention provides an image processing method, computer equipment and a vehicle, which can realize 360-degree panoramic vision and the functions of a driving recorder at the same time by using a panoramic camera, and save hardware cost.

In a first aspect, an embodiment of the present invention provides an image processing method applied in a vehicle, where the vehicle is provided with a display screen, and the image processing method includes:

the method comprises the steps that a plurality of fisheye cameras which are arranged on a vehicle and have different visual angles are used for obtaining multi-path images, and the fisheye cameras comprise front fisheye cameras for obtaining images in front of the vehicle;

carrying out distortion correction on the multi-path images by using a preset algorithm to obtain multi-path corrected images;

splicing the multiple paths of corrected images to obtain a panoramic image;

processing the image generated by the front fisheye camera to form an automobile data recorder image;

acquiring an instruction input by a user;

and outputting the panoramic image and the automobile data recorder image according to the instruction.

In a second aspect, an embodiment of the present invention provides a computer device, where the computer device includes:

a memory for storing computer instructions for an image processing method;

a processor for executing computer instructions to cause a computer device to implement the image processing method described above.

In a third aspect, an embodiment of the present invention provides a vehicle, where the vehicle includes a display screen, a plurality of fisheye cameras, and the computer device, so that the vehicle implements the image processing method described above.

According to the image processing method, the computer equipment and the vehicle, the plurality of fisheye cameras are used for acquiring the plurality of paths of images, the plurality of paths of images are spliced to acquire 360-degree panoramic images, the images acquired by the front fisheye camera of the images in front of the vehicle are used for acquiring the images of the automobile data recorder, the 360-degree panoramic view and the functions of the automobile data recorder can be realized simultaneously by only using the panoramic camera, the performance of the existing hardware is fully utilized, and the cost of the automobile data recorder is saved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It is to be understood that the drawings in the following description are merely exemplary of the invention and that other drawings may be derived from the structure shown in the drawings by those skilled in the art without the exercise of inventive faculty.

Fig. 1 is a flowchart of an image processing method according to an embodiment of the present invention.

Fig. 2 is a first sub-flowchart of an image processing method according to an embodiment of the present invention.

Fig. 3 is a second sub-flowchart of the image processing method according to the embodiment of the present invention.

Fig. 4 is a third sub-flowchart of an image processing method according to an embodiment of the present invention.

Fig. 5 is a fourth sub-flowchart of the image processing method according to the embodiment of the present invention.

Fig. 6 is a fifth sub-flowchart of an image processing method according to an embodiment of the present invention.

Fig. 7 is a sixth sub-flowchart of an image processing method according to an embodiment of the present invention.

Fig. 8 is a schematic view of a vehicle according to an embodiment of the present invention.

Fig. 9 is a schematic diagram of an internal structure of a computer device according to an embodiment of the present invention.

Reference numerals for the various elements in the figures

900 computer device 901 memory

902 processor 903 bus

904 display component 905 communication component

100 vehicle 101 fisheye camera

The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The terms "first," "second," "third," "fourth," and the like in the description and in the claims of the present application and in the drawings described above, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

It should be noted that the description relating to "first", "second", etc. in the present invention is for descriptive purposes only and is not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In addition, technical solutions between various embodiments may be combined with each other, but must be realized by a person skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present invention.

Please refer to fig. 1 in combination, which is a flowchart illustrating an image processing method applied in a vehicle according to an embodiment of the present invention. The vehicle is provided with a display screen. The image processing method applied to the vehicle provided by the first embodiment specifically includes the following steps.

And step S101, acquiring multi-path images by utilizing a plurality of fisheye cameras which are arranged on the vehicle and have different visual angles, wherein the fisheye cameras comprise front fisheye cameras for acquiring images in front of the vehicle. Specifically, a plurality of fisheye cameras that the visual angle is different are used for acquireing the environmental aspect around the vehicle, and leading fisheye camera is used for acquireing the environmental aspect directly in the front of the vehicle. Furthermore, a plurality of fisheye cameras with different visual angles are used for acquiring images of the appointed direction of the vehicle according to actual needs.

And S102, carrying out distortion correction on the multi-path images by using a preset algorithm to obtain multi-path corrected images. Specifically, the multipath image may be subjected to distortion correction using a warp and weft method commonly used for processing images to obtain a multipath corrected image. Other image correction algorithms that may be implemented to correct for distortion may also be used.

And step S103, splicing the multiple corrected images to obtain a panoramic image. Specifically, the multi-channel corrected images are spliced through image processing steps of feature point extraction and matching, image registration, image fusion and the like to obtain a panoramic image.

And step S104, processing the image generated by the front fisheye camera to form an automobile data recorder image. Referring to fig. 2 in combination, step S104 specifically includes the following steps.

Step S201, performing enhancement processing on the image generated by the front fisheye camera to form an enhanced image. Specifically, an image generated by the front fisheye camera is enhanced through an image enhancement algorithm such as contrast pull-up, Gamma correction, histogram equalization, histogram specification, a homomorphic filter or an HSV space-based color image enhancement method, so that an enhanced image is formed.

And step S202, forming a vehicle event data recorder image according to the enhanced image. Specifically, the enhanced image is output onto a display screen provided in the vehicle. With reference to fig. 3, step S202 specifically includes the following steps.

Step S301, acquiring the outline of an object beyond a preset distance in the field of view of the enhanced image. Specifically, the features of objects other than 10 meters in the enhanced image field of view are extracted by using image feature extraction algorithms such as LBP (Local Binary Patterns), HOG feature extraction algorithm (Histogram of Oriented Gradient), Scale-invariant feature transform (SIFT) and the like. Such as the outline of a tree, the outline of a car, and the outline of a license plate.

And step S302, judging whether a license plate exists according to the object contour. Referring to fig. 4 in combination, step S302 specifically includes the following steps.

Step S401, analyzing the object characteristics according to the object contour. Specifically, in this embodiment, the designated contour is a contour of a vehicle and a contour of a license plate.

In step S402, it is determined whether the object feature is a designated object feature. Specifically, it is determined whether there is a contour of the vehicle and a contour of the license plate.

And step S403, judging that the license plate exists when the object characteristics are the specified object characteristics.

And S404, judging that no license plate exists when the object features are not the specified object features.

Step S303, when the license plate exists, the number in the license plate is identified by adopting a license plate identification algorithm to form a license plate enhancement pattern. Referring to fig. 5 in combination, step S303 specifically includes the following steps.

Step S501, obtaining the number plate number shape in the enhanced image.

Step S502, comparing the number plate digital shape with the existing number plate digital shape in the number plate database to obtain a first matching shape. Referring to fig. 6 in combination, step S502 specifically includes the following steps.

Step S601, the number plate number shape is divided into a plurality of sub-shapes. Specifically, a row of numbers in a license plate is divided into a single number. For example, "yue B12345" can be divided into "yue", "B", "1", "2", "3", "4" and "5".

Step S602, the plurality of sub-shapes are respectively matched with the existing digital shapes in the shape database to obtain a plurality of sub-first matching shapes. Specifically, the shape database is a database that is set in advance for storing various shapes.

In step S603, the sub-first matching shapes are combined into a first matching shape.

And S503, acquiring license plate information according to the first matching shape. Referring to fig. 7 in combination, step S503 specifically includes the following steps.

And S701, calculating information of the to-be-determined license plate according to the first matching shape. Specifically, a numerical value corresponding to the shape is calculated according to an image processing algorithm.

Step S702, inquiring whether the license plate database has the information of the license plate to be determined. Specifically, the license plate database is a preset database for storing various license plate information.

And step S703, outputting the information of the license plate to be determined as the information of the license plate when the information of the license plate to be determined exists.

And step S704, recalculating the information of the undetermined license plate when the information of the undetermined license plate does not exist.

In the above embodiment, the accuracy of image recognition is improved by verifying the acquired license plate, and an example of displaying an incorrect shape of the image is prevented. Therefore, the reliability of the automobile data recorder image is higher.

Referring to fig. 3 again, in step S304, the license plate enhancement pattern is formed into a car recorder pattern.

In the above embodiment, generating the car recorder pattern according to different situations saves effort.

Referring to fig. 1 again, in step S105, an instruction input by the user is obtained. Specifically, the user inputs an instruction to output an image through an input device mounted on the vehicle, for example, presses a panoramic image output button on a touch screen mounted on the vehicle, and the image processing device acquires the panoramic image instruction.

And S106, outputting the panoramic image and the automobile data recorder image according to the instruction. The instruction includes panoramic picture output instruction and vehicle event data recorder image output instruction, exports panoramic picture and vehicle event data recorder image according to the instruction, specifically includes: and judging whether the instruction is a panoramic image output instruction and/or a vehicle event data recorder image output instruction. When the instruction is a panoramic image output instruction, the panoramic image is output to a display device mounted on the vehicle. And when the instruction is an automobile data recorder image output instruction, outputting the automobile data recorder image to a display device installed on the vehicle. And when the command is a panoramic image output command and a driving recorder image output command, outputting the panoramic image and the driving recorder image to a display device installed on the vehicle. Or when the instruction is not a panoramic image output instruction or a driving recorder image output instruction, outputting the panoramic image and the driving recorder image to a storage device installed on the vehicle. The display device can be an LED display screen or a liquid crystal display screen, and the storage device can be a USB flash disk or other vehicle-mounted devices with storage functions.

In this embodiment, the panoramic image and the image of the automobile data recorder are displayed according to the actual needs of the user, and if the images need to be displayed, the images are all displayed on a display screen arranged in the vehicle, so that the space in the vehicle is saved. Simultaneously, provide bigger environment visual angle for the driver, help the better environment of observing around of driver, promote the security of driving.

In the above embodiment, the image processing method obtains the multiple paths of images through the multiple fisheye cameras, splices the multiple paths of images to obtain the 360-degree panoramic image, and obtains the automobile data recorder image through the image obtained by the front fisheye camera of the image in front of the vehicle, so that the 360-degree panoramic view and the automobile data recorder function can be simultaneously realized by only using the panoramic camera, the performance of the existing hardware is fully utilized, and the cost of the automobile data recorder is saved. And the reusability of hardware is improved.

Referring to fig. 8, in the image processing method of the present embodiment, a plurality of fisheye cameras 101 with different visual angles are respectively disposed around a vehicle 100. Specifically, a fisheye camera 101 for acquiring an image in front of the vehicle is provided near the front bumper of the vehicle 100. Fisheye cameras 101 for acquiring images of both sides of the vehicle 100 are disposed below the rear view mirror of the vehicle 100. A fisheye camera 101 for acquiring an image behind the vehicle is provided near the rear bumper of the vehicle 100. In this embodiment, the installation position of the fisheye camera on the vehicle is only used for reference, and is not limited, and the position for installing the fisheye camera is selected according to specific situations in practical application.

In the above embodiment, the environmental conditions around the vehicle are acquired by the plurality of fisheye cameras, so that the acquired images can be spliced and enhanced by using an image processing method. The images of the fisheye cameras at the appointed positions can be selected according to actual needs to be subjected to enhancement processing to obtain related images. The utilization rate of hardware is improved.

Please refer to fig. 9, which is a schematic diagram illustrating an internal structure of a computer apparatus 900 according to an embodiment of the present invention. The computer device 900 comprises at least a memory 901, a processor 902. A memory 901 for storing computer instructions of the image processing method.

A processor 902 for executing computer instructions to cause a computer device to implement the image processing method described above.

The memory 901 includes at least one type of readable storage medium, which includes a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, and the like. The memory 901 may in some embodiments be an internal storage unit of the computer device 900, such as a hard disk of the computer device 900. The memory 901 may also be an external storage device of the computer device 900 in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital Card (SD), a Flash memory Card (Flash Card), etc., provided on the computer device 900. Further, the memory 901 may also include both internal storage units and external storage devices of the computer device 900. The memory 901 can be used not only for storing application software installed in the computer apparatus 900 and various types of data such as program instructions of an image processing method, etc., but also for temporarily storing data that has been output or is to be output. For example, a panoramic image and a driving record image generated by an image processing method.

Processor 902 may be, in some embodiments, a Central Processing Unit (CPU), controller, microcontroller, microprocessor or other data Processing chip that executes program instructions or processes data stored in memory 901. In particular, the processor 902 executes program instructions of the image processing method to control the computer device 900 image processing method.

Further, the computer device 900 may further include a bus 903 which may be a Peripheral Component Interconnect (PCI) standard bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 9, but this does not indicate only one bus or one type of bus.

Further, computer device 900 may also include a display component 904. The display component 904 may be an LED (Light Emitting Diode) display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light Emitting Diode) touch panel, or the like. The display component 904 may also be referred to as a display device or display unit, as appropriate, for displaying information processed in the computer device 900 and for displaying a visual user interface, among other things.

Further, the computer device 900 may also include a communication component 905, and the communication component 905 may optionally include a wired communication component and/or a wireless communication component (e.g., a WI-FI communication component, a bluetooth communication component, etc.), typically used for establishing a communication connection between the computer device 900 and other computer devices.

While fig. 9 illustrates only a computer device 900 having

components

901 and 905 and program instructions for an image processing method, those skilled in the art will appreciate that the architecture illustrated in fig. 9 is not intended to be limiting of the computer device 900, and may include fewer or more components than those illustrated, or some components may be combined, or a different arrangement of components.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product.

The computer apparatus 900 may be a general purpose computer, a special purpose computer, a network of computers, or other programmable devices. The program instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the program instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center by wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wirelessly (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can store or a data storage device, such as a server, a data center, etc., that is integrated with one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above described systems, apparatuses and units may refer to the corresponding processes in the above described method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described embodiment of a computer device for implementing an image processing method is merely illustrative, for example, the division of the unit is only a logical division, and there may be other divisions when the actual implementation is performed, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method of the embodiments of the present application. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program instructions.

The invention also provides a vehicle applying the image processing method. The vehicle includes display screen, a plurality of flake cameras and foretell computer equipment. The specific structure of the vehicle to which the image processing method is applied refers to the above embodiments, and since the vehicle to which the image processing method is applied adopts all technical solutions of all the above embodiments, at least all the beneficial effects brought by the technical solutions of the above embodiments are achieved, and details are not repeated herein.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, insofar as these modifications and variations of the invention fall within the scope of the claims of the invention and their equivalents, the invention is intended to include these modifications and variations.

The above-mentioned embodiments are only examples of the present invention, which should not be construed as limiting the scope of the present invention, and therefore, the present invention is not limited by the claims.

Claims

1. An image processing method, characterized in that the image processing method comprises:

the method comprises the steps that a plurality of fisheye cameras which are arranged on a vehicle and have different visual angles are used for obtaining multi-path images, wherein the fisheye cameras comprise front fisheye cameras for obtaining images in front of the vehicle;

splicing the multi-path corrected images to obtain a panoramic image;

acquiring an instruction input by a user; and

2. The image processing method according to claim 1, wherein processing the image generated by the front fisheye camera to form a vehicle event data recorder image specifically comprises:

enhancing the image generated by the front fisheye camera to form an enhanced image; and

and forming the automobile data recorder image according to the enhanced image.

3. The image processing method according to claim 2, wherein forming the vehicle event data recorder image from the enhanced image specifically comprises:

acquiring the outline of an object beyond a preset distance in the field of view of the enhanced image;

judging whether a license plate exists according to the object outline;

when the license plate exists, recognizing the number in the license plate by adopting a license plate recognition algorithm to form a license plate enhanced image; and

and forming the license plate enhanced image into an image of the automobile data recorder.

4. The image processing method of claim 3, wherein judging whether a license plate exists according to the object contour specifically comprises:

analyzing the object characteristics according to the object profile;

judging whether the object features are specified object features or not;

when the object feature is the specified object feature, judging that a license plate exists; or

And when the object feature is not the specified object feature, judging that no license plate exists.

5. The image processing method of claim 3, wherein recognizing the numbers in the license plate using a license plate recognition algorithm to form a license plate enhanced image, comprises:

acquiring the number plate digital shape in the enhanced image;

comparing the license plate digital shape with the existing license plate digital shape in a license plate database to obtain a first matching shape; and

and acquiring license plate information according to the first matching shape.

6. The image processing method of claim 5, wherein comparing the license plate digital shape to existing license plate digital shapes in a license plate database to obtain a first matching shape, comprises:

dividing the license plate number shape into a plurality of sub-shapes;

the sub shapes are respectively matched with the existing digital shapes in the shape database to obtain a plurality of sub first matching shapes; and

the sub first matching shapes are combined into the first matching shape.

7. The image processing method of claim 5, wherein obtaining license plate information according to the first matching shape specifically comprises:

calculating information of the undetermined license plate according to the first matching shape;

inquiring whether the license plate database has the information of the to-be-determined license plate;

when the information of the to-be-determined license plate exists, outputting the information of the to-be-determined license plate as the information of the license plate; or

And when the information of the undetermined license plate does not exist, recalculating the information of the undetermined license plate.

8. The image processing method according to claim 1, wherein the instruction includes a panoramic image output instruction and a drive recorder image output instruction, and the outputting the panoramic image and the drive recorder image according to the instruction specifically includes:

judging whether the instruction is the panoramic image output instruction and/or the automobile data recorder image output instruction;

outputting the panoramic image to a display device mounted on a vehicle when the instruction is the panoramic image output instruction;

when the instruction is the automobile data recorder image output instruction, outputting the automobile data recorder image to display equipment installed on a vehicle;

when the instruction is the panoramic image output instruction and the automobile data recorder image output instruction, outputting the panoramic image and the automobile data recorder image to a display device installed on a vehicle; or

And when the instruction is not the panoramic image output instruction or the automobile data recorder image output instruction, outputting the panoramic image and the automobile data recorder image to a storage device installed on a vehicle.

9. A computer device, characterized in that the computer device comprises:

a memory for storing computer instructions for an image processing method; and

a processor for executing the computer instructions to cause the computer device to implement the image processing method of any one of claims 1 to 8.

10. A vehicle, characterized in that the vehicle comprises a display screen, a plurality of fisheye cameras and the computer device of claim 9, so that the vehicle can realize the image processing method of any one of claims 1-8.