CN115147675A - Display method and device and electronic rearview mirror system - Google Patents
Display method and device and electronic rearview mirror system Download PDFInfo
- Publication number
- CN115147675A CN115147675A CN202210689210.0A CN202210689210A CN115147675A CN 115147675 A CN115147675 A CN 115147675A CN 202210689210 A CN202210689210 A CN 202210689210A CN 115147675 A CN115147675 A CN 115147675A
- Authority
- CN
- China
- Prior art keywords
- image
- fog
- foggy
- display method
- rearview mirror
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000012545 processing Methods 0.000 claims abstract description 35
- 230000004044 response Effects 0.000 claims abstract description 24
- 238000012549 training Methods 0.000 claims description 29
- 230000006870 function Effects 0.000 claims description 18
- 238000001914 filtration Methods 0.000 claims description 17
- 238000010801 machine learning Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 2
- 230000000875 corresponding effect Effects 0.000 description 14
- 238000004891 communication Methods 0.000 description 10
- 230000007613 environmental effect Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000010438 heat treatment Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000001965 increasing effect Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
- B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
- B60R1/26—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view to the rear of the vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/12—Mirror assemblies combined with other articles, e.g. clocks
- B60R2001/1253—Mirror assemblies combined with other articles, e.g. clocks with cameras, video cameras or video screens
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- Mechanical Engineering (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
The disclosure provides a display method and device and an electronic rearview mirror system. Specifically, the display method comprises the following steps: acquiring a rearview image acquired by means of a camera of the electronic rearview mirror system; judging whether the rear view image is a foggy image or not by utilizing a predetermined foggy image judgment model; responding to the fact that the rear-view image is a foggy image, acquiring and determining a fog source causing the foggy image according to fog condition information of a corresponding area of the rear-view image; sending a camera defogging signal in response to the fog source being lens fog; and in response to the fog source being ambient fog, performing defogging processing on the foggy image and sending a display signal. Through the mode, the identification of the foggy image and the processing mode of determining matching aiming at different fog sources are realized, the identification degree of objects in the image is improved, the electronic rearview mirror system can be ensured to realize accurate display under the foggy condition, and the driving safety of the vehicle is improved.
Description
Technical Field
The disclosure relates to the technical field of automobile safety, in particular to a display method and device and an electronic rearview mirror system.
Background
With the increasing application fields of cameras due to the continuous development of automobile electronic electrification and camera technologies, the physical rearview mirror tends to be gradually replaced by an electronic rearview mirror system. Compared with the traditional physical rearview mirror, the electronic rearview mirror has a plurality of advantages: when raining, the sight line cannot be influenced because rainwater hits the glass window and the lens; when external light is poor, the low-illumination function of the camera is easier to see objects clearly; rotating the display with the camera, etc.
Even so, electronic rearview mirror system is when the fog, because the fog shelters from, and the object is more difficult to discern, also will influence driving safety.
Disclosure of Invention
In view of this, an object of the present disclosure is to provide a display method and device and an electronic rearview mirror system, which increase the image accuracy of the electronic rearview mirror system under the condition of fogging and improve the driving safety.
In view of the above, in a first aspect, the present disclosure provides a display method applied to an electronic rearview mirror system, where the display method includes:
acquiring a rearview image acquired by means of a camera of the electronic rearview mirror system;
judging whether the rear view image is a foggy image or not by utilizing a predetermined foggy image judgment model;
in response to the fact that the rear-view image is a foggy image, acquiring and determining a fog source causing the foggy image according to fog condition information of a corresponding area of the rear-view image;
responding to the fog source is lens fog, and sending a camera defogging signal;
and in response to the fog source being ambient fog, performing defogging processing on the foggy image and sending a display signal.
Further, the display method further comprises:
acquiring vehicle speed information;
and responding to the vehicle speed information being larger than zero, sending a camera defogging signal, defogging the foggy image and sending a display signal.
Further, the fog image determination model includes:
acquiring the brightness and the saturation of an image to be judged;
calculating a difference between the brightness and the saturation, and comparing the difference with a preset fog threshold;
in response to the difference exceeding a preset fogging threshold; the image to be judged is a foggy image;
in response to the difference not exceeding a preset fogging threshold; the image to be judged is a normal image.
Further, the display method further comprises:
acquiring vehicle speed information;
adjusting the preset fogging threshold value according to the vehicle speed information; wherein the preset fogging threshold and the vehicle speed information are inversely related.
Further, the display method further comprises:
acquiring a plurality of historical rearview images under a foggy condition and a plurality of historical rearview images under a normal condition;
extracting the automobile body subimages in the historical rear view image to form a training data set;
and training an initial machine learning model through the training data set, and obtaining the fog image judgment model after the training is finished.
Further, the step of performing defogging processing on the foggy image and sending a display signal includes:
processing the foggy image through a first logarithmic function to obtain an enhanced dark area image;
filtering the enhanced dark area image by using a guided filtering algorithm to obtain a filtered image;
carrying out automatic gain adjustment on the enhanced dark area image and the filtering image and fusing to obtain a gain image;
processing the gain image through a second logarithmic function to obtain a defogged image; wherein the second logarithmic function is used to compensate for the reduced brightness of the first logarithmic function.
Further, the step of performing automatic gain adjustment on the enhanced dark region image and the filtered image and fusing the enhanced dark region image and the filtered image to obtain a gain image includes:
performing first automatic gain adjustment on the filtered image to obtain a gain first sub-image;
fusing the filtering image and the enhanced dark area image and then carrying out second automatic gain adjustment to obtain a gain second sub-image;
and fusing the gain first sub-image and the gain second sub-image to obtain the gain image.
In a second aspect, the present disclosure further provides a display device applied to an electronic rearview mirror system, including:
an acquisition module configured to acquire a rearview image acquired by means of a camera of the electronic rearview mirror system;
a judging module configured to judge whether the rear view image is a foggy image using a predetermined foggy image judging model;
the fog source analysis module is configured to respond to the rear-view image being a foggy image, acquire and determine a fog source causing the foggy image according to the fog condition information of the corresponding area of the rear-view image;
a processing module configured to: sending a camera defogging signal in response to the fog source being lens fog;
and responding to the fog source is the environmental fog, performing defogging processing on the foggy image and sending a display signal.
Further, the display device further includes:
a training module configured to: acquiring a plurality of historical rearview images under a foggy condition and a plurality of historical rearview images under a normal condition;
extracting the automobile body subimages in the historical rear view image to form a training data set;
and training an initial machine learning model through the training data set, and obtaining the fog image judgment model after the training is finished.
In a third aspect, the present disclosure also provides an electronic rearview mirror system comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method according to any one of the preceding claims when executing the program.
As can be seen from the above, the display method, device and electronic rearview mirror system provided by the present disclosure obtain a rearview image collected by means of a camera of the electronic rearview mirror system; judging whether the rear view image is a foggy image or not by utilizing a predetermined foggy image judgment model; responding to the fact that the rear-view image is a foggy image, acquiring and determining a fog source causing the foggy image according to fog condition information of a corresponding area of the rear-view image; sending a camera defogging signal in response to the fog source being lens fog; and in response to the fog source being ambient fog, performing defogging processing on the foggy image and sending a display signal. Through the mode, the identification of the foggy image and the processing mode of determining matching aiming at different fog sources are realized, the identification degree of objects in the image is improved, the electronic rearview mirror system can be ensured to realize accurate display under the foggy condition, and the driving safety of the vehicle is improved. In addition, the whole process runs automatically without the operation of a driver, so that the driver is prevented from dispersing attention due to fogging, and the driving safety is further ensured.
Drawings
In order to clearly illustrate the technical solutions of the present disclosure or related technologies, the drawings used in the embodiments or related technologies description will be briefly introduced below, and obviously, the drawings in the following description are only embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a schematic flowchart of a display method according to an embodiment of the disclosure;
fig. 2 is a schematic flow chart of constructing a fog image judgment model according to an embodiment of the present disclosure;
FIG. 3 is a schematic flow chart of a defogging method provided by the embodiment of the disclosure;
fig. 4 is a schematic partial structural diagram of a display device according to an embodiment of the present disclosure;
fig. 5 is a schematic partial structural view of an electronic rearview mirror system provided in the embodiment of the present disclosure.
Detailed Description
For the purpose of promoting a better understanding of the objects, aspects and advantages of the present disclosure, reference is made to the following detailed description taken in conjunction with the accompanying drawings.
It is to be noted that technical terms or scientific terms used in the embodiments of the present disclosure should have a general meaning as understood by those having ordinary skill in the art to which the present disclosure belongs, unless otherwise defined. The use of "first," "second," and similar terms in the embodiments of the disclosure is not intended to indicate any order, quantity, or importance, but rather is used to distinguish one element from another. The word "comprising" or "comprises", and the like, means that the element or item listed before the word covers the element or item listed after the word and its equivalents, but does not exclude other elements or items. The terms "connected" or "coupled" and the like are not restricted to physical or mechanical connections, but may include electrical connections, whether direct or indirect.
When the electronic rearview mirror system is in fog, objects are difficult to identify due to fog shielding, and driving safety is affected. For environmental fogging, such as the morning in the autumn and winter season after rain, the fog is diffused in the atmosphere, and the discrimination of objects cannot be improved by wiping the camera or the display screen. For camera fogging, for example, fog on a camera needs to be removed when the camera is driven out of an underground garage to a road on the ground in summer. However, for the electronic rearview mirror system, it is aimed to restore the image of the real world object to the maximum. Under the condition of fogging, the image with the fogging can be seen on the display screen, whether the image is fogged or not can not be distinguished, the source of the fogging can not be distinguished, and a targeted solution can not be adopted, so that the traffic hidden trouble is brought.
Therefore, aiming at the situation, the display method is applied to the electronic rearview mirror system, the identification degree of an object in a rearview mirror image is improved, and the purpose of improving driving safety is finally achieved.
Referring to fig. 1, the display method includes:
s101: acquiring a rearview image acquired by means of a camera of the electronic rearview mirror system;
here, the electronic rearview mirror system includes a camera and a display screen. The camera is located at the position below the front side of the front row of windows of the vehicle and is basically the same as the position of the physical rearview mirror. The display screen is typically located inside the cab for easy viewing by the driver, and is not particularly limited herein.
S102: judging whether the rear view image is a foggy image or not by utilizing a predetermined foggy image judgment model;
the predetermined fog image determination model may be a fog image determination rule or a fog image determination model trained in advance.
In a real scene, fog can be graded according to distance of horizontal visibility. For example, horizontal visibility distances between 1 and 10 kilometers are light fog; horizontal visibility distance is less than 1 kilometer, and is fog; the horizontal visibility is 200-500 m far away, and is fog.
It will be appreciated by those skilled in the art that different levels of fog will have different effects on the driver's judgment of real world objects. Thus, a foggy image is understood herein to be a rear view mirror image that affects the driver's judgment of real world objects, and is not understood to be a rear view mirror image formed in any foggy condition.
S1031: responding to the fact that the rear-view image is a normal image, and sending a display signal;
here, the aforementioned foggy image corresponds, and the normal image should be understood as a rear view mirror image that does not affect the driver's judgment of the real world object, and should not be understood as a rear view mirror image that does not include any fog.
For normal images, the display signal may be directly issued. Here, the display credits may be used to display a rear view mirror image on the display screen.
S1032: responding to the fact that the rear-view image is a foggy image, acquiring and determining a fog source causing the foggy image according to fog condition information of a corresponding area of the rear-view image;
here, the regional fog condition information may be obtained by obtaining weather information through networking, or may be obtained by obtaining the vehicle-mounted fog sensor, which is not specifically limited herein.
It should be noted that the fog sources of the foggy image are ambient fog and lens fog, respectively. Wherein, the environmental fog means that the air humidity is high and the environmental space has fog; lens fog refers to fog on a camera. If the regional fog condition information shows that the environmental space is fog, the fog source is environmental fog; otherwise, the fog source is lens fog.
It should be understood that the fog source may have both ambient fog and lens fog, and may be treated as ambient fog, as ambient fog is not eliminated.
The fog condition information is complex, so that the fog condition information has the characteristics of dispersion, regionality and the like, and particularly shows more obvious performance in mountainous areas. The rear view image is used as a starting point, and the rear view image is judged to be fogged and then the regional fog condition information is acquired, so that the lens fog can be effectively monitored, the regional fog condition information can be acquired in a targeted manner, the interference of the blind acquisition of the fog condition information on the display method is avoided, and the stability and the reliability of the display method are effectively guaranteed.
S1041: sending a camera defogging signal in response to the fog source being lens fog;
here, the electronic rearview mirror system includes a camera defogging component, such as a heater tube. When the controller of the heating pipe receives a defogging signal of the camera, the camera can be defogged by starting the heating pipe.
It should be understood that the heating tube is only an example and is not a limitation to the defogging component of the camera, and those skilled in the art may also use other defogging methods and components to perform defogging, and detailed description is omitted here.
S1042: and in response to the fog source being ambient fog, performing defogging processing on the foggy image and sending a display signal.
It should be noted that the method of the removal processing may be a defogging algorithm based on image enhancement, a defogging algorithm based on image restoration, a defogging algorithm based on deep learning, or the like, and details thereof are not described here.
And sending a display signal according to the defogged image, so that the display signal is displayed on a display screen, and the display device is helpful for a driver to identify real world objects and improves the driving safety.
Therefore, through the mode, the identification of the foggy image and the processing mode of determining matching aiming at different fog sources are realized, the identification degree of objects in the image is improved, the electronic rearview mirror system can be ensured to realize accurate display under the condition of fogging, and the driving safety of the vehicle is improved. In addition, the whole process runs automatically without the operation of a driver, so that the driver is prevented from dispersing attention due to fogging, and the driving safety is further ensured.
In some embodiments, the display method further comprises:
acquiring vehicle speed information;
here, the manner of acquiring the vehicle speed information is the prior art, and is not described here again.
And responding to the vehicle speed information being larger than zero, sending a camera defogging signal, defogging the foggy image and sending a display signal.
The defogging of the camera takes time, possibly several seconds or several minutes, and when the vehicle is in a driving state, the defogging processing is carried out on the foggy image and a display signal is sent out, so that the safe driving during the defogging period is ensured.
In some embodiments, the fog image determination model comprises:
acquiring the brightness and the saturation of an image to be judged;
calculating a difference between the brightness and the saturation, and comparing the difference with a preset fog threshold;
in response to the difference exceeding a preset fogging threshold; the image to be judged is a foggy image;
in response to the difference not exceeding a preset fogging threshold; the image to be judged is a normal image.
It should be noted that the fog concentration is in direct proportion to the difference between brightness and saturation, and the larger the difference between brightness and saturation is, the higher the fog concentration is, the lower the horizontal visibility is; conversely, a smaller difference between brightness and saturation indicates a smaller concentration of fog and a higher horizontal visibility.
For example, in an image of clear weather, the difference between brightness and saturation is close to zero, e.g., 0.18%, while in an image of fog, the difference between brightness and saturation is large, e.g., 6.36%.
Alternatively, the preset fogging threshold may be 3%, 5%, etc., which are not illustrated here. The person skilled in the art can adjust the preset fogging threshold according to the visibility requirement during driving.
Further, the display method further comprises:
acquiring vehicle speed information;
adjusting the preset fogging threshold value according to the vehicle speed information; wherein the preset fogging threshold and the vehicle speed information are inversely related.
It will be appreciated that as vehicle speed increases, the demand for horizontal visibility increases; the vehicle speed is reduced and the requirements for horizontal visibility are reduced.
When the vehicle speed increases, the preset fogging threshold value is reduced, so that the judgment accuracy of the fogging image is reduced, part of the rear view images which are suitable for low-speed driving but not suitable for high-speed driving can be judged as the fogging images, and the fogging images are displayed after defogging treatment, thereby being beneficial to ensuring the safety of high-speed driving.
On the contrary, when the vehicle speed is reduced, the preset fogging threshold is increased, so that the judgment accuracy of the fogging image is enhanced, the rear view image which is judged to be the fogging image under part of high-speed driving conditions can be treated as a normal image, and the driving safety can be guaranteed while computing resources are saved.
In some embodiments, as shown in fig. 2, the display method further includes:
s201: acquiring a plurality of historical rearview images under a foggy condition and a plurality of historical rearview images under a normal condition;
here, the foggy condition refers to a state that interferes with the recognition of real world objects by the driver, and does not refer to any foggy state. Correspondingly, the normal condition is a state that does not interfere with the recognition of real world objects by the driver, and may not be a clear state, but may be a light fog state.
S202: extracting the automobile body subimages in the historical rear view image to form a training data set;
it should be noted that, objects in the historical rear view image are various, and vehicle body subimages are extracted, so that the training difficulty can be reduced, and the training efficiency and accuracy can be improved.
S203: and training an initial machine learning model through the training data set, and obtaining the fog image judgment model after the training is finished.
Here, the initial machine learning model may be a neural network model, such as a LeNet model, alexNet model, *** LeNet model, or the like.
Furthermore, a vehicle body sub-image in the rear view image is extracted, and the vehicle body sub-image is input into the fog image judgment model, so that whether the rear view image is a fog image or not can be judged.
As an alternative embodiment, S201: the method comprises the following steps of obtaining a plurality of historical rearview images under the foggy condition and a plurality of historical rearview images under the normal condition, and replacing the historical rearview images with the historical rearview images under the foggy condition: and acquiring a plurality of historical rear-view images under the foggy condition and different vehicle speeds and a plurality of historical rear-view images under the normal condition. At this time, the fog image judgment model is obtained while considering the vehicle speed factor.
When the rear view image is judged, vehicle speed information needs to be acquired, and the rear view image under the vehicle speed condition is judged by using a fog image judgment model.
In some embodiments, the present disclosure also provides a specific defogging method.
Referring to fig. 3, the step of performing defogging processing on the foggy image and sending a display signal includes:
s301: processing the foggy image through a first logarithmic function to obtain an enhanced dark area image; such processing can remove the interfering signal, which is advantageous for the processing in step S302, and the overall picture effect becomes darker.
S302: filtering the enhanced dark area image by using a guided filtering algorithm to obtain a filtered image; in this way, a sharp outline of the image can be derived.
It should be noted that, for an input image p, an output image q is obtained by filtering through a guide image I, where p and I are input to the algorithm.
Illustratively, the leading image may be the same as the enhanced dark image, and the algorithm then becomes an edge preserving filter.
S303: carrying out automatic gain adjustment on the enhanced dark area image and the filtering image and fusing to obtain a gain image; here, by enhancing the luminance of the dark area by the automatic gain control, it is possible to prevent the image processing adjustment overshoot from causing the image flickering.
S304: processing the gain image through a second logarithmic function to obtain a defogged image; wherein the second logarithmic function is used to compensate for the reduced brightness of the first logarithmic function. And finally, improving the image exposure, and compensating the overall darkened image in the S301 so as to achieve the effect of overall brightness enhancement.
In some embodiments, the step of performing automatic gain adjustment on the enhanced dark region image and the filtered image and fusing the enhanced dark region image and the filtered image to obtain a gain image includes:
carrying out first automatic gain adjustment on the filtered image to obtain a gain first sub-image; here, the first automatic gain adjustment may be a Boost gain/Tone curve, which is not limited herein.
Fusing the filtering image and the enhanced dark area image, and then carrying out second automatic gain adjustment to obtain a second gain sub-image; here, the second automatic Gain adjustment may be Gain/Coring, which is not limited herein.
And fusing the gain first sub-image and the gain second sub-image to obtain the gain image.
And the mode of automatic gain and pattern fusion for multiple times is adopted, so that the effect of automatic gain can be effectively guaranteed.
It should be noted that the method of the embodiments of the present disclosure may be executed by a single device, such as a computer or a server. The method of the embodiment can also be applied to a distributed scene and is completed by the mutual cooperation of a plurality of devices. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of the embodiments of the present disclosure, and the multiple devices interact with each other to complete the method.
It should be noted that the above describes some embodiments of the disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments described above and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
Based on the same inventive concept, the disclosure also provides a display device corresponding to any of the above embodiments.
Referring to fig. 4, the display device, applied to an electronic rearview mirror system, includes:
an acquisition module 401 configured to acquire a rear view image acquired by means of a camera of the electronic rearview mirror system;
a judging module 402 configured to judge whether the rear view image is a foggy image by using a predetermined foggy image judgment model;
a fog source analysis module 403, configured to, in response to the rear-view image being a foggy image, acquire and determine a fog source causing the foggy image according to the fog condition information of the corresponding region of the rear-view image;
a processing module 404 configured to: sending a camera defogging signal in response to the fog source being lens fog;
and in response to the fog source being ambient fog, performing defogging processing on the foggy image and sending a display signal.
In some embodiments, the obtaining module 401 is further configured to obtain vehicle speed information;
the processing module further configured to: and responding to the vehicle speed information being larger than zero, sending a camera defogging signal, defogging the foggy image, and sending a display signal.
In some embodiments, the fog image determination model comprises:
acquiring the brightness and the saturation of an image to be judged;
calculating a difference between the brightness and the saturation, and comparing the difference with a preset fog threshold;
in response to the difference exceeding a preset fogging threshold; the image to be judged is a foggy image;
in response to the difference not exceeding a preset fogging threshold; the image to be judged is a normal image.
In some embodiments, further comprising:
an obtaining module 401 further configured to obtain vehicle speed information;
the judging module 402 is further configured to adjust the preset fogging threshold according to the vehicle speed information; wherein the preset fogging threshold value and the vehicle speed information are inversely correlated.
In some embodiments, the display device further comprises:
a training module configured to: acquiring a plurality of historical rearview images under a foggy condition and a plurality of historical rearview images under a normal condition;
extracting the automobile body subimages in the historical rear view image to form a training data set;
and training an initial machine learning model through the training data set, and obtaining the fog image judgment model after the training is finished.
In some embodiments, the processing module 404 is configured to:
processing the foggy image through a first logarithmic function to obtain an enhanced dark area image;
filtering the enhanced dark area image by using a guided filtering algorithm to obtain a filtered image;
carrying out automatic gain adjustment on the enhanced dark area image and the filtering image and fusing to obtain a gain image;
processing the gain image through a second logarithmic function to obtain a defogged image; wherein the second logarithmic function is used to compensate for the reduced brightness of the first logarithmic function.
In some embodiments, the processing module 404 is configured to:
carrying out first automatic gain adjustment on the filtered image to obtain a gain first sub-image;
fusing the filtering image and the enhanced dark area image, and then carrying out second automatic gain adjustment to obtain a second gain sub-image;
and fusing the gain first sub-image and the gain second sub-image to obtain the gain image.
For convenience of description, the above devices are described as being divided into various modules by functions, and are described separately. Of course, the functionality of the various modules may be implemented in the same one or more software and/or hardware implementations of the present disclosure.
The apparatus of the foregoing embodiment is used to implement the corresponding display method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiment, which are not described herein again.
Based on the same inventive concept, corresponding to any of the above embodiments, the present disclosure further provides an electronic rearview mirror system, including a memory, a processor, and a computer program stored on the memory and operable on the processor, wherein the processor implements the display method according to any of the above embodiments when executing the program.
Fig. 5 is a schematic diagram showing a more specific hardware structure of an electronic rearview mirror system provided by the embodiment, and the apparatus may include: a processor 1010, a memory 1020, an input/output interface 1030, a communication interface 1040, and a bus 1050. Wherein the processor 1010, memory 1020, input/output interface 1030, and communication interface 1040 are communicatively coupled to each other within the device via bus 1050.
The processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits, and is configured to execute related programs to implement the technical solutions provided in the embodiments of the present disclosure.
The Memory 1020 may be implemented in the form of a ROM (Read Only Memory), a RAM (Random Access Memory), a static Memory device, a dynamic Memory device, or the like. The memory 1020 may store an operating system and other application programs, and when the technical solution provided by the embodiments of the present specification is implemented by software or firmware, the relevant program codes are stored in the memory 1020 and called to be executed by the processor 1010.
The input/output interface 1030 is used for connecting an input/output module to input and output information. The i/o module may be configured as a component within the device (not shown) or may be external to the device to provide corresponding functionality. The input devices may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output devices may include a display, a speaker, a vibrator, an indicator light, etc.
The communication interface 1040 is used for connecting a communication module (not shown in the drawings) to implement communication interaction between the present apparatus and other apparatuses. The communication module can realize communication in a wired mode (such as USB, network cable and the like) and also can realize communication in a wireless mode (such as mobile network, WIFI, bluetooth and the like).
It should be noted that although the above-mentioned device only shows the processor 1010, the memory 1020, the input/output interface 1030, the communication interface 1040 and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, those skilled in the art will appreciate that the above-described apparatus may also include only those components necessary to implement the embodiments of the present description, and not necessarily all of the components shown in the figures.
The electronic rearview mirror system of the embodiment is used for realizing the corresponding display method in any one of the embodiments, has the beneficial effects of the corresponding method embodiment, and is not described again here.
Based on the same inventive concept, the present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the display method according to any of the above embodiments, corresponding to any of the above embodiment methods.
Computer-readable media of the present embodiments, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.
The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the display method according to any one of the above embodiments, and have the beneficial effects of the corresponding method embodiment, which are not described herein again.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, is limited to these examples; within the idea of the present disclosure, also technical features in the above embodiments or in different embodiments may be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of brevity.
In addition, well-known power/ground connections to Integrated Circuit (IC) chips and other components may or may not be shown in the provided figures for simplicity of illustration and discussion, and so as not to obscure the embodiments of the disclosure. Furthermore, devices may be shown in block diagram form in order to avoid obscuring embodiments of the present disclosure, and this also takes into account the fact that specifics with respect to implementation of such block diagram devices are highly dependent upon the platform within which the embodiments of the present disclosure are to be implemented (i.e., specifics should be well within purview of one skilled in the art). Where specific details (e.g., circuits) are set forth in order to describe example embodiments of the disclosure, it should be apparent to one skilled in the art that the embodiments of the disclosure can be practiced without, or with variation of, these specific details. Accordingly, the description is to be regarded as illustrative instead of restrictive.
While the present disclosure has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations thereof will be apparent to those skilled in the art in light of the foregoing description. For example, other memory architectures, such as Dynamic RAM (DRAM), may use the discussed embodiments.
The disclosed embodiments are intended to embrace all such alternatives, modifications and variances which fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalents, improvements, and the like that may be made within the spirit and principles of the embodiments of the disclosure are intended to be included within the scope of the disclosure.
Claims (10)
1. A display method is applied to an electronic rearview mirror system, wherein the display method comprises the following steps:
acquiring a rearview image acquired by means of a camera of the electronic rearview mirror system;
judging whether the rear view image is a foggy image or not by utilizing a predetermined foggy image judgment model;
responding to the fact that the rear-view image is a foggy image, acquiring and determining a fog source causing the foggy image according to fog condition information of a corresponding area of the rear-view image;
sending a camera defogging signal in response to the fog source being lens fog;
and in response to the fog source being ambient fog, performing defogging processing on the foggy image and sending a display signal.
2. The display method according to claim 1, wherein the display method further comprises:
acquiring vehicle speed information;
and responding to the vehicle speed information being larger than zero, sending a camera defogging signal, defogging the foggy image and sending a display signal.
3. The display method according to claim 1, wherein the fog image determination model includes:
acquiring the brightness and the saturation of an image to be judged;
calculating a difference between the brightness and the saturation, and comparing the difference with a preset fog threshold;
in response to the difference exceeding a preset fogging threshold; the image to be judged is a foggy image;
in response to the difference not exceeding a preset fogging threshold; the image to be judged is a normal image.
4. The display method according to claim 3, wherein the display method further comprises:
acquiring vehicle speed information;
adjusting the preset fogging threshold value according to the vehicle speed information; wherein the preset fogging threshold and the vehicle speed information are inversely related.
5. The display method according to claim 1, wherein the display method further comprises:
acquiring a plurality of historical rearview images under a foggy condition and a plurality of historical rearview images under a normal condition;
extracting a vehicle body subimage in the historical rear view image to form a training data set;
and training an initial machine learning model through the training data set, and obtaining the fog image judgment model after the training is finished.
6. The display method according to claim 1, wherein the step of performing defogging processing on the foggy image and emitting a display signal comprises:
processing the foggy image through a first logarithmic function to obtain an enhanced dark area image;
filtering the enhanced dark area image by using a guided filtering algorithm to obtain a filtered image;
carrying out automatic gain adjustment on the enhanced dark area image and the filtering image and fusing to obtain a gain image;
processing the gain image through a second logarithmic function to obtain a defogged image; wherein the second logarithmic function is used to compensate for the reduced brightness of the first logarithmic function.
7. The display method according to claim 6, wherein the step of performing automatic gain adjustment and fusion on the enhanced dark area image and the filtered image to obtain a gain image comprises:
carrying out first automatic gain adjustment on the filtered image to obtain a gain first sub-image;
fusing the filtering image and the enhanced dark area image, and then carrying out second automatic gain adjustment to obtain a second gain sub-image;
and fusing the gain first sub-image and the gain second sub-image to obtain the gain image.
8. A display device applied to an electronic rearview mirror system comprises:
an acquisition module configured to acquire a rearview image acquired by means of a camera of the electronic rearview mirror system;
a judging module configured to judge whether the rear view image is a foggy image using a predetermined foggy image judging model;
the fog source analysis module is configured to respond to the rear-view image being a foggy image, acquire and determine a fog source causing the foggy image according to the fog condition information of the corresponding area of the rear-view image;
a processing module configured to: sending a camera defogging signal in response to the fog source being lens fog;
and in response to the fog source being ambient fog, performing defogging processing on the foggy image and sending a display signal.
9. The display device according to claim 8, wherein the display device further comprises:
a training module configured to: acquiring a plurality of historical rearview images under a foggy condition and a plurality of historical rearview images under a normal condition;
extracting a vehicle body subimage in the historical rear view image to form a training data set;
and training an initial machine learning model through the training data set, and obtaining the fog image judgment model after the training is finished.
10. An electronic rearview mirror system comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the method of any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210689210.0A CN115147675A (en) | 2022-06-16 | 2022-06-16 | Display method and device and electronic rearview mirror system |
| PCT/CN2023/089173 WO2023241214A1 (en) | 2022-06-16 | 2023-04-19 | Display method and apparatus, and electronic rearview mirror system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210689210.0A CN115147675A (en) | 2022-06-16 | 2022-06-16 | Display method and device and electronic rearview mirror system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115147675A true CN115147675A (en) | 2022-10-04 |
Family
ID=83408556
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210689210.0A Pending CN115147675A (en) | 2022-06-16 | 2022-06-16 | Display method and device and electronic rearview mirror system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN115147675A (en) |
| WO (1) | WO2023241214A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023241214A1 (en) * | 2022-06-16 | 2023-12-21 | 中国第一汽车股份有限公司 | Display method and apparatus, and electronic rearview mirror system |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109905678B (en) * | 2019-03-21 | 2021-04-23 | 重庆工程职业技术学院 | Coal mine monitoring image defogging processing system |
| CN111791834B (en) * | 2019-04-08 | 2024-02-23 | 上海擎感智能科技有限公司 | Automobile and demisting method and device thereof |
| CN111275645A (en) * | 2020-01-20 | 2020-06-12 | 腾讯科技（深圳）有限公司 | Image defogging method, device and equipment based on artificial intelligence and storage medium |
| CN113343738A (en) * | 2020-02-17 | 2021-09-03 | 华为技术有限公司 | Detection method, device and storage medium |
| CN115147675A (en) * | 2022-06-16 | 2022-10-04 | 中国第一汽车股份有限公司 | Display method and device and electronic rearview mirror system |
-
2022
- 2022-06-16 CN CN202210689210.0A patent/CN115147675A/en active Pending
-
2023
- 2023-04-19 WO PCT/CN2023/089173 patent/WO2023241214A1/en unknown
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023241214A1 (en) * | 2022-06-16 | 2023-12-21 | 中国第一汽车股份有限公司 | Display method and apparatus, and electronic rearview mirror system |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2023241214A1 (en) | 2023-12-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10567674B2 (en) | Systems and methods for detecting objects in imaging systems | |
| CN107424133B (en) | Image defogging method and device, computer storage medium and mobile terminal | |
| KR101810727B1 (en) | Vehicle and method for operating a camera arrangement for a vehicle | |
| CN110135235B (en) | Glare processing method and device and vehicle | |
| CN110135302B (en) | Method, device, equipment and storage medium for training lane line recognition model | |
| JPWO2006098358A1 (en) | Image processing apparatus and method, program, and recording medium | |
| KR101534973B1 (en) | Image Processing Apparatus and Method for Removing Rain From Image Data | |
| WO2023241214A1 (en) | Display method and apparatus, and electronic rearview mirror system | |
| KR102521656B1 (en) | Method and apparatus of identifying object | |
| CN110562201A (en) | Control method and control device for windscreen wiper and vehicle | |
| CN111756962A (en) | Camera device and control method thereof | |
| CN117445794A (en) | Car lamp control method, device and storage medium in tunnel scene | |
| CN116524877A (en) | Vehicle-mounted screen brightness adjustment method and device, electronic equipment and storage medium | |
| KR101823655B1 (en) | System and method for detecting vehicle invasion using image | |
| CN116758875A (en) | Brightness adjusting method of vehicle display device and related equipment | |
| CN116923372A (en) | Driving control method, device, equipment and medium | |
| CN115334250A (en) | Image processing method and device and electronic equipment | |
| CN111756987B (en) | Control method and device of vehicle-mounted camera and vehicle-mounted image capturing system | |
| CN116547726A (en) | Correction of images of an imaging device during rain, incident light and dirt | |
| JP2018074191A (en) | On-vehicle video display system, on-vehicle video display method, and program | |
| CN114516270B (en) | Vehicle control method, apparatus, electronic device, and computer-readable medium | |
| KR101745261B1 (en) | Sun visor control apparatus and method | |
| US11790634B2 (en) | Image signal processing system, method, and program | |
| US11597261B2 (en) | Control apparatus, vehicle, and control method | |
| CN117141415A (en) | Vehicle windshield wiper control method, device, head-mounted display device and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |