WO2022206156A1 - 一种图像生成方法、装置、设备及存储介质 - Google Patents

一种图像生成方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2022206156A1
WO2022206156A1 PCT/CN2022/074059 CN2022074059W WO2022206156A1 WO 2022206156 A1 WO2022206156 A1 WO 2022206156A1 CN 2022074059 W CN2022074059 W CN 2022074059W WO 2022206156 A1 WO2022206156 A1 WO 2022206156A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
area
adjusted
scene
shadow
Prior art date
Application number
PCT/CN2022/074059
Other languages
English (en)
French (fr)
Inventor
程光亮
石建萍
安井裕司
松永英树
冨手要
Original Assignee
商汤集团有限公司
本田技研工业株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 商汤集团有限公司, 本田技研工业株式会社 filed Critical 商汤集团有限公司
Priority to JP2023559825A priority Critical patent/JP2024512102A/ja
Publication of WO2022206156A1 publication Critical patent/WO2022206156A1/zh
Priority to US18/474,189 priority patent/US20240013453A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/04—Context-preserving transformations, e.g. by using an importance map
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/40—Image enhancement or restoration using histogram techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/77—Retouching; Inpainting; Scratch removal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G06T5/94—Dynamic range modification of images or parts thereof based on local image properties, e.g. for local contrast enhancement
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • the embodiments of the present disclosure relate to the technical field of image processing, and relate to, but are not limited to, an image generation method, apparatus, device, and storage medium.
  • Embodiments of the present disclosure provide a technical solution for image generation.
  • An embodiment of the present disclosure provides an image generation method, the method includes:
  • the image rendering state of the to-be-adjusted area is related to the light irradiated on the target object in the original image, the target object includes a vehicle;
  • the image rendering state of the to-be-adjusted area is adjusted to the target rendering state to obtain the target image.
  • the image rendering state includes a shadow rendering state
  • the determining the target rendering state in the to-be-adjusted area includes: in the to-be-processed image, determining that the shadow rendering state is consistent with the scene information The matched reference area; the shadow rendering state of the reference area is determined as the target rendering state.
  • the image area without shadow state that matches the scene is used as the reference area, so as to perform histogram matching on the area to be adjusted based on the reference area, so that after processing
  • the picture chromaticity presented by the to-be-adjusted area is the same as the picture chromaticity of the reference area.
  • the determining, in the image to be processed, an area to be adjusted where the image rendering state does not match the scene information of the specific scene includes: when the scene information is a scene without a light source at night or a scene without a light source during the day In the case of a sunlight scene, the shadow area in the to-be-processed image is determined as the to-be-adjusted area. In this way, the image area in which the content of the screen is unreasonable can be accurately determined.
  • the determining, in the image to be processed, the area to be adjusted where the image rendering state does not match the scene information of the specific scene includes: when the scene information is a scene with a light source at night or a daytime scene In the case of a sunlight scene, the shadow area that can be generated by the target object is determined according to the direction in which the light illuminates the target object; the shadow area except the first shadow area in the shadow area in the image to be processed is determined.
  • the first shadow area is the intersection of the shadow area in the image to be processed and the shadow area that can be generated by the target object. In this way, by taking the shadow area in the image to be processed except the shadow area that can be generated by the target object as the area to be adjusted, the area that needs to be adjusted in the shadow rendering state can be more accurately determined.
  • the shadow rendering state of the reference area is a shadowless rendering state
  • the image rendering state of the to-be-adjusted area is adjusted to the target rendering state to obtain the target
  • the image includes: in the to-be-processed image, adjusting the image of the to-be-adjusted area to a shadow-free state according to the image of the reference area to obtain the target image.
  • the shadow rendering state in the generated target image is no shadow state, which more closely matches the scene with no light source at night and the scene without sunlight during the day.
  • adjusting the image of the to-be-adjusted area to a shadow-free state according to the image of the reference area, to obtain the target image includes: according to the reference The image of the area adjusts the grayscale image of the to-be-adjusted area to obtain an adjusted grayscale image; based on the adjusted grayscale image, an image of the replacement area whose shadow rendering state is a shadowless state is generated; in the to-be-processed image , the generated image of the replacement area is used to replace the image of the to-be-adjusted area to generate the target image.
  • the shadow rendering state in the generated target image matches the scene information, so that the generated target image is more realistic.
  • the adjusting the grayscale image of the to-be-adjusted area according to the image of the reference area to obtain an adjusted grayscale image includes: determining a histogram of the reference area according to the image of the reference area ; Based on the histogram of the reference area, adjust the histogram of the to-be-adjusted area to obtain an adjusted histogram; based on the adjusted histogram, determine the adjusted grayscale image. In this way, the shadow part in the image can be removed more naturally, so that the content of the obtained image is more realistic.
  • the image rendering state includes a shadow rendering state
  • determining an area to be adjusted where the image rendering state does not match the scene information includes: when the scene information is nighttime In the case of a light source scene or a daytime scene with sunlight, the shadow area that can be generated by the target object is determined according to the direction in which the light illuminates the target object; the shadow area that can be generated by the target object is determined as the area to be adjusted. In this way, the adjusted target image can be made more real and natural.
  • adjusting the image rendering state of the to-be-adjusted area to the target rendering state to obtain the target image includes: determining lighting parameters in the specific scene; Determine the illumination intensity of the target object surface capable of generating the shadow area based on the illumination parameters, render the to-be-adjusted area, and obtain the target image.
  • the target object that is closer to the artificial light source in the target image can generate a shadow area, and when there is natural light, the target object can generate a shadow area, thereby improving the fidelity of the generated target image.
  • the original image is an image collected in no rain/snow weather, and the specific scene is a rain/snow scene; in the to-be-processed image, the image rendering state of the to-be-adjusted area is adjusted Rendering the state for the target, obtaining a target image, including adding raindrops/snowflakes to the original image to obtain the target image. In this way, a more realistic target image can be obtained.
  • An embodiment of the present disclosure provides an image generation apparatus, the apparatus includes:
  • a first conversion module configured to convert the original image into an image to be processed in a specific scene
  • the first determining module is configured to, in the image to be processed, determine an area to be adjusted whose image rendering state does not match the scene information of the specific scene; Light correlation on a target object, the target object including a vehicle;
  • a second determining module configured to determine a target rendering state in the to-be-adjusted area; the target rendering state matches the scene information;
  • the first adjustment module is configured to, in the to-be-processed image, adjust the image rendering state of the to-be-adjusted area to the target rendering state to obtain a target image.
  • an embodiment of the present disclosure provides a computer storage medium, where computer-executable instructions are stored thereon, and after the computer-executable instructions are executed, the above-mentioned method steps can be implemented.
  • An embodiment of the present disclosure provides an electronic device, the electronic device includes a memory and a processor, where computer-executable instructions are stored in the memory, and the processor can implement the above-mentioned when executing the computer-executable instructions on the memory. described method steps.
  • An embodiment of the present disclosure provides a computer program, including computer-readable code, when the computer-readable code is executed in an electronic device, a processor in the electronic device executes the image for realizing any one of the above Generate method.
  • Embodiments of the present disclosure provide an image generation method, device, device, and storage medium.
  • an original image is converted to obtain a to-be-processed image in a specific scene; then, in the to-be-processed image, an image rendering state and scene information are determined.
  • the unmatched area to be adjusted then, by adjusting the rendering state of the area to be adjusted to the target rendering state that matches the scene information, a target image whose image rendering state matches the scene information is generated; in this way, the shadow in the generated target image
  • the rendering state is matched to the scene information, making the resulting target image more realistic.
  • FIG. 1A is a schematic diagram of a system architecture to which a trajectory prediction method according to an embodiment of the present disclosure can be applied;
  • FIG. 1B is a schematic diagram of an implementation flow of an image generation method provided by an embodiment of the present disclosure
  • FIG. 2 is a schematic flowchart of another implementation of the image generation method provided by the embodiment of the present disclosure.
  • 3A is a schematic diagram of the composition and structure of an image generation system provided by an embodiment of the present disclosure.
  • 3B is a schematic diagram of an application scenario of the image generation method provided by the embodiment of the present disclosure.
  • FIG. 4 is an implementation framework structure diagram of an image generation method provided by an embodiment of the present disclosure.
  • FIG. 5 is a schematic diagram of another application scenario of the image generation method provided by the embodiment of the present disclosure.
  • FIG. 6 is a schematic structural composition diagram of an image generating apparatus provided by an embodiment of the present disclosure.
  • FIG. 7 is a schematic structural diagram of a computer device provided by an embodiment of the present disclosure.
  • first ⁇ second ⁇ third is only used to distinguish similar objects, and does not represent a specific ordering of objects. It is understood that "first ⁇ second ⁇ third" Where permitted, the specific order or sequence may be interchanged to enable the embodiments of the disclosure described in some embodiments to be practiced in sequences other than those illustrated or described in some embodiments.
  • Gaussian blur It is a low-pass filter for the image.
  • the so-called “blur” can be understood as taking the average value of surrounding pixels for each pixel.
  • Autonomous vehicle A vehicle that contains sensors that sense the surrounding environment.
  • the vehicle coordinate system is fixed on the autonomous vehicle, wherein the x-axis is the direction of the car's advancing direction, the y-axis points to the left side of the vehicle's advancing direction, and the z-axis is perpendicular to the ground, which conforms to the right-handed coordinate system.
  • the origin of the coordinate system is on the ground below the midpoint of the rear axle.
  • the device provided by the embodiment of the present disclosure may be implemented as a notebook computer, a tablet computer, a desktop computer, a camera, and a mobile device (eg, a personal digital computer) with an image acquisition function.
  • a mobile device eg, a personal digital computer
  • Various types of user terminals such as assistants, dedicated messaging devices, portable game devices) can also be implemented as servers.
  • exemplary applications when the device is implemented as a terminal or a server will be described.
  • the method can be applied to a computer device, and the functions implemented by the method can be realized by calling a program code by a processor in the computer device.
  • the program code can be stored in a computer storage medium.
  • the computer device includes at least a processor and a storage medium. medium.
  • FIG. 1A is a schematic diagram of a system architecture to which an image generation method according to an embodiment of the present disclosure can be applied; as shown in FIG. 1A , the system architecture includes an image acquisition device 131 , a network 132 and an image generation terminal 133 .
  • the image capture device 131 and the image generation terminal 133 may establish a communication connection through the network 132, and the image capture device 131 reports the captured original image (or, the image generation terminal 133) to the image generation terminal 133 through the network 202.
  • the image generation terminal 133 responds to the received original image, first, converts the image into an image to be processed in a specific scene, and determines an unreasonable area to be adjusted in the image to be processed; then, determines the The target rendering state of the area to be adjusted, and the image rendering state of the area to be adjusted is adjusted to the target rendering state, to obtain the target image, and output the target image on the image display interface of the image generation terminal 133 .
  • the working state of the object in the target image is matched with the scene information, so that the generated target image is more in line with the real scene.
  • the image capture device 131 may be a capture device including a camera or the like.
  • the image generation terminal 133 may include a computer device with a certain computing capability, for example, the computer device includes a terminal device or a server or other processing device.
  • the network 132 can be wired or wireless. Wherein, when the image generation terminal 133 is a server, the image acquisition device 131 can communicate with the server through a wired connection, such as data communication through a bus; when the image generation terminal 133 is a terminal device, the image acquisition device 131 can be wirelessly connected The connection method is connected to the image generation terminal 133 for communication, and further data communication is performed.
  • the image generation terminal 133 may be a vision processing device with a video capture module, or a host with a camera.
  • the image generation method of the embodiment of the present disclosure may be executed by the image generation terminal 133 , and the above-mentioned system architecture may not include the network 132 and the image acquisition device 131 .
  • FIG. 1B is a schematic diagram of the implementation flow of the image generation method according to the embodiment of the present disclosure, as shown in FIG. 1B , and described in conjunction with the steps shown in FIG. 1B :
  • Step S101 converting the original image into an image to be processed in a specific scene.
  • the original image may be an image collected in any scene, an image with complex picture content, or an image with simple picture content, for example, an image collected on a road in the middle of the night, or an image collected during the day images on the road, etc.
  • the specific scene includes: a daytime scene, an evening scene, a rainy and snowy scene, or a sunny day scene, and the like.
  • the scene information of a specific scene includes the scene in which the image to be processed is collected; for example, the level of light in the scene, the location of the scene, and the objects in the scene; for example, if the image to be processed is an image of a road in a late-night scene, then the scene information This includes: how bright the road is, the location of the road, and objects such as vehicles and lights on the road.
  • the image to be processed may be an image in a specific scene; for example, a late night scene, an evening scene, or an early morning scene, and so on.
  • step S101 converting the original image into an image to be processed in a specific scene can be achieved by the following steps:
  • the original image is style-transformed according to a specific scene to obtain the image to be processed.
  • the process is as follows:
  • the first step is to get the original image.
  • the original image is an image collected in any scene, for example, a road image collected during the day or a road image collected at night.
  • the scene information of the original image is determined.
  • a trained discriminator is used to determine whether the scene information of the image is a specific scene. For example, if the specific scene is a scene with no light source at night, the discriminator determines whether the scene information of the image is a scene without light source at night.
  • the scene information of the original image is far away from the specific scene
  • the scene information of the original image is converted into the specific scene, that is, the original image is converted into the image under the specific scene, so as to obtain the specific scene image to be processed.
  • the original image is an image collected in a scene with sun during the day
  • the original image collected in a scene with sun during the day is converted into an image of a scene with no light source at night, which can be generated by inputting the original image. to generate a corresponding image to be processed at night without a light source.
  • the original image is an image collected in a scene with no light source at night
  • the original image collected in a scene with no light source at night is converted into an image of a scene with sun during the day, which can be generated by inputting the original image. to generate the corresponding images to be processed in the daytime scene with sun.
  • the original image is determined to be the image to be processed.
  • the scene information is the same as or very similar to a specific scene, it means that the original image has a specific scene, so the original image can be used as the image to be processed without performing image conversion on the original image.
  • the specific scene is a scene without light source in the middle of the night
  • the original image is an image collected in a scene where night falls, and the scene is similar to the specific scene and is night, then the original image is determined as the image to be processed.
  • the original image is acquired, by judging the scene information of the original image, if the scene information is not a specific scene, the original image is style-transformed, so as to obtain a waiting list with a specific scene. Process images.
  • Step S102 in the to-be-processed image, determine a to-be-adjusted area where the image rendering state does not match the scene information of the specific scene.
  • the image rendering state of the area to be adjusted is related to the light irradiated on the target object in the original image, and the target object includes a vehicle; in some possible implementations, the target object may be in the light Objects that can generate shadows under illumination can be any vehicles, traffic signs, pedestrians, or animals included in the picture content of the image to be processed; for example, the image to be processed is an image of a city road at night, which includes: buildings, Roads and vehicles parked on the side of the road, etc. Then the target objects that can generate shadows under the illumination of light are: buildings and vehicles.
  • the image rendering state is the rendering state of the target object when there is a light source or no light source, at least including: a shadow state and a shadow-free state.
  • the light sources include artificial light sources and natural light sources, and the artificial light sources include light sources such as street lights, building lights, or car lights.
  • the image rendering state that does not match the scene information is: under the scene information, the image rendering state generated by the object in the image does not conform to the actual scene.
  • the object in a night scene, if there is no light source (that is, no artificial light source and moonlight), then the object will not produce shadows, so the shadow rendering state that does not match the no light source scene at night is a shadowed state; In the scene, then the object will produce shadows, so the shadow rendering state that does not match the sun scene in the daytime is no shadow state.
  • no light source that is, no artificial light source and moonlight
  • the shaded area in the shaded state is the area to be adjusted.
  • Step S103 determining the target rendering state in the area to be adjusted.
  • the target rendering state that matches the scene information is a state in which the target object actually produces shadows in the scene information.
  • the scene information is a scene with no light source at night
  • the shadow-free state is the target rendering state that matches the scene information
  • the scene information is a scene with sun during the day or a scene with a light source at night
  • the state with shadow is the same as that of the scene with a light source at night.
  • the target rendering state that matches the scene information so that the determined target rendering state can perfectly match the scene information and conform to the actual scene.
  • the object will not produce shadows, so the rendering state of the target matching the scene without light source at night is the shadow-free state; in the scene with the sun during the day, due to the illumination of the sun, the target object will be Shadows are generated, so the render state of the target that matches the daytime sun scene is shaded.
  • the scene information is a nighttime scene without light source
  • the target rendering state matching the scene information is a shadowless state.
  • the rendering state of the target matching the scene information is a shadowed state.
  • the image to be processed as a road image if the image includes vehicles and trees parked on the side of the road, after segmenting the shadow area of the image to be processed, if there is no shadow of the vehicle around the vehicle, the The occupied image area is the area to be adjusted; the area where the shadow rendering state in the image is in the shadow state is the area with the target rendering state.
  • the to-be-adjusted area that does not match the scene is an image area that does not include raindrops.
  • the to-be-adjusted area that does not match the scene is an image area that does not include snowflakes.
  • the area to be adjusted is an unobstructed area (ie, an open-air area) in the original image. ; Then add raindrops/snowflakes in the area to be adjusted to obtain a target image that conforms to the rain/snow scene.
  • the area to be adjusted is the area where raindrops/snowflakes appear in the original image; then, in the area to be adjusted , the raindrops/snowflakes are removed to obtain the target image that matches the scene information.
  • a more realistic target image can be obtained by adding raindrops/snowflakes to the original image according to the scene information of the specific scene, or removing the raindrops/snowflakes.
  • the original image may be first converted into an image of a night scene, and the converted image may be adjusted. Shadowed areas in the image; then, in the resulting image, raindrops/snowflakes are added to get the target image.
  • Step S104 in the to-be-processed image, adjust the image rendering state of the to-be-adjusted area to the target rendering state to obtain a target image.
  • the target image is obtained by converting the image rendering state of the to-be-adjusted area so that the to-be-adjusted area has a target rendering state matching scene information.
  • the image rendering state of the area to be adjusted may be adjusted to the target rendering state in the image to be processed to obtain the target image; it may also be based on the target rendering state and the image of the area to be adjusted. content, generate an image area with the target rendering state, and replace the image area with the to-be-adjusted area in the to-be-processed image, thereby obtaining the target image.
  • the shadow rendering state of the area to be adjusted is a shadowless state, which may be in the image to be processed, and the shadowless state of the to-be-adjusted area is adjusted. Rendering the state for the target to obtain the target image; it is also possible to generate an image area with a shadow-free state according to the shadow-free state and the screen content of the area to be adjusted, and replace the image area with the to-be-adjusted area in the image to be processed, and The replaced image is smoothed to generate the target image.
  • the shadow rendering state in the generated target image is a shadowless state, which matches the scene without light source at night, thereby making the generated target image more realistic.
  • the region to be adjusted whose shadow rendering state does not match the scene information is determined in the image to be processed; then, the shadow rendering state of the to-be-adjusted region is adjusted to the target rendering state that matches the scene information. state to generate a target image whose shadow rendering state matches the scene information, making the generated target image more vivid.
  • the process of determining the area to be adjusted in the above step S102 can be implemented by an image segmentation network, and the neural network can be a network arbitrarily configured for image segmentation, and the implementation process is as follows:
  • the area to be adjusted and the reference area are marked in the form of detection frames through the neural network, so that the area to be adjusted and the reference area can be accurately predicted in the image to be processed.
  • the to-be-adjusted area and the reference area in the to-be-processed image are marked to mark the to-be-adjusted area and the reference area.
  • the image segmentation network can be any type of neural network, such as a fully convolutional neural network, an atrous convolutional network, or a parsing network.
  • Input the image to be processed into the trained image segmentation network, in the image segmentation network use a rectangular frame to mark the location of the area to be adjusted and the reference area in the image to be processed, so as to mark the area to be adjusted and the reference area.
  • the training process of the image segmentation network can be implemented by the following steps:
  • the first step is to input the training image into the image segmentation network to be trained, and predict the location information of the to-be-adjusted area where the shadow rendering state in the to-be-trained image does not match the scene information.
  • the image segmentation network to be trained is used to detect the area to be adjusted in the image to be trained, and the image segmentation network to be trained is trained by a large number of training images, that is, a large number of training images are input into the image segmentation network to be trained, To preliminarily predict the position of the region to be adjusted in the image to be trained.
  • the prediction loss of the position information is determined according to the real position information of the area to be adjusted in the training image.
  • the prediction loss is determined by using the difference between the actual location information of the area to be adjusted marked in the training image and the location information of the predicted area to be adjusted.
  • the accuracy of the multiple position information predicted by the image segmentation network to be trained is determined through the real position information of the multiple regions to be adjusted, thereby determining the prediction loss.
  • the network parameters of the image segmentation network to be trained are adjusted to obtain the image segmentation network.
  • the accuracy of each predicted position information is determined by combining the real position information of the area to be adjusted, and the accuracy is fed back to the image segmentation network, so that the image segmentation network can adjust parameters such as weights, etc. network parameters, thereby improving the accuracy of neural network detection.
  • the neural network to perform operations such as convolution and deconvolution to obtain the confidence of the position information of the 100 regions to be adjusted; since in the training stage, the parameters of the image segmentation network are Randomly initialized, so that the confidence level of the rough estimation of the position information of the 100 areas to be adjusted is also random, so if you want to improve the accuracy of the position information predicted by the image segmentation network, you need to tell the neural network 100 positions Information about which is right and which is wrong.
  • a comparison function is used to compare 100 pieces of position information with the real position information. If the similarity between the position information and the real position information is greater than the preset similarity threshold, output 1, otherwise output 0, so the comparison function will output 200 ratios.
  • the weight parameters of the image segmentation network are adjusted using the prediction loss corresponding to the position information, so as to obtain the trained image segmentation network.
  • the weight parameter is the weight of neurons in the neural network, and the like.
  • the prediction loss is the cross-entropy loss of positive samples and negative samples. The parameters such as the weight of the neural network are adjusted by the prediction loss, so that the adjusted prediction result of the image segmentation network is more accurate.
  • the above process is the process of training the image segmentation network. Based on the prediction of the input image to be processed, the predicted position of the area to be adjusted and the position of the real area to be adjusted are obtained, and multiple iterations are performed to make the image after training.
  • the prediction loss of the position information output by the segmentation network satisfies the convergence condition, so that the accuracy of the region to be adjusted detected by the neural network is higher.
  • step S103 when the image rendering state includes the shadow rendering state, the target rendering state to be switched for the region to be adjusted can be determined in the following two ways, that is, step S103 can be implemented in the following two ways:
  • Step S111 in the to-be-processed image, determine a reference area whose shadow rendering state matches the scene information.
  • the shadow rendering state includes: a shadowed state and a shadowless state, and the shadow rendering state of the area to be adjusted does not match the scene information, then the shadow rendering state of the reference area that is different from the shadow rendering state of the area to be adjusted is matches the scene information. For example, if the scene information is a scene with no light source at night, the area to be adjusted is the shadow area generated by the target object in the image in the case of a scene without light source at night; based on this, the reference area is the image area without shadow in the image.
  • a shadow-free area whose shadow rendering state is a shadow-free state is used as a reference area in the image to be processed.
  • the target object in a reasonable night scene image does not produce shadows; therefore, using the shadow-free area as a reference area enables the area to be adjusted to be adjusted according to the reference area. more in line with the laws of nature.
  • Step S112 determining the shadow rendering state of the reference area as the target rendering state.
  • the shadow rendering state of the reference region is used as the target rendering state to which the region to be adjusted needs to be converted.
  • the scene information is a scene with no light source at night or a scene without sunlight during the day
  • an image area whose shadow rendering state is no shadow state is determined as the reference area.
  • the image area in the unshaded state that matches the scene is used as the reference area, so as to perform histogram matching on the area to be adjusted based on the reference area, so that after processing
  • the picture chromaticity presented by the to-be-adjusted area is the same as the picture chromaticity of the reference area.
  • the scene information is a scene with a light source at night or a scene with sun during the day
  • an image area whose shadow rendering state is shadowed is determined as a reference area.
  • the shaded image area that matches the scene is used as the reference area, so that the area to be adjusted is histogram matched based on the reference area, so that the processed
  • the chromaticity of the picture presented in the adjustment area is the same as the chromaticity of the picture in the reference area.
  • the image area whose shadow rendering state matches the scene information is used as the reference area, so that the histogram of the area to be adjusted is matched with the reference area, so that the histograms of the area to be adjusted and the reference area are consistent, so that the target image
  • the picture effect is more natural.
  • the target rendering state is determined by determining an image region in the image to be processed whose shadow rendering state matches the scene information, so that the hue of the region to be adjusted after the shadow adjustment is consistent with the hue of other regions.
  • Method 2 By analyzing the lighting parameters of the scene information, the shadow rendering state matching the lighting parameters is determined, so as to obtain the target rendering state.
  • the illumination parameters include: illumination intensity and illumination angle of the light. If the light intensity is strong (for example, the light intensity is greater than a certain threshold), then the target rendering state is determined to be a shadowed state; if the light intensity is weak (for example, the light intensity is less than a certain threshold), then the target rendering state is determined to be a shadowless state. In this way, in the second manner, the target rendering state is set by analyzing the scene information of the image to be processed, so that the set target rendering state can be more in line with the requirements of the scene.
  • step S102 when the scene information is a scene with no light source at night or a scene without sunlight during the day, the area where the shadow is generated is used as the area to be adjusted, that is, step S102 can be implemented through the following process:
  • the shadow area in the image to be processed is determined as the area to be adjusted.
  • the target object if there is no light source at night or no sunlight during the day, the target object will not produce shadows; therefore, in the image to be processed, if the target object has shadows, it means that the shadow area is unreasonable; Based on this, among the target objects of the image to be processed, the shadowed object is selected as the target object, so that an unreasonable image area can be determined in the image to be processed subsequently.
  • the image to be processed is a road image in a scene with no light source at night
  • the target objects that can generate shadows under the illumination of light include vehicles.
  • the shadow rendering state of the shadow area generated by the vehicle is a shadow state, and the shadow area generated by the vehicle is used as the area to be adjusted.
  • FIG. 2 is a schematic flowchart of another implementation of the image generation method provided by the embodiments of the present disclosure, and the following description is made with reference to FIGS. 1 and 2:
  • Step S201 when the scene information is a scene with a light source at night or a scene with sunlight during the day, determine a shadow area that can be generated by the target object according to the direction in which the light illuminates the target object.
  • the shadow area that can be generated by the target object is determined by judging the direction in which the light illuminates the target object; if the light source in the scene with a light source at night is If the artificial light source is used, not only the direction in which the light illuminates the target object, but also the distance between the target object and the artificial light source should be further judged to determine the shadow area that the target object can produce.
  • the implementation process is shown in steps S221 and 222.
  • Step S202 Determine the shadow area except the first shadow area in the shadow area in the image to be processed as the area to be adjusted.
  • the first shadow area is an intersection of a shadow area in the image to be processed and a shadow area that can be generated by the target object.
  • the above steps S201 and S202 provide a method for determining the area to be adjusted in the image to be processed when the scene information is a scene with a light source at night or a scene with sunlight during the day.
  • the shadow area other than the shadow area that can be generated by the target object is used as the area to be adjusted, so that the area that needs to be adjusted in the shadow rendering state can be more accurately determined.
  • step S102 can be implemented through the following process :
  • the shadow area generated by the target object whose distance from the light source is greater than the preset distance threshold and the generated shadow area is determined as the area to be adjusted.
  • the shadow area generated by the target object capable of generating the shadow area may be an image area on a side away from the light source.
  • the distance between the artificial light source and the target object is greater than the preset distance threshold, indicating that the target object is not within the illumination range of the artificial light source, that is, when the artificial light source is in the working state, the target object cannot produce shadows.
  • the resulting shaded area is unreasonable.
  • the distance between the artificial light source and each target object is determined. First, determine the illumination range of the artificial light source; then, determine whether the target object is within the illumination range of the artificial light source by judging the distance between the artificial light source and each target object.
  • the image to be processed is an image of an urban road in a scene with street lights at night, there are street lights that are turned on in the scene, and the target objects are: buildings and trees.
  • the distance between the street light and the building and the tree is determined respectively, so as to judge whether the building and the tree are within the illumination range of the street light, that is, whether the building and the tree can produce shadows under the illumination of the street light.
  • the shadow area generated by the target object is the area to be adjusted.
  • the above provides a method for determining an area to be adjusted in an image to be processed when the scene information is a scene with artificial light sources at night.
  • this method when the scene information is a scene with artificial light sources at night, It is judged whether the target object in the image outside the light irradiation range has a shadow area, and the unreasonable shadow area is regarded as the area to be adjusted.
  • steps S103 and S104 can be realized by the following steps S203 and 204 respectively, and the processing of the area to be adjusted, so that Get the target image.
  • Step S203 determining that the shadow-free state is a target rendering state matching the scene information.
  • the target object when the scene information is a scene with no light source at night or a scene without sunlight during the day, the target object will not produce shadows; therefore, in the image to be processed, if there is no shadow, the image is reasonable. That is, it matches the scene information; based on this, in the to-be-processed image, a non-shaded area can be used as a reference area, that is, the target rendering state that matches the scene information is a shadow-free state.
  • Step S204 in the to-be-processed image, adjust the image of the to-be-adjusted area to a shadow-free state according to the image of the reference area to obtain the target image.
  • the shadow of the area is removed, thereby obtaining the target image.
  • the shadow of the to-be-adjusted area can be removed by matching the histogram of the shadow-free reference area with the histogram of the to-be-adjusted area, so that the shadow-free image area in the target image has The grayscale is consistent; other methods can also be used to remove shadows in the area to be adjusted; for example, the grayscale image of the area to be adjusted first performs maximum filtering, and then performs minimum filtering.
  • the image background of the area to be adjusted is dark and the target object is bright, you can perform the minimum filter first, and then the maximum filter to remove the shadow of the area to be adjusted; you can also use the preset image library to select The shadow-free image that matches the picture content of the area to be adjusted directly replaces the area to be adjusted; the shadow of the area to be adjusted can also be removed by decomposing the shadow image.
  • the target image in the image to be processed, by determining the area occupied by the area to be adjusted in the image to be processed, and adjusting the image of the area to be adjusted from a shadowed state to a shadowless state according to the image of the reference area, the obtained target image.
  • the target image in the to-be-processed image, can be obtained by performing shadow removal processing on the to-be-adjusted area without replacing the to-be-adjusted area with other areas. In this way, the shadow rendering state in the generated target image is no shadow state, which more closely matches the scene with no light source at night and the scene without sunlight during the day.
  • step S204 can be implemented by the following steps:
  • Step S241 Adjust the grayscale image of the to-be-adjusted area according to the image of the reference area to obtain an adjusted grayscale image.
  • a grayscale image of the reference region is determined.
  • the reference area may be determined from the image to be processed, that is, the image area in the image to be processed that has a shadow rendering state matching the scene as the reference area; it may also be that after setting the target rendering state, based on the target rendering state and The image area generated by the image content of the area to be adjusted.
  • a grayscale image of the reference region is obtained by determining the histogram of the reference region. For example, after the reference area is determined, a histogram of the reference area is generated according to an image of the reference area, and a grayscale image of the reference area can be generated based on the histogram.
  • the target rendering state is a no-shadow state.
  • the area with shadows in the image area is the area to be adjusted, and the area without shadows in the image area is the reference area; and by determining the histogram of the shadowless area, generate Grayscale image of the area.
  • the grayscale image of the to-be-adjusted area is adjusted to obtain an adjusted grayscale image.
  • the grayscale image of the area to be adjusted can be obtained by determining the histogram of the area to be adjusted. For example, after the region to be adjusted is determined, a histogram of the region to be adjusted is generated according to an image of the region to be adjusted, and a grayscale image of the region to be adjusted can be generated based on the histogram.
  • the grayscale image of the reference area is used as a reference to adjust the grayscale image of the area to be adjusted, so that the adjusted grayscale image obtained matches the grayscale image of the reference area, that is, the color corresponding to the two grayscale images is obtained.
  • the tones of the images are consistent.
  • the adjustment of the grayscale image of the area to be adjusted can be achieved through the following process:
  • the histogram of the reference area is determined according to the image of the reference area, and the histogram of the area to be adjusted is adjusted based on the histogram of the reference area to obtain an adjusted histogram; for example, the histogram of the area to be adjusted is compared with that of the reference area. Histogram, perform histogram matching to obtain an adjusted histogram.
  • the histogram of the area to be adjusted is converted to the target histogram, so that the histogram of the reference area matches the histogram of the area to be adjusted, that is, the histogram of the area to be adjusted Match the histogram of the reference area to keep the tones of the two areas consistent; in this way, the shadow rendering state of the area to be adjusted will be consistent with the shadow rendering state of the reference area, and both can match the scene information, which is reasonable in this scene. shadow state.
  • an adjusted grayscale image is determined. For example, based on the adjusted histogram, a grayscale image of the adjusted histogram can be generated, that is, an adjusted grayscale image can be obtained.
  • the reference area is a shadow-free area in the image to be processed.
  • Step S242 based on the adjusted grayscale image, generate an image of the replacement area in which the shadow rendering state is a shadow-free state.
  • the grayscale image is rendered in combination with the picture content of the area to be adjusted to generate an image of the replacement area in a shadow-free state.
  • Step S243 in the image to be processed, the generated image of the replacement area is used to replace the image of the area to be adjusted to generate a target image.
  • the image to be processed by determining the image of the replacement area in which the shadow rendering state does not match the scene information, and determining the area occupied by the area to be adjusted in the image to be processed; use the image of the replacement area to replace The image of the to-be-adjusted area is smoothed for the replaced image, thereby generating a target image.
  • the image to be processed is a road image
  • the area to be adjusted whose shadow rendering state does not match the scene information is the shadow area
  • the image of the replacement area is the shadow rendering state
  • the shadow rendering state is None
  • an image in a shadow state such a shadow-free image area is used to replace the image of the to-be-adjusted area, so as to obtain a target image whose shadow rendering state matches the scene information.
  • the shadow rendering state in the generated target image matches the scene information, so that the generated target image is more realistic.
  • step S243 after replacing the to-be-adjusted area with the replacement area, further smoothing is performed on the replaced image to generate the target image, that is, step S243 can be achieved by the following steps:
  • the first step is to adjust the shape and size of the image of the replacement area to be consistent with the image of the to-be-adjusted area to obtain the adjusted image of the replacement area.
  • the size information of the image of the replacement area is determined. After the replacement area is determined, size information of the replacement area needs to be determined, at least including: area, perimeter, length, width, and edge shape. Then, the area occupied by the image of the area to be adjusted in the image to be processed is determined. In some possible implementations, the area to be adjusted is marked with a detection frame through an image segmentation network, and the area of the detection frame can be used as the area of the area occupied by the area to be adjusted in the image to be processed. Finally, according to the area, the image size information of the replacement area is adjusted to obtain an image of the adjusted replacement area.
  • the image size information of the replacement area is adjusted according to the area of the area occupied by the image of the area to be adjusted in the image to be processed, so that the image size information of the adjusted replacement area is the same as the size of the image to be adjusted.
  • the image size of the area fits.
  • the image of the to-be-adjusted area in the to-be-processed image is replaced with the image of the adjusted replacement area, and the edge of the adjusted image of the replacement area is smoothed to generate the target image.
  • the image of the to-be-adjusted area is replaced with the image of the adjusted replacement area to generate a candidate image. That is, in the image to be processed, the adjusted replacement area is used to replace the to-be-adjusted area, so as to obtain a replaced image, that is, a candidate image.
  • the scene information is a scene with no light source at night
  • the image to be processed is a road image
  • the area to be adjusted is a shadow area whose shadow rendering state does not match the scene with no light source at night.
  • the replacement area has been adjusted to include a large tree without shadows, and by replacing The image of the area is resized, and the image of the adjusted replacement area can be obtained. Therefore, the image of the to-be-adjusted area is replaced by the image of the adjusted replacement area to generate the target image. Then, the candidate image is smoothed to generate the target image.
  • the area where the replacement operation occurs in the candidate object may be smoothed to eliminate noise in the area, or the entire candidate image may be smoothed to denoise the entire image. , so as to obtain the target image.
  • the image of the to-be-adjusted area is replaced with the image of the adjusted replacement area after size adjustment, and the replaced image is smoothed, so that the generated image The target image is more reasonable and clear.
  • the light source is an artificial light source as an example for illustration
  • the shadow area that can be generated by the target object within the illumination range of the light source is used as the area to be adjusted, which can be achieved by the following steps. :
  • Step S221 when the scene information is a scene with a light source at night or a scene with sunlight during the day, determine a shadow area that can be generated by the target object according to the direction in which the light illuminates the target object.
  • the shadow area that can be generated by the target object is determined.
  • the shadow rendering state that conforms to the actual scene is that the target object has shadows.
  • Step S222 determining the shadow area that can be generated by the target object as the area to be adjusted.
  • a target object that is closer to the artificial light source may be determined; then, among these closer target objects, a target object that does not generate a shadow area is determined. Since the distance between the target object and the artificial light source is relatively close, it means that the target object is within the illumination range of the artificial light source, then in the scene with artificial light source at night, the target object within the illumination range of the artificial light source can produce shadows; Objects do not produce shaded areas, so it is not reasonable. Therefore, the shadow area that can be generated by the target object that does not generate the shadow area needs to be the area to be adjusted.
  • the shadow area that can be generated by a target object that does not generate a shadow area may be determined based on the image area occupied by the target object and the illumination direction of the artificial light source; that is, in the image occupied by the target object
  • the shadow area and shadow position of the target object under the illumination of the artificial light source can be estimated; the image area corresponding to the shadow area and shadow position is determined as the target.
  • the shadow area that the object can produce may be determined based on the image area occupied by the target object and the illumination direction of the artificial light source; that is, in the image occupied by the target object
  • the shadow area and shadow position of the target object under the illumination of the artificial light source can be estimated; the image area corresponding to the shadow area and shadow position is determined as the target.
  • the shadow area that the object can produce.
  • the shadow area that can be generated by the target object within the illumination range of the light source is used as the area to be adjusted, or when there is moonlight in the night scene or in the daytime scene, the target object can be adjusted
  • the generated shadow area is determined as the area to be adjusted.
  • the relative position of the light source and the target object may vary.
  • the original image is an image collected in the morning with sunlight, and the current time is in the afternoon.
  • the irradiation direction of the sunlight and the relative position of the target object have changed.
  • the target image is obtained. In this way, the final adjusted target image can be made more realistic and natural.
  • the area to be adjusted can be determined in the following ways:
  • Step S231 in the image to be processed, if the scene information is a scene with moonlight at night, determine a target object that can be irradiated by moonlight.
  • Step S232 among the target objects that can be irradiated by moonlight, determine the shadow area that can be generated by the target object that fails to generate the shadow area as the area to be adjusted.
  • the area to be adjusted may be determined in the following manner:
  • Step S241 in the image to be processed, when the scene information is a scene with sunlight during the day, determine a target object that can be irradiated by sunlight.
  • Step S242 among the target objects that can be irradiated by sunlight, determine the shadow area that can be generated by the target object that fails to generate shadows as the to-be-adjusted area.
  • the adjustment of the area to be adjusted may be implemented through the following process to obtain the target image:
  • the first step is to determine the lighting parameters in a particular scene.
  • the artificial light source if it is for a scene with an artificial light source at night, it is determined that the artificial light source is in a working state, that is, it is in an enabled state, and the lighting parameters include: light intensity, light irradiation direction, and light brightness. If it is for a scene with moonlight at night or a scene with sunlight during the day, determine the illumination parameters of the moonlight or sunlight hitting the ground at the current moment.
  • the illumination intensity of the surface of the target object capable of generating the shadow area is determined based on the illumination parameters, the area to be adjusted is rendered, and the target image is obtained.
  • the image to be processed includes: a vehicle and a street lamp; wherein the artificial light source is a street lamp, and by analyzing the illumination intensity and light irradiation direction of the street lamp, it is determined that the vehicle is in the image to be processed.
  • the shadow area that can be generated in the image is rendered according to the light intensity of the vehicle surface to obtain the replacement area image.
  • the image of the to-be-adjusted area is replaced by the obtained image of the replacement area to generate the target image.
  • the target object in a scene with an artificial light source at night and the target object is within the illumination range of the artificial light source, or a scene with the moon at night, or a scene with sunlight during the day, in the image to be processed, according to the illumination parameters , rendering the shadow area that the target object can produce; thus, the target object in the target image that is closer to the artificial light source can produce a shadow area, and in the case of natural light, the target object produces a shadow area, thereby improving the generated target image fidelity.
  • Embodiments of the present disclosure provide a method for removing shadows in a nighttime scene without a light source using an image generation and histogram matching method.
  • By performing the processes of shadow region segmentation, histogram matching, and image generation on the obtained daytime scene image it can solve the problem of a daytime scene with sunlight.
  • the embodiments of the present disclosure can effectively remove shadows in the generated images, so that the generated night scene images are more realistic.
  • FIG. 3A is a schematic diagram of the composition and structure of an image generation system provided by an embodiment of the present disclosure, and the following description is made in conjunction with FIG. 3A :
  • the image generation system includes: a generator 301 and a discriminator 302 . Among them, first, the original image of the scene with sunlight during the day is used as input, and is input from the input terminal 303 to the generator 301;
  • the image 321 in FIG. 3B is the original image
  • the image 322 is a night image converted from a scene with sunlight in the daytime of the image 321 into a scene without light sources at night; Segment, and perform histogram matching on the shaded area and the non-shaded area to obtain the unshaded image 323; finally, use the GAN network to convert the daytime sun scene in the unshaded image 323 to the night without light source scene, and obtain the night image 324.
  • the night image 324 has no shadows compared to the image 322 , which is more in line with the scene of no light source at night, and the image picture is more realistic.
  • both the night scene image collected in the night scene with no light source and the generated night scene image are input into the discriminator 302 .
  • the discriminator 302 is used to distinguish whether the image of the night scene is the collected night scene image or the generated night scene image, that is, the real image 305 and the converted image 306 are obtained respectively;
  • FIG. 4 is an implementation frame structure diagram of the image generation method provided by the embodiment of the present disclosure, and the following description is made in conjunction with FIG. 4 :
  • the daytime image acquisition module 401 is configured to acquire an image of a scene with sunlight in the daytime to obtain a daytime image.
  • the shadow segmentation network module 402 is configured to divide the daytime image into shadow areas 431 and non-shadow areas 432 .
  • shadow regions are obtained from daytime images by an image segmentation network, resulting in shadow regions and non-shadow regions.
  • a network for image segmentation is trained by optimization and model (no specific network structure is limited here; for example, a fully convolutional network, a semantic segmentation network, and a Visual Geometry Group (VGG) network) , RefineNet, etc.) to obtain an image shadow extraction network that can accurately identify the shadow area of the image.
  • the image shadow extraction network is used to segment the shadow area and non-shadow area of the daytime image. Therefore, the shaded area and the non-shaded area in the daytime image can be obtained at the same time.
  • the histogram matching module 403 is configured to match the histogram of the shaded area 431 with the non-shaded area 432 through a histogram matching method.
  • the shaded area obtains a histogram distribution consistent with that of the non-shaded area, so that the shades of the shaded area and the non-shaded area are consistent. Since there is a big difference in the histogram distribution between the shaded area and the non-shaded area, the embodiment of the present disclosure uses the histogram matching method for the shaded area and the non-shaded area, so that the histogram of the shaded area is directed to the target histogram (non-shaded area) Transformation, in this way, can convert the image of the shaded area to the image of the unshaded area.
  • the daytime shadow-free image obtaining module 404 is configured to remove shadows in the daytime image by means of histogram matching to obtain a daytime shadow-free image.
  • the transformed shadowless area is pasted back into the original image, that is, the transformed shadowless area is pasted back into the daytime image to obtain a shadowless image of a scene with sunlight during the day.
  • the image generation network module 405 is configured to convert an image without shadows during the day from a scene with sunlight during the day to a scene without a light source at night to obtain a night image.
  • the image generation network shown in FIG. 3A is used to convert a scene with sun during the day without shadows in the daytime into a scene without light sources at night to obtain a night image.
  • the final result output module 406 is configured to use a smoothing technique to smooth the surrounding image of the replaced shadow area, so as to obtain the final night image.
  • FIG. 5 is a schematic diagram of another application scenario of the image generation method according to the embodiment of the present disclosure, wherein the original image 501 is For the images collected in the sunlight scene during the day, the generation process of the target image is as follows:
  • the shadow regions 502, 503 and 504 are marked with rectangular boxes.
  • the regions marked by the rectangular frame that is, the shadow regions 502 , 503 and 504 , corresponding to the regions to be adjusted in the above embodiment
  • the regions marked by the rectangular frame are deducted and enlarged to obtain the enlarged shadow regions 511 , 512 and 513 in sequence.
  • the histogram of the shadowed images 521, 522 and 523 at night is converted to the histogram of the shadowless area by means of histogram matching to obtain the shadow-removed images 531, 532 and 533 (corresponding to the target in the above embodiment). image).
  • an image segmentation network is used to obtain the shadow area and the non-shadow area. Then, the histogram matching of the shadow area and the non-shadow area is used to obtain the image after removing the shadow; The image of the scene is transformed to a night scene without light source, resulting in a night scene image that does not contain shadow areas. In this way, the shadow of the image generated at night is effectively removed, so that the generated night scene image is more realistic and more consistent with the authenticity of the night scene.
  • FIG. 6 is a schematic structural diagram of an image generating apparatus according to an embodiment of the present disclosure.
  • the apparatus 600 includes:
  • a first conversion module 601 configured to convert an original image into an image to be processed in a specific scene
  • the first determining module 602 is configured to, in the image to be processed, determine an area to be adjusted whose image rendering state does not match the scene information of the specific scene; ray correlation on the target object, the target object includes a vehicle;
  • the second determination module 603 is configured to determine the target rendering state in the to-be-adjusted area; the target rendering state matches the scene information;
  • the first adjustment module 604 is configured to, in the to-be-processed image, adjust the image rendering state of the to-be-adjusted area to the target rendering state to obtain a target image.
  • the image rendering state includes a shadow rendering state
  • the second determining module 603 includes:
  • a first determination submodule configured to determine, in the to-be-processed image, a reference area where the shadow rendering state matches the scene information
  • the second determining submodule is configured to determine the shadow rendering state of the reference area as the target rendering state.
  • the first determining module 602 is further configured to:
  • the shadow area in the image to be processed is determined as the area to be adjusted.
  • the first determining module 602 includes:
  • a third determining submodule configured to determine a shadow area that can be generated by the target object according to the direction in which the light illuminates the target object when the scene information is a scene with a light source at night or a scene with sunlight during the day;
  • a fourth determination sub-module configured to determine the shadow area except the first shadow area in the shadow area in the image to be processed as the area to be adjusted, and the first shadow area is the shadow in the image to be processed The intersection of the area and the shadow area that the target object can produce.
  • the shadow rendering state of the reference area is a shadowless rendering state
  • the first adjustment module 604 includes:
  • the first adjustment sub-module is configured to, in the image to be processed, adjust the image of the to-be-adjusted area to a shadow-free state according to the image of the reference area to obtain the target image.
  • the first adjustment sub-module includes:
  • a first adjustment unit configured to adjust the grayscale image of the to-be-adjusted area according to the image of the reference area to obtain an adjusted grayscale image
  • a first generating unit configured to generate, based on the adjusted grayscale image, an image of the replacement area in which the shadow rendering state is a shadow-free state
  • the second generating unit is configured to replace the image of the to-be-adjusted area with the generated image of the replacement area in the to-be-processed image to generate the target image.
  • the first adjustment unit includes:
  • a first determining subunit configured to determine a histogram of the reference area according to an image of the reference area
  • a first adjustment subunit configured to adjust the histogram of the to-be-adjusted area based on the histogram of the reference area to obtain an adjusted histogram
  • the second determination subunit is configured to determine the adjusted grayscale image based on the adjusted histogram.
  • the image rendering state includes a shadow rendering state
  • the first determining module 602 includes:
  • a fifth determination sub-module configured to determine a shadow area that can be generated by the target object according to the direction in which the light illuminates the target object when the scene information is a scene with a light source at night or a scene with sunlight during the day;
  • the sixth determination sub-module is configured to determine the shadow area that can be generated by the target object as the area to be adjusted.
  • the first adjustment module 604 includes:
  • a seventh determination sub-module configured to determine lighting parameters in the specific scene
  • the first rendering sub-module is configured to determine, based on the illumination parameters, the illumination intensity of the surface of the target object capable of generating a shadow area, render the to-be-adjusted area, and obtain the target image.
  • the original image is an image collected in no rain/snow weather
  • the specific scene is a rain/snow scene
  • the first adjustment module 604 includes:
  • the first adding submodule is configured to add raindrops/snowflakes to the original image to obtain a target image.
  • the above-mentioned image generation method is implemented in the form of a software function module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
  • the technical solutions of the embodiments of the present disclosure essentially or the parts that make contributions to the prior art can be embodied in the form of a software product, and the computer software product is stored in a storage medium and includes several instructions for A computer device (which may be a terminal, a server, etc.) is caused to execute all or part of the methods described in the various embodiments of the present disclosure.
  • the aforementioned storage medium includes: a U disk, a mobile hard disk, a read only memory (Read Only Memory, ROM), a magnetic disk or an optical disk and other media that can store program codes.
  • ROM Read Only Memory
  • an embodiment of the present disclosure further provides a computer program product, the computer program product includes computer-executable instructions, and after the computer-executable instructions are executed, can implement the steps in the image generation method provided by the embodiment of the present disclosure.
  • the embodiments of the present disclosure further provide a computer storage medium, where computer-executable instructions are stored on the computer storage medium, and when the computer-executable instructions are executed by a processor, the image generation methods provided by the above embodiments are implemented. step.
  • FIG. 7 is a schematic structural diagram of a computer device according to an embodiment of the present disclosure.
  • the computer device 700 includes: a processor 701 , at least one communication bus, Communication interface 702 , at least one external communication interface and memory 703 .
  • the communication interface 702 is configured to realize the connection communication between these components.
  • the communication interface 702 may include a display screen, and the external communication interface may include a standard wired interface and a wireless interface.
  • the processor 701 is configured to execute the image processing program in the memory, so as to implement the steps of the image generation method provided by the above embodiments.
  • the disclosed apparatus and method may be implemented in other manners.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components may be combined, or Can be integrated into another system, or some features can be ignored, or not implemented.
  • the coupling, or direct coupling, or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical or other forms. of.
  • the unit described above as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit; it may be located in one place or distributed to multiple network units; Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • each functional unit in each embodiment of the present disclosure may be all integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above integration
  • the unit can be implemented either in the form of hardware or in the form of hardware plus software functional units.
  • the above-mentioned integrated units of the present disclosure are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.
  • the technical solutions of the embodiments of the present disclosure essentially or the parts that make contributions to the prior art can be embodied in the form of a software product, and the computer software product is stored in a storage medium and includes several instructions for A computer device (which may be a personal computer, a server, or a network device, etc.) is caused to execute all or part of the methods described in the various embodiments of the present disclosure.
  • the aforementioned storage medium includes various media that can store program codes, such as a removable storage device, a ROM, a magnetic disk, or an optical disk.
  • Embodiments of the present disclosure provide an image generation method, apparatus, device, and storage medium, wherein an original image is converted into a to-be-processed image in a specific scene; in the to-be-processed image, an image rendering state and the specific scene are determined.
  • the image rendering state of the to-be-adjusted area is related to the light irradiated on the target object in the original image, and the target object includes a vehicle; determine the target rendering in the to-be-adjusted area
  • the target rendering state matches the scene information; in the to-be-processed image, the image rendering state of the to-be-adjusted area is adjusted to the target rendering state to obtain the target image.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)

Abstract

本公开实施例提供一种图像生成方法、装置、设备及存储介质，其中，将原始图像转换为特定场景下的待处理图像；在所述待处理图像中，确定图像渲染状态与所述特定场景的场景信息不匹配的待调整区域；所述待调整区域的图像渲染状态与照射到原始图像中的目标对象上的光线相关，所述目标对象包括车辆；确定所述待调整区域内的目标渲染状态；所述目标渲染状态与所述场景信息相匹配；在所述待处理图像中，将所述待调整区域的图像渲染状态调整为所述目标渲染状态，得到目标图像。如此，生成的目标图像中的阴影渲染状态与场景信息相匹配，从而使得生成的目标图像更加真实。

Description

一种图像生成方法、装置、设备及存储介质
相关申请的交叉引用
本公开基于申请号为202110351943.9、申请日为2021年3月31日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本公开作为参考。
技术领域
本公开实施例涉及图像处理技术领域，涉及但不限于一种图像生成方法、装置、设备及存储介质。
背景技术
在相关技术的图像生成方法中，当需要将白天采集的一张道路场景的图像通过风格转换，转换为夜晚场景的图像时，由于白天采集的图像中存在阴影，转换后的图像中仍然存在阴影，使得生成的图像的真实性不高。
发明内容
本公开实施例提供一种图像生成技术方案。
本公开实施例的技术方案是这样实现的：
本公开实施例提供一种图像生成方法，所述方法包括：
将原始图像转换为特定场景下的待处理图像；
在所述待处理图像中，确定图像渲染状态与所述特定场景的场景信息不匹配的待调整区域；所述待调整区域的图像渲染状态与照射到原始图像中的目标对象上的光线相关，所述目标对象包括车辆；
确定所述待调整区域内的目标渲染状态；所述目标渲染状态与所述场景信息相匹配；
在所述待处理图像中，将所述待调整区域的图像渲染状态调整为所述目标渲染状态，得到目标图像。
在一些实施例中，所述图像渲染状态包括阴影渲染状态，所述确定所述待调整区域内的目标渲染状态，包括：在所述待处理图像中，确定阴影渲染状态与所述场景信息相匹配的参考区域；将所述参考区域的阴影渲染状态，确定为所述目标渲染状态。如此，无论是在夜晚无光源场景或白天无阳光场景的情况下，将与场景匹配的无阴影状态的图像区域作为参考区域，以便于基于参考区域对待调整区域进行直方图匹配，从而使得处理后的待调整区域呈现的画面色度与参考区域的画面色度相同。
在一些实施例中，所述在所述待处理图像中，确定图像渲染状态与所述特定场景的场景信息不匹配的待调整区域，包括：在所述场景信息为夜晚无光源场景或白天无阳光场景的情况下，将所述待处理图像中的阴影区域，确定为所述待调整区域。如此，能够准确地确定出画面内容不合理的图像区域。
在一些实施例中，所述在在所述待处理图像中，确定图像渲染状态与所述特定场景的场景信息不匹配的待调整区域，包括：在所述场景信息为夜晚有光源场景或白天有阳光场景的情况下，根据光线照射所述目标对象的方向，确定所述目标对象能够产生的阴影区域；将所述待处理图像中的阴影区域中除第一阴影区域之外的阴影区域确定为待调整区域，所述第一阴影区域为所述待处理图像中的阴影区域与所述目标对象能够产生的阴影区域的交集。如此，通过将待处理图像中除去目标对象能够产生的阴影区域之外的阴影区域，作为待调整区域，从而能够更加准确的确定出阴影渲染状态需要调整的区域。
在一些实施例中，所述参考区域的阴影渲染状态为无阴影渲染状态，所述在所述待处理图像中，将所述待调整区域的图像渲染状态调整为所述目标渲染状态，得到目标图像，包括：在所述待处理图像中，根据所述参考区域的图像，将所述待调整区域的图像调整为无阴影状态，得到所述目标图像。如此，生成的目标图像中阴影渲染状态为无阴影状态，与夜晚无光源场景以及白天无阳光场景更加匹配。
在一些实施例中，所述在所述待处理图像中，根据所述参考区域的图像，将所述待调整区域的图像调整为无阴影状态，得到所述目标图像，包括：根据所述参考区域的图像调整所述待调整区域的灰度图像，得到已调整灰度图；基于所述已调整灰度图，生成阴影渲染状态为无阴影状态的替换区域的图像；在所述待处理图像中，采用生成的替换区域的图像替换所述待调整区域的图像，生成所述目标图像。如此，生成的目标图像中阴影渲染状态与场景信息匹配，使得生成的目标图像更加逼真。
在一些实施例中，所述根据所述参考区域的图像调整所述待调整区域的灰度图像，得到已调整灰度图，包括：根据所述参考区域的图像确定所述参考区域的直方图；基于所述参考区域的直方图，调整所述待调整区域的直方图，得到已调整直方图；基于所述已调整直方图，确定所述已调整灰度图。如此，能够更加自然地去掉图像中的阴影部分，使得到的图的画面内容更加逼真。
在一些实施例中，所述图像渲染状态包括阴影渲染状态，在所述待处理图像中，确定图像渲染状态与所述场景信息不匹配的待调整区域，包括：在所述场景信息为夜晚有光源场景或白天有阳光场景的情况下，根据光线照射所述目标对象的方向，确定所述目标对象能够产生的阴影区域；将所述目标对象能够产生的阴影区域确定为待调整区域。如此，如此，能够使得调整后的目标图像更加真实自然。
在一些实施例中，所述在所述待处理图像中，将所述待调整区域的图像渲染状态调整为所述目标渲染状态，得到目标图像，包括：确定所述特定场景中的光照参数；基于所述光照参数确定能够产生阴影区域的目标对象表面的光照强度，渲染所述待调整区域，得到所述目标图像。如此，能够使得目标图像中距离人造光源较近的目标对象产生阴影区域，以及，在有自然光的情况下目标对象产生阴影区域，进而提高了生成的目标图像的逼真度。
在一些实施例中，所述原始图像为无雨/雪天气下采集的图像，所述特定场景为雨/雪场景；在所述待处理图像中，将所述待调整区域的图像渲染状态调整为所述目标渲染状态，得到目标图像，包括在所述原始图像中增加雨滴/雪花，得到目标图像。如此，能够得到更加逼真的目标图像。
本公开实施例提供一种图像生成装置，所述装置包括：
第一转换模块，配置为将原始图像转换为特定场景下的待处理图像；
第一确定模块，配置为在所述待处理图像中，确定图像渲染状态与所述特定场景的场景信息不匹配的待调整区域；所述待调整区域的图像渲染状态与照射到原始图像中的目标对象上的光线相关，所述目标对象包括车辆；
第二确定模块，配置为确定所述待调整区域内的目标渲染状态；所述目标渲染状态与所述场景信息相匹配；
第一调整模块，配置为在所述待处理图像中，将所述待调整区域的图像渲染状态调整为所述目标渲染状态，得到目标图像。
对应地，本公开实施例提供一种计算机存储介质，所述计算机存储介质上存储有计算机可执行指令，该计算机可执行指令被执行后，能够实现上述所述的方法步骤。
本公开实施例提供一种电子设备，所述电子设备包括存储器和处理器，所述存储器上存储有计算机可执行指令，所述处理器运行所述存储器上的计算机可执行指令时可实现上述所述的方法步骤。
本公开实施例提供一种计算机程序，包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行用于实现上述任意一项所述的图像生成方法。
本公开实施例提供一种图像生成方法、装置、设备及存储介质，首先对原始图像进行转换，得到特定场景下的待处理图像；然后，在待处理图像中，确定出图像渲染状态与场景信息不匹配的待调整区域；然后，通过将待调整区域的渲染状态调整为与场景信息相匹配的目标渲染状态，生成图像渲染状态与场景信息匹配的目标图像；如此，生成的目标图像中的阴影渲染状态与场景信息相匹配，从而使得生成的目标图像更加真实。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，这些附图示出了符合本公开实施例的实施例，并与说明书一起用于说明本公开实施例的技术方案。
图1A为可以应用本公开实施例的轨迹预测方法的一种***架构示意图；
图1B为本公开实施例提供的图像生成方法的实现流程示意图；
图2为本公开实施例提供的图像生成方法的另一实现流程示意图；
图3A为本公开实施例提供的图像生成***的组成结构示意图；
图3B为本公开实施例提供的图像生成方法的应用场景示意图；
图4为本公开实施例提供的图像生成方法的实现框架结构图；
图5为本公开实施例提供的图像生成方法的另一应用场景示意图；
图6为本公开实施例提供的图像生成装置结构组成示意图；
图7为本公开实施例提供的计算机设备的组成结构示意图。
具体实施方式
在以下的描述中，所涉及的术语“第一\第二\第三”仅仅是是区别类似的对象，不代表针对对象的特定排序，可以理解地，“第一\第二\第三”在允许的情况下可以互换特定的顺序或先后次序，以使在一些实施例中描述的本公开实施例能够以除了在在一些实施例中图示或描述的以外的顺序实施。
除非另有定义，本文所使用的所有的技术和科学术语与属于本公开的技术领域的技术人员通常理解的含义相同。本文中所使用的术语只是为了描述本公开实施例的目的，不是旨在限制本公开。
对本公开实施例进行进一步详细说明之前，对本公开实施例中涉及的名词和术语进行说明，本公开实施例中涉及的名词和术语适配置为如下的解释。
1)高斯模糊：对于图像来说就是一个低通滤波器。所谓"模糊"，可以理解成每一个像素都取周边像素的平均值。
2)自主车辆(ego vehicle)：包含感知周围环境传感器的车辆。车辆坐标系固连在自主车辆上，其中，x轴为汽车前进的方向，y轴指向车辆前进方向的左侧，z轴垂直于地面向上，符合右手坐标系。坐标系原点位于后轴中点下方的大地上。
下面说明本公开实施例提供的图像生成的设备的示例性应用，本公开实施例提供的设备可以实施为具有图像采集功能的笔记本电脑，平板电脑，台式计算机，相机，移动设备(例如，个人数字助理，专用消息设备，便携式游戏设备)等各种类型的用户终端，也可以实施为服务器。下面，将说明设备实施为终端或服务器时示例性应用。
该方法可以应用于计算机设备，该方法所实现的功能可以通过计算机设备中的处理器调用程序代码来实现，当然程序代码可以保存在计算机存储介质中，可见，该计算机设备至少包括处理器和存储介质。
图1A为可以应用本公开实施例的图像生成方法的一种***架构示意图；如图1A所示，该***架构中包括：图像采集设备131、网络132和图像生成终端133。为实现支撑一个示例性应用，图像采集设备131和图像生成终端133可以通过网络132建立通信连接，图像采集设备131通过网络202向图像生成终端133上报采集到的原始图像(或者，图像生成终端133自动获取原始图像)，图像生成终端133响应于接收到的原始图像，首先，将该图像转换为特定场景下待处理图像，并在待处理图像中确定不合理的待调整区域；然后，确定该待调整区域的目标渲染状态，并待调整区域的图像渲染状态调整为目标渲染状态，得到目标图像，并在图像生成终端133的图像显示界面上输出目标图像。如此，使得目标图像中的对象的工作状态与场景信息相匹配，从而使得生成的目标图像更加符合真实场景。
作为示例，图像采集设备131可以为包括摄像头等的采集设备。图像生成终端133可以包括具有一定计算能力的计算机设备，该计算机设备例如包括：终端设备或服务器或其它处理设备。网络132可以采用有线连接或无线连接方式。其中，在图像生成终端133为服务器时，图像采集设备131可以通过有线连接的方式与服务器通信连接，例如通过总线进行数据通信；在图像生成终端133为终端设备时，图像采集设备131可以通过无线连接的方式与图像生成终端133通信连接，进而进行数据通信。
或者，在一些场景中，图像生成终端133可以是带有视频采集模组的视觉处理设备，可以是带有摄像头的主机。这时，本公开实施例的图像生成方法可以由图像生成终端133执行，上述***架构可以不包含网络132和图像采集设备131。
为使本公开实施例的目的、技术方案和优点更加清楚，下面将结合本公开实施例中的附图，对发明的具体技术方案做进一步详细描述。以下实施例配置为说明本公开，但不用来限制本公开的范围。
在以下的描述中，涉及到“一些实施例”，其描述了所有可能实施例的子集，但是可以理解，“一些实施例”可以是所有可能实施例的相同子集或不同子集，并且可以在不冲突的情况下相互结合。
图1B为本公开实施例图像生成方法的实现流程示意图，如图1B所示，结合如图1B所示步骤进行说明：
步骤S101，将原始图像转换为特定场景下的待处理图像。
在一些实施例中，原始图像可以是任意场景下采集的图像，可以是包括画面内容复杂的图像还可以是包括画面内容简单的图像，比如，在深夜采集的道路上的图像，或者在白天采集的道路上的图像等。特定场景包括：白天场景、傍晚场景、雨雪场景或者晴天场景等。特定场景的场景信息包括采集该待处理图像的场景；比如，场景中光线明暗程度、场景所在的位置以及场景中的对象等；比如，待处理图像是深夜场景下的道路的图像，那么场景信息包括：该道路的明亮程度、该道路的位置以及道路上的车辆和路灯等对象。
在一些可能的实现方式中，待处理图像可以是在特定场景下的图像；比如，深夜场景、傍晚场景或凌晨场景等。
在步骤S101中，将原始图像转换为特定场景下的待处理图像，可以通过以下步骤实现：
通过获取目标对象，按照特定场景对原始图像进行风格转换，得到待处理图像，过程如下：
第一步，获取原始图像。
在一些可能的实现方式中，原始图像为采集到的任意场景下的图像，比如，在白天采集的道路图像或者在夜晚采集的道路图像等。
第二步，确定原始图像的场景信息。
在一些可能的实现方式中，获取到原始图像之后，通过训练好的判别器，判断该图像的场景信息是否为特定场景。比如，特定场景为夜晚无光源场景，通过判别器判断该图像的场景信息是否为夜晚无光源场景。
第三步，在所述场景信息与所述特定场景不匹配的情况下，将原始图像进行风格转换，得到特定场景下的待处理图像。
在一些可能的实现方式中，在原始图像的场景信息与特定场景相差较远的情况下，将原始图像的场景信息转换为特定场景，即将原始图像转换为特定场景下的图像，从而得到特定场景下的待处理图像。比如，特定场景为夜晚无光源场景，原始图像为白天有太阳场景下采集到的图像，那么将白天有太阳场景下采集的原始图像转换为夜晚无光源场景的图像，可以通过将原始图像输入生成器，生成对应的夜晚无光源场景的待处理图像。或者，特定场景为白天有太阳场景，原始图像为夜晚无光源场景下采集到的图像，那么将夜晚无光源场景下采集的原始图像转换为白天有太阳场景的图像，可以通过将原始图像输入生成器，生成对应的白天有太阳场景的待处理图像。
在一些实施例中，在场景信息与特定场景相匹配的情况下，将原始图像确定为待处理图像。
比如，场景信息与特定场景相同或极为相近的情况下，说明原始图像即具有特定场景，所以不需要对原始图像进行图像转换，即可将原始图像作为待处理图像。在一个具体例子中，特定场景为深夜无光源场景，原始图像为夜幕降临的场景下采集到的图像，该场景与特定场景相似均为夜晚，那么将原始图像确定为待处理图像。
通过上述第一步至第三步，在获取原始图像之后，通过对原始图像的场景信息进行判断，在场景信息不是特定场景的情况下，对原始图像进行风格转换，从而得到具有特定场景的待处理图像。
步骤S102，在所述待处理图像中，确定图像渲染状态与所述特定场景的场景信息不匹配的待调整区域。
在一些实施例中，所述待调整区域的图像渲染状态与照射到原始图像中的目标对象上的光线相关，所述目标对象包括车辆；在一些可能的实现方式中，目标对象可以是在光线照射下能够产生阴影的对象，可以是待处理图像的画面内容中包括的任意车辆、交通标识、行人或者动物等；比如，待处理图像是夜晚的城市道路图像，该图像中包括：建筑物、路面和停靠在路边的车辆等。那么在光线照射下能够产生阴影的目标对象为：建筑物和车辆。
图像渲染状态为在有光源或者无光源的情况下目标对象的渲染状态，至少包括：有阴影状态和无阴影状态。其中，光源包括人造光源和自然光源，人造光源包括：路灯、建筑物灯或者车灯等光源。与场景信息不相匹配的图像渲染状态为：在该场景信息下，图像中物体产生图像渲染状态与实际场景不符合。比如，在夜晚场景下，若无光源(即无人造光源以及月光)，那么物体将不产生阴影，所以与夜晚无光源场景不匹配的阴影渲染状态为有阴影状态；在白天有太阳光照射的场景下，那么物体将会产生阴影，所以与白天有太阳场景不匹配的阴影渲染状态为无阴影状态。
在一些可能的实现方式中，通过对待处理图像中的阴影区域和非阴影区域进行分割，判断这些阴影区域和非阴影区域的阴影渲染状态是否和场景信息相匹配。比如，场景信息为夜晚无光源场景，与该场景不匹配的阴影渲染状态为有阴影状态；通过对处理图像中的阴影区域和非阴影区域进行分割之后，确定待处理图像中阴影渲染状态为有阴影状态的阴影区域为待调整区域。
步骤S103，确定待调整区域内的目标渲染状态。
在一些实施例中，与场景信息相匹配的目标渲染状态为：在该场景信息中目标对象实际产生阴影的状态。在场景信息为夜晚无光源场景的情况下，确定无阴影状态为与场景信息相匹配的目标渲染状态；在场景信息为白天有太阳场景或夜晚有光源场景的情况下，确定有阴影状态为与场景信息相匹配的目标渲染状态，这样确定出的目标渲染状态能够完美的匹配场景信息，符合实际场景。比如，在夜晚无光源场景下，那么物体将不产生阴影，所以与夜晚无光源场景匹配的目标渲染状态为无阴影状态；在白天有太阳场景下，由于太阳光的照射，那么目标对象将会产生阴影，所以与白天有太阳场景相匹配的目标渲染状态为有阴影状态。
在一个具体例子中，以场景信息为夜晚无光源场景，与所述场景信息相匹配的目标渲染状态为无阴影状态。以待处理图像为道路图像为例，如果该图像中包括停放在路边的车辆和树木，通过对该待处理图像进行阴影区域分割之后，如果车辆周围产生了车辆的阴影，那么该阴影所占据的图像区域即为待调整区域；该图像中区域的阴影渲染状态为无阴影状态的区域，即为具有目标渲染状态的区域。
在另一例子中，以场景信息为夜晚有光源场景的情况下，比如，存在路灯等人造光源，与所述场景信息相匹配的目标渲染状态为有阴影状态。以待处理图像为道路图像为例，如果该图像中包括停放在路边的车辆和树木，通过对该待处理图像进行阴影区域分割之后，如果车辆周围未产生车辆的阴影，那么该车辆周围所占据的图像区域即为待调整区域；该图像中阴影渲染状态为有阴影状态的区域，即为具有目标渲染状态的区域。
在一些实施例中，在特定场景为雨天场景的情况下，与该场景不匹配的待调整区域为不包括雨滴的图像区域。在特定场景为雪天场景的情况下，与该场景不匹配的待调整区域为不包括雪花的图像区域。
在一例子中，以场景信息为雨/雪场景为例，在原始图像为无雨/雪天气下采集的图像的情况下，待调整区域为该原始图像中的无遮挡区域(即露天区域)；那么在该待调整区域中增加雨滴/雪花，得到符合雨/雪场景的目标图像。
或者，以场景信息为无雨/雪场景为例，在原始图像为雨/雪天气下采集的图像的情况下，待调整区域为原始图像中出现雨滴/雪花的区域；那么在该待调整区域中，将雨滴/雪花去除，得到符合场景信息的目标图像。如此，针对雨/雪场景下的图像，按照特定场景的场景信息对原始图像增加雨滴/雪花，或者去掉雨滴/雪花，能够得到更加逼真的目标图像。
在其他实施例中，如果场景信息为夜晚的雨/雪场景，原始图像为白天无雨/雪天气下采集的图像，那么可以首先将该原始图像转换为夜晚场景的图像，并调整转换后的图像中的阴影区域；然后，在得到的图像中，增加雨滴/雪花，得到目标图像。
步骤S104，在所述待处理图像中，将所述待调整区域的图像渲染状态调整为所述目标渲染状态，得到目标图像。
在一些实施例中，在待处理图像中，通过对待调整区域的图像渲染状态的转换，使得待调整区域具有与场景信息匹配的目标渲染状态，从而得到目标图像。在一些可能的实现方式中，可以是在待处理图像中，将该待调整区域的图像渲染状态调整为目标渲染状态，以得到目标图像；还可以是按照目标渲染状态，和待调整区域的画面内容，生成具有目标渲染状态的图像区域，将该图像区域替换待处理图像中的待调整区域，从而得到目标图像。
比如，场景信息为夜晚无光源场景，如果目标渲染状态为有阴影状态，那么待调整区域的阴影渲染状态为无阴影状态，可以是在待处理图像中，将该待调整区域的无阴影状态调整为目标渲染状态，以得到目标图像；还可以是按照无阴影状态，和待调整区域的画面内容，生成具有无阴影状态的图像区域，将该图像区域替换待处理图像中的待调整区域，并针对替换后的图像进行平滑处理，从而生成目标图像。如此，生成的目标图像中阴影渲染状态为无阴影状态，与夜晚无光源场景相匹配，从而使得生成的目标图像更加逼真。
在本公开实施例中，通过在待处理图像中，确定出阴影渲染状态与场景信息不匹配的待调整区域；然后，通过将待调整区域的阴影渲染状态调整为与场景信息相匹配的目标渲染状态，生成阴影渲染状态与场景信息匹配的目标图像，使得生成的目标图像更加生动逼真。
在一些实施例中，上述步骤S102中确定待调整区域的过程，可以通过图像分割网络来实现，该神经网络可以是任意配置为图像分割的网络，实现过程如下：
通过该神经网络以检测框的方式标注待调整区域和参考区域，从而能够准确的在待处理图像中预测出待调整区域和参考区域，即可通过该图像分割网络，采用检测框，对所述待处理图像中的待调整区域和参考区域进行标注，以标注出待调整区域和参考区域。
在一些可能的实现方式中，图像分割网络可以是任意类型的神经网络，比如，全卷积神经网络，空洞卷积网络，或解析网络等。将待处理图像输入已训练的图像分割网络，在图像分割网络中采用矩形框对待处理图像中的待调整区域和参考区域所在的位置进行标注，从而标注出待调整区域和参考区域。
在一些实施例中，图像分割网络的训练过程可以通过以下步骤实现：
第一步，将训练图像输入待训练图像分割网络中，预测所述待训练图像中的阴影渲染状态与场景信息不匹配的待调整区域的位置信息。
在一些可能的实现方式中，待训练图像分割网络用于检测待训练图像中的待调整区域，通过大量的训练图像对待训练图像分割网络进行训练，即将大量的训练图像输入待训练图像分割网络，以初步预测待训练图像中的待调整区域的位置。
第二步，根据训练图像中的待调整区域的真实位置信息，确定该位置信息的预测损失。
在一些可能的实现方式中，利用训练图像中标注的真实待调整区域的位置信息和预测待调整区域的位置信息的差值，确定预测损失。通过多个待调整区域的真实位置信息，确定待训练图像分割网络预测的多个位置信息的准确度， 从而确定预测损失。
第三步，根据预测损失，对待训练图像分割网络的网络参数进行调整，得到该图像分割网络。
在一些可能的实现方式中，通过结合待调整区域的真实位置信息确定预测的每一个位置信息的准确度，将这一准确度反馈给图像分割网络，以使图像分割网络调整如权值参数等网络参数，从而提升神经网络检测的准确度。比如，得到100个待调整区域的位置信息，首先使用神经网络进行卷积反卷积等操作，得到这100个待调整区域的位置信息的置信度；由于在训练阶段，图像分割网络的参数是随机初始化的，这样就导致这100个待调整区域的位置信息的粗略估计的置信度也是随机的，那么如果想要提升图像分割网络预测的位置信息的正确度，就需要告诉神经网络100个位置信息哪些是对的哪些是错的。基于此，采用对比函数，将100个位置信息与真实位置信息进行比较，如果位置信息与真实位置信息的相似度大于预设相似度阈值输出1，否则输出0，这样对比函数将输出200个比对值(0，1)值；接下来，将这200个比对值输入待训练图像分割网络，以使待训练图像分割网络中采用损失函数对位置信息进行监督，从而对于对比值为1的位置信息增大该位置信息的置信度，对于对比值为0的位置信息减小该位置信息的置信度；如此，得到每一位置信息的置信度，即得到位置信息的检测结果。最终，采用位置信息对应的预测损失，对图像分割网络的权值参数进行调整，从而得到已训练的图像分割网络。比如，所述权值参数为神经网络中神经元权重等。预测损失为正样本和负样本的交叉熵损失。采用该预测损失对神经网络的权重等参数进行调整，从而使得调整后的图像分割网络预测结果更加准确。
上述过程为对图像分割网络进行训练的过程，基于对输入的待处理图像进行预测，得到的预测的待调整区域的位置和真实待调整区域的位置，进行多次迭代，以使训练后的图像分割网络输出的位置信息的预测损失满足收敛条件，从而使得该神经网络检测的待调整区域的准确度更高。
在一些实施例中，在图像渲染状态包括阴影渲染状态情况下，可以通过以下两种方式确定待调整区域需要切换的目标渲染状态，即步骤S103可以通过以下两种方式实现：
方式一：步骤S111，在所述待处理图像中，确定阴影渲染状态与所述场景信息相匹配的参考区域。
在一些实施例中，阴影渲染状态包括：有阴影状态和无阴影状态，待调整区域的阴影渲染状态与场景信息不匹配，那么与待调整区域的阴影渲染状态不同的参考区域的阴影渲染状态是与场景信息相匹配的。比如，场景信息为夜晚无光源场景，那么待调整区域为在夜晚无光源场景的情况下，图像中目标对象产生的阴影区域；基于此，参考区域为图像中无阴影的图像区域。
在一些可能的实现方式中，在场景信息为夜晚无光源场景的情况下，通过在待处理图像中，将阴影渲染状态为无阴影状态的无阴影区域，作为参考区域。这样，在夜晚无光源场景下，由于光线昏暗，合理的夜晚场景图像中目标对象是不产生阴影的；所以，将无阴影区域作为参考区域，能够使得待调整区域按照该参考区域进行调整之后，更加符合自然规律。
步骤S112，将参考区域的阴影渲染状态，确定为目标渲染状态。
在一些实施例中，在待处理图像中，确定出参考区域之后，将该参考区域的阴影渲染状态作为待调整区域需要转换到的目标渲染状态。
在一些可能的实现方式中，在场景信息为夜晚无光源场景或白天无阳光场景的情况下，在待处理图像中，将阴影渲染状态为无阴影状态的图像区域，确定为所述参考区域。这样，无论是在夜晚无光源场景或白天无阳光场景的情况下，将与场景匹配的无阴影状态的图像区域作为参考区域，以便于基于参考区域对待调整区域进行直方图匹配，从而使得处理后的待调整区域呈现的画面色度与参考区域的画面色度相同。
在场景信息为夜晚有光源场景或白天有太阳场景的情况下，在待处理图像中，将阴影渲染状态为有阴影的图像区域，确定为参考区域。这样，无论是在夜晚有光源场景或白天有太阳场景的情况下，将与场景匹配的有阴影的图像区域作为参考区域，以便基于参考区域对待调整区域进行直方图匹配，从而使得处理后的待调整区域呈现的画面色度与参考区域的画面色度相同。如此，将阴影渲染状态与所述场景信息相匹配的图像区域作为参考区域，以便将待调整区域与该参考区域进行直方图匹配，使得待调整区域与参考区域的直方图一致，从而使得目标图像的画面效果更加自然。
在方式一中，通过在待处理图像中确定出阴影渲染状态匹配场景信息的图像区域，来确定目标渲染状态，使得阴影调整后的待调整区域的色调与其他区域的色调一致。
方式二：通过分析场景信息的光照参数，确定与该光照参数匹配的阴影渲染状态，从而得到目标渲染状态。
在一些实施例中，光照参数包括：光照强度和光线照射角度等。如果光照强度较强(比如，光照强度大于一定阈 值)，那么确定目标渲染状态为有阴影状态；如果光照强度较弱(比如，光照强度小于一定阈值)，那么确定目标渲染状态为无阴影状态。如此，在方式二中，通过分析待处理图像的场景信息来设定目标渲染状态，能够使得设定的目标渲染状态更加符合场景的需求。
在一些实施例中，在场景信息为夜晚无光源场景或白天无阳光场景的情况下，将产生阴影的区域作为待调整区域，即步骤S102可以通过以下过程实现：
在所述场景信息为夜晚无光源场景或白天无阳光场景的情况下，将所述待处理图像中的阴影区域，确定为所述待调整区域。
在一些实施例中，如果在夜晚无光源场景或白天无阳光场景下，那么目标对象将不会产生阴影；所以在待处理图像中，如果目标对象存在阴影，说明该阴影区域是不合理的；基于此，在该待处理图像的目标对象中，将产生了阴影的对象作为目标对象，并挑选出来，以便于后续能够在该待处理图像中确定出不合理的图像区域。在一个具体例子中，如果待处理图像是在夜晚无光源场景下的道路图像，光线照射下能够产生阴影的目标对象包括车辆。其中，车辆产生的阴影区域的阴影渲染状态为有阴影状态，该车辆产生的阴影区域作为待调整区域。
在一些实施例中，图2为本公开实施例提供的图像生成方法的另一实现流程示意图，结合图1和2进行以下说明：
步骤S201，在所述场景信息为夜晚有光源场景或白天有阳光场景的情况下，根据光线照射所述目标对象的方向，确定所述目标对象能够产生的阴影区域。
在一些实施例中，在白天有阳光场景或者夜晚的光源为月光的情景下，通过判断光线照射目标对象的方向，来确定目标对象能够产生的阴影区域；如果是夜晚有光源场景中的光源为人造光源，那么不仅要判断光线照射目标对象的方向，还要进一步判断目标对象与人造光源的距离，来确定目标对象能够产生的阴影区域，实现过程如步骤S221和222所示。
步骤S202，将所述待处理图像中的阴影区域中除第一阴影区域之外的阴影区域确定为待调整区域。
在一些实施例中，第一阴影区域为所述待处理图像中的阴影区域与所述目标对象能够产生的阴影区域的交集。
上述步骤S201和步骤S202提供了一种在场景信息为夜晚有光源场景或白天有阳光场景的情况下，在待处理图像中确定待调整区域的方式，在该方式中，通过将待处理图像中除去目标对象能够产生的阴影区域之外的阴影区域，作为待调整区域，从而能够更加准确的确定出阴影渲染状态需要调整的区域。
在其他实施例中，在场景信息为夜晚有光源场景的情况下，以光源为人造光源为例进行说明，将光源照射范围外有阴影的区域作为待调整区域，那么步骤S102可以通过以下过程实现：
在所述待处理图像中，将与光源的距离大于预设距离阈值、且产生的阴影区域的目标对象所产生的阴影区域，确定为待调整区域。
在一些实施例中，能够产生阴影区域的目标对象所产生的阴影区域可以是，远离所述光源的一侧的图像区域。人造光源与目标对象之间的距离大于预设距离阈值，说明目标对象不在该人造光源的照射范围内，即在该人造光源处于工作状态下，目标对象不能够产生阴影，进一步说明由该目标对象所产生的阴影区域是不合理的。在夜晚有人造光源的情况下，确定人造光源与每一目标对象之间的距离。首先，确定该人造光源的照射范围；然后，通过判断人造光源与每一目标对象之间的距离，来确定目标对象是否在人造光源的照射范围内。在一个具体例子中，如果待处理图像是在夜晚有路灯场景下的城市道路图像，该场景中存在处于已开启状态的路灯，目标对象为：建筑物和树。分别确定路灯与建筑物和树之间的距离，从而判断建筑和树是否在路灯的照射范围内，即在该路灯的照射下，建筑物和树是否能够产生阴影。当目标对象不在路灯的照射范围内时，由目标对象产生的阴影区域为待调整区域。
上述提供了一种实现在场景信息为夜晚有人造光源场景的情况下，在待处理图像中确定待调整区域的方式，在该方式中，通过在场景信息为夜晚有人造光源场景的情况下，判断图像中的光线照射范围以外的目标对象是否产生了阴影区域，将该不合理的阴影区域作为待调整区域。
在通过步骤S201和步骤S202，确定出阴影渲染状态为有阴影状态的图像区域为待调整区域之后，步骤S103和步骤S104可以分别通过以下步骤S203和204实现，对该待调整区域的处理，从而得到目标图像。
步骤S203，确定无阴影状态为与场景信息相匹配的目标渲染状态。
在一些实施例中，在场景信息为夜晚无光源场景或白天无阳光场景的情况下，那么目标对象将不会产生阴影；所以在待处理图像中，如果不存在阴影，说明图像是合理的，即是与场景信息相匹配的；基于此，在该待处理图像中，可以将非阴影区域作为参考区域，即与场景信息相匹配的目标渲染状态为无阴影状态。
步骤S204，在所述待处理图像中，根据参考区域的图像，将所述待调整区域的图像调整为无阴影状态，得到所述目标图像。
在一些实施例中，在待处理图像中，通过对待调整区域的有阴影状态进行调整，去除该区域的阴影，从而得到目标图像。在一些可能的实现方式中，可以通过将无阴影状态的参考区域的直方图，与待调整区域的直方图进行匹配，以去除待调整区域的阴影，使得目标图像中的无阴影的图像区域的灰度一致；还可以采用其他方式去除待调整区域中的阴影；比如，对待调整区域的灰度图像先执行最大滤波，再执行最小滤波。如果待调整区域的图像背景较暗且目标对象较亮，可以先对待调整区域执行最小滤波，再进行最大滤波，以实现对待调整区域进行阴影去除；还可以是采用从预设图像库中，选择和待调整区域的画面内容匹配的无阴影图像直接替换该待调整区域；还可以是通过阴影图像分解去除待调整区域的阴影等。
在本公开实施例中，在待处理图像中，通过确定出待调整区域在待处理图像中占据的区域，按照参考区域的图像将待调整区域的图像从有阴影状态调整为无阴影状态，得到目标图像。或者是，在待处理图像中，通过对待调整区域进行去阴影处理，不需要采用其他区域对该待调整区域进行替换，即可得到目标图像。如此，生成的目标图像中阴影渲染状态为无阴影状态，与夜晚无光源场景以及白天无阳光场景更加匹配。
在一些可能的实现方式中，通过按照参考区域的灰度调整待调整区域的灰度，生成目标图像，即步骤S204可以通过以下步骤实现：
步骤S241，根据所述参考区域的图像调整所述待调整区域的灰度图像，得到已调整灰度图。
在一些实施例中，首先，确定参考区域的灰度图像。参考区域可以是从待处理图像中确定出的，即将待处理图像中具有与场景相匹配的阴影渲染状态的图像区域作为参考区域；还可以是设定目标渲染状态之后，基于该目标渲染状态以及待调整区域的图像画面内容生成的图像区域。
在一些可能的实现方式中，通过确定参考区域的直方图，以得到参考区域的灰度图像。比如，在确定出参考区域之后，根据参考区域的图像生成该参考区域的直方图，基于该直方图可以生成该参考区域的灰度图像。在一个具体例子中，如果场景信息为夜晚无光源场景，那么目标渲染状态为无阴影状态。在待处理图像中，在无人造光源的情况下，图像区域中具有阴影的区域为待调整区域，图像区域中无阴影的区域即为参考区域；并通过确定该无阴影区域的直方图，生成该区域的灰度图像。
然后，基于参考区域的灰度图像，调整待调整区域的灰度图像，得到已调整灰度图。待调整区域的灰度图像可以通过确定待调整区域的直方图来得到。比如，在确定出待调整区域之后，根据待调整区域的图像生成该待调整区域的直方图，基于该直方图可以生成该待调整区域的灰度图像。将参考区域的灰度图像作为参考，来调整待调整区域的灰度图像，以使得到的已调整灰度图与参考区域的灰度图像相匹配，即使得这两个灰度图像对应的彩色图像的色调一致。
在一些可能的实现方式中，可以通过以下过程实现对待调整区域的灰度图像的调整：
首先，根据参考区域的图像确定参考区域的直方图，以基于参考区域的直方图，调整待调整区域的直方图，得到已调整直方图；比如，将待调整区域的直方图，与参考区域的直方图，进行直方图匹配，得到已调整直方图。通过将参考区域的直方图作为目标直方图，将待调整区域的直方图向目标直方图进行转换，以使参考区域的直方图与待调整区域的直方图相匹配，即将待调整区域的直方图匹配到参考区域的直方图，使两区域的色调保持一致；这样，待调整区域的阴影渲染状态将会与参考区域的阴影渲染状态保持一致，均能够匹配场景信息，即是该场景下合理的阴影状态。
然后，基于已调整直方图，确定已调整灰度图。比如，基于该已调整直方图，可以生成该已调整直方图的灰度图像，即得到已调整灰度图。在一些可能的实现方式中，在夜晚无光源场景的情况下，那么参考区域即为待处理图像中的无阴影区域。通过确定无阴影区域的直方图，并将待调整区域的直方图转化为无阴影区域的直方图，即可去除待调整区域的阴影，得到已调整直方图。
步骤S242，基于已调整灰度图，生成阴影渲染状态为无阴影状态的替换区域的图像。
在一些实施例中，通过已调整直方图生成该已调整直方图的灰度图像之后，结合待调整区域的画面内容，对该灰度图像进行渲染生成无阴影状态的替换区域的图像。
步骤S243，在待处理图像中，采用生成的替换区域的图像替换待调整区域的图像，生成目标图像。
在一些实施例中，在待处理图像中，通过确定出阴影渲染状态与场景信息不匹配的替换区域的图像，并确定出待调整区域在待处理图像中占据的区域；采用替换区域的图像替换该待调整区域的图像，并针对替换后的图像进行平滑处理，从而生成目标图像。
在一个具体例子中，以场景信息为夜晚无光源场景为例，待处理图像为道路图像，阴影渲染状态与场景信息不匹配的待调整区域为阴影区域；替换区域的图像为阴影渲染状态为无阴影状态的图像，采用这样无阴影的图像区域替换该待调整区域的图像，以得到阴影渲染状态与场景信息匹配的目标图像。如此，生成的目标图像中阴影渲染状态与场景信息匹配，使得生成的目标图像更加逼真。
在一些可能的实现方式中，通过采用替换区域替换待调整区域后，对替换后的图像进行进一步的平滑处理，生成目标图像，即步骤S243可以通过以下步骤实现：
第一步，将所述替换区域的图像的形状和尺寸调整至与所述待调整区域的图像的一致，得到已调整的替换区域的图像。
在一些可能的实现方式中，首先，确定替换区域的图像的尺寸信息。当确定出替换区域之后，需要确定出替换区域的尺寸信息，至少包括：面积、周长、长度、宽度以及边缘形状等。然后，确定待调整区域的图像在待处理图像中所占据的面积。在一些可能的实现方式中，通过图像分割网络将待调整区域用检测框标注出，可以将检测框的面积作为待调整区域在待处理图像中所占据的区域的面积。最后，根据该面积，对替换区域的图像尺寸信息进行调整，得到已调整的替换区域的图像。在一些可能的实现方式中，按照待调整区域的图像在待处理图像中所占据的区域的面积，对替换区域的图像尺寸信息进行调整，从而使得已调整替换区域的图像尺寸信息与该待调整区域的图像大小相契合。
第二步，采用已调整的替换区域的图像替换待处理图像中的待调整区域的图像，并对已调整的替换区域的图像的边缘进行平滑处理，生成目标图像。
在一些可能的实现方式中，首先，采用已调整替换区域的图像替换所述待调整区域的图像，生成候选图像。即，在待处理图像中，采用已调整替换区域替换待调整区域，从而得到替换后的图像，即候选图像。在一个具体例子中，如果场景信息为夜晚无光源场景，待处理图像为道路图像，待调整区域为阴影渲染状态与夜晚无光源场景不匹配的阴影区域。比如，在无光源的情况下，图像中存在大树的影子，说明大树的影子所占据的图像区域的呈现方式并不合理；那么已调整替换区域为包括无影子的大树，通过对替换区域的图像进行尺寸调整，即可得到已调整的替换区域的图像。从而采用已调整的替换区域的图像替换待调整区域的图像，生成目标图像。然后，对候选图像进行平滑处理，生成目标图像。在一些可能的实现方式中，可以是通过对候选对象中发生替换操作的区域进行平滑处理，以消除图像在该区域的噪声，还可以是对整个候选图像进行平滑处理，对整个图像进行降噪，从而得到目标图像。
在本公开实施例中，通过对替换区域的图像进行尺寸伸缩，采用尺寸调整后的已调整的替换区域的图像替换待调整区域的图像，并对替换后的图像进行平滑处理，从而使得生成的目标图像更加的合理且清晰。
在一些实施例中，在场景信息为夜晚有光源场景的情况下，以光源为人造光源为例进行说明，将光源照射范围内目标对象能够产生的阴影区域作为待调整区域，可以通过以下步骤实现：
步骤S221，在所述场景信息为夜晚有光源场景或白天有阳光场景的情况下，根据光线照射所述目标对象的方向，确定所述目标对象能够产生的阴影区域。
在一些实施例中，通过分析光线照射目标对象的方向，在待处理图像中，确定出目标对象能够产生的阴影区域。在夜晚有光源场景或白天有阳光场景下，符合实际场景的阴影渲染状态为目标对象有阴影。
步骤S222，将所述目标对象能够产生的阴影区域确定为待调整区域。
在一些实施例中，首先可以在待处理图像包括的目标对象中，确定出距离人造光源较近的目标对象；然后，在这些距离较近的目标对象中，确定未产生阴影区域的目标对象。由于目标对象与人造光源的距离较近，说明目标对象在人造光源的照射范围内，那么在夜晚有人造光源的场景下，人造光源照射范围内的目标对象，是能够产生阴影的；由于该目标对象未产生阴影区域，因此不合理。因此需要将该未产生阴影区域的目标对象所能够产生的阴影区域为待调整区域。
在一些实施例中，未产生阴影区域的目标对象能够产生的阴影区域，可以是基于该目标对象所占据的图像区域，以及人造光源的照射方向来确定的；即在该目标对象所占据的图像区域的基础上，通过分析人造光源的照射方向，能够预估该目标对象在该人造光源的照射下所产生的阴影面积和阴影位置；将该阴影面积和阴影位置对应的图像区域确定为该目标对象所能够产生的阴影区域。
在夜晚场景中存在人造光源的情况下，将光源照射范围内目标对象能够产生的阴影区域作为需要进行调整的待调整区域，或者是在夜晚场景存在月光或白天有阳光场景下，将目标对象能够产生的阴影区域确定为待调整区域。
在一种可能的实现方式中，由于光源和目标对象的相对位置可以发生变化。比如，原始图像是有阳光的早晨采集 的图像，而当前时刻为下午，显然太阳光的照射方向与目标对象的相对位置发生变化，这种情况下，需要按照当前时刻光源的照射方向，重新确定目标对象能够产生的阴影区域，从而确定出待调整区域。然后，通过确定当前时刻光源的光照参数，并按照光源照射到目标对象表面的光照强度，渲染该待调整区域，从而得到目标图像。如此，能够使得最后调整后的目标图像更加真实自然。
在一些实施例中，待处理图像中的待调整区域可以是多个，比如，在场景信息为夜晚有人造光源的场景下，在该待处理图像中既存在离人造光源较近的目标对象，也存在也人造光源较远的人造对象；那么在待处理图像中，将该人造光源的照射范围内目标对象能够产生的阴影的区域，和光源照射范围外有阴影的区域均作为需要进行调整的待调整区域。
在一些实施例中，在场景信息为夜晚有光源场景的情况下，以光源为月光为例进行说明，可以通过以下方式确定待调整区域：
步骤S231，在待处理图像中，在所述场景信息为夜晚有月光的场景的情况下，确定能够被月光照射到的目标对象。
步骤S232，在能够被月光照射到的目标对象中，确定未能产生阴影区域的目标对象所能够产生的阴影区域为待调整区域。
在一些实施例中，在场景信息为白天有太阳场景的情况下，可以通过以下方式确定待调整区域：
步骤S241，在待处理图像中，在场景信息为白天有阳光场景的情况下，确定能够被阳光照射到的目标对象。
步骤S242，在能够被阳光照射到的目标对象中，确定未能产生阴影的目标对象所能够产生的阴影区域为所述待调整区域。
在一些实施例中，在确定出待调整区域之后，可以通过以下过程实现对待调整区域的调整，以得到目标图像：
第一步，确定特定场景中的光照参数。
在一些实施例中，如果是针对夜晚有人造光源的场景下，那么确定人造光源处于工作状态即处于已开启状态，光照参数包括：光照强度、光线照射方向和光照亮度等。如果是针对夜晚有月光的场景或者白天有阳光的场景，那么确定当前时刻月光或者阳光照射到地面的光照参数。
第二步，基于所述光照参数确定能够产生阴影区域的目标对象表面的光照强度，渲染所述待调整区域，得到所述目标图像。
在一个具体例子中，在夜晚有人造光源的场景下，待处理图像中包括：车辆和路灯；其中，人造光源为路灯，通过分析该路灯的光照强度和光线照射方向，确定车辆在待处理图像中能够产生的阴影区域，根据车辆表面的光照强度渲染车辆在待处理图像中能够产生的阴影区域从而得到替换区域图像。采用得到的替换区域的图像替换待调整区域的图像，生成目标图像。
在本公开实施例中，在夜晚有人造光源的场景且目标对象在该人造光源的照射范围内，或夜晚有月亮的场景，或白天有阳光的场景下，在待处理图像中，按照光照参数，对目标对象能够产生的阴影区域进行渲染；从而能够使得目标图像中距离人造光源较近的目标对象产生阴影区域，以及，在有自然光的情况下目标对象产生阴影区域，进而提高了生成的目标图像的逼真度。
下面，将说明本公开实施例在一个实际的应用场景中的示例性应用，以白天有阳光场景下采集的图像转换为夜晚无光源场景、与所述场景信息相匹配的目标渲染状态为无阴影状态、待调整区域为阴影区域为例，进行说明。
本公开实施例提供一种图像生成和直方图匹配方法的夜晚无光源场景阴影去除方法，通过对获得的白天场景图像进行阴影区域分割、直方图匹配和图像生成等过程，能够解决白天有阳光场景的图像直接转换到夜晚无光源场景出现不合理的阴影区域的问题。本公开实施例能够有效去除生成的图像中的阴影，使得生成的夜晚场景图像更加真实。
图3A为本公开实施例提供的图像生成***的组成结构示意图，结合图3A进行以下说明：
本公开实施例提供的图像生成***包括：生成器301和判别器302。其中，首先，将白天有阳光场景的原始图像作为输入，从输入端303输入到生成器301；
其次，通过生成器301生成夜晚场景图像(对应于上述实施例中的待处理图像，如图3B中的图像322)，并将生成的夜晚场景图像通过输出端304输出到判别器302；
在一些实施例中，图3B中的图像321为原始图像，图像322为将该图像321的白天有阳光场景转换为夜晚无光源场景的夜晚图像；通过对图像321进行阴影区域和非阴影区域的分割，并对阴影区域和非阴影区域进行直方图匹配，得到去阴影图像323；最后，采用GAN网络将去阴影图像323的白天有太阳场景转换为夜晚无光源场景，得到夜晚图像324。 从图3B可以看出，夜晚图像324相较于图像322没有阴影存在，更加符合夜晚无光源场景，图像画面更加逼真。
在一些可能的实现方式中，将在夜晚无光源场景下采集到的夜晚场景图像和生成的夜晚场景图像均输入到判别器302中。
然后通过判别器302来区分夜晚场景的图像是采集到的夜晚场景图像还是生成的夜晚场景图像，即分别得到真实图像305和转换图像306；
最后，通过不断优化生成器和判别器的损失函数，使得生成器生成的夜晚场景更加真实。
在一些实施例中，获得白天有阳光场景的白天图像之后，在进行白天图像转夜晚图像的之前，首先对该白天图像进行去阴影操作，然后，再对去阴影后的图像进行场景转换，则得到更加真实的夜晚图像，如图4所示，图4为本公开实施例提供的图像生成方法的实现框架结构图，结合图4进行以下说明：
白天图像获取模块401，配置为获取白天有阳光场景的图像，得到白天图像。
阴影分割网络模块402，配置为将白天图像划分为阴影区域431和非阴影区域432。
在一些实施例中，通过图像分割网络从白天图像中获得阴影区域，从而得到阴影区域和非阴影区域。在本公开实施例中，通过优化和模型训练用于图像分割的网络(此处不限定特定的网络结构；比如，全卷积网络、语义分割网络、视觉几何组(Visual Geometry Group，VGG)网络、细化网络(RefineNet)等)，获得一个能够准确识别出图像阴影区域的图像阴影提取网络。利用该图像阴影提取网络对白天图像的阴影区域和非阴影区域进行分割。因此，能够同时获得白天图像中的阴影区域和非阴影区域。
直方图匹配模块403，配置为通过直方图匹配方法，将阴影区域431的直方图与非阴影区域432进行匹配。
在一些实施例中，通过直方图匹配方法，使得阴影区域获得与非阴影区域一致的直方图分布，以使阴影区域与非阴影区域的色调一致。由于阴影区域和非阴影区域在直方图分布上存在较大差别，本公开实施例对阴影区域和非阴影区域通过直方图匹配的方法，使得阴影区域的直方图向目标直方图(非阴影区域)转化，这样，能够将阴影区域的图像转换为无阴影区域的图像。
白天无阴影图像获得模块404，配置为通过直方图匹配的方式，将白天图像中的阴影去除，得到白天无阴影图像。
在一些实施例中，将转化后的无阴影区域贴回到原始图像中，即将转化后的无阴影区域贴回到白天图像中，获得白天有阳光场景的无阴影的图像。
图像生成网络模块405，配置为将白天无阴影图像从白天有阳光场景转换为夜晚无光源场景，得到夜晚图像。
在一些实施例中，采用图3A所示的图像生成网络，将白天无阴影图像的白天有太阳场景转换为夜晚无光源场景，得到夜晚图像。
最终结果输出模块406，配置为使用平滑技术对替换阴影区域的周围图像进行平滑处理，从而得到最终的夜晚图像。
在一些实施例中，利用去除阴影的区域替换原始图像中的阴影区域之后的图像如图5所示，图5为本公开实施例图像生成方法的另一应用场景示意图，其中，原始图像501为白天有阳光场景下采集的图像，目标图像的生成过程如下：
首先，通过对原始图像501进行阴影检测，利用矩形框标出阴影区域502、503和504。
然后，对矩形框标出的区域(即阴影区域502、503和504，对应于上述实施例中的待调整区域)进行扣取并放大，依次得到放大的阴影区域511、512和513。并转换为夜晚场景图像，依次得到夜晚有阴影图像521、522和523。
最后，采用直方图匹配的方式，将夜晚有阴影图像521、522和523的直方图向无阴影区域的直方图进行转换，得到去除阴影图像531、532和533(对应于上述实施例中的目标图像)。
在本公开实施例中，采用图像分割网络获得阴影区域和非阴影区域然后，通过阴影区域和非阴影区域的直方图匹配，获得去除阴影后的图像；最后，通过图像生成的方法将白天有太阳场景的图像转化到夜晚无光源场景，生成不包含阴影区域的夜晚场景图像。如此，有效去除夜晚生成图像的阴影，使得生成的夜晚场景图像更加真实，更加符合夜晚场景的真实性。
本公开实施例提供一种图像生成装置，图6为本公开实施例图像生成装置结构组成示意图，如图6所示，所述装置600包括：
第一转换模块601，配置为将原始图像转换为特定场景下的待处理图像；
第一确定模块602，配置为在所述待处理图像中，确定图像渲染状态与所述特定场景的场景信息不匹配的待调整区域；所述待调整区域的图像渲染状态与照射到原始图像中的目标对象上的光线相关，所述目标对象包括车辆；
第二确定模块603，配置为确定所述待调整区域内的目标渲染状态；所述目标渲染状态与所述场景信息相匹配；
第一调整模块604，配置为在所述待处理图像中，将所述待调整区域的图像渲染状态调整为所述目标渲染状态，得到目标图像。
在上述装置中，所述图像渲染状态包括阴影渲染状态，所述第二确定模块603，包括：
第一确定子模块，配置为在所述待处理图像中，确定阴影渲染状态与所述场景信息相匹配的参考区域；
第二确定子模块，配置为将所述参考区域的阴影渲染状态，确定为所述目标渲染状态。
在上述装置中，所述第一确定模块602，还配置为：
在所述场景信息为夜晚无光源场景或白天无阳光场景的情况下，将所述待处理图像中的阴影区域，确定为所述待调整区域。
在上述装置中，所述第一确定模块602，包括：
第三确定子模块，配置为在所述场景信息为夜晚有光源场景或白天有阳光场景的情况下，根据光线照射所述目标对象的方向，确定所述目标对象能够产生的阴影区域；
第四确定子模块，配置为将所述待处理图像中的阴影区域中除第一阴影区域之外的阴影区域确定为待调整区域，所述第一阴影区域为所述待处理图像中的阴影区域与所述目标对象能够产生的阴影区域的交集。
在上述装置中，所述参考区域的阴影渲染状态为无阴影渲染状态，所述第一调整模块604，包括：
第一调整子模块，配置为在所述待处理图像中，根据所述参考区域的图像，将所述待调整区域的图像调整为无阴影状态，得到所述目标图像。
在上述装置中，所述第一调整子模块，包括：
第一调整单元，配置为根据所述参考区域的图像调整所述待调整区域的灰度图像，得到已调整灰度图；
第一生成单元，配置为基于所述已调整灰度图，生成阴影渲染状态为无阴影状态的替换区域的图像；
第二生成单元，配置为在所述待处理图像中，采用生成的替换区域的图像替换所述待调整区域的图像，生成所述目标图像。
在上述装置中，所述第一调整单元，包括：
第一确定子单元，配置为根据所述参考区域的图像确定所述参考区域的直方图；
第一调整子单元，配置为基于所述参考区域的直方图，调整所述待调整区域的直方图，得到已调整直方图；
第二确定子单元，配置为基于所述已调整直方图，确定所述已调整灰度图。
在上述装置中，所述图像渲染状态包括阴影渲染状态，所述第一确定模块602，包括：
第五确定子模块，配置为在所述场景信息为夜晚有光源场景或白天有阳光场景的情况下，根据光线照射所述目标对象的方向，确定所述目标对象能够产生的阴影区域；
第六确定子模块，配置为将所述目标对象能够产生的阴影区域确定为待调整区域。
在上述装置中，所述第一调整模块604，包括：
第七确定子模块，配置为确定所述特定场景中的光照参数；
第一渲染子模块，配置为基于所述光照参数确定能够产生阴影区域的目标对象表面的光照强度，渲染所述待调整区域，得到所述目标图像。
在上述装置中，所述原始图像为无雨/雪天气下采集的图像，所述特定场景为雨/雪场景，所述第一调整模块604，包括：
第一添加子模块，配置为在所述原始图像中增加雨滴/雪花，得到目标图像。
需要说明的是，以上装置实施例的描述，与上述方法实施例的描述是类似的，具有同方法实施例相似的有益效果。对于本公开装置实施例中未披露的技术细节，请参照本公开方法实施例的描述而理解。
需要说明的是，本公开实施例中，如果以软件功能模块的形式实现上述的图像生成方法，并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。基于这样的理解，本公开实施例的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是终端、服务器等)执行本公开各个实施例所述方法的全部或部分。而前述的存储介质包括：U盘、运动硬盘、只读存储器(Read Only Memory，ROM)、磁碟或者光盘等各种可以存储程序代码的介质。这样，本公开实施例不限制于任何特定的硬件和软件结合。
对应地，本公开实施例再提供一种计算机程序产品，所述计算机程序产品包括计算机可执行指令，该计算机可执 行指令被执行后，能够实现本公开实施例提供的图像生成方法中的步骤。
相应的，本公开实施例再提供一种计算机存储介质，所述计算机存储介质上存储有计算机可执行指令，所述该计算机可执行指令被处理器执行时实现上述实施例提供的图像生成方法的步骤。
相应的，本公开实施例提供一种计算机设备，图7为本公开实施例计算机设备的组成结构示意图，如图7所示，所述计算机设备700包括：一个处理器701、至少一个通信总线、通信接口702、至少一个外部通信接口和存储器703。其中，通信接口702配置为实现这些组件之间的连接通信。其中，通信接口702可以包括显示屏，外部通信接口可以包括标准的有线接口和无线接口。其中所述处理器701，配置为执行存储器中图像处理程序，以实现上述实施例提供的图像生成方法的步骤。
以上图像生成装置、计算机设备和存储介质实施例的描述，与上述方法实施例的描述是类似的，具有同相应方法实施例相似的技术描述和有益效果，限于篇幅，可案件上述方法实施例的记载，故在此不再赘述。对于本公开图像生成装置、计算机设备和存储介质实施例中未披露的技术细节，请参照本公开方法实施例的描述而理解。
应理解，说明书通篇中提到的“一个实施例”或“一实施例”意味着与实施例有关的特定特征、结构或特性包括在本公开的至少一个实施例中。因此，在整个说明书各处出现的“在一个实施例中”或“在一实施例中”未必一定指相同的实施例。此外，这些特定的特征、结构或特性可以任意适合的方式结合在一个或多个实施例中。应理解，在本公开的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本公开实施例的实施过程构成任何限定。上述本公开实施例序号仅仅为了描述，不代表实施例的优劣。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。
在本公开所提供的几个实施例中，应该理解到，所揭露的设备和方法，可以通过其它的方式实现。以上所描述的设备实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，如：多个单元或组件可以结合，或可以集成到另一个***，或一些特征可以忽略，或不执行。另外，所显示或讨论的各组成部分相互之间的耦合、或直接耦合、或通信连接可以是通过一些接口，设备或单元的间接耦合或通信连接，可以是电性的、机械的或其它形式的。
上述作为分离部件说明的单元可以是、或也可以不是物理上分开的，作为单元显示的部件可以是、或也可以不是物理单元；既可以位于一个地方，也可以分布到多个网络单元上；可以根据实际的需要选择其中的部分或全部单元来实现本实施例方案的目的。
另外，在本公开各实施例中的各功能单元可以全部集成在一个处理单元中，也可以是各单元分别单独作为一个单元，也可以两个或两个以上单元集成在一个单元中；上述集成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能单元的形式实现。本领域普通技术人员可以理解：实现上述方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成，前述的程序可以存储于计算机可读取存储介质中，该程序在执行时，执行包括上述方法实施例的步骤；而前述的存储介质包括：移动存储设备、只读存储器(Read Only Memory，ROM)、磁碟或者光盘等各种可以存储程序代码的介质。
或者，本公开上述集成的单元如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。基于这样的理解，本公开实施例的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机、服务器、或者网络设备等)执行本公开各个实施例所述方法的全部或部分。而前述的存储介质包括：移动存储设备、ROM、磁碟或者光盘等各种可以存储程序代码的介质。以上所述，仅为本公开的具体实施方式，但本公开的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本公开揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本公开的保护范围之内。因此，本公开的保护范围应以所述权利要求的保护范围为准。
工业实用性
本公开实施例提供一种图像生成方法、装置、设备及存储介质，其中，将原始图像转换为特定场景下的待处理图 像；在所述待处理图像中，确定图像渲染状态与所述特定场景的场景信息不匹配的待调整区域；所述待调整区域的图像渲染状态与照射到原始图像中的目标对象上的光线相关，所述目标对象包括车辆；确定所述待调整区域内的目标渲染状态；所述目标渲染状态与所述场景信息相匹配；在所述待处理图像中，将所述待调整区域的图像渲染状态调整为所述目标渲染状态，得到目标图像。

Claims (13)

  1. 一种图像生成方法，所述方法由电子设备执行，所述方法包括：
    将原始图像转换为特定场景下的待处理图像；
    在所述待处理图像中，确定图像渲染状态与所述特定场景的场景信息不匹配的待调整区域；所述待调整区域的图像渲染状态与照射到原始图像中的目标对象上的光线相关，所述目标对象包括车辆；
    确定所述待调整区域内的目标渲染状态；所述目标渲染状态与所述场景信息相匹配；
    在所述待处理图像中，将所述待调整区域的图像渲染状态调整为所述目标渲染状态，得到目标图像。
  2. 根据权利要求1所述的方法，其中，所述图像渲染状态包括阴影渲染状态，所述确定所述待调整区域内的目标渲染状态，包括：
    在所述待处理图像中，确定阴影渲染状态与所述场景信息相匹配的参考区域；
    将所述参考区域的阴影渲染状态，确定为所述目标渲染状态。
  3. 根据权利要求2所述的方法，其中，所述在所述待处理图像中，确定图像渲染状态与所述特定场景的场景信息不匹配的待调整区域，包括：
    在所述场景信息为夜晚无光源场景或白天无阳光场景的情况下，将所述待处理图像中的阴影区域，确定为所述待调整区域。
  4. 根据权利要求2或3所述的方法，其中，所述在在所述待处理图像中，确定图像渲染状态与所述特定场景的场景信息不匹配的待调整区域，包括：
    在所述场景信息为夜晚有光源场景或白天有阳光场景的情况下，根据光线照射所述目标对象的方向，确定所述目标对象能够产生的阴影区域；
    将所述待处理图像中的阴影区域中除第一阴影区域之外的阴影区域确定为待调整区域，所述第一阴影区域为所述待处理图像中的阴影区域与所述目标对象能够产生的阴影区域的交集。
  5. 根据权利要求3或4所述的方法，其中，所述参考区域的阴影渲染状态为无阴影渲染状态，所述在所述待处理图像中，将所述待调整区域的图像渲染状态调整为所述目标渲染状态，得到目标图像，包括：
    在所述待处理图像中，根据所述参考区域的图像，将所述待调整区域的图像调整为无阴影状态，得到所述目标图像。
  6. 根据权利要求5所述的方法，其中，所述在所述待处理图像中，根据所述参考区域的图像，将所述待调整区域的图像调整为无阴影状态，得到所述目标图像，包括：
    根据所述参考区域的图像调整所述待调整区域的灰度图像，得到已调整灰度图；
    基于所述已调整灰度图，生成阴影渲染状态为无阴影状态的替换区域的图像；
    在所述待处理图像中，采用生成的替换区域的图像替换所述待调整区域的图像，生成所述目标图像。
  7. 根据权利要求6所述的方法，其中，所述根据所述参考区域的图像调整所述待调整区域的灰度图像，得到已调整灰度图，包括：
    根据所述参考区域的图像确定所述参考区域的直方图；
    基于所述参考区域的直方图，调整所述待调整区域的直方图，得到已调整直方图；
    基于所述已调整直方图，确定所述已调整灰度图。
  8. 根据权利要求1至7任一项所述的方法，其中，所述图像渲染状态包括阴影渲染状态，在所述待处理图像中，确定图像渲染状态与所述场景信息不匹配的待调整区域，包括：
    在所述场景信息为夜晚有光源场景或白天有阳光场景的情况下，根据光线照射所述目标对象的方向，确定所述目标对象能够产生的阴影区域；
    将所述目标对象能够产生的阴影区域确定为待调整区域。
  9. 根据权利要求8所述的方法，其中，所述在所述待处理图像中，将所述待调整区域的图像渲染状态调整为所述目标渲染状态，得到目标图像，包括：
    确定所述特定场景中的光照参数；
    基于所述光照参数确定能够产生阴影区域的目标对象表面的光照强度，渲染所述待调整区域，得到所述目标图像。
  10. 根据权利要求1至9任一项所述的方法，其中，所述原始图像为无雨/雪天气下采集的图像，所述特定场景为雨/雪场景；
    在所述待处理图像中，将所述待调整区域的图像渲染状态调整为所述目标渲染状态，得到目标图像，包括：
    在所述原始图像中增加雨滴/雪花，得到目标图像。
  11. 一种图像生成装置，其中，所述装置包括：
    第一转换模块，配置为将原始图像转换为特定场景下的待处理图像；
    第一确定模块，配置为在所述待处理图像中，确定图像渲染状态与所述特定场景的场景信息不匹配的待调整区域；所述待调整区域的图像渲染状态与照射到原始图像中的目标对象上的光线相关，所述目标对象包括车辆；
    第二确定模块，配置为确定所述待调整区域内的目标渲染状态；所述目标渲染状态与所述场景信息相匹配；
    第一调整模块，配置为在所述待处理图像中，将所述待调整区域的图像渲染状态调整为所述目标渲染状态，得到目标图像。
  12. 一种计算机存储介质，其中，所述计算机存储介质上存储有计算机可执行指令，该计算机可执行指令被执行后，能够实现权利要求1至10任一项所述的方法步骤。
  13. 一种电子设备，其中，所述电子设备包括存储器和处理器，所述存储器上存储有计算机可执行指令，所述处理器运行所述存储器上的计算机可执行指令时可实现权利要求1至10任一项所述的方法步骤。
PCT/CN2022/074059 2021-03-31 2022-01-26 一种图像生成方法、装置、设备及存储介质 WO2022206156A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2023559825A JP2024512102A (ja) 2021-03-31 2022-01-26 画像生成方法、装置、機器及び記憶媒体
US18/474,189 US20240013453A1 (en) 2021-03-31 2023-09-25 Image generation method and apparatus, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110351943.9A CN112991158B (zh) 2021-03-31 2021-03-31 一种图像生成方法、装置、设备及存储介质
CN202110351943.9 2021-03-31

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/474,189 Continuation US20240013453A1 (en) 2021-03-31 2023-09-25 Image generation method and apparatus, and storage medium

Publications (1)

Publication Number Publication Date
WO2022206156A1 true WO2022206156A1 (zh) 2022-10-06

Family

ID=76338790

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/074059 WO2022206156A1 (zh) 2021-03-31 2022-01-26 一种图像生成方法、装置、设备及存储介质

Country Status (4)

Country Link
US (1) US20240013453A1 (zh)
JP (1) JP2024512102A (zh)
CN (1) CN112991158B (zh)
WO (1) WO2022206156A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112991158B (zh) * 2021-03-31 2024-05-31 商汤集团有限公司 一种图像生成方法、装置、设备及存储介质
US12015855B2 (en) 2021-11-16 2024-06-18 Samsung Electronics Co., Ltd. System and method for synthesizing low-light images
CN116245716A (zh) * 2023-05-06 2023-06-09 中国第一汽车股份有限公司 车辆的路况图像的处理方法及装置

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102136132A (zh) * 2010-01-27 2011-07-27 中国科学院自动化研究所 一种暗环境下人眼视觉***感知效果的合成方法
CN106327454A (zh) * 2015-07-03 2017-01-11 株式会社理光 合成图像的方法和装置
CN109887066A (zh) * 2019-02-25 2019-06-14 网易（杭州）网络有限公司 光照效果处理方法及装置、电子设备、存储介质
CN111260769A (zh) * 2020-01-09 2020-06-09 北京中科深智科技有限公司 一种基于动态光照变化的实时渲染方法及装置
US10885701B1 (en) * 2017-12-08 2021-01-05 Amazon Technologies, Inc. Light simulation for augmented reality applications
CN112419328A (zh) * 2019-08-22 2021-02-26 北京市商汤科技开发有限公司 图像处理方法及装置、电子设备和存储介质
CN112991158A (zh) * 2021-03-31 2021-06-18 商汤集团有限公司 一种图像生成方法、装置、设备及存储介质

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101923011B1 (ko) * 2014-10-10 2018-11-28 엠파이어 테크놀로지 디벨롭먼트 엘엘씨 장면 이미지 생성기
CN109377502A (zh) * 2018-10-15 2019-02-22 深圳市中科明望通信软件有限公司 一种图像处理方法、图像处理装置及终端设备
CN111583097A (zh) * 2019-02-18 2020-08-25 北京三星通信技术研究有限公司 图像处理方法、装置、电子设备及计算机可读存储介质
CN110264413B (zh) * 2019-05-17 2021-08-31 北京达佳互联信息技术有限公司 一种图像处理方法、装置、电子设备及存储介质
CN110490960B (zh) * 2019-07-11 2023-04-07 创新先进技术有限公司 一种合成图像生成方法及装置
CN111951157A (zh) * 2020-09-02 2020-11-17 深圳传音控股股份有限公司 图像处理方法、设备及存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102136132A (zh) * 2010-01-27 2011-07-27 中国科学院自动化研究所 一种暗环境下人眼视觉***感知效果的合成方法
CN106327454A (zh) * 2015-07-03 2017-01-11 株式会社理光 合成图像的方法和装置
US10885701B1 (en) * 2017-12-08 2021-01-05 Amazon Technologies, Inc. Light simulation for augmented reality applications
CN109887066A (zh) * 2019-02-25 2019-06-14 网易（杭州）网络有限公司 光照效果处理方法及装置、电子设备、存储介质
CN112419328A (zh) * 2019-08-22 2021-02-26 北京市商汤科技开发有限公司 图像处理方法及装置、电子设备和存储介质
CN111260769A (zh) * 2020-01-09 2020-06-09 北京中科深智科技有限公司 一种基于动态光照变化的实时渲染方法及装置
CN112991158A (zh) * 2021-03-31 2021-06-18 商汤集团有限公司 一种图像生成方法、装置、设备及存储介质

Also Published As

Publication number Publication date
CN112991158A (zh) 2021-06-18
JP2024512102A (ja) 2024-03-18
CN112991158B (zh) 2024-05-31
US20240013453A1 (en) 2024-01-11

Similar Documents

Publication Publication Date Title
WO2022206156A1 (zh) 一种图像生成方法、装置、设备及存储介质
CN110399856B (zh) 特征提取网络训练方法、图像处理方法、装置及其设备
Huang et al. An efficient visibility enhancement algorithm for road scenes captured by intelligent transportation systems
CN107301623B (zh) 一种基于暗通道和图像分割的交通图像去雾方法及***
CN106791471B (zh) 图像优化方法、图像优化装置和终端
US20060215882A1 (en) Image processing apparatus and method, recording medium, and program
Peng et al. Image haze removal using airlight white correction, local light filter, and aerial perspective prior
CN115861380B (zh) 雾天低照度场景下端到端无人机视觉目标跟踪方法及装置
KR20210006276A (ko) 플리커를 약화시키기 위한 이미지 처리 방법
CN114331873B (zh) 一种基于区域划分的非均匀光照彩色图像校正方法
Lou et al. Integrating haze density features for fast nighttime image dehazing
CN113505702A (zh) 基于双神经网络优化的路面病害识别方法及***
CN117274107B (zh) 低照度场景下端到端色彩及细节增强方法、装置及设备
CN112863194B (zh) 一种图像处理方法、装置、终端及介质
US10474906B2 (en) High dynamic range video of fast moving objects without blur
CN102622594A (zh) 一种基于红外和帧差的近距离目标提取方法
CN117611501A (zh) 一种低照度图像增强方法、装置、设备及可读存储介质
Guislain et al. Detecting and correcting shadows in urban point clouds and image collections
WO2023110880A1 (en) Image processing methods and systems for low-light image enhancement using machine learning models
TWM625817U (zh) 具時序平滑性之影像模擬系統
Subhashini et al. Implementation of Satellite Road Image Denoising using Iterative Domain Guided Image Filtering with Gray World Optimization.
Khade et al. Efficient single image haze removal using CLAHE and Dark Channel Prior for Internet of Multimedia Things
JP2002366959A (ja) 画像処理装置および画像処理方法
Rajkumar et al. Vehicle Detection and Tracking System from CCTV Captured Image for Night Vision
US20230186612A1 (en) Image processing methods and systems for generating a training dataset for low-light image enhancement using machine learning models

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22778344

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023559825

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 13.02.2024)