CN115546036A

CN115546036A - Image enhancement method, device, equipment and computer readable storage medium

Info

Publication number: CN115546036A
Application number: CN202110729526.3A
Authority: CN
Inventors: 范世炜
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-06-29
Filing date: 2021-06-29
Publication date: 2022-12-30

Abstract

The application provides an image enhancement method, device, equipment and computer readable storage medium; the method comprises the following steps: acquiring a coding information sequence of an image to be enhanced, wherein the coding information sequence is a sequence formed by coding information of each pixel point; traversing the coding information sequence, acquiring enhanced information and filtering information corresponding to the enhanced information aiming at the coding information of the traversed pixel points, and enhancing the coding information of the traversed pixel points by combining the enhanced information and the filtering information to obtain the enhanced coding information; when the coding information sequence is traversed, obtaining an enhanced coding information sequence corresponding to the coding information sequence; and decoding an enhanced image corresponding to the image to be enhanced based on the enhanced coding information sequence. By the method and the device, the computing efficiency of the image enhancement process can be improved.

Description

Image enhancement method, device, equipment and computer readable storage medium

Technical Field

The present application relates to image processing technologies in the field of computer applications, and in particular, to an image enhancement method, apparatus, device, and computer-readable storage medium.

Background

Dark scene enhancement refers to processing for improving the brightness of an image on the premise of not influencing chromaticity (hue and saturation) aiming at a low-light image; in the field of artificial intelligence, the visual effect under normal and sufficient light can be achieved through dark scene enhancement.

Generally, in order to perform dark scene enhancement processing on an image, a pixel point is generally taken as a unit, the image is traversed for one time, an enhancement matrix of the image is obtained, then the enhancement matrix is traversed to obtain a filtering matrix of the image, and dark scene enhancement is performed by combining the enhancement matrix and the filtering matrix; because the traversal of the full-image pixel points is performed twice in the process of performing the dark scene enhancement on the image, the resource consumption is large, and the efficiency of the dark scene enhancement of the image is low.

Disclosure of Invention

The embodiment of the application provides an image enhancement method, an image enhancement device, image enhancement equipment and a computer-readable storage medium, and the dark scene enhancement efficiency of an image can be improved.

The technical scheme of the embodiment of the application is realized as follows:

the embodiment of the application provides an image enhancement method, which comprises the following steps:

acquiring a coding information sequence of an image to be enhanced, wherein the coding information sequence is a sequence formed by coding information of each pixel point;

traversing the coding information sequence, acquiring enhanced information and filtering information corresponding to the enhanced information aiming at the traversed coding information of the pixel points, and enhancing the coding information of the traversed pixel points by combining the enhanced information and the filtering information to obtain the enhanced coding information;

when traversing the coding information sequence, obtaining an enhanced coding information sequence corresponding to the coding information sequence;

and decoding an enhanced image corresponding to the image to be enhanced based on the enhanced coding information sequence.

An embodiment of the present application provides an image enhancement apparatus, including:

the image coding module is used for acquiring a coding information sequence of an image to be enhanced, wherein the coding information sequence is a sequence formed by coding information of each pixel point;

the image enhancement module is used for traversing the coding information sequence, acquiring enhancement information and filtering information corresponding to the enhancement information aiming at the traversed coding information of the pixel points, and enhancing the traversed coding information of the pixel points by combining the enhancement information and the filtering information to obtain the enhancement coding information;

the image enhancement module is also used for obtaining an enhanced coding information sequence corresponding to the coding information sequence when the coding information sequence is traversed;

and the image decoding module is used for decoding the enhanced image corresponding to the image to be enhanced based on the enhanced coding information sequence.

In this embodiment of the present application, the image enhancement module is further configured to generate a cycle traversal instruction based on a quantity threshold, where the quantity threshold is determined based on a register specification and an occupied space of the encoding information of the pixel point; responding to the periodic traversal instruction, sequentially traversing the coding information of the pixel points with the number threshold and the coding information of the associated pixel points corresponding to the coding information of each traversed pixel point from the coding information sequence; and acquiring the enhanced information and the filtering information corresponding to the enhanced information based on the associated pixel point coding information corresponding to the coding information of each traversed pixel point.

In this embodiment of the application, the image enhancement module is further configured to shift the coding information of each traversed pixel and the coding information of the associated pixel respectively; the shifted coding information of each traversed pixel point and the shifted coding information of the associated pixel point are subjected to equalization processing, and the equalization processing result is determined to be initial enhancement information; mapping the initial enhancement information based on a filtering mapping relation to map initial filtering information; and performing reverse shift on the initial enhancement information to obtain the enhancement information, and performing reverse shift on the initial filtering information to obtain the filtering information.

In this embodiment of the application, the image enhancement module is further configured to perform equalization processing on the coding information of each traversed pixel and the coding information of the associated pixel, and determine an equalization processing result as the enhancement information; and mapping the enhanced information based on a filtering mapping relation, and determining a mapping result as the filtering information.

In this embodiment of the present application, the image enhancement module is further configured to obtain a first weighting result of the enhancement information and the enhancement weight; acquiring a second weighting result of the filtering information and the filtering weight; and determining a combination result of the first weighting result and the second weighting result as the enhanced coding information, wherein the enhanced coding information is a result of enhancing the coding information of the traversed pixel points.

In an embodiment of the present application, the image enhancement module is further configured to store the enhancement weights; obtaining the enhancement information and the stored first weighting result of the enhancement weight.

In this embodiment of the application, the image enhancement module is further configured to obtain a pixel point layout of the image to be enhanced; based on the pixel point layout, determining storage position information corresponding to each group of pixel points in the image to be enhanced; storing storage position information corresponding to each group of pixel points in the image to be enhanced; responding to the periodic traversal instruction, and acquiring row and column information to be traversed; determining a target group and offset position information based on the row and column information to be traversed; determining target position information corresponding to the target group based on the stored storage position information corresponding to each group of pixel points in the image to be enhanced; determining to-be-traversed position information based on the target position information and the bias position information; sequentially traversing the coding information of the pixel points with the number threshold and the coding information of the associated pixel points corresponding to the coding information of each traversed pixel point from the position information to be traversed of the image coding information.

In an embodiment of the present application, the image encoding module is configured to present a first enhancement control corresponding to the image to be enhanced; and responding to a first enhancement operation acted on the first enhancement control, and acquiring the coded information sequence of the image to be enhanced.

In an embodiment of the application, the image enhancement apparatus further includes an image rendering module configured to render the enhanced image.

In this embodiment of the present application, the image enhancement apparatus further includes an image determination module, configured to present a second enhancement control for the played video information; and responding to a second enhancement operation acted on the second enhancement control, and determining a frame image corresponding to the video information as the image to be enhanced.

In an embodiment of the present application, the image presenting module is further configured to replace presentation of the frame image with the enhanced image.

In this embodiment of the present application, the image encoding module is further configured to obtain a color channel image of the image to be enhanced; acquiring a minimum color channel value of each pixel point based on the color channel image, and constructing a gray image corresponding to the image to be enhanced based on the minimum color channel value of each pixel point; and acquiring the minimum gray value of each pixel point in a sliding window based on the gray image, and constructing the coding information sequence corresponding to the image to be enhanced based on the minimum gray value of each pixel point in the sliding window.

In this embodiment of the present application, the image decoding module is further configured to decode an initial enhanced image based on the enhanced coding information sequence; and performing post-processing on the initial enhanced image to obtain the enhanced image corresponding to the image to be enhanced, wherein the post-processing comprises at least one of denoising processing, sharpening processing and correction processing.

a memory for storing executable instructions;

and the processor is used for realizing the image enhancement method provided by the embodiment of the application when executing the executable instructions stored in the memory.

The embodiment of the application provides a computer-readable storage medium, which stores executable instructions for causing a processor to execute the method for enhancing the image provided by the embodiment of the application.

The embodiment of the application has at least the following beneficial effects: in the process of enhancing the coding information of each pixel point, aiming at the coding information of the pixel points traversed from the coding information sequence, the enhancement information is calculated, meanwhile, the filtering information corresponding to the enhancement information is calculated, and the coding information of each pixel point is enhanced based on the enhancement information and the filtering information; therefore, the enhancement of the coding information of each pixel point can be completed through one-time traversal, so that the resource consumption is less, and the dark scene enhancement efficiency of the image can be improved.

Drawings

FIG. 1 is a schematic flow diagram of an exemplary dark scene enhancement;

FIG. 2 is a schematic diagram of an alternative architecture of an image enhancement system provided by an embodiment of the present application;

fig. 3 is a schematic structural diagram of a terminal in fig. 2 according to an embodiment of the present disclosure;

FIG. 4 is a schematic flow chart of an alternative image enhancement method provided by the embodiment of the present application;

FIG. 5 is a schematic flow chart of another alternative image enhancement method provided in the embodiment of the present application;

FIG. 6 is a schematic flow chart of still another alternative image enhancement method provided in the embodiments of the present application;

FIG. 7 is a schematic view of an exemplary scene triggering dark scene enhancement provided by an embodiment of the present application;

FIG. 8 is a flow chart of an exemplary image enhancement provided by an embodiment of the present application;

fig. 9 is a flowchart of a process of S802 in fig. 8 according to an embodiment of the present disclosure;

FIG. 10 is a block diagram of an exemplary single instruction multiple data stream processing system according to an embodiment of the present disclosure;

fig. 11 is a processing manner of a single instruction and single data stream according to an embodiment of the present application.

Detailed Description

In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be described in further detail with reference to the attached drawings, the described embodiments should not be considered as limiting the present application, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

In the following description, references to the terms "first", "second", and the like, are only to distinguish similar objects and do not denote a particular order, but rather the terms "first", "second", and the like may be used interchangeably with the order specified, where permissible, to enable embodiments of the present application described herein to be practiced otherwise than as specifically illustrated or described herein.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the application.

Before further detailed description of the embodiments of the present application, terms and expressions referred to in the embodiments of the present application will be described, and the terms and expressions referred to in the embodiments of the present application will be used for the following explanation.

1) Cloud Technology refers to a hosting Technology for unifying serial resources such as hardware, software and network in a wide area network or a local area network to realize calculation, storage, processing and sharing of data.

2) The low-light image is an image with brightness lower than a brightness threshold value, which is collected in a scene with dark illumination; for example, images obtained by taking pictures or taking pictures at night; in a low-light image, due to the fact that light is dark, visual quality of the image is greatly reduced, detail loss and contrast reduction on the image are caused, on one hand, subjective visual perception of people is affected, and on the other hand, performance of subsequent image processing is also affected.

3) YUV (luminance chrominance), which is a color coding method, in which "Y" represents luminance, i.e., a gray scale value, and "U" and "V" represent chrominance; in YUV, color is represented by both luminance and chrominance. Wherein, brightness, also called lightness, represents the brightness of color; chroma is the property of a color excluding brightness, and indicates the hue and saturation of the color, where hue refers to the relative brightness of an image, and appears as a color on a color image, and saturation refers to the vividness of the color. In addition, a commonly used YUV format is, for example, "NV12".

4) Single Instruction Multiple Data (SIMD) is an extension of the basic Instruction set of a processor, and is mainly used for parallel operation instructions for continuously storing Data; for example, in the image processing related to the embodiment of the present application, data represents a pixel component by less than or equal to 8 bits (bit) in a memory, and when a processor is usually used for calculation, only the lower 8 bits of a 32-bit or 64-bit register can be used, which causes resource waste; and the 64-bit register can be disassembled into 8-bit registers for use by using SIMD, so that 8 data processing operations can be completed simultaneously by one instruction.

5) The enhancement matrix represents a global dark scene value, global variables obtained by integrally evaluating the image by using a dark channel contained in YUV coding are used for controlling the overall brightness, chromaticity and intensity of the image, ensuring that the overall brightness is not abnormal and keeping the color reduction degree; the enhancement information sequence related to the embodiment of the application is an enhancement matrix. The dark channel acquisition process comprises the following steps: calculating the minimum value of three channels corresponding to each pixel point, and taking the minimum channel value as the value of the pixel point to obtain a gray scale map with the same size as the original image; and then smoothing by using minimum filtering, namely taking a rectangular window with a certain size by taking each pixel point as the center in the obtained gray-scale image, and taking the minimum value of the gray-scale values in the rectangular window to replace the value of the central pixel point, thereby obtaining a dark channel image corresponding to the original image.

6) A filter matrix, which represents information for processing dark channel to obtain local variables by using Gaussian filter and the like, and is used for distinguishing dark region and bright region in the image during dark scene enhancement processing; the filter matrix is combined with the enhancement matrix to jointly control the enhancement effect of the dark scene; the filtering information sequence related to the embodiment of the present application is a filtering matrix.

7) A processor increment for measuring the performance of the processor; the CPU increment is taken as an example for explanation, and represents the increment absolute value of the CPU occupancy rate of the host process on the computer after starting a certain function or a certain algorithm; for example, before the dark scene enhancement is started, the CPU occupancy of the video conference is 5%, and after the dark scene enhancement is started, the CPU occupancy of the video conference is 7%, and at this time, the CPU increment is 2%. The CPU increment reflects the performance of a certain function or a certain algorithm on the CPU, and the lower the value is, the better the performance is. In addition, since the number of cores differs from computer device to computer device, this value needs to be divided by the number of cores.

8) The register is a small storage area used for storing data in the processor and is used for temporarily storing the data participating in the operation and the operation result; in the computer field, registers are elements internal to a processor.

9) The control is triggerable information displayed in the forms of buttons, icons, links, texts, selection boxes, input boxes, tabs and the like; the triggering mode can be contact triggering, non-contact triggering, command receiving triggering and the like; in addition, the various controls in the embodiments of the present application may be a single control, or may be a collective name of multiple controls.

10 Operation) is a manner for triggering the device to execute processing, such as click operation, double-click operation, long-press operation, slide operation, gesture operation, received trigger instruction, and the like; in addition, various operations in the embodiments of the present application may be a single operation or may be a collective term for a plurality of operations; and various operations in the embodiments of the present application may be touch operations and may also be non-touch operations.

11 In response to a condition or state that indicates that the executed process is dependent on, one or more operations performed may be in real-time or may have a set delay when the dependent condition or state is satisfied; there is no restriction on the order of execution of the operations performed unless otherwise specified.

It should be noted that cloud computing (cloud computing) is a computing model, which distributes computing tasks over a resource pool formed by a large number of computers, so that various application systems can obtain computing power, storage space and information services as required. The network that provides the resources is called the "cloud"; resources in the "cloud" appear to the user as being infinitely expandable and available at any time, available on demand, expandable at any time, and paid for on-demand. The cloud computing resource pool mainly comprises computing equipment (a virtualization machine comprising an operating system), storage equipment and network equipment, wherein a Platform as a Service (Platform as a Service) layer can be deployed on the IaaS layer according to logic function division, a Software as a Service (SaaS) layer can be deployed on the PaaS layer, and the SaaS layer can also be directly deployed on the IaaS layer.

Generally, in order to perform dark scene enhancement processing on an image, a pixel point is usually taken as a unit, the image is traversed for one time to obtain an enhancement matrix of the image, then the enhancement matrix is traversed to obtain a filter matrix of the image, and the dark scene enhancement is performed by combining the enhancement matrix and the filter matrix; referring to FIG. 1, FIG. 1 is a schematic flow diagram of an exemplary dark scene enhancement; as shown in fig. 1, the exemplary dark scene enhancement process includes:

s101, starting.

S102, traversing the coding matrix of the image and extracting the enhancement matrix.

Here, by traversing the encoding matrix of the image, the global dark scene value is extracted, and the enhancement matrix of the image is also obtained.

S103, traversing the enhancement matrix, obtaining a filter matrix and combining the enhancement matrix to carry out dark scene enhancement.

Here, by traversing the enhancement matrix, local dark scene values are mapped out, and a filter matrix is also obtained.

And S104, ending.

Because the traversal of the full-image pixel points is performed twice in the process of performing dark scene enhancement on the image in the steps S101 to S104, the operation of reading the pixel points is repeated, the jump access of the memory of the computer is frequently performed, more CPU resources are occupied, the resource consumption is higher, and therefore, the dark scene enhancement efficiency of the image is lower.

Based on this, embodiments of the present application provide an image enhancement method, apparatus, device and computer-readable storage medium, which can improve dark scene enhancement efficiency of an image and reduce load of a processor. An exemplary application of the image enhancement device provided by the embodiment of the present application is described below, and the image enhancement device provided by the embodiment of the present application may be implemented as various types of user terminals such as a notebook computer, a tablet computer, a desktop computer, a set-top box, a mobile device, a vehicle-mounted terminal, a smart television (for example, a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, and a portable game device), and may also be implemented as a server. Next, an exemplary application when the image enhancement apparatus is implemented as a terminal will be explained.

Referring to fig. 2, fig. 2 is a schematic diagram of an alternative architecture of an image enhancement system provided by an embodiment of the present application; as shown in fig. 2, in order to support an image enhancement application, in the image enhancement system 100, the terminal 200 (image enhancement device, illustratively, the terminal 200-1 and the terminal 200-2) is connected to the server 400 through the network 300, and the network 300 may be a wide area network or a local area network, or a combination of both. In addition, in the image enhancement system 100, a database 500 is further included for providing data support to the server 400.

The terminal 200 is configured to obtain a coding information sequence of an image to be enhanced after a dark scene enhancement option (see the terminal 200-1) is selected in the setting options for the presented video information or an image to be enhanced is selected on the interface and a dark scene enhancement button (see the terminal 200-2) is clicked, where the coding information sequence is a sequence formed by coding information of each pixel point; traversing the coding information sequence, acquiring enhanced information and filtering information corresponding to the enhanced information aiming at the coding information of the traversed pixel points, and enhancing the coding information of the traversed pixel points by combining the enhanced information and the filtering information to obtain the enhanced coding information; when the coding information sequence is traversed, obtaining an enhanced coding information sequence corresponding to the coding information sequence; and decoding an enhanced image corresponding to the image to be enhanced based on the enhanced coding information sequence.

A server 400 for providing a computing service to the terminal 200 through the network 300.

In some embodiments, the server 400 may be an independent physical server, may also be a server cluster or a distributed system formed by a plurality of physical servers, and may also be a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a Network service, cloud communication, a middleware service, a domain name service, a security service, a CDN (Content Delivery Network), a big data and artificial intelligence platform, and the like. The terminal 200 may be, but is not limited to, a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a vehicle-mounted terminal, a smart television, and the like. The terminal and the server may be directly or indirectly connected through wired or wireless communication, and the embodiment of the present application is not limited.

Referring to fig. 3, fig. 3 is a schematic structural diagram of a terminal in fig. 2 according to an embodiment of the present disclosure, where the terminal 200 shown in fig. 3 includes: at least one processor 210, memory 250, at least one network interface 220, and a user interface 230. The various components in terminal 200 are coupled together by a bus system 240. It is understood that the bus system 240 is used to enable communications among the components. The bus system 240 includes a power bus, a control bus, and a status signal bus in addition to a data bus. For clarity of illustration, however, the various buses are designated as bus system 240 in FIG. 3.

The Processor 210 may be an integrated circuit chip having Signal processing capabilities, such as a general purpose Processor, a Digital Signal Processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like, wherein the general purpose Processor may be a microprocessor or any conventional Processor, or the like.

The user interface 230 includes one or more output devices 231, including one or more speakers and/or one or more visual display screens, that enable the presentation of media content. The user interface 230 also includes one or more input devices 232, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

The memory 250 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid state memory, hard disk drives, optical disk drives, and the like. Memory 250 optionally includes one or more storage devices physically located remotely from processor 210.

The memory 250 includes volatile memory or nonvolatile memory, and can also include both volatile and nonvolatile memory. The nonvolatile Memory may be a Read Only Memory (ROM), and the volatile Memory may be a Random Access Memory (RAM). The memory 250 described in embodiments herein is intended to comprise any suitable type of memory.

In some embodiments, memory 250 is capable of storing data, examples of which include programs, modules, and data structures, or a subset or superset thereof, to support various operations, as exemplified below.

An operating system 251 including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

a network communication module 252 for communicating to other computer devices via one or more (wired or wireless) network interfaces 220, exemplary network interfaces 220 including: bluetooth, wireless-compatibility authentication (Wi-Fi), and Universal Serial Bus (USB), etc.;

in some embodiments, the terminal 200 further includes a presentation module 253 for enabling presentation of information (e.g., a user interface for operating peripherals and displaying content and information) via one or more output devices 231 (e.g., a display screen, speakers, etc.) associated with the user interface 230;

an input processing module 254 for detecting one or more user inputs or interactions from one of the one or more input devices 232 and translating the detected inputs or interactions.

In some embodiments, the image enhancement apparatus provided by the embodiments of the present application may be implemented in software, and fig. 3 shows an image enhancement apparatus 255 stored in the memory 250, which may be software in the form of programs and plug-ins, and includes the following software modules: an image encoding module 2551, an image enhancement module 2552, an image decoding module 2553, an image rendering module 2554 and an image determination module 2555, which are logical and therefore can be combined arbitrarily or further split depending on the functionality implemented. The functions of the respective modules will be explained below.

In other embodiments, the image enhancement Device provided in the embodiments of the present Application may be implemented in hardware, and for example, the image enhancement Device provided in the embodiments of the present Application may be a processor in the form of a hardware decoding processor, which is programmed to perform the image enhancement method provided in the embodiments of the present Application, for example, the processor in the form of the hardware decoding processor may employ one or more Application Specific Integrated Circuits (ASICs), DSPs, programmable Logic Devices (PLDs), complex Programmable Logic Devices (CPLDs), field Programmable Gate Arrays (FPGAs), or other electronic components.

In the following, the image enhancement method provided by the embodiment of the present application will be described in conjunction with an exemplary application and implementation of the server provided by the embodiment of the present application.

Referring to fig. 4, fig. 4 is an alternative flowchart of an image enhancement method provided in an embodiment of the present application, which will be described with reference to the steps shown in fig. 4.

S401, acquiring a coding information sequence of the image to be enhanced.

In the embodiment of the application, the image enhancement equipment encodes each pixel point of the image to be enhanced based on the brightness of the pixel point to obtain the encoding information of each pixel point; and the sequence formed by the coding information of each pixel point is the coding information sequence corresponding to the image to be enhanced. Here, the image enhancement device may be a coding information sequence directly acquired, or may be a coding information sequence obtained by coding an acquired image to be enhanced, where the coding is luminance-based coding, for example, YUV coding and extracting a Y channel value.

It should be noted that the coded information sequence is a luminance-based coded image corresponding to the image to be enhanced, and the coded information of the pixel points in the coded information sequence corresponds to the information of the pixel points in the image to be enhanced one by one; and, the coding information of each pixel in the coding information sequence is obtained based on the brightness of each pixel, for example, the Y channel value of the pixel in the YUV image corresponding to the image to be enhanced, or the dark channel value of the pixel of the image to be enhanced.

S402, traversing the coding information sequence, acquiring the enhanced information and the filtering information corresponding to the enhanced information aiming at the coding information of the traversed pixel point, and enhancing the coding information of the traversed pixel point by combining the enhanced information and the filtering information to obtain the enhanced coding information.

In the embodiment of the present application, after the image enhancement device obtains the encoded information sequence, in order to perform dark scene enhancement processing by using a pixel as a unit, the encoded information sequence is traversed by using the pixel as a unit. When traversing the coding information sequence, the image enhancement device acquires enhancement information corresponding to the coding information of the pixel point aiming at the coding information of the traversed pixel point, acquires filtering information corresponding to the coding information of the pixel point based on the enhancement information corresponding to the coding information of the pixel point, combines the enhancement information corresponding to the coding information of the pixel point and the filtering information corresponding to the coding information of the pixel point, and determines the combination result as the enhancement result corresponding to the coding information of the pixel point, wherein the enhancement result is the enhancement coding information.

It should be noted that the enhancement information is an overall evaluation result of the luminance of the pixel points, and an enhancement information sequence corresponding to the image to be enhanced and formed by the enhancement information of each pixel point represents a global dark scene value, and is used to control the overall luminance, chromaticity and intensity of the image, ensure that the overall luminance is within a preset luminance range, and maintain the color reproduction. The filter information is a local evaluation result of the brightness of the pixel points, and a filter information sequence corresponding to the image to be enhanced and formed by the filter information of each pixel point represents a local dark scene value and is used for controlling the brightness enhancement amplitude of the dark part area and the brightness area of the image, wherein the brightness enhancement amplitude is inversely proportional to the brightness of the image, for example, the brightness of the dark part area of the image is improved, and the brightness area of the image is finely adjusted; and, the filtering information is mapped based on the enhancement information. In addition, the enhancement encoded information is encoded information enhanced for each pixel, here, enhancement, i.e., dark scene enhancement.

And S403, when the coded information sequence is traversed, obtaining an enhanced coded information sequence corresponding to the coded information sequence.

In the embodiment of the application, the image enhancement device performs dark scene enhancement on the coding information of each traversed pixel point by adopting the processing process described in the step S402 to obtain corresponding enhanced coding information; therefore, when the image enhancement equipment finishes traversing the coding information sequence and finishes enhancing the coding information of each pixel point, the enhanced coding information of each pixel point is obtained; and the enhanced coding information of each pixel point forms an enhanced coding information sequence corresponding to the coding information sequence. The coding information in the coding information sequence corresponds to the enhanced coding information in the enhanced coding information sequence one by one, and the enhanced coding information sequence corresponds to the image to be enhanced one by one on the basis of pixel points.

S404, decoding an enhanced image corresponding to the image to be enhanced based on the enhanced coding information sequence.

In the embodiment of the present application, the image enhancement device decodes the obtained enhancement coding information sequence, and thus obtains an enhanced image corresponding to the image to be enhanced, where the enhanced image is the enhanced image to be enhanced.

It should be noted that if the coding information sequence is directly obtained by the image enhancement device without coding, S404 is not executed, and the enhancement coding information sequence is directly returned.

It can be understood that, in the process of enhancing the coding information of each pixel point, for the coding information of the pixel points traversed from the coding information sequence, the filtering information corresponding to the enhancement information is also calculated while the enhancement information is calculated, and the coding information of each pixel point is enhanced based on the enhancement information and the filtering information; therefore, the enhancement of the coding information of each pixel point can be completed through one-time traversal, so that the resource consumption is less, and the dark scene enhancement efficiency of the image can be improved.

In the embodiment of the present application, S402 may be implemented by S4021 to S4023; that is to say, the image enhancement device traverses the coding information sequence, and obtains the enhancement information and the filtering information corresponding to the enhancement information for the coding information of the traversed pixel point, including S4021 to S4023, and the following steps are described separately.

S4021, generating a cycle traversal instruction based on the quantity threshold value.

It should be noted that the coding information sequence of the image to be enhanced is stored in the memory of the image enhancement device, and the image enhancement device traverses the coding information sequence in the memory through the processor. Here, the image enhancement device acquires an occupation space of the register specification and the coding information of the pixel points in the processor, determines a reading number corresponding to one instruction cycle, that is, a number threshold, based on the occupation space of the register specification and the coding information of the pixel points, and finally generates a single traversal instruction for acquiring the coding information of the pixel points of the number threshold, that is, a cycle traversal instruction.

Illustratively, the obtaining process of the quantity threshold value can be realized by equation (1), where equation (1) is:

wherein c is a quantity threshold; v is a register specification, e.g., 64 bits, 32 bits; o is the occupied space of the encoded information of the pixel point, for example, 8 bits.

S4022, responding to the periodic traversal instruction, sequentially traversing the coding information of the pixels with the threshold number and the coding information of the associated pixels corresponding to the coding information of each traversed pixel from the coding information sequence.

In the embodiment of the application, when traversing the coding information sequence, the image enhancement device responds to a periodic traversal instruction for acquiring data each time, and traverses the coding information of a threshold number of pixel points from the coding information sequence at one time, and associated pixel point coding information corresponding to the coding information of each traversed pixel point in the coding information of the traversed pixel points of the threshold number of pixel points.

It should be noted that the associated pixel coding information may be coding information of a pixel adjacent to the coding information of each traversed pixel, or coding information of a pixel in a preset sliding window, and the like, which is not specifically limited in this embodiment of the present application.

S4023, acquiring the enhanced information and the filtering information corresponding to the enhanced information based on the associated pixel point coding information corresponding to the coding information of each traversed pixel point.

It should be noted that the image enhancement device may randomly select the coding information of one pixel from the coding information of each traversed pixel and the coding information of the associated pixel to determine the coding information of the traversed pixel as the enhancement information corresponding to the coding information of the traversed pixel, may also determine the enhancement information corresponding to the coding information of the traversed pixel based on the coding information of each traversed pixel and the overall coding information of the associated pixel, and the like, which is not specifically limited in this embodiment of the present application. In addition, the image enhancement device judges the brightness intensity of the enhancement information and determines the filtering information corresponding to the enhancement information based on the judgment result; here, the image enhancement apparatus, when determining the filter information based on the determination result, obtains the filter information lower than the enhancement information if it is determined that the luminance of the pixel point is high based on the enhancement information, and obtains the filter information higher than the enhancement information if it is determined that the luminance of the pixel point is low based on the enhancement information.

It can be understood that, when the encoded information sequence is traversed, the traversal data is read once in a SIMD mode, so that the storage efficiency of the register is improved, the resource consumption caused by interaction in the traversal process is reduced, the traversal parallelism is improved, and the traversal efficiency is improved.

In the embodiment of the present application, S4023 may be realized by S40231 to S40234; that is to say, the image enhancement device obtains the enhancement information and the filtering information corresponding to the enhancement information, including S40231 to S40234, based on the associated pixel encoding information corresponding to the encoding information of each traversed pixel, and the following steps are respectively described.

S40231, shifting the coding information of each traversed pixel and the coding information of the associated pixel.

It should be noted that, when the associated pixel point coding information belongs to a data type with a decimal point (for example, a floating point type, a double floating point type, or the like), in order to improve the efficiency of calculating the coding information of each traversed pixel point and the coding information of the associated pixel point, the image enhancement device shifts the associated pixel point coding information to convert the associated pixel point coding information into integer type data, and shifts the coding information of each traversed pixel point to convert the coding information of each traversed pixel point into integer type data; therefore, the shifted coding information of the associated pixel points and the shifted coding information of each traversed pixel point are data of integer type.

S40232, equalizing the code information of each traversed pixel after shifting and the code information of the related pixel after shifting, and determining the equalizing result as initial enhancement information.

In the embodiment of the application, the image enhancement device performs equalization processing on the shifted coding information of each traversed pixel and the shifted coding information of the associated pixel to obtain the overall evaluation result of the representation image. Here, since the image enhancement device performs the equalization processing on the code information of each traversed pixel after the shift and the code information of the associated pixel after the shift, the "code information of each traversed pixel after the equalization processing and the code information of the associated pixel after the shift" are only the initial enhancement information of the traversed pixel.

S40233 maps the initial enhancement information based on the filter mapping relationship to obtain initial filter information.

In the embodiment of the application, the image enhancement device can acquire a filtering mapping relation, wherein the filtering mapping relation is used for determining a bright and dark area of an image; here, the image enhancement apparatus maps the initial enhancement information to obtain a characterization image local evaluation result. Here, since the image enhancement device performs mapping on the initial enhancement information, the mapped information is only the initial filtering information of the pixel point.

S40234, performing reverse shift on the initial enhancement information to obtain enhancement information, and performing reverse shift on the initial filtering information to obtain filtering information.

In the embodiment of the application, the image enhancement device performs inverse shift on the initial enhancement information and the initial filtering information to obtain enhancement information corresponding to the initial enhancement information and filtering information corresponding to the initial filtering information. The reverse shift is the reverse process corresponding to the shift in S40231.

In the embodiment of the application, S4023 may be realized by S40235 and S40236; that is to say, the image enhancement device obtains the enhancement information and the filtering information corresponding to the enhancement information, including S40235 and S40236, based on the associated pixel encoding information corresponding to the encoding information of each traversed pixel, and the following steps are respectively described.

S40235, equalizing the coding information of each traversed pixel and the coding information of the associated pixels, and determining the equalization processing result as enhancement information.

S40236, mapping the enhanced information based on the filtering mapping relation, and determining the mapping result as filtering information.

It should be noted that the image enhancement device may also directly calculate the coding information of each traversed pixel and the coding information of the associated pixel, so as to obtain the enhancement information and the filtering information corresponding to the coding information of the traversed pixel.

In the embodiment of the present application, S402 may be implemented by S4021 to S4023; that is to say, the image enhancement device enhances the coding information of the traversed pixels in combination with the enhancement information and the filtering information to obtain enhanced coding information, which includes S4021 to S4023, and the following steps are described separately.

S4021, acquiring the enhancement information and a first weighting result of the enhancement weight.

The image enhancement device can acquire the weight corresponding to the enhancement information, which is the enhancement weight; the image enhancement apparatus acquires a combination result of enhancement information and enhancement weights, which is a first weighting result.

S4022, acquiring a second weighting result of the filtering information and the filtering weight.

The image filtering device may obtain a weight corresponding to the filtering information, where the weight corresponding to the filtering information is a filtering weight; the image filtering apparatus acquires a combination result of the filter information and the filter weight, which is a second weighting result.

S4023, determining a combination result of the first weighting result and the second weighting result as the enhancement encoded information.

It should be noted that the process of obtaining the enhanced coding information is a process of enhancing the coding information of the traversed pixel, and the enhanced coding information is a result of enhancing the coding information of the traversed pixel.

In the embodiment of the application, S4024 is further included before S4021; that is, before the image enhancement apparatus acquires the enhancement information and the first weighting result of the enhancement weight, the image enhancement method further includes S4024, which is explained below.

S4024, storing the enhancement weight.

In the embodiment of the application, the image enhancement device stores the experience value or the experience parameter so as to realize the solidification of the experience value or the experience parameter; for example, for the enhancement weight, the enhancement weight is solidified by storing the enhancement weight.

Accordingly, in the embodiment of the present application, S4021 may be implemented by S40211; that is, the sample acquiring apparatus acquires the enhancement information and the first weighting result of the enhancement weight, including S40211, which is explained below separately.

S40211, acquiring the enhanced information and the stored first weighting result of the enhanced weight.

It should be noted that, since the image enhancement device extracts and stores the enhancement weights, here, the stored enhancement weights are directly obtained, and the stored enhancement weights are combined with the enhancement information to obtain the first weighting result.

In the embodiment of the present application, the empirical values or empirical parameters such as the filtering mapping relationship and the filtering weight may be subjected to a curing process before participating in the calculation.

It can be understood that the calculation efficiency in the image enhancement process can be improved by solidifying the empirical values or empirical parameters such as the filtering mapping relation, the filtering weight and the enhancement weight, and thus, the image enhancement efficiency can be improved.

In the embodiment of the present application, S402 may further include S405 to S407; that is, before the image enhancement apparatus traverses the encoded information sequence, the image enhancement method further includes S405 to S407, and the following steps are separately described.

S405, obtaining pixel point layout of the image to be enhanced.

It should be noted that the pixel layout is the position information of each pixel in the image to be enhanced.

S406, based on the pixel point layout, determining storage position information corresponding to each group of pixel points in the image to be enhanced.

In the embodiment of the present application, the image enhancement device groups the pixels of the image to be enhanced based on the pixel layout (for example, in a process of determining a row of pixels or a column of pixels), determines each group (row or column) of pixels, and determines the storage location of the coding information of each group of pixels in the memory, where the coding information of each group of pixels is stored in the memory, that is, the storage location information.

It should be noted that the image enhancement device may determine the relative storage location information of the last pixel point in a group as the storage location information corresponding to the group of pixel points.

Illustratively, for an image to be enhanced with 128 × 256 pixel point layout, the storage location information for the first row of pixel points (one group) is 256, and the storage location information for the second row of pixel points is 512, …, and the storage location information of each group of pixel points is sequentially obtained.

And S407, storing storage position information corresponding to each group of pixel points in the image to be enhanced.

In the embodiment of the application, the image enhancement device stores the acquired storage position information corresponding to each group of pixel points, so as to perform traversal processing according to the stored storage position information corresponding to each group of pixel points.

Accordingly, in the embodiment of the present application, S4022 may be realized by S40221 to S40223; that is to say, in response to the periodic traversal instruction, the image enhancement device sequentially traverses the encoding information of a threshold number of pixel points from the encoding information sequence and the associated pixel point encoding information corresponding to the encoding information of each traversed pixel point, including S40221 to S40223, and the following steps are respectively explained.

S40221, responding to the periodic traversing instruction, acquiring row and column information to be traversed, and determining a target group and offset position information based on the row and column information to be traversed.

It should be noted that the row and column information to be traversed is the row and column position of the pixel point in the pixel point layout. Here, the image enhancement device may determine a row in the row and column information to be traversed as the target group, where a column in the row and column information to be traversed is offset position information; and determining the column in the row and column information to be traversed as a target group, wherein the behavior in the row and column information to be traversed is biased to position information.

S40222, determining target position information corresponding to the target group based on the stored storage position information corresponding to each group of pixel points in the image to be enhanced, and determining position information to be traversed based on the target position information and the offset position information.

It should be noted that, the image enhancement apparatus determines, from the storage location information corresponding to each group of pixel points in the image to be enhanced, the storage location information corresponding to the target group, which is the target location information corresponding to the target group. In addition, the position information to be traversed is a superposition result of the target position information and the offset position information.

S40223, sequentially traversing the coding information of a threshold number of pixel points and the coding information of the associated pixel point corresponding to the coding information of each traversed pixel point from the position information to be traversed of the image coding information.

It can be understood that the pixel points of the image to be enhanced are grouped by rows or columns, and then the traversal position of the pixel point to be traversed is determined according to the relative position information of the determined group, so that the calculation consumption of the position of each pixel point in the traversal process is reduced.

Referring to fig. 5, fig. 5 is a schematic flow chart of another alternative image enhancement method provided in the embodiment of the present application; as shown in fig. 5, in the embodiment of the present application, S401 may be implemented by S4011; that is, the image enhancement apparatus acquires a coding information sequence of an image to be enhanced, including S4011, which is explained below.

S4011, presenting a first enhancement control corresponding to the image to be enhanced, and responding to a first enhancement operation acted on the first enhancement control to obtain a coding information sequence of the image to be enhanced.

It should be noted that the first enhancement control is used to trigger dark scene enhancement processing of the image to be enhanced, and the image enhancement device obtains dark scene operation performed by the user for the image to be enhanced by presenting the first enhancement control corresponding to the image to be enhanced. When a user triggers the first enhancement control to perform dark scene enhancement processing on an image to be enhanced, the image enhancement equipment receives a first enhancement operation acting on the first enhancement control; at this time, the image enhancement device triggers a dark scene enhancement process of the image to be enhanced in response to the first enhancement operation, so as to obtain a coding information sequence of the image to be enhanced and perform a subsequent enhancement processing process on the coding information sequence.

With continued reference to fig. 5, in the embodiment of the present application, S408 may be further included after S404; that is, after the image enhancement apparatus decodes the enhanced image corresponding to the image to be enhanced based on the enhancement coded information sequence, the image enhancement method further includes S408, which is explained below.

And S408, presenting the enhanced image.

In the embodiment of the application, the image enhancement device informs the user of the dark scene processing result of the image to be processed by presenting the enhanced image.

It should be noted that the first enhancement control may also be presented by another device, and then when receiving a first enhancement request generated by another device in response to a first enhancement operation acting on the first enhancement control, the image enhancement device acquires a sequence of encoded information of an image to be enhanced in response to the first enhancement request. Thus, after the image enhancement device obtains the enhanced image, the enhanced image is sent to another device to present the enhanced image on the other device.

Referring to fig. 6, fig. 6 is a schematic flowchart of yet another alternative image enhancement method provided in the embodiment of the present application; as shown in fig. 6, in the embodiment of the present application, S401 may further include S409; that is, the image enhancement apparatus may further include S409 before acquiring the coding information sequence of the image to be enhanced, which is explained below.

And S409, presenting a second enhancement control aiming at the played video information, and determining a frame image corresponding to the video information as an image to be enhanced in response to a second enhancement operation acted on the second enhancement control.

It should be noted that the played video information may be real-time or historical, and this is not specifically limited in this embodiment of the application. Here, the second enhancement control is used to trigger dark scene enhancement processing of the video information, and the image enhancement device obtains dark scene operation performed by the user on the video information by presenting the second enhancement control corresponding to the video information. When a user triggers a second enhancement control to perform dark scene enhancement processing on the video information, the image enhancement equipment also receives second enhancement operation acting on the second enhancement control, and at the moment, the image enhancement equipment responds to the second enhancement operation to perform dark scene enhancement processing on a current video frame image of the video information, so that the current frame image is taken as an image to be processed; or at this time, the image enhancement device responds to the second enhancement operation and performs dark scene enhancement processing from the next frame of the current video frame of the video information, so as to take the next frame of image as the image to be processed.

With continued reference to fig. 6, in the embodiment of the present application, S410 may be further included after S404; that is, after the image enhancement apparatus decodes the enhanced image corresponding to the image to be enhanced based on the enhancement coded information sequence, the image enhancement method further includes S410, which is explained below.

And S410, replacing the presentation of the frame image by the enhanced image.

It should be noted that the second enhancement control may also be presented by another device, and then when receiving a second enhancement request generated by another device in response to a second enhancement operation acting on the second enhancement control, the image enhancement device acquires the coding information sequence of the image to be enhanced in response to the second enhancement request. Thus, after the image enhancement device obtains the enhanced image, the enhanced image is sent to the other device for rendering the enhanced image on the other device.

In an embodiment of the present application, an image enhancement apparatus obtains a coding information sequence of an image to be enhanced, including: the image enhancement device acquires a color channel image of an image to be enhanced; acquiring a minimum color channel value of each pixel point based on the color channel image, and constructing a gray image corresponding to the image to be enhanced based on the minimum color channel value of each pixel point; and acquiring the minimum gray value of each pixel point in the sliding window based on the gray image, and constructing a coding information sequence corresponding to the image to be enhanced based on the minimum gray value of each pixel point in the sliding window.

In the embodiment of the present application, S404 may be implemented by S4041 and S4042; that is, the image enhancement apparatus decodes an enhanced image corresponding to the image to be enhanced based on the enhancement coded information sequence, including S4041 and S4042, and the following steps are separately described.

S4041, based on the enhancement coding information sequence, an initial enhancement image is decoded.

It should be noted that the image enhancement apparatus decodes the enhancement encoded information sequence, and the obtained decoding result is the initial enhancement image.

S4042, carrying out post-processing on the initial enhanced image to obtain an enhanced image corresponding to the image to be enhanced.

In the embodiment of the application, the image enhancement device may directly determine the initial enhanced image as the image to be enhanced after the dark scene is enhanced, or may perform post-processing on the initial enhanced image, and determine the post-processed initial enhanced image as the enhanced image corresponding to the image to be enhanced. Wherein the post-processing includes at least one of a denoising process, a sharpening process, and a correction process.

It should be noted that, the denoising process includes a bilateral filtering denoising algorithm, an "NL-Mean" denoising algorithm, and a "BM3D" denoising algorithm; a correction process such as gamma correction to raise the brightness of the initial enhanced image; and sharpening is carried out to improve the loss of the initial enhanced image.

Next, an exemplary application of the embodiment of the present application in a practical application scenario will be described.

Referring to fig. 7, fig. 7 is a schematic view of an exemplary scene triggering dark scene enhancement provided by an embodiment of the present application; as shown in fig. 7, in the video interface 7-1 of the video conference, a setting box 7-2 is presented for conference video (video information); for clicking the video option 7-21 in the setting frame 7-2, presenting a video image 7-22 of the conference video and a dark scene enhancement option 7-23 (a second enhancement control) corresponding to the video image 7-22 in the setting frame 7-2; when the user clicks (second enhancement operation) on option 7-23, dark scene enhancement is performed on the frame image of the conference video presented in the video interface 7-1. Here, the dark scene enhancement is triggered by a user operation, and the processing speed is faster than the processing of determining whether to perform the dark scene enhancement based on the brightness or the content of the frame image.

It should be noted that, when dark scene enhancement is performed on a frame image of a conference video, referring to fig. 8, fig. 8 is an exemplary image enhancement flowchart provided in an embodiment of the present application; as shown in fig. 8, S801 to S803 are included, in which:

s801, acquiring a video image (image to be enhanced).

S802, performing a traversal for the video image, and calculating an enhancement value (enhancement information), a filtering value (filtering information), and an enhancement result (enhancement coding information) for each pixel.

It should be noted that, by using the image enhancement method provided in the embodiment of the present application, S102 and S103 in fig. 1 are circularly combined, so as to reduce the number of times of memory access and reduce multiplication operations, thereby reducing the memory access overhead of the processor and reducing the overhead of the multiplier.

And S803, finishing. And outputting an enhanced video image (enhanced image), wherein the enhanced video image corresponds to an image formed by the enhanced results of all the pixel points.

Referring to fig. 9, fig. 9 is a flowchart of a process of S802 in fig. 8 according to an embodiment of the present disclosure; as shown in fig. 9, S802 may be implemented by S901 to S903, where:

and S901, converting the video image into an image in a YUV format.

S902, traversing the YUV format image:

1. through SIMD, in one instruction cycle (cycle traversal instruction), simultaneously acquiring dark channel values of 8 (quantity threshold) pixel points and dark channel values (associated pixel point coding information) of 4 pixel points around each pixel point (up, down, left and right), and performing balanced weighted calculation (balanced processing) on the dark channel values of 5 pixel points (the dark channel value of each pixel point and the dark channel values of the 4 pixel points around) to obtain an enhanced value;

2. mapping the enhanced value to obtain a filtering value;

3. and weighting and calculating the filtering value and the enhancement value to obtain the enhancement result of each pixel point.

And when traversing is finished, obtaining an enhanced image in a YUV format.

It should be noted that, when traversing the image in YUV format, the temporary storage processing is performed on the position of each row or each column of pixel points, so that the temporary storage result and the offset information are directly accumulated each time the information of the pixel points is taken, and the multiplication processing is reduced; in addition, during calculation, the empirical value is also solidified, so that repeated calculation is avoided; and during calculation, the calculation efficiency is improved by converting the data to be calculated from a floating point type to an integer type through a processing mode of firstly shifting, then calculating and finally shifting.

And S903, carrying out format conversion on the enhanced image in the YUV format to obtain an enhanced video image. Here, the enhanced image in YUV format is converted into an image in video image format.

It should be noted that, when the image enhancement method provided by the embodiment of the present application is used for dark scene enhancement, if the obtained image is directly in the YUV format, S902 is directly executed, and S903 is not executed any more; and if the acquired image is not in the YUV format, performing S901 to S903.

It should be further noted that, referring to fig. 10, fig. 10 is a processing manner of single instruction multiple data streams according to an embodiment of the present application; as shown in fig. 10, a single instruction cycle 10-1 can obtain dark channel values of 8 pixels and dark channel values of 4 pixels around each pixel 10-2; compared with the single-instruction single-data-stream processing mode in fig. 11 (only the dark channel values of 1 pixel and the dark channel values of 4 surrounding pixels are obtained in one instruction cycle 11-1), the parallelism is higher.

In the embodiment of the present application, when the processor of the image enhancement device includes a CPU and a GPU (graphics processing Unit), the increment of the CPU can be further reduced.

The image enhancement method provided by the embodiment of the application has a cross-platform property, and can be applied to a MacOSX desktop end, a Windows desktop end, an Android mobile end and an iOS mobile end. Next, based on the image enhancement method provided by the embodiment of the present application, the application results (CPU increments) of the "MacOSX" desktop end, the "Windows desktop end, the" Android "mobile end, and the" iOS "mobile end are respectively explained:

"iOS" mobile side: the resolution is 640 × 360, the frame rate is 30FPS (Frames Per Second transmitted frame), when dark scene enhancement is started by selecting options 7-23 in fig. 7, the flow of dark scene enhancement described in fig. 1 is adopted, and the average CPU increment Per core is 1.99; by adopting the image enhancement method provided by the embodiment of the application, the average CPU increment per core is 0.48.

"iOS" mobile side: the resolution is 1280 × 720, the frame rate is 30FPS, when the dark scene enhancement is started by selecting options 7-23 in fig. 7, the flow of the dark scene enhancement described in fig. 1 is adopted, and the average CPU increment per core is 0.22; by adopting the image enhancement method provided by the embodiment of the application, the average CPU increment per core is 0.04.

The Android mobile terminal: the resolution is 640 × 360, the frame rate is 30FPS, when the dark scene enhancement is started in the check option 7-23 in fig. 7, the flow of the dark scene enhancement described in fig. 1 is adopted, and the average CPU increment per core is 8.18; by adopting the image enhancement method provided by the embodiment of the application, the average CPU increment per core is 0.4;

the Android mobile terminal: the resolution is 640 × 360, the frame rate is 30FPS, when the dark scene enhancement is started in the check option 7-23 in fig. 7, the flow of the dark scene enhancement described in fig. 1 is adopted, and the average CPU increment per core is 2.91; by adopting the image enhancement method provided by the embodiment of the application, the average CPU increment per core is 1.04.

"Windows" desktop end: the resolution is 1280 × 720, the frame rate is 30FPS, when the dark scene enhancement is started according to the check option 7-23 in fig. 7, the dark scene enhancement flow described in fig. 1 is adopted, and the average CPU increment per core is 2.13; by adopting the image enhancement method provided by the embodiment of the application, the average CPU increment per core is 0.18.

A "Windows" desktop end; the resolution is 1280 × 720, the frame rate is 30FPS, when the dark scene enhancement is started in the check option 7-23 in fig. 7, the flow of the dark scene enhancement described in fig. 1 is adopted, and the average CPU increment per core is 4.8; by adopting the image enhancement method provided by the embodiment of the application, the average CPU increment per core is 2.11.

"MacOSX" desktop side: the resolution is 1280 × 720, the frame rate is 30FPS, when the dark scene enhancement is started in the check option 7-23 in fig. 7, the flow of the dark scene enhancement described in fig. 1 is adopted, and the average CPU increment per core is 5.87; by adopting the image enhancement method provided by the embodiment of the application, the average CPU increment per core is 3.27.

"MacOSX" desktop side: the resolution is 1280 × 720, the frame rate is 30FPS, when the dark scene enhancement is started according to the check option 7-23 in fig. 7, the dark scene enhancement flow described in fig. 1 is adopted, and the average CPU increment per core is 4.44; by adopting the image enhancement method provided by the embodiment of the application, the average CPU increment per core is 1.6.

In summary, when the image enhancement method provided by the embodiment of the present application starts dark scene enhancement, compared with the dark scene enhancement scheme described in fig. 1, the CPU increment is lower, and some are even close to 0, so that the energy consumption is less. In addition, when the image enhancement method provided by the embodiment of the application is executed in a device comprising a CPU and a GPU, the increment of the CPU is lower.

Continuing with the exemplary structure of the image enhancement device 255 provided by the embodiments of the present application as software modules, in some embodiments, as shown in fig. 3, the software modules stored in the image enhancement device 255 of the memory 250 may include:

the image coding module 2551 is configured to obtain a coding information sequence of an image to be enhanced, where the coding information sequence is a sequence formed by coding information of each pixel;

the image enhancement module 2552 is configured to traverse the coding information sequence, acquire enhancement information and filtering information corresponding to the enhancement information for the traversed coding information of the pixel point, and enhance the traversed coding information of the pixel point by combining the enhancement information and the filtering information to obtain enhanced coding information;

the image enhancement module 2552 is further configured to, when the coded information sequence is traversed, obtain an enhanced coded information sequence corresponding to the coded information sequence;

an image decoding module 2553, configured to decode an enhanced image corresponding to the image to be enhanced based on the enhancement coding information sequence.

In this embodiment of the application, the image enhancement module 2552 is further configured to generate a cycle traversal instruction based on a quantity threshold, where the quantity threshold is determined based on a register specification and an occupied space of the encoding information of the pixel point; responding to the periodic traversal instruction, sequentially traversing the coding information of the pixel points with the number threshold and the coding information of the associated pixel points corresponding to the coding information of each traversed pixel point from the coding information sequence; and acquiring the enhanced information and the filtering information corresponding to the enhanced information based on the associated pixel point coding information corresponding to the coding information of each traversed pixel point.

In this embodiment of the application, the image enhancement module 2552 is further configured to shift the coding information of the pixel point and the coding information of the associated pixel point of each traversal respectively; balancing the coding information of the pixel points traversed by each shift and the coding information of the associated pixel points shifted, and determining a balancing result as initial enhancement information; mapping the initial enhancement information based on a filtering mapping relation to map out initial filtering information; and performing reverse shift on the initial enhancement information to obtain the enhancement information, and performing reverse shift on the initial filtering information to obtain the filtering information.

In this embodiment of the application, the image enhancement module 2552 is further configured to perform equalization processing on the coding information of the pixel point and the coding information of the associated pixel point of each traversal, and determine an equalization processing result as the enhancement information; and mapping the enhanced information based on a filtering mapping relation, and determining a mapping result as the filtering information.

In this embodiment of the application, the image enhancement module 2552 is further configured to obtain a first weighting result of the enhancement information and the enhancement weight; acquiring a second weighting result of the filtering information and the filtering weight; and determining a combination result of the first weighting result and the second weighting result as the enhanced coding information, wherein the enhanced coding information is a result of enhancing the coding information of the traversed pixel points.

In this embodiment of the present application, the image enhancement module 2552 is further configured to store the enhancement weights; obtaining the first weighting result of the enhancement information and the stored enhancement weight.

In this embodiment of the application, the image enhancement module 2552 is further configured to obtain a pixel point layout of the image to be enhanced; based on the pixel point layout, determining storage position information corresponding to each group of pixel points in the image to be enhanced; storing storage position information corresponding to each group of pixel points in the image to be enhanced; responding to the periodic traversal instruction, and acquiring row and column information to be traversed; determining a target group and offset position information based on the row and column information to be traversed; determining target position information corresponding to the target group based on the stored storage position information corresponding to each group of pixel points in the image to be enhanced; determining to-be-traversed position information based on the target position information and the bias position information; sequentially traversing the coding information of the pixel points with the number threshold and the coding information of the associated pixel points corresponding to the coding information of each traversed pixel point from the position information to be traversed of the image coding information.

In this embodiment of the application, the image encoding module 2551 is configured to present a first enhancement control corresponding to the image to be enhanced; and responding to a first enhancement operation acted on the first enhancement control, and acquiring the coded information sequence of the image to be enhanced.

In the embodiment of the present application, the image enhancement device 255 further includes an image rendering module 2554 for rendering the enhanced image.

In this embodiment of the present application, the image enhancement apparatus 255 further includes an image determining module 2555, configured to present a second enhancement control for the played video information; and responding to a second enhancement operation acted on the second enhancement control, and determining a frame image corresponding to the video information as the image to be enhanced.

In this embodiment, the image rendering module 2554 is further configured to replace the rendering of the frame image with the enhanced image.

In this embodiment of the application, the image coding module 2551 is further configured to obtain a color channel image of the image to be enhanced; acquiring a minimum color channel value of each pixel point based on the color channel image, and constructing a gray image corresponding to the image to be enhanced based on the minimum color channel value of each pixel point; and acquiring the minimum gray value of each pixel point in a sliding window based on the gray image, and constructing the coding information sequence corresponding to the image to be enhanced based on the minimum gray value of each pixel point in the sliding window.

In this embodiment of the application, the image decoding module 2553 is further configured to decode an initial enhanced image based on the enhancement coding information sequence; and performing post-processing on the initial enhanced image to obtain the enhanced image corresponding to the image to be enhanced, wherein the post-processing comprises at least one of denoising processing, sharpening processing and correction processing.

Embodiments of the present application provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, so that the computer device executes the image enhancement method described in the embodiment of the present application.

Embodiments of the present application provide a computer-readable storage medium having stored therein executable instructions that, when executed by a processor, cause the processor to perform an image enhancement method provided by embodiments of the present application, for example, the image enhancement method as shown in fig. 4.

In some embodiments, the computer-readable storage medium may be memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

In some embodiments, executable instructions may be written in any form of programming language (including compiled or interpreted languages), in the form of programs, software modules, scripts or code, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

By way of example, executable instructions may correspond, but do not necessarily have to correspond, to files in a file system, and may be stored in a portion of a file that holds other programs or data, such as in one or more scripts in a hypertext Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code).

By way of example, executable instructions may be deployed to be executed on one computer device or on multiple computer devices at one site or distributed across multiple sites and interconnected by a communication network.

In summary, according to the embodiment of the present application, in the process of enhancing the coding information of each pixel, for the coding information of the pixel traversed from the coding information sequence, the filtering information corresponding to the enhancement information is calculated while the enhancement information is calculated, and the coding information of each pixel is enhanced based on the enhancement information and the filtering information; therefore, the enhancement of the coding information of each pixel point can be completed through one-time traversal, so that the resource consumption is less, and the dark scene enhancement efficiency of the image can be improved. In addition, the parallelism of the processor in the dark scene enhancement process can be improved by adopting SIMD; and by solidifying the empirical value or the empirical parameter, the processing of calculating the empirical value or the empirical parameter can be avoided; the calculation efficiency is improved by converting the floating point type into the shaping and then calculating; in conclusion, the processing efficiency of the processor in the dark scene enhancement process can be improved, and the increment of the processor is improved.

The above description is only an example of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, and improvement made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. An image enhancement method, comprising:

traversing the coding information sequence, acquiring enhanced information and filtering information corresponding to the enhanced information aiming at the traversed coding information of the pixel points, and enhancing the traversed coding information of the pixel points by combining the enhanced information and the filtering information to obtain enhanced coding information;

2. The method according to claim 1, wherein the traversing the coding information sequence, and for the coding information of the traversed pixel point, obtaining enhancement information and filtering information corresponding to the enhancement information includes:

generating a cycle traversal instruction based on a quantity threshold, wherein the quantity threshold is determined based on the register specification and the occupied space of the coding information of the pixel points;

responding to the periodic traversal instruction, sequentially traversing the coding information of the pixel points with the number threshold and the coding information of the associated pixel points corresponding to the coding information of each traversed pixel point from the coding information sequence;

and acquiring the enhanced information and the filtering information corresponding to the enhanced information based on the associated pixel point coding information corresponding to the coding information of each traversed pixel point.

3. The method according to claim 2, wherein said obtaining the enhancement information and the filtering information corresponding to the enhancement information based on the coding information of the associated pixel corresponding to the coding information of the pixel in each traversal comprises:

shifting the coding information of each traversed pixel point and the coding information of the associated pixel point respectively;

balancing the coding information of the pixel points traversed by each shift and the coding information of the associated pixel points shifted, and determining a balancing result as initial enhancement information;

mapping the initial enhancement information based on a filtering mapping relation to map initial filtering information;

and performing reverse shift on the initial enhancement information to obtain the enhancement information, and performing reverse shift on the initial filtering information to obtain the filtering information.

4. The method according to claim 2, wherein the obtaining the enhancement information and the filtering information corresponding to the enhancement information based on the associated pixel point coding information corresponding to the coding information of the pixel point of each traversal comprises:

the coding information of each traversed pixel point and the coding information of the associated pixel point are subjected to equalization processing, and the equalization processing result is determined to be the enhancement information;

and mapping the enhanced information based on a filtering mapping relation, and determining a mapping result as the filtering information.

5. The method according to any one of claims 1 to 4, wherein the enhancing the coding information of the traversed pixel point by combining the enhancement information and the filtering information to obtain enhanced coding information includes:

acquiring a first weighting result of the enhancement information and the enhancement weight;

acquiring a second weighting result of the filtering information and the filtering weight;

and determining a combination result of the first weighting result and the second weighting result as the enhanced coding information, wherein the enhanced coding information is a result of enhancing the coding information of the traversed pixel points.

6. The method of claim 5, wherein before obtaining the first weighting result of the enhancement information and the enhancement weight, the method further comprises:

storing the enhancement weights;

the obtaining of the first weighting result of the enhancement information and the enhancement weight includes:

obtaining the enhancement information and the stored first weighting result of the enhancement weight.

7. The method of claim 2, wherein prior to said traversing said encoded information sequence, said method further comprises:

obtaining the pixel point layout of the image to be enhanced;

based on the pixel point layout, determining storage position information corresponding to each group of pixel points in the image to be enhanced;

storing storage position information corresponding to each group of pixel points in the image to be enhanced;

the step of sequentially traversing the coding information of the number threshold number of the pixel points and the coding information of the associated pixel point corresponding to the coding information of each traversed pixel point from the coding information sequence in response to the periodic traversal instruction includes:

responding to the periodic traversing instruction, and acquiring row and column information to be traversed;

determining a target group and offset position information based on the row and column information to be traversed;

determining target position information corresponding to the target group based on the stored storage position information corresponding to each group of pixel points in the image to be enhanced;

determining to-be-traversed position information based on the target position information and the bias position information;

sequentially traversing the coding information of the pixel points with the number threshold and the coding information of the associated pixel points corresponding to the coding information of each traversed pixel point from the position information to be traversed of the image coding information.

8. The method according to any one of claims 1 to 4 and 7, wherein the obtaining of the coded information sequence of the image to be enhanced comprises:

presenting a first enhancement control corresponding to the image to be enhanced;

responding to a first enhancement operation acted on the first enhancement control, and acquiring the coding information sequence of the image to be enhanced;

after the enhanced image corresponding to the image to be enhanced is decoded based on the enhanced coding information sequence, the method further includes:

presenting the enhanced image.

9. The method according to any one of claims 1 to 4 and 7, wherein before the obtaining of the coded information sequence of the image to be enhanced, the method further comprises:

presenting a second enhancement control for the played video information;

in response to a second enhancement operation acted on the second enhancement control, determining a frame image corresponding to the video information as the image to be enhanced;

replacing presentation of the frame image with the enhanced image.

10. The method according to any one of claims 1 to 4 and 7, wherein the obtaining of the coded information sequence of the image to be enhanced comprises:

acquiring a color channel image of the image to be enhanced;

acquiring a minimum color channel value of each pixel point based on the color channel image, and constructing a gray image corresponding to the image to be enhanced based on the minimum color channel value of each pixel point;

and acquiring the minimum gray value of each pixel point in a sliding window based on the gray image, and constructing the coding information sequence corresponding to the image to be enhanced based on the minimum gray value of each pixel point in the sliding window.

11. The method according to any one of claims 1 to 4 and 7, wherein said decoding an enhanced image corresponding to the image to be enhanced based on the enhancement coding information sequence comprises:

decoding an initial enhanced image based on the enhanced coding information sequence;

and performing post-processing on the initial enhanced image to obtain the enhanced image corresponding to the image to be enhanced, wherein the post-processing comprises at least one of denoising processing, sharpening processing and correction processing.

12. An image enhancement apparatus, comprising:

13. An image enhancement device, characterized by comprising:

a memory for storing executable instructions;

a processor for implementing the image enhancement method of any one of claims 1 to 11 when executing executable instructions stored in the memory.

14. A computer-readable storage medium having stored thereon executable instructions for implementing, when executed by a processor, the image enhancement method of any one of claims 1 to 11.