WO2022156167A1 - Image processing method and apparatus, and electronic device, computer-readable storage medium, computer program and program product - Google Patents

Image processing method and apparatus, and electronic device, computer-readable storage medium, computer program and program product Download PDF

Info

Publication number
WO2022156167A1
WO2022156167A1 PCT/CN2021/107017 CN2021107017W WO2022156167A1 WO 2022156167 A1 WO2022156167 A1 WO 2022156167A1 CN 2021107017 W CN2021107017 W CN 2021107017W WO 2022156167 A1 WO2022156167 A1 WO 2022156167A1
Authority
WO
WIPO (PCT)
Prior art keywords
hair
segmented
color
image
mask
Prior art date
Application number
PCT/CN2021/107017
Other languages
French (fr)
Chinese (zh)
Inventor
李祯禧
蓝劲鹏
孙文秀
Original Assignee
深圳市慧鲤科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市慧鲤科技有限公司 filed Critical 深圳市慧鲤科技有限公司
Publication of WO2022156167A1 publication Critical patent/WO2022156167A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G06V40/162—Detection; Localisation; Normalisation using pixel segmentation or colour matching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds

Definitions

  • the present application relates to the technical field of image processing, and in particular, to an image processing method and apparatus, an electronic device, a computer-readable storage medium, a computer program, and a program product.
  • the semantic information in the image is the pixel point of the object to be segmented, and then the pixel point to be segmented can be distinguished from the pixel point whose semantic information is not the object to be segmented.
  • the segmentation accuracy obtained by the current segmentation process is low.
  • Embodiments of the present application provide an image processing method and apparatus, an electronic device, a computer-readable storage medium, a computer program, and a program product.
  • an image processing method comprising:
  • the image to be processed includes the hair region of the object to be segmented
  • Pixels whose color values in the hair region are outside the hair color value range are determined as background pixels.
  • the image processing device determines whether the pixels in the hair area are hair pixels according to the range of hair color values, and can determine background pixels in the hairy area, thereby improving the segmentation accuracy of the object to be segmented in the image to be processed.
  • the obtaining of the range of hair color values of the object to be segmented includes:
  • the hair color value range is obtained according to the color value of the pixel corresponding to the second hair mask.
  • the image processing apparatus obtains a second hair mask including hair pixels with high confidence by reducing the first hair mask.
  • the image processing device obtains the hair color value range according to the color values of the pixels corresponding to the second hair mask, which can improve the accuracy of the hair color value range.
  • the object to be segmented includes a person to be segmented, and the hair includes hair.
  • the Methods also include:
  • the position of the face area determine the hair area of the person to be segmented from the image to be processed
  • the reducing the hair area in the first hair mask to obtain a second hair mask including:
  • hair pixels in the hair region of the person to be segmented are determined to obtain the second hair mask.
  • the image processing device determines the hair pixels in the hair region of the person to be segmented according to the hair mask, and can determine the hair pixels to be segmented. High-confidence hair pixels in the hair region of the character, and then obtain the second hair mask.
  • performing face detection processing on the to-be-processed image to obtain the position of the face region of the person to be segmented includes:
  • the position of the face frame of the person to be divided and the size of the face frame of the person to be divided are used as the position of the face region of the person to be divided.
  • the selection of the face frame with the largest scale in the at least one face frame as the face frame of the character to be segmented includes:
  • the face frame with the largest area in the at least one face frame is selected as the face frame of the person to be segmented.
  • the method before determining that the pixels whose color values in the hair region are outside the range of the hair color values are used as background pixels, the method further includes:
  • the first portrait mask determine the portrait pixels in the hair region of the person to be segmented, and obtain the second portrait mask
  • the determining that the color value in the hair area is outside the range of the hair color value is used as the background pixel, including:
  • the second portrait mask determine the portrait pixels in the at least one first pixel to obtain at least one second pixel
  • the at least one second pixel point in the at least one first pixel point is removed to obtain the background pixel point.
  • the image processing device can remove the portrait pixels in the at least one first pixel by removing at least one second pixel in the at least one first pixel to obtain background pixels, thereby improving the background. Pixel accuracy.
  • the range of the hair color value is obtained according to the color value of the pixel corresponding to the second hair mask, including:
  • a hair color value interval with the largest number of corresponding pixel points and the length of which is the length of the hair color value interval is obtained as the hair color value range.
  • the image to be processed further includes a non-to-be-segmented object area
  • the non-to-be-segmented object area is an area of the to-be-processed image other than the pixel area covered by the to-be-segmented object.
  • pixel area the method further includes:
  • the non-to-be-segmented object area and the pixel point area covered by the to-be-segmented object are obtained by performing segmentation processing on the to-be-processed image.
  • the pixel point area covered by the object to be segmented includes the non-hairy pixels in the hollow area, and the non-to-be-segmented object area does not include the non-hair pixels in the hollow area.
  • the image processing device can improve the color retention effect by changing the color of the background pixels and the color of the non-to-be-segmented object area, and obtain a color-retained image of the to-be-segmented object.
  • the method further includes:
  • changing the color of the background pixels and the color of the non-to-be-segmented object area to obtain a color-retained image of the to-be-segmented object includes:
  • Grayscale processing is performed on the background pixel points and the non-to-be-segmented object area to obtain the color-retained image.
  • the method further includes:
  • the optimized foreground mask determine the optimized background area in the to-be-processed image
  • the embodiments of the present application further provide an image processing apparatus, the apparatus comprising:
  • a first acquiring part configured to acquire an image to be processed; the image to be processed includes the hair region of the object to be segmented;
  • a second acquiring part configured to acquire the range of hair color values of the object to be segmented
  • the processing part is configured to determine the pixel points in the hair area whose color values are outside the hair color value range as background pixels.
  • the second obtaining part is further configured as:
  • the hair color value range is obtained according to the color value of the pixel corresponding to the second hair mask.
  • the object to be segmented includes a person to be segmented, and the hair includes hair;
  • the processing part is further configured to perform face detection processing on the to-be-processed image to obtain the to-be-segmented image before reducing the hair region in the first hair mask to obtain the second hair mask the location of the face area of the character;
  • the second acquisition part is also configured as:
  • hair pixels in the hair region of the person to be segmented are determined to obtain the second hair mask.
  • processing part is further configured as:
  • the position of the face frame of the person to be divided and the size of the face frame of the person to be divided are used as the position of the face region of the person to be divided.
  • processing part is further configured as:
  • the face frame with the largest area in the at least one face frame is selected as the face frame of the person to be segmented.
  • the processing part is further configured to, before determining that the pixels whose color values in the hair region are outside the range of the hair color values are used as background pixels, perform the processing on the pixels.
  • the to-be-processed image is subjected to portrait segmentation to obtain a first portrait mask;
  • the first portrait mask determine the portrait pixels in the hair region of the person to be segmented, and obtain the second portrait mask
  • the second obtaining part is further configured as:
  • a hair color value interval with the largest number of corresponding pixel points and the length of which is the length of the hair color value interval is obtained as the hair color value range.
  • the image to be processed further includes a non-to-be-segmented object area, and the non-to-be-segmented object area is an area of the to-be-processed image other than the pixel area covered by the to-be-segmented object. pixel area;
  • the processing part is further configured to change the color of the background pixels and the color of the non-to-be-segmented object area to obtain a color-retained image of the to-be-segmented object;
  • the color of the object area is the same as the color of the background pixels.
  • processing part is further configured as:
  • processing part is further configured as:
  • Grayscale processing is performed on the background pixel points and the non-to-be-segmented object area to obtain the color-retained image.
  • processing part is further configured as:
  • the optimized foreground mask determine the optimized background area in the to-be-processed image
  • an electronic device comprising: a processor and a memory, the memory being configured to store computer program code, the computer program code comprising computer instructions, the processor executing the computer instructions
  • the electronic device executes the method of the first aspect.
  • another electronic device comprising: a processor, a transmitting device, an input device, an output device, and a memory, the memory being configured to store computer program code, the computer program code including computer instructions, in which When the processor executes the computer instructions, the electronic device executes the method of the first aspect.
  • a fifth aspect provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, the computer program includes program instructions, and when the program instructions are executed by a processor, the The processor performs the method of the first aspect as described above.
  • a computer program product in a sixth aspect, includes a computer program or instructions, and when the computer program or instructions are run on a computer, the computer executes the method of the first aspect.
  • a computer program in a seventh aspect, includes a computer program or instructions, and when the computer program or instructions are run on a computer, the computer executes the method of the first aspect.
  • FIG. 1A is a schematic diagram 1 of a hollow area provided by an embodiment of the present application.
  • FIG. 1B is a second schematic diagram of a hollow area provided by an embodiment of the present application.
  • FIG. 2 is a schematic flowchart of an image processing method provided by an embodiment of the present application.
  • FIG. 3 is a schematic diagram of a second hair mask provided in an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a second portrait mask provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of an optimized foreground mask provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a to-be-processed image provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a middle portrait segmentation mask provided by an embodiment of the present application.
  • FIG. 8 is a schematic diagram of an optimized foreground mask provided by an embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application.
  • FIG. 10 is a schematic diagram of a hardware structure of an image processing apparatus according to an embodiment of the present application.
  • At least one (item) refers to one or more
  • multiple refers to two or more
  • at least two (item) refers to two or more
  • three or more, "and/or” is used to describe the association relationship of related objects, indicating that three kinds of relationships can exist, for example, "A and/or B” can mean: only A exists, only B exists, and at the same time There are three cases of A and B, where A and B can be singular or plural.
  • the character “/" can indicate that the related objects before and after are an "or” relationship, which refers to any combination of these items, including any combination of single item(s) or plural items(s).
  • At least one (a) of a, b or c can mean: a, b, c, "a and b", “a and c", “b and c", or "a and b and c” ", where a, b, c can be single or multiple.
  • Segmentation processing refers to determining that the semantic information in the image is the pixel point of the object to be segmented. For example, when the object to be segmented is a portrait, by performing portrait segmentation processing on the image, it can be determined that the semantic information in the image is the pixels of the portrait, and then the portrait pixels can be distinguished from non- portrait pixels.
  • the pixels whose semantic information is the object to be segmented are called foreground pixels, and the pixels whose semantic information is not the object to be segmented are called background pixels.
  • non-hairy pixels refer to pixels whose semantic information is not hairy.
  • FIG. 1A shows the hollowed-out area of the hair of the portrait obtained by adopting the method for segmenting the object to be segmented in the related art.
  • the embodiments of the present application provide a technical solution to reduce the number of background pixels in the hollow area that are mistakenly identified as foreground pixels, thereby improving segmentation accuracy.
  • the mask refers to an image-specific semantic mask.
  • Elements in the image-specific semantic mask correspond one-to-one with pixels in the image.
  • the values of the elements in the mask include a first value and a second value, wherein the first value is different from the second value.
  • the first value indicates that the semantics of the pixel corresponding to the element is a specific semantic
  • the second value indicates that the semantics of the pixel corresponding to the element is not a specific semantic.
  • the first value is 1, and the second value is 0.
  • image I includes pixel A, pixel B, pixel C, and pixel D.
  • the hair mask of image 1 includes element a, element b, element c, and element d, that is, the specific semantic is hair.
  • the element a corresponds to the pixel point A
  • the element b corresponds to the pixel point B
  • the element c corresponds to the pixel point C
  • the element d corresponds to the pixel point D.
  • the first value is 1 and the second value is 0. If the value of element a is 1, the value of element b is 0, the value of element c is 1, and the value of element d is 0.
  • the semantics of pixel A is hair, that is, pixel A is a hair pixel; the semantics of pixel B is not hair, that is, pixel B is not a hair pixel; the semantics of pixel C is hair, that is, pixel C is hair Pixel point; the semantics of pixel point D is not hair, that is, pixel point D is not a hair pixel point.
  • [ ⁇ , ⁇ ] represents a value interval that is greater than or equal to ⁇ and less than or equal to ⁇ .
  • the execution subject of the embodiments of the present application is an image processing apparatus, where the image processing apparatus may be any electronic device that can execute the technical solutions disclosed in the method embodiments of the present application.
  • the image processing apparatus may be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, a vehicle devices, wearables, etc.
  • UE User Equipment
  • PDA Personal Digital Assistant
  • a handheld device a computing device
  • vehicle devices wearables, etc.
  • it can be one of the following: mobile phone, computer, server, tablet.
  • FIG. 2 is a schematic flowchart of an image processing method provided by an embodiment of the present application.
  • the image to be processed includes an object to be segmented, where the object to be segmented may be any object.
  • the object to be segmented may be a person; for another example, the object to be segmented may be a sweater; for another example, the object to be segmented may be a sweater and a person.
  • the object to be segmented includes a hair region, where the hair region includes at least one of the following: a body hair region, a fluff region, and a feather region.
  • the hair region includes at least one of the following: a body hair region, a fluff region, and a feather region.
  • the hair area can be a body hair area; for another example, assuming that the object to be segmented is a dandelion, the hair area can be a fluff area; for another example, assuming that the object to be segmented is a bird, the hair area can be a feather area;
  • the hair region may include body hair regions and feather regions; for another example, assuming that the objects to be segmented are people, birds, and dandelions, the hair regions may include body hair regions, feather regions, and fluff regions.
  • the image processing apparatus receives the to-be-processed image input by the user through the input component, and acquires the to-be-processed image.
  • the above input components include: keyboard, mouse, touch screen, touch pad, audio input and so on.
  • the image processing apparatus receives the to-be-processed image sent by the terminal to acquire the to-be-processed image.
  • the terminal may be any of the following: a mobile phone, a computer, a tablet computer, a server, and a wearable device.
  • the image processing apparatus is loaded with a camera assembly, wherein the camera assembly includes a camera.
  • the image processing apparatus collects an image containing the object to be segmented as an image to be processed by using a camera assembly.
  • the image processing apparatus selects one frame of image as the image to be processed from the acquired video stream.
  • the color value may be one of the following: a pixel value and a grayscale value.
  • the hair color value of the object to be segmented refers to the color value of the hair pixels of the object to be segmented in the image to be processed.
  • the hair color value of the object to be segmented may be the pixel value of Zhang San's hair pixel in the image to be processed; for another example, suppose that the object to be segmented is a bird, and the hair of the object to be segmented is a feather of a bird. Then the hair color value of the object to be segmented can be the gray value of the feather pixel of the bird in the image to be processed.
  • the range of hair color values is used to determine whether a pixel is a hair pixel.
  • a pixel whose color value is within the hair color value range can be regarded as a hair pixel. That is, the range of hair color values can be regarded as a reference range of color values of hair pixels of the object to be segmented.
  • the range of hair color values is the gray value range of hair pixels, and the gray value range of hair pixels is [50, 70]. Then the pixels whose gray values are within [50, 70] are hair pixels, and those whose gray values are outside [50, 70] are not hair pixels.
  • the range of coat color values can be a single number.
  • the range of fur color value is [60, 60], in this case, the fur color value is 60. Then the pixels whose color value is not equal to 60 are not hair pixels, and the pixels whose color value is 60 are hair pixels.
  • the image processing device obtains the hair pixel points of the object to be segmented by performing hair segmentation processing on the image to be processed.
  • the histogram of the color values of the hair pixels of the object to be divided is counted, and the hair color value range with the largest number of pixels in the histogram is used as the hair color value range of the object to be divided.
  • the image processing apparatus takes the hair color value range input by the user through the input component as the hair color value range of the object to be segmented.
  • the image processing apparatus uses the range of hair color values sent by the terminal as the range of hair color values of the object to be divided.
  • S203 Determine the pixel points in the hair region whose color values are outside the hair color value range, and obtain background pixels.
  • the image processing device can determine whether the pixels in the hair area are hair pixels according to the hair color value range and the color values of the pixels in the hair area.
  • pixels whose color values are determined to be within the range of hair color values are hair pixels; pixels whose color values are determined to be outside the range of hair color values are not hair pixels (ie, background pixels).
  • the hair region in this embodiment is obtained by performing hair segmentation processing on the image.
  • the hair region is a hair region.
  • the hair region can be determined.
  • the execution body of the hair segmentation process performed on the image to be processed may be an image processing device, or may not be an image processing device.
  • the terminal determines the hair pixels in the image a by performing hair segmentation processing on the image a, and marks the hair pixels in the image a to obtain the image to be processed.
  • the terminal sends the image to be processed to the image processing device, so that the hair region has been marked in the image to be processed obtained by the image processing device.
  • the image processing device determines whether the pixels in the hair area are hair pixels according to the range of hair color values, and can Determine the background pixels in the hair area, thereby improving the segmentation accuracy of the object to be segmented in the image to be processed.
  • the image processing apparatus performs the following steps in the process of performing step 202:
  • the hair segmentation process is used to determine pixels whose semantic information is hair from the image.
  • the hair segmentation process is hair segmentation process.
  • the semantic information in the image is the pixel points of hair; for another example, it is assumed that the hair segmentation processing is feather segmentation processing.
  • feather segmentation processing By performing feather segmentation processing on the image, the semantic information in the image can be determined as the pixels of feathers.
  • the elements in the hair mask correspond to the pixels in the image to be processed one-to-one, and the pixels corresponding to the elements with the first value in the hair mask are hair pixels.
  • the pixel corresponding to the element whose value is the second value is not a hair pixel.
  • the hair region in the first hair mask refers to the region formed by the elements whose value is the first value. Reducing the hair area in the first hair mask refers to reducing the border of the hair area toward the center of the hair area.
  • the edge of the hair area obtained by the hair segmentation process is prone to segmentation errors (that is, the non-hair pixels are identified as hair pixels), the confidence level of the pixels at the edge of the hair area as hair pixels is low (the hair area will be described below).
  • the pixels at the edge are called low-confidence hair pixels), and the pixels other than the low-confidence hair pixels in the hair area are hair pixels with higher confidence (hereinafter referred to as high-confidence hair pixels).
  • the image processing device obtains the second hair mask by reducing the hair area in the first hair mask, and can update the value of the non-hair pixel point in the first hair mask to the second value, so that the low
  • the confidence hair pixels are eliminated from the pixels corresponding to the first hair mask, and the high confidence hair pixels are retained in the pixels corresponding to the second hair mask.
  • the image processing apparatus may reduce the border of the hair region toward the center to obtain the second hair mask. Assuming that the first mask includes the hair area a, the image processing apparatus obtains the hair area b in the second hair mask by reducing the hair area a toward the center by n pixels.
  • the area of the hairy area b/the area of the hairy area a m, optionally, the value range of m is [10/11, 12/13].
  • the image processing apparatus calculates the average value of the color values of the pixels corresponding to the second hair mask as the hair color value range.
  • the image processing apparatus calculates the median value of the color values of the pixel points corresponding to the second hair mask as the hair color value range.
  • the image processing device counts a histogram of the color values of the pixels corresponding to the second hair mask, and uses the hair color value range with the largest number of pixels in the histogram as the hair color of the object to be segmented value range.
  • the image processing device obtains the hair color value range according to the color values of the pixels corresponding to the second hair mask, which can increase the hair color value range accuracy.
  • the object to be segmented includes a person to be segmented
  • the hair of the object to be segmented includes the hair of the person to be segmented
  • S304 Perform face detection processing on the above-mentioned image to be processed to obtain the position of the above-mentioned face region of the person to be segmented.
  • the image processing apparatus may perform face detection processing on the image to be processed to determine whether the image to be processed includes a face region.
  • face detection processing on the image to be processed to determine whether the image to be processed includes a face region.
  • the position of the human face region can also be determined.
  • the position of the face region refers to the coordinates of the face region in the pixel coordinate system of the image to be processed.
  • the image processing apparatus may obtain a face frame including the face region by performing face detection processing on the image to be processed, and use the position of the face frame in the image to be processed as the position of the face region.
  • the position of the face frame in the image to be processed includes: coordinates of any vertex in the face frame in the pixel coordinate system of the image to be processed, and the size of the face frame.
  • the position of the face frame includes: the coordinates of the upper left vertex of the face frame in the pixel coordinate system of the image to be processed, the length of the face frame, and the width of the face frame.
  • the image processing apparatus obtains face feature data of the image to be processed by performing face feature extraction processing on the image to be processed. Further, it is possible to determine whether the image to be processed includes a human face region according to the face feature data, and determine the position of the human face region if the image to be processed includes a human face region.
  • the image processing apparatus uses a face detection neural network to process the image to be processed, so as to determine whether the image to be processed contains a face area, and the image to be processed is If the face area is included, the position of the face area is determined.
  • the face detection neural network is obtained by training a training image with annotation information as training data, wherein the annotation information includes whether the training image contains a human face, and the position of the face frame corresponding to the face area.
  • the image processing apparatus regards the face area as the face area of the person to be segmented; when the number of face areas contained in the image to be processed exceeds 1, the image processing apparatus One can be selected from all face regions as the face region of the person to be segmented.
  • S305 Determine the hair region of the person to be segmented from the image to be processed according to the position of the face region.
  • the hair region of the person to be segmented refers to the hair region at the top of the head of the person to be segmented. Since the relative positional relationship between the hair region and the face region is an empirical relationship, and the relationship between the size of the face region and the size of the hair region is also an empirical relationship, the image processing device can determine the to-be-segmented based on the position of the face region The character's hair area.
  • the position of the face region includes: the coordinates of the upper left vertex of the face frame in the pixel coordinate system, the length of the face frame and the width of the face frame.
  • the image processing device obtains the coordinates of the upper left vertex of the hair frame in the pixel coordinate system according to the coordinates of the upper left vertex of the face frame and the positional relationship between the face frame and the hair frame, wherein the hair frame contains the hair region of the person to be segmented.
  • the image processing device obtains the length of the hair frame according to the length of the face frame, and obtains the width of the hair frame according to the width of the face frame. In this way, the image processing apparatus can determine the position of the hair frame and the size of the hair frame, and then determine the hair region from the image to be processed.
  • the abscissa of the upper left vertex of the face frame is x f
  • the ordinate of the upper left vertex of the face frame is y f
  • the length of the face frame is h f
  • the width of the face frame is w f
  • the upper left of the hair frame is The abscissa of the vertex is x h
  • the ordinate of the upper left vertex of the hair frame is y h
  • the length of the hair frame is h h
  • the width of the hair frame is w h .
  • ⁇ , ⁇ , ⁇ , and ⁇ are all positive numbers.
  • the value ranges of ⁇ , ⁇ , ⁇ , and ⁇ are all [1.1, 1.2].
  • the image processing apparatus After obtaining the hair region of the person to be segmented, the image processing apparatus performs the following steps in the process of performing step 302:
  • step 302 The implementation manner of this step is the same as that of step 302 .
  • step 302 is that in this step, the first hair mask is the hair mask of the image to be processed, and the hair area in the first hair mask is the hair area.
  • S307. Determine the hair pixels in the hair region of the person to be segmented according to the hair mask, and obtain the second hair mask.
  • the pixels corresponding to the first hair mask may also include hair pixels other than the hair region of the person to be segmented. Obviously, determining the hair color value range of the person to be segmented according to the color values of the hair pixels other than the hair region of the person to be segmented will reduce the accuracy of the hair color value range of the person to be segmented.
  • the image processing device determines the hair pixels in the hair region of the person to be segmented according to the hair mask, and can determine the high-confidence hair pixels in the hair region of the person to be segmented. Confidence hair pixels, and then get the second hair mask. That is, the pixels corresponding to the second hair mask are high-confidence hair pixels in the hair region of the person to be segmented.
  • the hair mask includes element a, element b, and element c, wherein the value of element a is the first value, the value of element b is the second value, and the value of element b is the third value.
  • the hair region of the person to be segmented includes pixel A, pixel B, pixel C, and pixel D. If the pixel point corresponding to element a is pixel point A, the pixel point corresponding to element b is pixel point B, and the pixel point corresponding to element c is pixel point C, then pixel point A and pixel point C are the characters to be segmented. High-confidence hair pixels in the hair region of .
  • the image processing device obtains the hair color value range of the object to be segmented according to the color value of the pixel point corresponding to the second hair mask, which can improve the accuracy of the hair color value range.
  • the image processing apparatus performs the following steps in the process of performing step 304:
  • Each of the above-mentioned at least one face frame includes a face area, and any two face frames include different face areas.
  • S309 Select the face frame with the largest scale in the at least one face frame as the face frame of the person to be segmented.
  • the scale of the near object in the image is large, and the scale of the distant object in the image is small.
  • "far” refers to the distance between the real object corresponding to the object and the imaging device that collects the above image in the image
  • “near” refers to the distance between the real object corresponding to the object in the image and the imaging device that collects the above image. close distance.
  • the face contained in the face frame with the largest scale is the face closest to the imaging device that collects the image to be processed.
  • the image processing apparatus selects the face frame with the largest scale from at least one face frame as the face frame of the person to be segmented, so that the above-mentioned photographing object can be regarded as the person to be segmented.
  • the image processing device determines that the face contained in the face frame with the largest scale in the photo is Xiaoming's face, and performs segmentation processing on Xiaoming's face area.
  • the image processing apparatus separately determines the measure of each face frame, and uses the face frame with the largest measurement as the face frame with the largest scale.
  • the measurement of the face frame refers to the maximum side length of the face frame. For example, the length of the face frame aa is 10, and the width of the face frame aa is 6, then the measurement of the face frame aa is 10; for another example, the length of the face frame bb is 7, and the width of the face frame bb is 9, then The measure of the face frame bb is 9.
  • the image processing apparatus performs the following steps in the process of performing step 309:
  • the image processing apparatus before performing step 203, the image processing apparatus further performs the following steps:
  • S312 Perform portrait segmentation processing on the above-mentioned to-be-processed image to obtain a first portrait mask.
  • the portrait segmentation processing in this step is used to determine pixels whose semantic information in the image to be processed is a portrait (hereinafter referred to as portrait pixels).
  • the image processing apparatus determines the portrait pixels in the image to be processed by executing step 312, and obtains the first portrait mask. That is, the pixel corresponding to the element whose value is the first value in the first portrait mask is the portrait pixel.
  • the implementation manner of determining the high-confidence hair pixels in the hair region of the person to be segmented is the same as in step 307 , according to the hair mask.
  • the image processing device determines the portrait pixels in the hair region of the person to be segmented according to the first portrait mask, and obtains the second portrait mask. That is, the pixel corresponding to the element whose value is the first value in the second portrait mask is the portrait pixel in the hair region of the person to be segmented.
  • the image processing apparatus After obtaining the second portrait mask, the image processing apparatus performs the following steps in the process of performing step 203:
  • S314 Determine the pixels whose color values are outside the range of the hair color values in the hair region, and obtain at least one first pixel.
  • step 203 The implementation of this step is the same as that of step 203 . It should be understood that, in this step, what is obtained is not a background pixel, but at least one first pixel.
  • the image processing apparatus determines, from the at least one first pixel point, a pixel point corresponding to an element in the second portrait mask whose value is the first value, as a second pixel point.
  • the second portrait mask includes element a and element b, wherein the value of element a is the first value, and the value of element b is the second value.
  • At least one first pixel point includes pixel point A and pixel point B. If the pixel point A is the pixel point corresponding to the element a, and the pixel point B is the pixel point corresponding to the element b, then at least one second pixel point includes the pixel point A.
  • the hair pixel is determined as the first pixel. In this way, at least one of the first pixels will include hair pixels.
  • the image processing apparatus can determine at least one hair pixel (ie, the second pixel) in the first pixel by executing step 315 .
  • the hair region (ie, the hair region) of the person to be segmented includes pixel a, pixel b, and pixel c, where pixel a and pixel b are both hair pixels, and pixel c is a background pixel. Since the color value of the pixel point b is outside the range of the hair color value, the image processing apparatus determines the pixel point b as the background pixel point, and uses the pixel point b as the second pixel point. At this time, the at least one second pixel point includes pixel point b and pixel point c.
  • the second portrait mask includes element U, element V, and element W, wherein the value of element U and the value of element V are both the first value, the value of element W is the second value, and the value corresponding to element U
  • the pixel point is the pixel point u
  • the pixel point corresponding to the element V is the pixel point v
  • the pixel point corresponding to the element W is the pixel point w.
  • the image processing device can determine that the pixel point v is a portrait pixel point (ie, a hair pixel point) of the hair region, thereby determining that the pixel point v is the second pixel point.
  • the image processing apparatus can improve the accuracy of the background pixel by performing step 316 .
  • the image processing apparatus performs the following steps in the process of performing step 303:
  • the length of the hair color value interval refers to the length of the interval corresponding to the hair color value range.
  • the range of fur color values is [50, 80], at this time, the span of the fur color value range is 30.
  • the color values include grayscale values
  • the color value histogram includes a grayscale histogram
  • the color-retaining object In the field of image processing, in order to highlight an object in an image (hereinafter referred to as a color-retaining object), the color-retaining object is usually subjected to color-retaining processing.
  • the color-retaining object by changing the color of the pixel area (hereinafter referred to as the non-to-be-segmented object area) in the image except for the color-retaining object, and retaining the pixel area covered by the color-retaining object (hereinafter referred to as the foreground area) Color to achieve the effect of highlighting objects with color.
  • a portrait-retaining image can be obtained by performing a portrait color-retaining process on the image, retaining the color of the portrait, and changing the color of the non-portrait area to gray.
  • the portrait in the color-preserving portrait image, the portrait can be highlighted, and the non-to-be-segmented object area can be weakened.
  • the image processing apparatus can also perform color retention processing on the object to be segmented, so as to improve the color retention effect.
  • the image processing apparatus further performs the following steps:
  • the image processing device can change the color of the background pixels and the color of the non-to-be-divided object area to any color.
  • the image processing device may change the color of the background pixels and the color of the non-to-be-divided object area to black; for another example, the image processing device may change both the background pixel color and the color of the non-to-be-divided object area to white
  • the image processing device may change the color of the background pixels and the color of the non-to-be-divided object area to red.
  • the image processing apparatus will also retain the color of the foreground area. In this way, a color image of the object to be segmented can be obtained.
  • the image processing apparatus may also perform beautification processing on the foreground area.
  • the image processing apparatus after performing step 320, the image processing apparatus further performs the following steps:
  • the image processing device can further highlight the object to be segmented in the color image by blurring the background pixels in the color image and the non-to-be-segmented area in the color image, so as to obtain the color-retention blur of the object to be segmented. image.
  • the image processing apparatus performs the following steps in the process of performing step 320:
  • the image processing apparatus when the image processing apparatus performs step 313 to obtain the second hair mask, the image processing apparatus further performs the following steps:
  • the image processing device determines the union of the elements with the first value in the second hair mask and the second portrait mask by performing fusion processing on the second hair mask and the second portrait mask, to obtain The optimized foreground mask.
  • Figure 3 shows the second hair mask
  • Figure 4 shows the second portrait mask.
  • the image processing device takes the union of the second hair mask and the second portrait mask to obtain the optimized foreground mask shown in FIG. 5 .
  • S324 Determine the optimized background area in the above-mentioned image to be processed according to the above-mentioned optimized foreground mask.
  • the optimized background area includes the pixels corresponding to the elements in the optimized foreground mask whose value is the second value, that is, the pixels other than the pixels corresponding to the optimized foreground mask.
  • the first value is 1, and the second value is 0.
  • the optimized background area includes pixels corresponding to elements whose value is 0.
  • the portrait mask of the person to be segmented obtained is called an intermediate portrait segmentation mask.
  • the intermediate portrait segmentation mask is processed to obtain an optimized foreground mask, and the background pixels in the intermediate portrait segmentation mask that are mistakenly identified as foreground pixels can be corrected.
  • pixel A, pixel B, pixel C, pixel D, pixel E, pixel H, pixel J, and pixel K are all background pixels
  • the pixel point F, the pixel point G, the pixel point I, the pixel point L, the pixel point M, the pixel point N, the pixel point O, and the pixel point P are all portrait pixels
  • the pixel point J and the pixel point K are the hollow area. background pixels inside.
  • the optimized foreground mask shown in FIG. 8 is obtained.
  • the pixels corresponding to the elements with a value of 1 are portrait pixels
  • the pixel corresponding to the element whose value is 0 is the background pixel.
  • pixel point J and pixel point K are corrected as background pixels.
  • the image processing apparatus obtains a color-retaining image by changing the color of the optimized background area, which can improve the color-retaining effect.
  • the image processing apparatus may perform the following steps in the process of performing step 322:
  • S327 Determine the optimized background area in the above-mentioned image to be processed according to the above-mentioned optimized foreground mask.
  • the image processing apparatus obtains the color-preserving image by executing steps 326 to 328, which can reduce the amount of data processing and improve the processing speed.
  • the writing order of each step does not mean a strict execution order and constitutes any limitation on the implementation process, and the execution order of each step should be determined by its function and possible internal logic.
  • FIG. 9 is a schematic structural diagram of an image processing apparatus provided by an embodiment of the application.
  • the image processing apparatus 1 includes: a first acquisition part 11, a second acquisition part 12, and a processing part 13, wherein:
  • the first acquisition part 11 is configured to acquire an image to be processed; the image to be processed includes the hair region of the object to be segmented;
  • the second acquiring part 12 is configured to acquire the range of hair color values of the object to be segmented
  • the processing part 13 is configured to determine the pixels whose color values are outside the range of the hair color values in the hair area as background pixels.
  • the second obtaining part 12 is further configured as:
  • the hair color value range is obtained according to the color value of the pixel corresponding to the second hair mask.
  • the object to be segmented includes a person to be segmented, and the hair includes hair;
  • the processing part 13 is further configured to perform face detection processing on the to-be-processed image before the hair region in the first hair mask is reduced to obtain the second hair mask to obtain the to-be-processed image. the position of the face area of the segmented character; and, also configured to determine the hair area of the character to be segmented from the to-be-processed image according to the location of the face area;
  • the second obtaining part 12 is further configured to:
  • hair pixels in the hair region of the person to be segmented are determined to obtain the second hair mask.
  • processing part 13 is further configured to:
  • the position of the face frame of the person to be divided and the size of the face frame of the person to be divided are used as the position of the face region of the person to be divided.
  • processing part 13 is further configured to:
  • the face frame with the largest area in the at least one face frame is selected as the face frame of the person to be segmented.
  • the processing part 13 is further configured to, before determining that the pixel points in the hair area whose color values are outside the hair color value range are used as background pixels, Performing portrait segmentation processing on the to-be-processed image to obtain a first portrait mask;
  • the first portrait mask determine the portrait pixels in the hair region of the person to be segmented, and obtain the second portrait mask
  • the second obtaining part 12 is further configured as:
  • a hair color value interval with the largest number of corresponding pixel points and the length of which is the length of the hair color value interval is obtained as the hair color value range.
  • the image to be processed further includes a non-to-be-segmented object area, and the non-to-be-segmented object area is an area of the to-be-processed image other than the pixel area covered by the to-be-segmented object. pixel area;
  • the processing part 13 is further configured to change the color of the background pixels and the color of the non-to-be-segmented object area to obtain a color-retained image of the to-be-segmented object;
  • the color of the segmented object area is the same as the color of the background pixels.
  • processing part 13 is further configured to:
  • processing part 13 is further configured to:
  • Grayscale processing is performed on the background pixel points and the non-to-be-segmented object area to obtain the color-retained image.
  • processing part 13 is further configured to:
  • the optimized foreground mask determine the optimized background area in the to-be-processed image
  • first acquisition part 11 and the second acquisition part 12 may be the same data interface, the first acquisition part 11 and the second acquisition part 12 may also be different data interfaces, and the processing part 13 may be a processor.
  • the functions or modules included in the apparatus provided in the embodiments of the present application may be used to execute the methods described in the above method embodiments, and reference may be made to the descriptions in the above method embodiments for specific implementation.
  • a "part" may be a part of a circuit, a part of a processor, a part of a program or software, etc., of course, a unit, a module or a non-modularity.
  • FIG. 10 is a schematic diagram of a hardware structure of an image processing apparatus according to an embodiment of the present application.
  • the image processing device 2 includes a processor 21 , a memory 22 , an input device 23 , and an output device 24 .
  • the processor 21, the memory 22, the input device 23, and the output device 24 are coupled through a connector, and the connector includes various types of interfaces, transmission lines, or buses, etc., which are not limited in this embodiment of the present application. It should be understood that, in various embodiments of the present application, coupling refers to mutual connection in a specific manner, including direct connection or indirect connection through other devices, such as various interfaces, transmission lines, and buses.
  • the processor 21 may be one or more graphics processing units (graphics processing units, GPUs).
  • the GPU may be a single-core GPU or a multi-core GPU.
  • the processor 21 may be a processor group composed of multiple GPUs, and the multiple processors are coupled to each other through one or more buses.
  • the processor may also be another type of processor, etc., which is not limited in this embodiment of the present application.
  • the memory 22 may be configured to store computer program instructions, as well as various types of computer program code, including program code configured to execute the aspects of the present application.
  • the memory includes, but is not limited to, random access memory (RAM), read-only memory (read-only memory, ROM), erasable programmable read-only memory (erasable programmable read only memory, EPROM) ), or a portable disc read-only memory (CD-ROM), which is configured for associated instructions and data.
  • RAM random access memory
  • ROM read-only memory
  • EPROM erasable programmable read only memory
  • CD-ROM portable disc read-only memory
  • the input device 23 is configured to input data and/or signals
  • the output device 24 is configured to output data and/or signals.
  • the input device 23 and the output device 24 may be independent devices or may be an integral device.
  • the memory 22 may be configured not only to store relevant instructions, but also to store relevant data.
  • the memory 22 may be configured to store images to be processed obtained through the input device 23 , or The memory 22 may also be configured to store background pixels and the like obtained by the processor 21, and the data stored in the memory is not limited in this embodiment of the present application.
  • FIG. 10 only shows a simplified design of an image processing apparatus.
  • the image processing apparatus may also include other necessary elements, including but not limited to any number of input/output devices, processors, memories, etc., and all image processing apparatuses that can implement the embodiments of the present application are included in this application. within the scope of protection of the application.
  • the disclosed system, apparatus and method may be implemented in other manners.
  • the apparatus embodiments described above are only illustrative.
  • the division of the parts is only a logical function division.
  • there may be other division methods for example, multiple parts or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented.
  • the shown or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or parts, and may be in electrical, mechanical or other forms.
  • the parts described as separate parts may or may not be physically separated, and the parts shown as parts may or may not be physical parts, that is, they may be located in one place, or may be distributed to multiple network parts. Some or all of them may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • each functional part in each embodiment of the present application may be integrated into one processing part, or each part may exist physically alone, or two or more parts may be integrated into one part.
  • the above-mentioned embodiments it may be implemented in whole or in part by software, hardware, firmware or any combination thereof.
  • software it can be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated.
  • the computer may be a general purpose computer, special purpose computer, computer network, or other programmable device.
  • the computer instructions may be stored in or transmitted over a computer-readable storage medium.
  • the computer instructions can be sent from a website site, computer, server, or data center via wired (eg, coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.) another website site, computer, server or data center for transmission.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes an integration of one or more available media.
  • the available media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, digital versatile discs (DVDs)), or semiconductor media (eg, solid state disks (SSDs)) )Wait.
  • the process can be completed by instructing the relevant hardware by a computer program, and the program can be stored in a computer-readable storage medium.
  • the program When the program is executed , which may include the processes of the foregoing method embodiments.
  • the aforementioned storage medium includes: read-only memory (read-only memory, ROM) or random access memory (random access memory, RAM), magnetic disk or optical disk and other media that can store program codes.
  • the embodiments of the present application disclose an image processing method and apparatus, an electronic device, a computer-readable storage medium, a computer program, and a program product.
  • the method includes: acquiring an image to be processed; the image to be processed includes a hair area of an object to be segmented; acquiring a range of hair color values of the object to be segmented; determining that the color value in the hair area is within the range of the hair color value
  • the outer pixels are used as background pixels. Since the hair color value range can be regarded as the reference range of the color values of the hair pixels of the object to be segmented, the image processing device can determine whether the pixels in the hair area are hair pixels according to the hair color value range, and can determine the background in the hair area. pixel points, thereby improving the segmentation accuracy of the object to be segmented in the image to be processed.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)
  • Color Image Communication Systems (AREA)

Abstract

An image processing method and apparatus, and an electronic device, a computer-readable storage medium, a computer program and a program product. The method comprises: acquiring an image to be subjected to processing, wherein said image comprises a hair region of an object to be segmented (S201); acquiring a hair color value range of said object (S202); and determining a pixel point, with a color value in the hair region being beyond the hair color value range, as a background pixel point (S203).

Description

图像处理方法及装置、电子设备、计算机可读存储介质、计算机程序及程序产品Image processing method and apparatus, electronic device, computer-readable storage medium, computer program and program product
相关申请的交叉引用CROSS-REFERENCE TO RELATED APPLICATIONS
本申请基于申请号为202110090411.4、申请日为2021年01月22日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本申请作为参考。This application is based on the Chinese patent application with the application number of 202110090411.4 and the filing date of January 22, 2021, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is incorporated herein by reference.
技术领域technical field
本申请涉及图像处理技术领域，尤其涉及一种图像处理方法及装置、电子设备、计算机可读存储介质、计算机程序及程序产品。The present application relates to the technical field of image processing, and in particular, to an image processing method and apparatus, an electronic device, a computer-readable storage medium, a computer program, and a program product.
背景技术Background technique
通过对图像进行分割处理，可确定图像中语义信息为待分割对象的像素点，进而将待分割像素点与语义信息不是待分割对象的像素点区分开。但通过目前的分割处理得到的分割准确度低。By segmenting the image, it can be determined that the semantic information in the image is the pixel point of the object to be segmented, and then the pixel point to be segmented can be distinguished from the pixel point whose semantic information is not the object to be segmented. However, the segmentation accuracy obtained by the current segmentation process is low.
发明内容SUMMARY OF THE INVENTION
本申请实施例提供一种图像处理方法及装置、电子设备、计算机可读存储介质、计算机程序及程序产品。Embodiments of the present application provide an image processing method and apparatus, an electronic device, a computer-readable storage medium, a computer program, and a program product.
第一方面，提供了一种图像处理方法，所述方法包括：In a first aspect, an image processing method is provided, the method comprising:
获取待处理图像；所述待处理图像包含待分割对象的毛区域；Obtain an image to be processed; the image to be processed includes the hair region of the object to be segmented;
获取所述待分割对象的毛颜色值范围；Obtain the range of hair color values of the object to be segmented;
确定所述毛区域内颜色值处于所述毛颜色值范围之外的像素点，作为背景像素点。Pixels whose color values in the hair region are outside the hair color value range are determined as background pixels.
在该方面中，由于毛颜色值范围可视为待分割对象的毛像素点的颜色值的参考范围，图像处理装置依据毛颜色值范围判断毛区域内的像素点是否为毛像素点，可确定毛区域内的背景像素点，从而提高待处理图像中待分割对象的分割准确度。In this aspect, since the range of hair color values can be regarded as the reference range of the color values of the hair pixels of the object to be divided, the image processing device determines whether the pixels in the hair area are hair pixels according to the range of hair color values, and can determine background pixels in the hairy area, thereby improving the segmentation accuracy of the object to be segmented in the image to be processed.
结合本申请任一实施方式，所述获取所述待分割对象的毛颜色值范围，包括：In conjunction with any embodiment of the present application, the obtaining of the range of hair color values of the object to be segmented includes:
对所述待处理图像进行毛分割处理，得到第一毛掩码；Perform hair segmentation processing on the to-be-processed image to obtain a first hair mask;
缩小所述第一毛掩码中的毛区域，得到第二毛掩码；reducing the hair area in the first hair mask to obtain a second hair mask;
依据所述第二毛掩码所对应的像素点的颜色值，得到所述毛颜色值范围。The hair color value range is obtained according to the color value of the pixel corresponding to the second hair mask.
在该种实施方式中，图像处理装置通过缩小第一毛掩码，得到包含高置信度毛像素点的第二毛掩码。这样，图像处理装置依据第二毛掩码所对应的像素点的颜色值，得到毛颜色值范围，可提高毛颜色值范围的准确度。In this embodiment, the image processing apparatus obtains a second hair mask including hair pixels with high confidence by reducing the first hair mask. In this way, the image processing device obtains the hair color value range according to the color values of the pixels corresponding to the second hair mask, which can improve the accuracy of the hair color value range.
结合本申请任一实施方式，所述待分割对象包括待分割人物，所述毛包括头发，在所述缩小所述第一毛掩码中的毛区域，得到第二毛掩码之前，所述方法还包括：With reference to any embodiment of the present application, the object to be segmented includes a person to be segmented, and the hair includes hair. Before the hair area in the first hair mask is reduced to obtain the second hair mask, the Methods also include:
对所述待处理图像进行人脸检测处理，得到所述待分割人物的人脸区域的位置；Performing face detection processing on the to-be-processed image to obtain the position of the face region of the to-be-segmented character;
依据所述人脸区域的位置，从所述待处理图像中确定所述待分割人物的头发区域；According to the position of the face area, determine the hair area of the person to be segmented from the image to be processed;
所述缩小所述第一毛掩码中的毛区域，得到第二毛掩码，包括：The reducing the hair area in the first hair mask to obtain a second hair mask, including:
缩小所述第一毛掩码中的头发区域，得到头发掩码；reducing the hair area in the first hair mask to obtain a hair mask;
依据所述头发掩码，确定所述待分割人物的头发区域中的头发像素点，得到所述第二毛掩码。According to the hair mask, hair pixels in the hair region of the person to be segmented are determined to obtain the second hair mask.
在该种实施方式中，由于头发掩码所对应的像素点均为高置信度头发像素点，图像处 理装置依据头发掩码，确定待分割人物的头发区域中的头发像素点，可确定待分割人物的头发区域中的高置信度头发像素点，进而得到第二毛掩码。In this embodiment, since the pixels corresponding to the hair mask are all high-confidence hair pixels, the image processing device determines the hair pixels in the hair region of the person to be segmented according to the hair mask, and can determine the hair pixels to be segmented. High-confidence hair pixels in the hair region of the character, and then obtain the second hair mask.
结合本申请任一实施方式，所述对所述待处理图像进行人脸检测处理，得到所述待分割人物的人脸区域的位置，包括：In combination with any of the embodiments of the present application, performing face detection processing on the to-be-processed image to obtain the position of the face region of the person to be segmented includes:
对所述待处理图像进行人脸检测处理，得到至少一个人脸框；performing face detection processing on the to-be-processed image to obtain at least one face frame;
选取所述至少一个人脸框中尺度最大的人脸框，作为所述待分割人物的人脸框；Select the face frame with the largest scale in the at least one face frame as the face frame of the character to be segmented;
将所述待分割人物的人脸框的位置和所述待分割人物的人脸框的尺寸，作为所述待分割人物的人脸区域的位置。The position of the face frame of the person to be divided and the size of the face frame of the person to be divided are used as the position of the face region of the person to be divided.
结合本申请任一实施方式，所述选取所述至少一个人脸框中尺度最大的人脸框，作为所述待分割人物的人脸框，包括：In combination with any embodiment of the present application, the selection of the face frame with the largest scale in the at least one face frame as the face frame of the character to be segmented includes:
选取所述至少一个人脸框中面积最大的人脸框，作为所述待分割人物的人脸框。The face frame with the largest area in the at least one face frame is selected as the face frame of the person to be segmented.
结合本申请任一实施方式，在所述确定所述毛区域内颜色值处于所述毛颜色值范围之外的像素点，作为背景像素点之前，所述方法还包括：With reference to any of the embodiments of the present application, before determining that the pixels whose color values in the hair region are outside the range of the hair color values are used as background pixels, the method further includes:
对所述待处理图像进行人像分割处理，得到第一人像掩码；Performing portrait segmentation processing on the to-be-processed image to obtain a first portrait mask;
依据所述第一人像掩码，确定所述待分割人物的头发区域中的人像像素点，得到第二人像掩码；According to the first portrait mask, determine the portrait pixels in the hair region of the person to be segmented, and obtain the second portrait mask;
所述确定所述毛区域内颜色值处于所述毛颜色值范围之外的像素点，作为背景像素点，包括：The determining that the color value in the hair area is outside the range of the hair color value is used as the background pixel, including:
确定所述毛区域内颜色值处于所述毛颜色值范围之外的像素点，得到至少一个第一像素点；Determining the pixels in the hair region whose color values are outside the hair color value range to obtain at least one first pixel;
依据所述第二人像掩码，确定所述至少一个第一像素点中的人像像素点，得到至少一个第二像素点；According to the second portrait mask, determine the portrait pixels in the at least one first pixel to obtain at least one second pixel;
去除所述至少一个第一像素点中的所述至少一个第二像素点，得到所述背景像素点。The at least one second pixel point in the at least one first pixel point is removed to obtain the background pixel point.
在该种实施方式中，图像处理装置通过去除至少一个第一像素点中的至少一个第二像素点，可将至少一个第一像素点中的人像像素点去除，得到背景像素点，从而提高背景像素点的准确度。In this embodiment, the image processing device can remove the portrait pixels in the at least one first pixel by removing at least one second pixel in the at least one first pixel to obtain background pixels, thereby improving the background. Pixel accuracy.
结合本申请任一实施方式，所述依据所述第二毛掩码所对应的像素点的颜色值，得到所述毛颜色值范围，包括：In combination with any embodiment of the present application, the range of the hair color value is obtained according to the color value of the pixel corresponding to the second hair mask, including:
获取毛颜色值区间长度；Get the length of the hair color value interval;
统计所述第二毛掩码所对应的像素点的颜色直方图；Counting the color histogram of the pixel corresponding to the second hair mask;
依据所述颜色直方图，得到所对应的像素点的数量最多、且长度为所述毛颜色值区间长度的毛颜色值区间，作为所述毛颜色值范围。According to the color histogram, a hair color value interval with the largest number of corresponding pixel points and the length of which is the length of the hair color value interval is obtained as the hair color value range.
结合本申请任一实施方式，所述待处理图像还包括非待分割对象区域，所述非待分割对象区域为所述待处理图像中除所述待分割对象所覆盖的像素点区域之外的像素点区域；所述方法还包括：With reference to any embodiment of the present application, the image to be processed further includes a non-to-be-segmented object area, and the non-to-be-segmented object area is an area of the to-be-processed image other than the pixel area covered by the to-be-segmented object. pixel area; the method further includes:
改变所述背景像素点的颜色和非待分割对象区域的颜色，得到所述待分割对象的留色图像；在所述留色图像中，所述非待分割对象区域的颜色和所述背景像素点的颜色相同。Change the color of the background pixels and the color of the non-to-be-segmented object area to obtain a color-retained image of the to-be-segmented object; in the color-retained image, the color of the non-to-be-segmented object area and the background pixels The dots are the same color.
在该种实施方式中，非待分割对象区域和待分割对象所覆盖的像素点区域，是通过对待处理图像进行分割处理得到的。In this embodiment, the non-to-be-segmented object area and the pixel point area covered by the to-be-segmented object are obtained by performing segmentation processing on the to-be-processed image.
在待分割对象的边缘存在毛，且边缘毛存在镂空区域的情况下，分割处理难以将镂空区域中的非毛像素点区域从镂空区域中准确分割出来，进而将非毛像素点区域误识别为前景像素点。因此，在待处理图像中，待分割对象覆盖的像素点区域包括镂空区域中的非毛像素点，非待分割对象区域不包含镂空区域中的非毛像素点。When there is hair on the edge of the object to be segmented, and there is a hollow area in the edge hair, it is difficult for the segmentation process to accurately segment the non-hair pixel area in the hollow area from the hollow area, and then the non-hair pixel area is misidentified as Foreground pixels. Therefore, in the image to be processed, the pixel point area covered by the object to be segmented includes the non-hairy pixels in the hollow area, and the non-to-be-segmented object area does not include the non-hair pixels in the hollow area.
图像处理装置通过改变背景像素点的颜色和非待分割对象区域的颜色，可提升留色效果，得到待分割对象的留色图像。The image processing device can improve the color retention effect by changing the color of the background pixels and the color of the non-to-be-segmented object area, and obtain a color-retained image of the to-be-segmented object.
结合本申请任一实施方式，所述方法还包括：In conjunction with any embodiment of the present application, the method further includes:
对所述留色图像中的所述背景像素点和所述留色图像中的所述非待分割对象区域进行虚化处理，得到所述待分割对象的留色虚化图像。Perform a blurring process on the background pixels in the color-retained image and the non-to-be-segmented object area in the color-retained image to obtain a color-retained blurred image of the object to be segmented.
结合本申请任一实施方式，所述改变所述背景像素点的颜色和非待分割对象区域的颜色，得到所述待分割对象的留色图像，包括：In combination with any of the embodiments of the present application, changing the color of the background pixels and the color of the non-to-be-segmented object area to obtain a color-retained image of the to-be-segmented object includes:
对所述背景像素点和所述非待分割对象区域进行灰度化处理，得到所述留色图像。Grayscale processing is performed on the background pixel points and the non-to-be-segmented object area to obtain the color-retained image.
结合本申请任一实施方式，所述方法还包括：In conjunction with any embodiment of the present application, the method further includes:
对所述第二毛掩码与所述第二人像掩码进行融合处理，得到优化后的前景掩码；Perform fusion processing on the second hair mask and the second portrait mask to obtain an optimized foreground mask;
根据所述优化后的前景掩码，确定所述待处理图像中的优化背景区域；According to the optimized foreground mask, determine the optimized background area in the to-be-processed image;
改变所述优化背景区域的颜色，得到留色图像。Change the color of the optimized background area to obtain a color-retained image.
第二方面，本申请实施例还提供了一种图像处理装置，所述装置包括：In a second aspect, the embodiments of the present application further provide an image processing apparatus, the apparatus comprising:
第一获取部分，被配置为获取待处理图像；所述待处理图像包含待分割对象的毛区域；a first acquiring part, configured to acquire an image to be processed; the image to be processed includes the hair region of the object to be segmented;
第二获取部分，被配置为获取所述待分割对象的毛颜色值范围；a second acquiring part, configured to acquire the range of hair color values of the object to be segmented;
处理部分，被配置为确定所述毛区域内颜色值处于所述毛颜色值范围之外的像素点，作为背景像素点。The processing part is configured to determine the pixel points in the hair area whose color values are outside the hair color value range as background pixels.
结合本申请任一实施方式，所述第二获取部分，还被配置为：With reference to any embodiment of the present application, the second obtaining part is further configured as:
对所述待处理图像进行毛分割处理，得到第一毛掩码；Perform hair segmentation processing on the to-be-processed image to obtain a first hair mask;
缩小所述第一毛掩码中的毛区域，得到第二毛掩码；reducing the hair area in the first hair mask to obtain a second hair mask;
依据所述第二毛掩码所对应的像素点的颜色值，得到所述毛颜色值范围。The hair color value range is obtained according to the color value of the pixel corresponding to the second hair mask.
结合本申请任一实施方式，所述待分割对象包括待分割人物，所述毛包括头发；With reference to any embodiment of the present application, the object to be segmented includes a person to be segmented, and the hair includes hair;
所述处理部分，还被配置为在所述缩小所述第一毛掩码中的毛区域，得到第二毛掩码之前，对所述待处理图像进行人脸检测处理，得到所述待分割人物的人脸区域的位置；The processing part is further configured to perform face detection processing on the to-be-processed image to obtain the to-be-segmented image before reducing the hair region in the first hair mask to obtain the second hair mask the location of the face area of the character;
以及被配置为依据所述人脸区域的位置，从所述待处理图像中确定所述待分割人物的头发区域；and is configured to determine the hair region of the person to be segmented from the image to be processed according to the position of the face region;
所述第二获取部分，还被配置为：The second acquisition part is also configured as:
缩小所述第一毛掩码中的头发区域，得到头发掩码；reducing the hair area in the first hair mask to obtain a hair mask;
依据所述头发掩码，确定所述待分割人物的头发区域中的头发像素点，得到所述第二毛掩码。According to the hair mask, hair pixels in the hair region of the person to be segmented are determined to obtain the second hair mask.
结合本申请任一实施方式，所述处理部分，还被配置为：With reference to any embodiment of the present application, the processing part is further configured as:
对所述待处理图像进行人脸检测处理，得到至少一个人脸框；performing face detection processing on the to-be-processed image to obtain at least one face frame;
选取所述至少一个人脸框中尺度最大的人脸框，作为所述待分割人物的人脸框；Select the face frame with the largest scale in the at least one face frame as the face frame of the character to be segmented;
将所述待分割人物的人脸框的位置和所述待分割人物的人脸框的尺寸，作为所述待分割人物的人脸区域的位置。The position of the face frame of the person to be divided and the size of the face frame of the person to be divided are used as the position of the face region of the person to be divided.
结合本申请任一实施方式，所述处理部分，还被配置为：With reference to any embodiment of the present application, the processing part is further configured as:
选取所述至少一个人脸框中面积最大的人脸框，作为所述待分割人物的人脸框。The face frame with the largest area in the at least one face frame is selected as the face frame of the person to be segmented.
结合本申请任一实施方式，所述处理部分，还被配置为在所述确定所述毛区域内颜色值处于所述毛颜色值范围之外的像素点，作为背景像素点之前，对所述待处理图像进行人像分割处理，得到第一人像掩码；With reference to any embodiment of the present application, the processing part is further configured to, before determining that the pixels whose color values in the hair region are outside the range of the hair color values are used as background pixels, perform the processing on the pixels. The to-be-processed image is subjected to portrait segmentation to obtain a first portrait mask;
以及依据所述第一人像掩码，确定所述待分割人物的头发区域中的人像像素点，得到第二人像掩码；And according to the first portrait mask, determine the portrait pixels in the hair region of the person to be segmented, and obtain the second portrait mask;
以及确定所述毛区域内颜色值处于所述毛颜色值范围之外的像素点，得到至少一个第一像素点；and determining the pixels whose color values in the hair area are outside the range of the hair color values to obtain at least one first pixel;
以及依据所述第二人像掩码，确定所述至少一个第一像素点中的人像像素点，得到至少一个第二像素点；and determining a portrait pixel in the at least one first pixel according to the second portrait mask to obtain at least one second pixel;
以及去除所述至少一个第一像素点中的所述至少一个第二像素点，得到所述背景像素 点。and removing the at least one second pixel point in the at least one first pixel point to obtain the background pixel point.
结合本申请任一实施方式，所述第二获取部分，还被配置为：With reference to any embodiment of the present application, the second obtaining part is further configured as:
获取毛颜色值区间长度；Get the length of the hair color value interval;
统计所述第二毛掩码所对应的像素点的颜色直方图；Counting the color histogram of the pixel corresponding to the second hair mask;
依据所述颜色直方图，得到所对应的像素点的数量最多、且长度为所述毛颜色值区间长度的毛颜色值区间，作为所述毛颜色值范围。According to the color histogram, a hair color value interval with the largest number of corresponding pixel points and the length of which is the length of the hair color value interval is obtained as the hair color value range.
结合本申请任一实施方式，所述待处理图像还包括非待分割对象区域，所述非待分割对象区域为所述待处理图像中除所述待分割对象所覆盖的像素点区域之外的像素点区域；With reference to any embodiment of the present application, the image to be processed further includes a non-to-be-segmented object area, and the non-to-be-segmented object area is an area of the to-be-processed image other than the pixel area covered by the to-be-segmented object. pixel area;
所述处理部分，还被配置为改变所述背景像素点的颜色和非待分割对象区域的颜色，得到所述待分割对象的留色图像；在所述留色图像中，所述非待分割对象区域的颜色和所述背景像素点的颜色相同。The processing part is further configured to change the color of the background pixels and the color of the non-to-be-segmented object area to obtain a color-retained image of the to-be-segmented object; The color of the object area is the same as the color of the background pixels.
结合本申请任一实施方式，所述处理部分，还被配置为：With reference to any embodiment of the present application, the processing part is further configured as:
对所述留色图像中的所述背景像素点和所述留色图像中的所述非待分割对象区域进行虚化处理，得到所述待分割对象的留色虚化图像。Perform a blurring process on the background pixels in the color-retained image and the non-to-be-segmented object area in the color-retained image to obtain a color-retained blurred image of the object to be segmented.
结合本申请任一实施方式，所述处理部分，还被配置为：With reference to any embodiment of the present application, the processing part is further configured as:
对所述背景像素点和所述非待分割对象区域进行灰度化处理，得到所述留色图像。Grayscale processing is performed on the background pixel points and the non-to-be-segmented object area to obtain the color-retained image.
结合本申请任一实施方式，所述处理部分，还被配置为：With reference to any embodiment of the present application, the processing part is further configured as:
对所述第二毛掩码与所述第二人像掩码进行融合处理，得到优化后的前景掩码；Perform fusion processing on the second hair mask and the second portrait mask to obtain an optimized foreground mask;
根据所述优化后的前景掩码，确定所述待处理图像中的优化背景区域；According to the optimized foreground mask, determine the optimized background area in the to-be-processed image;
改变所述优化背景区域的颜色，得到留色图像。Change the color of the optimized background area to obtain a color-retained image.
第三方面，提供了一种电子设备，其中，包括：处理器和存储器，所述存储器被配置为存储计算机程序代码，所述计算机程序代码包括计算机指令，在所述处理器执行所述计算机指令的情况下，所述电子设备执行如上述第一方面的方法。In a third aspect, an electronic device is provided, comprising: a processor and a memory, the memory being configured to store computer program code, the computer program code comprising computer instructions, the processor executing the computer instructions In this case, the electronic device executes the method of the first aspect.
第四方面，提供了另一种电子设备，包括：处理器、发送装置、输入装置、输出装置和存储器，所述存储器被配置为存储计算机程序代码，所述计算机程序代码包括计算机指令，在所述处理器执行所述计算机指令的情况下，所述电子设备执行如上述第一方面的方法。In a fourth aspect, another electronic device is provided, comprising: a processor, a transmitting device, an input device, an output device, and a memory, the memory being configured to store computer program code, the computer program code including computer instructions, in which When the processor executes the computer instructions, the electronic device executes the method of the first aspect.
第五方面，提供了一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机程序，所述计算机程序包括程序指令，在所述程序指令被处理器执行的情况下，所述处理器执行如上述第一方面的方法。A fifth aspect provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, the computer program includes program instructions, and when the program instructions are executed by a processor, the The processor performs the method of the first aspect as described above.
第六方面，提供了一种计算机程序产品，所述计算机程序产品包括计算机程序或指令，在所述计算机程序或指令在计算机上运行的情况下，所述计算机执行上述第一方面的方法。In a sixth aspect, a computer program product is provided, the computer program product includes a computer program or instructions, and when the computer program or instructions are run on a computer, the computer executes the method of the first aspect.
第七方面，提供了一种计算机程序，所述计算机程序产品包括计算机程序或指令，在所述计算机程序或指令在计算机上运行的情况下，所述计算机执行上述第一方面的方法。In a seventh aspect, a computer program is provided, the computer program product includes a computer program or instructions, and when the computer program or instructions are run on a computer, the computer executes the method of the first aspect.
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，而非限制本申请。It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
附图说明Description of drawings
为了更清楚地说明本申请实施例或背景技术中的技术方案，下面将对本申请实施例或背景技术中所需要使用的附图进行说明。In order to more clearly describe the technical solutions in the embodiments of the present application or the background technology, the accompanying drawings required in the embodiments or the background technology of the present application will be described below.
此处的附图被并入说明书中并构成本说明书的一部分，这些附图示出了符合本申请的实施例，并与说明书一起用于说明本申请的技术方案。The accompanying drawings, which are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the description, serve to explain the technical solutions of the present application.
图1A为本申请实施例提供的一种镂空区域示意图一；1A is a schematic diagram 1 of a hollow area provided by an embodiment of the present application;
图1B为本申请实施例提供的一种镂空区域示意图二；FIG. 1B is a second schematic diagram of a hollow area provided by an embodiment of the present application;
图2为本申请实施例提供的一种图像处理方法的流程示意图；2 is a schematic flowchart of an image processing method provided by an embodiment of the present application;
图3为本申请实施例提供的一种第二毛掩码示意图；3 is a schematic diagram of a second hair mask provided in an embodiment of the present application;
图4为本申请实施例提供的一种第二人像掩码示意图；4 is a schematic diagram of a second portrait mask provided by an embodiment of the present application;
图5为本申请实施例提供的一种优化后的前景掩码示意图；5 is a schematic diagram of an optimized foreground mask provided by an embodiment of the present application;
图6为本申请实施例提供的一种待处理图像示意图；FIG. 6 is a schematic diagram of a to-be-processed image provided by an embodiment of the present application;
图7为本申请实施例提供的一种中间人像分割掩码示意图；7 is a schematic diagram of a middle portrait segmentation mask provided by an embodiment of the present application;
图8为本申请实施例提供的一种优化后的前景掩码示意图；8 is a schematic diagram of an optimized foreground mask provided by an embodiment of the present application;
图9为本申请实施例提供的一种图像处理装置的结构示意图；FIG. 9 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application;
图10为本申请实施例提供的一种图像处理装置的硬件结构示意图。FIG. 10 is a schematic diagram of a hardware structure of an image processing apparatus according to an embodiment of the present application.
具体实施方式Detailed ways
为了使本技术领域的人员更好地理解本申请方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。In order to make those skilled in the art better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only It is a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别不同对象，而不是用于描述特定顺序。此外，术语“包括”和“具有”以及它们任何变形，意图在于覆盖不排他的包含。例如包含了一系列步骤或部分的过程、方法、***、产品或设备没有限定于已列出的步骤或部分，而是可选地还包括没有列出的步骤或部分，或可选地还包括对于这些过程、方法、产品或设备固有的其他步骤或部分。The terms "first", "second" and the like in the description and claims of the present application and the above drawings are used to distinguish different objects, rather than to describe a specific order. Furthermore, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or parts is not limited to the listed steps or parts, but optionally also includes unlisted steps or parts, or optionally also includes For other steps or parts inherent to these processes, methods, products or devices.
应当理解，在本申请实施例中，“至少一个(项)”是指一个或者多个，“多个”是指两个或两个以上，“至少两个(项)”是指两个或三个及三个以上，“和/或”，用于描述关联对象的关联关系，表示可以存在三种关系，例如，“A和/或B”可以表示：只存在A，只存在B以及同时存在A和B三种情况，其中A，B可以是单数或者复数。字符“/”可表示前后关联对象是一种“或”的关系，是指这些项中的任意组合，包括单项(个)或复数项(个)的任意组合。例如，a，b或c中的至少一项(个)，可以表示：a，b，c，“a和b”，“a和c”，“b和c”，或“a和b和c”，其中a，b，c可以是单个，也可以是多个。字符“/”还可表示数学运算中的除号，例如，a/b＝a除以b；6/3＝2。“以下至少一项(个)”或其类似表达。It should be understood that, in the embodiments of the present application, "at least one (item)" refers to one or more, "multiple" refers to two or more, and "at least two (item)" refers to two or more Three or more, "and/or", is used to describe the association relationship of related objects, indicating that three kinds of relationships can exist, for example, "A and/or B" can mean: only A exists, only B exists, and at the same time There are three cases of A and B, where A and B can be singular or plural. The character "/" can indicate that the related objects before and after are an "or" relationship, which refers to any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (a) of a, b or c, can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c" ", where a, b, c can be single or multiple. The character "/" can also represent a division sign in a mathematical operation, eg, a/b=a divided by b; 6/3=2. "At least one (a) of the following" or similar expressions thereof.
在本文中提及“实施例”意味着，结合实施例描述的特定特征、结构或特性可以包含在本申请的至少一个实施例中。在说明书中的各个位置出现该短语并不一定均是指相同的实施例，也不是与其它实施例互斥的独立的或备选的实施例。本领域技术人员显式地和隐式地理解的是，本文所描述的实施例可以与其它实施例相结合。Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor a separate or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
分割处理指，确定图像中语义信息为待分割对象的像素点。例如，在待分割对象为人像的情况下，通过对图像进行人像分割处理，可确定图像中语义信息为人像的像素点，进而将人像像素点与非人像像素点区分开。Segmentation processing refers to determining that the semantic information in the image is the pixel point of the object to be segmented. For example, when the object to be segmented is a portrait, by performing portrait segmentation processing on the image, it can be determined that the semantic information in the image is the pixels of the portrait, and then the portrait pixels can be distinguished from non- portrait pixels.
为表述方便，下文将语义信息为待分割对象的像素点称为前景像素点，并将语义信息不是待分割对象的像素点称为背景像素点。For convenience of expression, hereinafter, the pixels whose semantic information is the object to be segmented are called foreground pixels, and the pixels whose semantic information is not the object to be segmented are called background pixels.
但在待分割对象的边缘存在毛，且边缘毛存在镂空区域的情况下，目前，相关技术中的待分割对象分割处理方法难以将镂空区域中的非毛像素点区域从镂空区域中准确分割出来，进而将非毛像素点区域误识别为前景像素点，从而导致对待分割对象的分割准确度低。上述非毛像素点指语义信息不是毛的像素点。However, when there is hair on the edge of the object to be segmented, and the edge hair has a hollow area, at present, it is difficult for the segmentation processing method of the object to be segmented in the related art to accurately segment the non-hair pixel area in the hollow area from the hollow area. , and then mistakenly identify the non-hairy pixel area as a foreground pixel, resulting in low segmentation accuracy of the object to be segmented. The above-mentioned non-hairy pixels refer to pixels whose semantic information is not hairy.
例如，假设待分割对象为人像。在图1中，人像的头发区域中存在镂空区域，在镂空区域中，头发与头发之间存在非头发像素点。本申请实施例中，非头发像素点指语义信息不是头发的像素点。由于非头发像素点为背景像素点，而人像分割处理方法会将非头发像素点区域误识别为头发像素点，对图1A的人像进行分割处理得到的分割准确度低。示例 性地，图1B是采用相关技术中的待分割对象分割处理方法，得到的人像的头发镂空区域。For example, suppose the object to be segmented is a portrait. In Figure 1, there is a hollow area in the hair area of the portrait, and in the hollow area, there are non-hair pixels between the hair and the hair. In the embodiments of the present application, non-hair pixels refer to pixels whose semantic information is not hair. Since the non-hair pixels are background pixels, and the portrait segmentation processing method will misidentify the non-hair pixel area as hair pixels, the segmentation accuracy obtained by segmenting the portrait in FIG. 1A is low. Illustratively, FIG. 1B shows the hollowed-out area of the hair of the portrait obtained by adopting the method for segmenting the object to be segmented in the related art.
基于此，本申请实施例提供了一种技术方案，以减少镂空区域中被误识别为前景像素点的背景像素点的数量，从而提高分割准确度。Based on this, the embodiments of the present application provide a technical solution to reduce the number of background pixels in the hollow area that are mistakenly identified as foreground pixels, thereby improving segmentation accuracy.
在进行接下来的描述之前，首先对下面将要出现的掩码进行定义。本申请实施例中，掩码指图像特定语义掩码。图像特定语义掩码中的元素与图像中的像素点一一对应。掩码中元素的取值包括第一值和第二值，其中，第一值与第二值不同。第一值表示与该元素对应的像素点的语义是特定语义，第二值表示与该元素对应的像素点的语义不是特定语义。可选的，第一值为1，第二值为0。Before proceeding to the following description, the mask that will appear below is first defined. In this embodiment of the present application, the mask refers to an image-specific semantic mask. Elements in the image-specific semantic mask correspond one-to-one with pixels in the image. The values of the elements in the mask include a first value and a second value, wherein the first value is different from the second value. The first value indicates that the semantics of the pixel corresponding to the element is a specific semantic, and the second value indicates that the semantics of the pixel corresponding to the element is not a specific semantic. Optionally, the first value is 1, and the second value is 0.
例如，图像I包括像素点A、像素点B、像素点C和像素点D。图像1的头发掩码包括元素a、元素b、元素c和元素d，即特定语义为头发。其中，元素a与像素点A对应，元素b与像素点B对应，元素c与像素点C对应，元素d与像素点D对应。假设第一值为1，第二值为0。若元素a的取值1，元素b的取值为0，元素c的取值为1，元素d的取值为0。那么，像素点A的语义为头发，即像素点A为头发像素点；像素点B的语义不是头发，即像素点B不是头发像素点；像素点C的语义为头发，即像素点C为头发像素点；像素点D的语义不是头发，即像素点D不是头发像素点。For example, image I includes pixel A, pixel B, pixel C, and pixel D. The hair mask of image 1 includes element a, element b, element c, and element d, that is, the specific semantic is hair. The element a corresponds to the pixel point A, the element b corresponds to the pixel point B, the element c corresponds to the pixel point C, and the element d corresponds to the pixel point D. Suppose the first value is 1 and the second value is 0. If the value of element a is 1, the value of element b is 0, the value of element c is 1, and the value of element d is 0. Then, the semantics of pixel A is hair, that is, pixel A is a hair pixel; the semantics of pixel B is not hair, that is, pixel B is not a hair pixel; the semantics of pixel C is hair, that is, pixel C is hair Pixel point; the semantics of pixel point D is not hair, that is, pixel point D is not a hair pixel point.
为表述方便，本申请实施例中，[α，β]表示大于或等于α且小于或等于β的取值区间。本申请实施例的执行主体为图像处理装置，其中，图像处理装置可以是任意一种可执行本申请方法实施例所公开的技术方案的电子设备。可选的，图像处理装置可以是用户设备(User Equipment，UE)、移动设备、用户终端、终端、蜂窝电话、无绳电话、个人数字处理(Personal Digital Assistant，PDA)、手持设备、计算设备、车载设备、可穿戴设备等。例如，可以是以下中的一种：手机、计算机、服务器、平板电脑。For convenience of expression, in the embodiments of the present application, [α, β] represents a value interval that is greater than or equal to α and less than or equal to β. The execution subject of the embodiments of the present application is an image processing apparatus, where the image processing apparatus may be any electronic device that can execute the technical solutions disclosed in the method embodiments of the present application. Optionally, the image processing apparatus may be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, a vehicle devices, wearables, etc. For example, it can be one of the following: mobile phone, computer, server, tablet.
应理解，本申请方法实施例还可以通过处理器执行计算机程序代码的方式实现。下面结合本申请实施例中的附图对本申请实施例进行描述。请参阅图2，图2是本申请实施例提供的一种图像处理方法的流程示意图。It should be understood that, the method embodiments of the present application may also be implemented by a processor executing computer program codes. The embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. Please refer to FIG. 2 , which is a schematic flowchart of an image processing method provided by an embodiment of the present application.
S201、获取待处理图像，其中，上述待处理图像包含待分割对象的毛区域。S201. Acquire an image to be processed, wherein the image to be processed includes a hair region of an object to be segmented.
本申请实施例中，待处理图像包含待分割对象，其中，待分割对象可以是任意物体。例如，待分割对象可以是人物；又例如，待分割对象可以是毛衣；再例如，待分割对象可以是毛衣和人物。In this embodiment of the present application, the image to be processed includes an object to be segmented, where the object to be segmented may be any object. For example, the object to be segmented may be a person; for another example, the object to be segmented may be a sweater; for another example, the object to be segmented may be a sweater and a person.
本申请实施例中，待分割对象包含毛区域，其中，毛区域包括以下至少一种：体毛区域、绒毛区域、羽毛区域。例如，假设待分割对象为人物，毛区域可以是体毛区域；又例如，假设待分割对象为蒲公英，毛区域可以是绒毛区域；再例如，假设待分割对象为鸟，毛区域可以是羽毛区域；再例如，假设待分割对象为人物和鸟，毛区域可以包括体毛区域和羽毛区域；再例如，假设待分割对象为人物、鸟和蒲公英，毛区域可以包括体毛区域、羽毛区域和绒毛区域。In this embodiment of the present application, the object to be segmented includes a hair region, where the hair region includes at least one of the following: a body hair region, a fluff region, and a feather region. For example, suppose that the object to be segmented is a person, and the hair area can be a body hair area; for another example, assuming that the object to be segmented is a dandelion, the hair area can be a fluff area; for another example, assuming that the object to be segmented is a bird, the hair area can be a feather area; For another example, assuming that the objects to be segmented are people and birds, the hair region may include body hair regions and feather regions; for another example, assuming that the objects to be segmented are people, birds, and dandelions, the hair regions may include body hair regions, feather regions, and fluff regions.
在一种获取待处理图像的实现方式中，图像处理装置接收用户通过输入组件输入的待处理图像，来获取待处理图像。上述输入组件包括：键盘、鼠标、触控屏、触控板和音频输入器等。In an implementation manner of acquiring the to-be-processed image, the image processing apparatus receives the to-be-processed image input by the user through the input component, and acquires the to-be-processed image. The above input components include: keyboard, mouse, touch screen, touch pad, audio input and so on.
在另一种获取待处理图像的实现方式中，图像处理装置接收终端发送的待处理图像，来获取待处理图像。可选的，终端可以是以下任意一种：手机、计算机、平板电脑、服务器、可穿戴设备。In another implementation manner of acquiring the to-be-processed image, the image processing apparatus receives the to-be-processed image sent by the terminal to acquire the to-be-processed image. Optionally, the terminal may be any of the following: a mobile phone, a computer, a tablet computer, a server, and a wearable device.
在又一种获取待处理图像的实现方式中，图像处理装置装载有摄像组件，其中，摄像组件包括摄像头。图像处理装置通过使用摄像组件，采集包含待分割对象的图像作为待处理图像。In yet another implementation manner of acquiring the image to be processed, the image processing apparatus is loaded with a camera assembly, wherein the camera assembly includes a camera. The image processing apparatus collects an image containing the object to be segmented as an image to be processed by using a camera assembly.
在又一种获取待处理图像的实现方式中，图像处理装置从获取到的视频流中，选取一帧图像作为待处理图像。In another implementation manner of acquiring the image to be processed, the image processing apparatus selects one frame of image as the image to be processed from the acquired video stream.
S202、获取上述待分割对象的毛颜色值范围。S202: Obtain the range of hair color values of the object to be segmented.
本申请实施例中，颜色值可以是以下中的一种：像素值、灰度值。待分割对象的毛颜色值指待处理图像中待分割对象的毛像素点的颜色值。例如，假设待分割对象为张三，待分割对象的毛为张三的头发。那么待分割对象的毛颜色值可以是待处理图像中张三的头发像素点的像素值；又例如，假设待分割对象为鸟，待分割对象的毛为鸟的羽毛。那么待分割对象的毛颜色值可以是待处理图像中鸟的羽毛像素点的灰度值。In this embodiment of the present application, the color value may be one of the following: a pixel value and a grayscale value. The hair color value of the object to be segmented refers to the color value of the hair pixels of the object to be segmented in the image to be processed. For example, it is assumed that the object to be segmented is Zhang San, and the hair of the object to be segmented is Zhang San's hair. Then, the hair color value of the object to be segmented may be the pixel value of Zhang San's hair pixel in the image to be processed; for another example, suppose that the object to be segmented is a bird, and the hair of the object to be segmented is a feather of a bird. Then the hair color value of the object to be segmented can be the gray value of the feather pixel of the bird in the image to be processed.
本申请实施例中，毛颜色值范围用于判断像素点是否为毛像素点，具体的，颜色值处于毛颜色值范围内的像素点可视为毛像素点。即毛颜色值范围可视为待分割对象的毛像素点的颜色值的参考范围。例如，假设毛颜色值范围为头发像素点的灰度值范围，且头发像素点的灰度值范围为[50，70]。那么灰度值处于[50，70]内的像素点为头发像素点，灰度值处于[50，70]外不是头发像素点。In the embodiment of the present application, the range of hair color values is used to determine whether a pixel is a hair pixel. Specifically, a pixel whose color value is within the hair color value range can be regarded as a hair pixel. That is, the range of hair color values can be regarded as a reference range of color values of hair pixels of the object to be segmented. For example, it is assumed that the range of hair color values is the gray value range of hair pixels, and the gray value range of hair pixels is [50, 70]. Then the pixels whose gray values are within [50, 70] are hair pixels, and those whose gray values are outside [50, 70] are not hair pixels.
应理解，毛颜色值范围可以是一个数值。例如，毛颜色值范围为[60，60]，此时，毛颜色值为60。那么颜色值不等于60的像素点均不是毛像素点，颜色值为60的像素点为毛像素点。It should be understood that the range of coat color values can be a single number. For example, the range of fur color value is [60, 60], in this case, the fur color value is 60. Then the pixels whose color value is not equal to 60 are not hair pixels, and the pixels whose color value is 60 are hair pixels.
在一种获取待分割对象的毛颜色值范围的实现方式中，图像处理装置通过对待处理图像进行毛分割处理，得到待分割对象的毛像素点。统计待分割对象的毛像素点的颜色值的直方图，并将直方图中像素点数量最多的毛颜色值范围，作为待分割对象的毛颜色值范围。In an implementation manner of acquiring the hair color value range of the object to be segmented, the image processing device obtains the hair pixel points of the object to be segmented by performing hair segmentation processing on the image to be processed. The histogram of the color values of the hair pixels of the object to be divided is counted, and the hair color value range with the largest number of pixels in the histogram is used as the hair color value range of the object to be divided.
在另一种获取待分割对象的毛颜色值范围的实现方式中，图像处理装置将用户通过输入组件输入的毛颜色值范围，作为待分割对象的毛颜色值范围。In another implementation manner of acquiring the hair color value range of the object to be segmented, the image processing apparatus takes the hair color value range input by the user through the input component as the hair color value range of the object to be segmented.
在又一种获取待分割对象的毛颜色值范围的实现方式中，图像处理装置将终端发送的毛颜色值范围，作为待分割对象的毛颜色值范围。In yet another implementation manner of acquiring the range of hair color values of the object to be divided, the image processing apparatus uses the range of hair color values sent by the terminal as the range of hair color values of the object to be divided.
S203、确定上述毛区域内颜色值处于上述毛颜色值范围之外的像素点，得到背景像素点。S203: Determine the pixel points in the hair region whose color values are outside the hair color value range, and obtain background pixels.
图像处理装置依据毛颜色值范围和毛区域内像素点的颜色值，可判断毛区域内的像素点是否为毛像素点。这里，确定颜色值处于毛颜色值范围内的像素点为毛像素点；确定颜色值处于毛颜色值范围外的像素点不是毛像素点(即为背景像素点)。The image processing device can determine whether the pixels in the hair area are hair pixels according to the hair color value range and the color values of the pixels in the hair area. Here, pixels whose color values are determined to be within the range of hair color values are hair pixels; pixels whose color values are determined to be outside the range of hair color values are not hair pixels (ie, background pixels).
可选的，本实施例中的毛区域通过对图像进行毛分割处理得到。例如，假设毛区域为头发区域。通过对待处理图像进行头发分割处理，可确定头发区域。Optionally, the hair region in this embodiment is obtained by performing hair segmentation processing on the image. For example, assume that the hair region is a hair region. By performing hair segmentation processing on the image to be processed, the hair region can be determined.
应理解，对待处理图像进行的毛分割处理的执行主体可以是图像处理装置，也可以不是图像处理装置。例如，终端通过对图像a进行毛分割处理，确定图像a中的毛像素点，并在图像a中将毛像素点标记出来，得到待处理图像。终端将待处理图像发送至图像处理装置，这样，在图像处理装置获取到的待处理图像中，已将毛区域标记出来。It should be understood that the execution body of the hair segmentation process performed on the image to be processed may be an image processing device, or may not be an image processing device. For example, the terminal determines the hair pixels in the image a by performing hair segmentation processing on the image a, and marks the hair pixels in the image a to obtain the image to be processed. The terminal sends the image to be processed to the image processing device, so that the hair region has been marked in the image to be processed obtained by the image processing device.
本实施例中，由于毛颜色值范围可视为待分割对象的毛像素点的颜色值的参考范围，图像处理装置依据毛颜色值范围，判断毛区域内的像素点是否为毛像素点，可确定毛区域内的背景像素点，从而提高待处理图像中待分割对象的分割准确度。In this embodiment, since the range of hair color values can be regarded as the reference range of the color values of the hair pixels of the object to be segmented, the image processing device determines whether the pixels in the hair area are hair pixels according to the range of hair color values, and can Determine the background pixels in the hair area, thereby improving the segmentation accuracy of the object to be segmented in the image to be processed.
作为一种可选的实施方式，图像处理装置在执行步骤202的过程中执行以下步骤：As an optional implementation manner, the image processing apparatus performs the following steps in the process of performing step 202:
S301、对上述待处理图像进行毛分割处理，得到第一毛掩码。S301. Perform hair segmentation processing on the image to be processed to obtain a first hair mask.
本申请实施例中，毛分割处理用于从图像中确定语义信息为毛的像素点。例如，假设毛分割处理为头发分割处理。通过对图像进行头发分割处理，可确定图像中语义信息为头发的像素点；又例如，假设毛分割处理为羽毛分割处理。通过对图像进行羽毛分割处理，可确定图像中语义信息为羽毛的像素点。In this embodiment of the present application, the hair segmentation process is used to determine pixels whose semantic information is hair from the image. For example, it is assumed that the hair segmentation process is hair segmentation process. By performing hair segmentation processing on the image, it can be determined that the semantic information in the image is the pixel points of hair; for another example, it is assumed that the hair segmentation processing is feather segmentation processing. By performing feather segmentation processing on the image, the semantic information in the image can be determined as the pixels of feathers.
本申请实施例中，毛掩码中的元素与待处理图像中的像素点一一对应，且毛掩码中取值为第一值的元素所对应的像素点是毛像素点，毛掩码中取值为第二值的元素所对应的像素点不是毛像素点。In the embodiment of the present application, the elements in the hair mask correspond to the pixels in the image to be processed one-to-one, and the pixels corresponding to the elements with the first value in the hair mask are hair pixels. The pixel corresponding to the element whose value is the second value is not a hair pixel.
S302、缩小上述第一毛掩码中的毛区域，得到第二毛掩码。S302, reducing the hair region in the first hair mask to obtain a second hair mask.
本步骤中，第一毛掩码中的毛区域指取值为第一值的元素所构成的区域。缩小第一毛掩码中的毛区域指，将毛区域的边界向毛区域的中心缩小。In this step, the hair region in the first hair mask refers to the region formed by the elements whose value is the first value. Reducing the hair area in the first hair mask refers to reducing the border of the hair area toward the center of the hair area.
由于通过毛分割处理得到的毛区域的边缘易出现分割误差(即将非毛像素点识别为毛像素点)，毛区域的边缘处的像素点为毛像素点的置信度较低(下文将毛区域边缘处的像素点，称为低置信度毛像素点)，毛区域内除低置信度毛像素点之外的像素点，为毛像素点的置信度较高(下文将称为高置信度毛像素点)，图像处理装置通过缩小第一毛掩码中的毛区域，得到第二毛掩码，可将第一毛掩码中非毛像素点的取值更新为第二值，从而将低置信度毛像素点从第一毛掩码所对应的像素点中剔除，并在第二毛掩码所对应的像素点中保留高置信度毛像素点。Since the edge of the hair area obtained by the hair segmentation process is prone to segmentation errors (that is, the non-hair pixels are identified as hair pixels), the confidence level of the pixels at the edge of the hair area as hair pixels is low (the hair area will be described below). The pixels at the edge are called low-confidence hair pixels), and the pixels other than the low-confidence hair pixels in the hair area are hair pixels with higher confidence (hereinafter referred to as high-confidence hair pixels). pixel), the image processing device obtains the second hair mask by reducing the hair area in the first hair mask, and can update the value of the non-hair pixel point in the first hair mask to the second value, so that the low The confidence hair pixels are eliminated from the pixels corresponding to the first hair mask, and the high confidence hair pixels are retained in the pixels corresponding to the second hair mask.
可选的，图像处理装置可将毛区域的边界向中心缩小，得到第二毛掩码。假设第一掩码包括毛区域a，图像处理装置通过将毛区域a向中心缩小n个像素点，得到第二毛掩码中的毛区域b。毛区域b的面积/毛区域a的面积＝m，可选的，m的取值范围为[10/11，12/13]。Optionally, the image processing apparatus may reduce the border of the hair region toward the center to obtain the second hair mask. Assuming that the first mask includes the hair area a, the image processing apparatus obtains the hair area b in the second hair mask by reducing the hair area a toward the center by n pixels. The area of the hairy area b/the area of the hairy area a=m, optionally, the value range of m is [10/11, 12/13].
S303、依据上述第二毛掩码所对应的像素点的颜色值，得到上述毛颜色值范围。S303. Obtain the range of hair color values according to the color values of the pixels corresponding to the second hair mask.
在一些实施例中，图像处理装置计算第二毛掩码所对应的像素点的颜色值的均值，作为毛颜色值范围。In some embodiments, the image processing apparatus calculates the average value of the color values of the pixels corresponding to the second hair mask as the hair color value range.
在又一些实施例中，图像处理装置计算第二毛掩码所对应的像素点的颜色值的中值，作为毛颜色值范围。In still other embodiments, the image processing apparatus calculates the median value of the color values of the pixel points corresponding to the second hair mask as the hair color value range.
在另一些实施例中，图像处理装置统计第二毛掩码所对应的像素点的颜色值的直方图，并将直方图中像素点数量最多的毛颜色值范围，作为待分割对象的毛颜色值范围。In other embodiments, the image processing device counts a histogram of the color values of the pixels corresponding to the second hair mask, and uses the hair color value range with the largest number of pixels in the histogram as the hair color of the object to be segmented value range.
由于第二毛掩码所对应的像素点均为高置信度毛像素点，图像处理装置依据第二毛掩码所对应的像素点的颜色值，得到毛颜色值范围，可提高毛颜色值范围的准确度。Since the pixels corresponding to the second hair mask are all high-confidence hair pixels, the image processing device obtains the hair color value range according to the color values of the pixels corresponding to the second hair mask, which can increase the hair color value range accuracy.
作为一种可选的实施方式，待分割对象包括待分割人物，待分割对象的毛包括待分割人物的头发。图像处理装置在执行步骤302之前，还执行以下步骤：As an optional implementation manner, the object to be segmented includes a person to be segmented, and the hair of the object to be segmented includes the hair of the person to be segmented. Before performing step 302, the image processing apparatus further performs the following steps:
S304、对上述待处理图像进行人脸检测处理，得到上述待分割人物的人脸区域的位置。S304. Perform face detection processing on the above-mentioned image to be processed to obtain the position of the above-mentioned face region of the person to be segmented.
本申请实施例中，图像处理装置可通过对待处理图像进行人脸检测处理，确定待处理图像是否包含人脸区域。这里，在确定待处理图像包含人脸区域的情况下，还可确定人脸区域的位置。In this embodiment of the present application, the image processing apparatus may perform face detection processing on the image to be processed to determine whether the image to be processed includes a face region. Here, when it is determined that the image to be processed contains a human face region, the position of the human face region can also be determined.
本申请实施例中，人脸区域的位置指人脸区域在待处理图像的像素坐标系下的坐标。可选的，图像处理装置通过对待处理图像进行人脸检测处理，可得到包含人脸区域的人脸框，并将人脸框在待处理图像中的位置作为人脸区域的位置。In the embodiment of the present application, the position of the face region refers to the coordinates of the face region in the pixel coordinate system of the image to be processed. Optionally, the image processing apparatus may obtain a face frame including the face region by performing face detection processing on the image to be processed, and use the position of the face frame in the image to be processed as the position of the face region.
可选的，人脸框在待处理图像中的位置包括：人脸框中任意一个顶点在待处理图像的像素坐标系下的坐标，以及人脸框的尺寸。例如，人脸框的位置包括：人脸框的左上顶点在待处理图像的像素坐标系下的坐标、人脸框的长以及人脸框的宽。Optionally, the position of the face frame in the image to be processed includes: coordinates of any vertex in the face frame in the pixel coordinate system of the image to be processed, and the size of the face frame. For example, the position of the face frame includes: the coordinates of the upper left vertex of the face frame in the pixel coordinate system of the image to be processed, the length of the face frame, and the width of the face frame.
在一种对待处理图像进行人脸检测处理的实现方式中，图像处理装置通过对待处理图像进行人脸特征提取处理，得到待处理图像的人脸特征数据。进而可依据该人脸特征数据，确定待处理图像中是否包含人脸区域，并在待处理图像中包含人脸区域的情况下，确定人脸区域的位置。In an implementation manner of performing face detection processing on the image to be processed, the image processing apparatus obtains face feature data of the image to be processed by performing face feature extraction processing on the image to be processed. Further, it is possible to determine whether the image to be processed includes a human face region according to the face feature data, and determine the position of the human face region if the image to be processed includes a human face region.
在另一种对待处理图像进行人脸检测处理的实现方式中，图像处理装置使用人脸检测神经网络对待处理图像进行处理，以确定待处理图像中是否包含人脸区域，并在待处理图像中包含人脸区域的情况下，确定人脸区域的位置。该人脸检测神经网络以带有标注信息的训练图像为训练数据训练得到，其中，标注信息包括训练图像中是否包含人脸，以及与人脸区域对应的人脸框的位置。In another implementation manner of performing face detection processing on the image to be processed, the image processing apparatus uses a face detection neural network to process the image to be processed, so as to determine whether the image to be processed contains a face area, and the image to be processed is If the face area is included, the position of the face area is determined. The face detection neural network is obtained by training a training image with annotation information as training data, wherein the annotation information includes whether the training image contains a human face, and the position of the face frame corresponding to the face area.
在待处理图像包含1个人脸区域的情况下，图像处理装置将该人脸区域作为待分割人物的人脸区域；在待处理图像包含的人脸区域的数量超过1的情况下，图像处理装置可从所有人脸区域中选取一个作为待分割人物的人脸区域。In the case that the image to be processed contains one face area, the image processing apparatus regards the face area as the face area of the person to be segmented; when the number of face areas contained in the image to be processed exceeds 1, the image processing apparatus One can be selected from all face regions as the face region of the person to be segmented.
S305、依据上述人脸区域的位置，从上述待处理图像中确定上述待分割人物的头发区域。S305. Determine the hair region of the person to be segmented from the image to be processed according to the position of the face region.
本申请实施例中，待分割人物的头发区域指待分割人物的头顶处的头发区域。由于头发区域与人脸区域之间的相对位置关系为经验关系，人脸区域的尺寸与头发区域的尺寸之间的关系也为经验关系，图像处理装置依据人脸区域的位置，可确定待分割人物的头发区域。In the embodiment of the present application, the hair region of the person to be segmented refers to the hair region at the top of the head of the person to be segmented. Since the relative positional relationship between the hair region and the face region is an empirical relationship, and the relationship between the size of the face region and the size of the hair region is also an empirical relationship, the image processing device can determine the to-be-segmented based on the position of the face region The character's hair area.
在一些实施例中，人脸区域的位置包括：人脸框的左上顶点在像素坐标系下的坐标、人脸框的长和人脸框的宽。图像处理装置依据人脸框的左上顶点的坐标、人脸框和头发框的位置关系，得到头发框左上顶点在像素坐标系下的坐标，其中，头发框包含待分割人物的头发区域。图像处理装置依据人脸框的长得到头发框的长，并依据人脸框的宽得到头发框的宽。这样，图像处理装置可确定头发框的位置以及头发框的尺寸，进而从待处理图像中确定头发区域。In some embodiments, the position of the face region includes: the coordinates of the upper left vertex of the face frame in the pixel coordinate system, the length of the face frame and the width of the face frame. The image processing device obtains the coordinates of the upper left vertex of the hair frame in the pixel coordinate system according to the coordinates of the upper left vertex of the face frame and the positional relationship between the face frame and the hair frame, wherein the hair frame contains the hair region of the person to be segmented. The image processing device obtains the length of the hair frame according to the length of the face frame, and obtains the width of the hair frame according to the width of the face frame. In this way, the image processing apparatus can determine the position of the hair frame and the size of the hair frame, and then determine the hair region from the image to be processed.
可选的，假设人脸框左上顶点的横坐标为x f，人脸框左上顶点的纵坐标为y f，人脸框的长为h f，人脸框的宽为w f，头发框左上顶点的横坐标为x h，头发框左上顶点的纵坐标为y h，头发框的长为h h，头发框的宽为w h。 Optionally, it is assumed that the abscissa of the upper left vertex of the face frame is x f , the ordinate of the upper left vertex of the face frame is y f , the length of the face frame is h f , the width of the face frame is w f , and the upper left of the hair frame is The abscissa of the vertex is x h , the ordinate of the upper left vertex of the hair frame is y h , the length of the hair frame is h h , and the width of the hair frame is w h .
x f，y f，h f，w f，x h，y h，h h，w h满足下式： x f , y f , h f , w f , x h , y h , h h , w h satisfy the following equations:
Figure PCTCN2021107017-appb-000001
Figure PCTCN2021107017-appb-000001
其中，α、β、δ和ε均为正数。可选的，α、β、δ和ε的取值范围均为[1.1，1.2]。where α, β, δ, and ε are all positive numbers. Optionally, the value ranges of α, β, δ, and ε are all [1.1, 1.2].
在得到待分割人物的头发区域后，图像处理装置在执行步骤302的过程中执行以下步骤：After obtaining the hair region of the person to be segmented, the image processing apparatus performs the following steps in the process of performing step 302:
S306、缩小上述第一毛掩码中的头发区域，得到头发掩码。S306 , reducing the hair region in the first hair mask to obtain a hair mask.
本步骤的实现方式与步骤302的实现方式相同。而与步骤302不同的是，本步骤中第一毛掩码即为待处理图像的头发掩码，第一毛掩码中的毛区域为头发区域。The implementation manner of this step is the same as that of step 302 . The difference from step 302 is that in this step, the first hair mask is the hair mask of the image to be processed, and the hair area in the first hair mask is the hair area.
S307、依据上述头发掩码，确定上述待分割人物的头发区域中的头发像素点，得到上述第二毛掩码。S307. Determine the hair pixels in the hair region of the person to be segmented according to the hair mask, and obtain the second hair mask.
由于待处理图像中除待分割人物的头发区域之外的头发像素点，第一毛掩码所对应的像素点中，也可能会包含除待分割人物的头发区域之外的头发像素点。显然，依据除待分割人物的头发区域之外的头发像素点的颜色值，确定待分割人物的头发毛颜色值范围，会降低待分割人物的头发毛颜色值范围的准确度。Since there are hair pixels in the image to be processed other than the hair region of the person to be segmented, the pixels corresponding to the first hair mask may also include hair pixels other than the hair region of the person to be segmented. Obviously, determining the hair color value range of the person to be segmented according to the color values of the hair pixels other than the hair region of the person to be segmented will reduce the accuracy of the hair color value range of the person to be segmented.
由于头发掩码所对应的像素点均为高置信度头发像素点，图像处理装置依据头发掩码，确定待分割人物的头发区域中的头发像素点，可确定待分割人物的头发区域中的高置信度头发像素点，进而得到第二毛掩码。即第二毛掩码所对应的像素点为待分割人物的头发区域中的高置信度头发像素点。Since the pixels corresponding to the hair mask are all high-confidence hair pixels, the image processing device determines the hair pixels in the hair region of the person to be segmented according to the hair mask, and can determine the high-confidence hair pixels in the hair region of the person to be segmented. Confidence hair pixels, and then get the second hair mask. That is, the pixels corresponding to the second hair mask are high-confidence hair pixels in the hair region of the person to be segmented.
例如，头发掩码包括元素a、元素b和元素c，其中，元素a的取值为第一值，元素b的取值为第二值，元素的取值为第三值。待分割人物的头发区域包括像素点A、像素点B、像素点C和像素点D。若元素a所对应的像素点为像素点A，元素b所对应的像素点为像素点B，元素c所对应的像素点为像素点C，那么，像素点A和像素点C为待分割人物的头发区域中的高置信度头发像素点。For example, the hair mask includes element a, element b, and element c, wherein the value of element a is the first value, the value of element b is the second value, and the value of element b is the third value. The hair region of the person to be segmented includes pixel A, pixel B, pixel C, and pixel D. If the pixel point corresponding to element a is pixel point A, the pixel point corresponding to element b is pixel point B, and the pixel point corresponding to element c is pixel point C, then pixel point A and pixel point C are the characters to be segmented. High-confidence hair pixels in the hair region of .
这样，图像处理装置再依据第二毛掩码所对应的像素点的颜色值得到待分割对象的毛颜色值范围，可提高毛颜色值范围的准确度。In this way, the image processing device obtains the hair color value range of the object to be segmented according to the color value of the pixel point corresponding to the second hair mask, which can improve the accuracy of the hair color value range.
作为一种可选的实施方式，图像处理装置在执行步骤304的过程中执行以下步骤：As an optional implementation manner, the image processing apparatus performs the following steps in the process of performing step 304:
S308、对上述待处理图像进行人脸检测处理，得到至少一个人脸框。S308. Perform face detection processing on the above image to be processed to obtain at least one face frame.
本步骤中的人脸检测处理的实现方式可参见步骤304中人脸检测处理的实现方式。上述至少一个人脸框中每个人脸框均包含一个人脸区域，且任意两个人脸框所包含的人脸区域不同。For the implementation manner of the face detection processing in this step, reference may be made to the implementation manner of the face detection processing in step 304 . Each of the above-mentioned at least one face frame includes a face area, and any two face frames include different face areas.
S309、选取上述至少一个人脸框中尺度最大的人脸框，作为上述待分割人物的人脸框。S309: Select the face frame with the largest scale in the at least one face frame as the face frame of the person to be segmented.
本申请实施例中，近处的物体在图像中尺度大，远处的物体在图像中尺度小。本申请实施例中的“远”指图像中，物体对应的真实物体与采集上述图像的成像设备之间的距离远，“近”指图像中物体对应的真实物体与采集上述图像的成像设备之间的距离近。尺度最大的人脸框包含的人脸为距离采集待处理图像的成像设备最近的人脸。In the embodiment of the present application, the scale of the near object in the image is large, and the scale of the distant object in the image is small. In the embodiments of this application, "far" refers to the distance between the real object corresponding to the object and the imaging device that collects the above image in the image, and "near" refers to the distance between the real object corresponding to the object in the image and the imaging device that collects the above image. close distance. The face contained in the face frame with the largest scale is the face closest to the imaging device that collects the image to be processed.
在某些场景下拍摄人物照的情况下，由于拍摄场景中有很多除拍摄对象之外的人，拍摄得到的图像中会包含除拍摄对象的人脸之外的人脸。因此，图像处理装置从至少一个人脸框中选取尺度最大的人脸框，作为待分割人物的人脸框，可实现将上述拍摄对象作为待分割人物。In the case of taking a picture of a person in some scenes, since there are many people other than the subject in the shooting scene, the captured image will contain faces other than the face of the subject. Therefore, the image processing apparatus selects the face frame with the largest scale from at least one face frame as the face frame of the person to be segmented, so that the above-mentioned photographing object can be regarded as the person to be segmented.
例如，小明在景点旅游时，觉得景点的风景很好，故请小红为他拍一张照片。但是景点的游客众多，在小红给小明拍的照片中，小明后面有很多游客。图像处理装置在对小明的照片进行处理的情况下，确定照片中尺度最大的人脸框所包含的人脸为小明的人脸，并对小明的人脸区域进行分割处理。For example, when Xiao Ming was traveling in a scenic spot, he felt that the scenery of the scenic spot was very good, so he asked Xiao Hong to take a photo for him. But there are many tourists in the scenic spot. In the photos Xiaohong took of Xiaoming, there are many tourists behind Xiaoming. In the case of processing Xiaoming's photo, the image processing device determines that the face contained in the face frame with the largest scale in the photo is Xiaoming's face, and performs segmentation processing on Xiaoming's face area.
在一些实施例中，图像处理装置分别确定每个人脸框的测度，并将测度最大的人脸框作为尺度最大的人脸框。其中，人脸框的测度指人脸框的最大边长。例如，人脸框aa的长为10，人脸框aa的宽6，那么人脸框aa的测度为10；又例如，人脸框bb的长为7，人脸框bb的宽9，那么人脸框bb的测度为9。In some embodiments, the image processing apparatus separately determines the measure of each face frame, and uses the face frame with the largest measurement as the face frame with the largest scale. The measurement of the face frame refers to the maximum side length of the face frame. For example, the length of the face frame aa is 10, and the width of the face frame aa is 6, then the measurement of the face frame aa is 10; for another example, the length of the face frame bb is 7, and the width of the face frame bb is 9, then The measure of the face frame bb is 9.
S310、将上述待分割人物的人脸框的位置和上述待分割人物的人脸框的尺寸，作为上述待分割人物的人脸区域的位置。S310. Use the position of the face frame of the person to be divided and the size of the face frame of the person to be divided as the position of the face region of the person to be divided.
在一些实施例中，图像处理装置在执行步骤309的过程中执行以下步骤：In some embodiments, the image processing apparatus performs the following steps in the process of performing step 309:
S311、选取上述至少一个人脸框中面积最大的人脸框，作为上述待分割人物的人脸框。S311. Select the face frame with the largest area in the at least one face frame as the face frame of the person to be segmented.
在一些实施例中，图像处理装置在执行步骤203之前，还执行以下步骤：In some embodiments, before performing step 203, the image processing apparatus further performs the following steps:
S312、对上述待处理图像进行人像分割处理，得到第一人像掩码。S312: Perform portrait segmentation processing on the above-mentioned to-be-processed image to obtain a first portrait mask.
本步骤中的人像分割处理用于确定待处理图像中语义信息为人像的像素点(下文将称为人像像素点)。图像处理装置通过执行步骤312，确定待处理图像的中的人像像素点，得到第一人像掩码。即第一人像掩码中取值为第一值的元素所对应的像素点为人像像素点。The portrait segmentation processing in this step is used to determine pixels whose semantic information in the image to be processed is a portrait (hereinafter referred to as portrait pixels). The image processing apparatus determines the portrait pixels in the image to be processed by executing step 312, and obtains the first portrait mask. That is, the pixel corresponding to the element whose value is the first value in the first portrait mask is the portrait pixel.
S313、依据上述第一人像掩码，确定上述待分割人物的头发区域中的人像像素点，得到第二人像掩码。S313. Determine the portrait pixels in the hair region of the person to be segmented according to the first portrait mask to obtain a second portrait mask.
与步骤307中，依据头发掩码，确定待分割人物的头发区域中的高置信度头发像素点的实现方式相同。图像处理装置依据第一人像掩码，确定待分割人物的头发区域中的人像像素点，得到第二人像掩码。即第二人像掩码中取值为第一值的元素所对应的像素点为，待分割人物的头发区域中的人像像素点。The implementation manner of determining the high-confidence hair pixels in the hair region of the person to be segmented is the same as in step 307 , according to the hair mask. The image processing device determines the portrait pixels in the hair region of the person to be segmented according to the first portrait mask, and obtains the second portrait mask. That is, the pixel corresponding to the element whose value is the first value in the second portrait mask is the portrait pixel in the hair region of the person to be segmented.
在得到第二人像掩码后，图像处理装置在执行步骤203的过程中执行以下步骤：After obtaining the second portrait mask, the image processing apparatus performs the following steps in the process of performing step 203:
S314、确定上述毛区域内颜色值处于上述毛颜色值范围之外的像素点，得到至少一个第一像素点。S314: Determine the pixels whose color values are outside the range of the hair color values in the hair region, and obtain at least one first pixel.
本步骤的实现方式与步骤203的实现方式相同。应理解，在本步骤中，得到的不是背景像素点，而是至少一个第一像素点。The implementation of this step is the same as that of step 203 . It should be understood that, in this step, what is obtained is not a background pixel, but at least one first pixel.
S315、依据上述第二人像掩码，确定上述至少一个第一像素点中的人像像素点，得到至少一个第二像素点。S315. Determine a portrait pixel in the at least one first pixel according to the second portrait mask to obtain at least one second pixel.
图像处理装置从至少一个第一像素点中，确定与第二人像掩码中取值为第一值的元素对应的像素点，作为第二像素点。例如，第二人像掩码包括元素a和元素b，其中，元素a的取值为第一值，元素b的取值为第二值。至少一个第一像素点包括像素点A和像素点B。 若像素点A为元素a所对应的像素点，像素点B为元素b所对应的像素点，那么至少一个第二像素点包括像素点A。The image processing apparatus determines, from the at least one first pixel point, a pixel point corresponding to an element in the second portrait mask whose value is the first value, as a second pixel point. For example, the second portrait mask includes element a and element b, wherein the value of element a is the first value, and the value of element b is the second value. At least one first pixel point includes pixel point A and pixel point B. If the pixel point A is the pixel point corresponding to the element a, and the pixel point B is the pixel point corresponding to the element b, then at least one second pixel point includes the pixel point A.
由于图像处理装置在执行步骤314的过程中，可能将待分割人物的头发像素点误识别为非头发像素点，从而确定该头发像素点为第一像素点。这样，至少一个第一像素点将包括头发像素点。Since the image processing apparatus may mistakenly identify the hair pixel of the person to be segmented as a non-hair pixel in the process of executing step 314, the hair pixel is determined as the first pixel. In this way, at least one of the first pixels will include hair pixels.
因为待分割人物的头发像素点属于待分割人物的人像像素点，图像处理装置通过执行步骤315，可确定至少一个第一像素点中的头发像素点(即第二像素点)。Because the hair pixels of the person to be segmented belong to the portrait pixels of the person to be segmented, the image processing apparatus can determine at least one hair pixel (ie, the second pixel) in the first pixel by executing step 315 .
例如，待分割人物的头发区域(即毛区域)包括像素点a、像素点b和像素点c，其中，像素点a和像素点b均为头发像素点，像素点c为背景像素点。由于像素点b的颜色值处于毛颜色值范围外，图像处理装置确定像素点b为背景像素点，并将像素点b作为第二像素点。此时，至少一个第二像素点包括像素点b和像素点c。For example, the hair region (ie, the hair region) of the person to be segmented includes pixel a, pixel b, and pixel c, where pixel a and pixel b are both hair pixels, and pixel c is a background pixel. Since the color value of the pixel point b is outside the range of the hair color value, the image processing apparatus determines the pixel point b as the background pixel point, and uses the pixel point b as the second pixel point. At this time, the at least one second pixel point includes pixel point b and pixel point c.
第二人像掩码包括元素U、元素V和元素W，其中，元素U的取值和元素V的取值均为第一值，元素W的取值为第二值，且元素U所对应的像素点为像素点u，元素V所对应的像素点为像素点v，元素W所对应的像素点为像素点w。这样，图像处理装置依据第二人像掩码，可确定像素点v为头发区域的人像像素点(即头发像素点)，从而确定像素点v为第二像素点。The second portrait mask includes element U, element V, and element W, wherein the value of element U and the value of element V are both the first value, the value of element W is the second value, and the value corresponding to element U The pixel point is the pixel point u, the pixel point corresponding to the element V is the pixel point v, and the pixel point corresponding to the element W is the pixel point w. In this way, according to the second portrait mask, the image processing device can determine that the pixel point v is a portrait pixel point (ie, a hair pixel point) of the hair region, thereby determining that the pixel point v is the second pixel point.
S316、去除上述至少一个第一像素点中的上述至少一个第二像素点，得到上述背景像素点。S316: Remove the at least one second pixel point from the at least one first pixel point to obtain the background pixel point.
由于第二像素点为头发像素点，图像处理装置通过执行步骤316，可提高背景像素点的准确度。Since the second pixel is a hair pixel, the image processing apparatus can improve the accuracy of the background pixel by performing step 316 .
在一些实施例中，图像处理装置在执行步骤303的过程中执行以下步骤：In some embodiments, the image processing apparatus performs the following steps in the process of performing step 303:
S317、获取毛颜色值区间长度。S317. Obtain the length of the hair color value interval.
本申请实施例中，毛颜色值区间长度指毛颜色值范围所对应的区间的长度。例如，毛颜色值范围为[50，80]，此时，毛颜色值范围的跨度为30。In the embodiments of the present application, the length of the hair color value interval refers to the length of the interval corresponding to the hair color value range. For example, the range of fur color values is [50, 80], at this time, the span of the fur color value range is 30.
S318、统计上述第二毛掩码所对应的像素点的颜色直方图。S318: Count the color histograms of the pixels corresponding to the second hair mask.
S319、依据上述颜色直方图，得到所对应的像素点的数量最多、且长度为上述毛颜色值区间长度的毛颜色值区间，作为上述毛颜色值范围。S319. According to the color histogram, obtain a hair color value interval with the largest number of corresponding pixel points and the length of which is the length of the hair color value interval, as the hair color value range.
在一些实施例中，上述颜色值包括灰度值，上述颜色值直方图包括灰度直方图。In some embodiments, the color values include grayscale values, and the color value histogram includes a grayscale histogram.
在图像处理领域，为突出图像中某个对象(下文将称为留色对象)，通常会对该留色对象进行留色处理。这里，通过改变图像中除留色对象之外的像素点区域(下文将称为非待分割对象区域)的颜色，并保留留色对象所覆盖的像素点区域(下文将称为前景区域)的颜色，以取得突出留色对象的效果。In the field of image processing, in order to highlight an object in an image (hereinafter referred to as a color-retaining object), the color-retaining object is usually subjected to color-retaining processing. Here, by changing the color of the pixel area (hereinafter referred to as the non-to-be-segmented object area) in the image except for the color-retaining object, and retaining the pixel area covered by the color-retaining object (hereinafter referred to as the foreground area) Color to achieve the effect of highlighting objects with color.
例如，在留色对象是人像的情况下，可通过对图像进行人像留色处理，保留人像的颜色，并将非人像区域的颜色变为灰色，得到人像留色图像。这样，在人像留色图像中，可突出人像，弱化非待分割对象区域。For example, in the case where the color-retaining object is a portrait, a portrait-retaining image can be obtained by performing a portrait color-retaining process on the image, retaining the color of the portrait, and changing the color of the non-portrait area to gray. In this way, in the color-preserving portrait image, the portrait can be highlighted, and the non-to-be-segmented object area can be weakened.
在改变非待分割对象区域的颜色之前，需要先对图像进行留色对象分割处理，以从图像中确定前景区域和非待分割对象区域。因此，对留色对象的分割准确度直接影响留色对象的留色效果。图像处理装置基于前文所提供的技术方案，还可对待分割对象进行留色处理，以提升留色效果。Before changing the color of the non-to-be-segmented object area, it is necessary to perform color-retained object segmentation processing on the image to determine the foreground area and the non-to-be-segmented target area from the image. Therefore, the segmentation accuracy of the color-retaining objects directly affects the color-retaining effect of the color-retaining objects. Based on the technical solutions provided above, the image processing apparatus can also perform color retention processing on the object to be segmented, so as to improve the color retention effect.
在一些实施例中，图像处理装置还执行以下步骤：In some embodiments, the image processing apparatus further performs the following steps:
S320、改变上述背景像素点的颜色和非待分割对象区域的颜色，得到上述待分割对象的留色图像，其中，在上述留色图像中，上述非待分割对象区域的颜色和背景像素点的颜色相同。S320, changing the color of the above-mentioned background pixels and the color of the non-to-be-segmented object area, to obtain the above-mentioned color-retained image of the to-be-segmented object, wherein, in the above-mentioned color-retained image, the color of the above-mentioned non-to-be-segmented object area and the background pixel point Same color.
本步骤中，图像处理装置可将背景像素点的颜色和非待分割对象区域的颜色改变为任意颜色。例如，图像处理装置可将背景像素点的颜色和非待分割对象区域的颜色均改变为 黑色；又例如，图像处理装置可将背景像素点的颜色和非待分割对象区域的颜色均改变为白色；再例如，图像处理装置可将背景像素点的颜色和非待分割对象区域的颜色均改变为红色。In this step, the image processing device can change the color of the background pixels and the color of the non-to-be-divided object area to any color. For example, the image processing device may change the color of the background pixels and the color of the non-to-be-divided object area to black; for another example, the image processing device may change both the background pixel color and the color of the non-to-be-divided object area to white For another example, the image processing device may change the color of the background pixels and the color of the non-to-be-divided object area to red.
可选的，在将背景像素点的颜色和非待分割对象区域的颜色改变为相同的同时，图像处理装置还将保留前景区域的颜色。这样，可得到待分割对象的留色图像。Optionally, while changing the color of the background pixels and the color of the non-to-be-divided object area to be the same, the image processing apparatus will also retain the color of the foreground area. In this way, a color image of the object to be segmented can be obtained.
可选的，在前景区域为人像的情况下，在将背景像素点的颜色和非待分割对象区域的颜色改变为相同的同时，图像处理装置还可对前景区域进行美颜处理。Optionally, when the foreground area is a portrait, while changing the color of the background pixels and the color of the non-to-be-divided object area to be the same, the image processing apparatus may also perform beautification processing on the foreground area.
在一些实施例中，图像处理装置在执行步骤320后还执行以下步骤：In some embodiments, after performing step 320, the image processing apparatus further performs the following steps:
S321、对上述留色图像中的上述背景像素点和上述留色图像中的上述非待分割对象区域进行虚化处理，得到上述待分割对象的留色虚化图像。S321 , performing a blurring process on the background pixels in the color-retaining image and the non-object-to-be-segmented region in the color-retaining image, to obtain a color-retaining blurred image of the object to be segmented.
图像处理装置通过对留色图像中的背景像素点，以及留色图像中的非待分割对象区域进行虚化处理，可更加突出留色图像中的待分割对象，得到待分割对象的留色虚化图像。The image processing device can further highlight the object to be segmented in the color image by blurring the background pixels in the color image and the non-to-be-segmented area in the color image, so as to obtain the color-retention blur of the object to be segmented. image.
在一些实施例中，图像处理装置在执行步骤320的过程中执行以下步骤：In some embodiments, the image processing apparatus performs the following steps in the process of performing step 320:
S322、对上述背景像素点和上述非待分割对象区域进行灰度化处理，得到上述留色图像。S322 , performing grayscale processing on the background pixels and the non-to-be-segmented area to obtain the color-retained image.
在一些实施例中，在图像处理装置执行步骤313得到第二毛掩码的情况下，图像处理装置还执行以下步骤：In some embodiments, when the image processing apparatus performs step 313 to obtain the second hair mask, the image processing apparatus further performs the following steps:
S323、对上述第二毛掩码与上述第二人像掩码进行融合处理，得到优化后的前景掩码。S323. Perform fusion processing on the second hair mask and the second portrait mask to obtain an optimized foreground mask.
本步骤中，图像处理装置通过对第二毛掩码与第二人像掩码进行融合处理，确定第二毛掩码与第二人像掩码中取值为第一值的元素的并集，得到优化后的前景掩码。In this step, the image processing device determines the union of the elements with the first value in the second hair mask and the second portrait mask by performing fusion processing on the second hair mask and the second portrait mask, to obtain The optimized foreground mask.
例如，图3所示为第二毛掩码，图4所示为第二人像掩码。图像处理装置取第二毛掩码和第二人像掩码的并集，得到图5所示的优化后的前景掩码。For example, Figure 3 shows the second hair mask, and Figure 4 shows the second portrait mask. The image processing device takes the union of the second hair mask and the second portrait mask to obtain the optimized foreground mask shown in FIG. 5 .
S324、根据上述优化后的前景掩码，确定上述待处理图像中的优化背景区域。S324: Determine the optimized background area in the above-mentioned image to be processed according to the above-mentioned optimized foreground mask.
本步骤中，优化背景区域包括优化后的前景掩码中取值为第二值的元素所对应的像素点，即为优化后的前景掩码所对应的像素点之外的像素点。例如，在图5所示的优化后的前景掩码中，第一值为1，第二值为0。优化背景区域包括取值为0的元素所对应的像素点。In this step, the optimized background area includes the pixels corresponding to the elements in the optimized foreground mask whose value is the second value, that is, the pixels other than the pixels corresponding to the optimized foreground mask. For example, in the optimized foreground mask shown in FIG. 5 , the first value is 1, and the second value is 0. The optimized background area includes pixels corresponding to elements whose value is 0.
S325、改变上述优化背景区域的颜色，得到留色图像。S325 , changing the color of the above-mentioned optimized background area to obtain a color-preserving image.
若将通过对待处理图像进行人像分割处理，得到的待分割人物的人像掩码称为中间人像分割掩码。基于本申请实施例提供的技术方案，对中间人像分割掩码进行处理，得到优化后的前景掩码，可修正中间人像分割掩码中被误识别为前景像素点的背景像素点。If the image to be processed is subjected to portrait segmentation processing, the portrait mask of the person to be segmented obtained is called an intermediate portrait segmentation mask. Based on the technical solutions provided by the embodiments of the present application, the intermediate portrait segmentation mask is processed to obtain an optimized foreground mask, and the background pixels in the intermediate portrait segmentation mask that are mistakenly identified as foreground pixels can be corrected.
例如，在图6所示的待处理图像中，像素点A、像素点B、像素点C、像素点D、像素点E、像素点H、像素点J、像素点K均为背景像素点，像素点F、像素点G、像素点I、像素点L、像素点M、像素点N、像素点O、像素点P均为人像像素点，其中，像素点J和像素点K即为镂空区域内的背景像素点。通过对该待处理图像进行分割处理，得到图7所示的中间人像分割掩码，其中，取值为1的元素所对应的像素点为人像像素点，取值为0的元素所对应的像素点为背景像素点。显然，在图7所示的中间人像分割掩码中，像素点J和像素点K被误判为人像像素点。基于本申请实施例提供的技术方案，得到图8所示的优化后的前景掩码，在该优化后的前景掩码中，取值为1的元素所对应的像素点为人像像素点，取值为0的元素所对应的像素点为背景像素点。显然，在该优化后的前景掩码中，像素点J和像素点K被修正为背景像素点。For example, in the to-be-processed image shown in FIG. 6, pixel A, pixel B, pixel C, pixel D, pixel E, pixel H, pixel J, and pixel K are all background pixels, The pixel point F, the pixel point G, the pixel point I, the pixel point L, the pixel point M, the pixel point N, the pixel point O, and the pixel point P are all portrait pixels, and the pixel point J and the pixel point K are the hollow area. background pixels inside. By performing segmentation processing on the to-be-processed image, the middle portrait segmentation mask shown in FIG. 7 is obtained, wherein the pixel corresponding to the element whose value is 1 is the portrait pixel, and the pixel corresponding to the element whose value is 0 is the portrait pixel. Points are background pixels. Obviously, in the middle portrait segmentation mask shown in Fig. 7, pixel point J and pixel point K are misjudged as portrait pixels. Based on the technical solutions provided in the embodiments of the present application, the optimized foreground mask shown in FIG. 8 is obtained. In the optimized foreground mask, the pixels corresponding to the elements with a value of 1 are portrait pixels, and The pixel corresponding to the element whose value is 0 is the background pixel. Obviously, in the optimized foreground mask, pixel point J and pixel point K are corrected as background pixels.
因此，图像处理装置通过改变优化背景区域的颜色得到留色图像，可提升留色效果。Therefore, the image processing apparatus obtains a color-retaining image by changing the color of the optimized background area, which can improve the color-retaining effect.
可选的，在图像处理装置执行步骤313得到第二毛掩码的情况下，图像处理装置可在执行步骤322的过程中执行以下步骤：Optionally, when the image processing apparatus performs step 313 to obtain the second hair mask, the image processing apparatus may perform the following steps in the process of performing step 322:
S326、对上述第二毛掩码与上述第二人像掩码进行融合处理，得到优化后的前景掩码。S326. Perform fusion processing on the second hair mask and the second portrait mask to obtain an optimized foreground mask.
本步骤的实现方式可参见步骤323。For the implementation of this step, refer to step 323.
S327、根据上述优化后的前景掩码，确定上述待处理图像中的优化背景区域。S327: Determine the optimized background area in the above-mentioned image to be processed according to the above-mentioned optimized foreground mask.
本步骤的实现方式可参见步骤323。For the implementation of this step, refer to step 323.
S328、对上述优化背景区域进行灰度化处理，得到上述留色图像。S328 , performing grayscale processing on the above-mentioned optimized background area to obtain the above-mentioned color-retained image.
图像处理装置通过执行步骤326～步骤328得到留色图像，可减少数据处理量，提升处理速度。The image processing apparatus obtains the color-preserving image by executing steps 326 to 328, which can reduce the amount of data processing and improve the processing speed.
需要说明的是，本申请实施例的技术方案可以应用在人像虚化、人像虚化留色和视频虚化等方面。It should be noted that the technical solutions of the embodiments of the present application can be applied to portrait blurring, portrait blurring and color retention, and video blurring.
本领域技术人员可以理解，上述方法中，各步骤的撰写顺序并不意味着严格的执行顺序而对实施过程构成任何限定，各步骤的执行顺序应当以其功能和可能的内在逻辑确定。Those skilled in the art can understand that, in the above method, the writing order of each step does not mean a strict execution order and constitutes any limitation on the implementation process, and the execution order of each step should be determined by its function and possible internal logic.
上述详细阐述了本申请实施例的方法，下面提供了本申请实施例的装置。The methods of the embodiments of the present application are described in detail above, and the apparatuses of the embodiments of the present application are provided below.
请参阅图9，图9为本申请实施例提供的一种图像处理装置的结构示意图，该图像处理装置1包括：第一获取部分11、第二获取部分12、处理部分13，其中：Please refer to FIG. 9. FIG. 9 is a schematic structural diagram of an image processing apparatus provided by an embodiment of the application. The image processing apparatus 1 includes: a first acquisition part 11, a second acquisition part 12, and a processing part 13, wherein:
第一获取部分11，被配置为获取待处理图像；所述待处理图像包含待分割对象的毛区域；The first acquisition part 11 is configured to acquire an image to be processed; the image to be processed includes the hair region of the object to be segmented;
第二获取部分12，被配置为获取所述待分割对象的毛颜色值范围；The second acquiring part 12 is configured to acquire the range of hair color values of the object to be segmented;
处理部分13，被配置为确定所述毛区域内颜色值处于所述毛颜色值范围之外的像素点，作为背景像素点。The processing part 13 is configured to determine the pixels whose color values are outside the range of the hair color values in the hair area as background pixels.
结合本申请任一实施方式，所述第二获取部分12，还被配置为：With reference to any embodiment of the present application, the second obtaining part 12 is further configured as:
对所述待处理图像进行毛分割处理，得到第一毛掩码；Perform hair segmentation processing on the to-be-processed image to obtain a first hair mask;
缩小所述第一毛掩码中的毛区域，得到第二毛掩码；reducing the hair area in the first hair mask to obtain a second hair mask;
依据所述第二毛掩码所对应的像素点的颜色值，得到所述毛颜色值范围。The hair color value range is obtained according to the color value of the pixel corresponding to the second hair mask.
结合本申请任一实施方式，所述待分割对象包括待分割人物，所述毛包括头发；With reference to any embodiment of the present application, the object to be segmented includes a person to be segmented, and the hair includes hair;
所述处理部分13，还被配置为在所述缩小所述第一毛掩码中的毛区域，得到第二毛掩码之前，对所述待处理图像进行人脸检测处理，得到所述待分割人物的人脸区域的位置；以及，还被配置为依据所述人脸区域的位置，从所述待处理图像中确定所述待分割人物的头发区域；The processing part 13 is further configured to perform face detection processing on the to-be-processed image before the hair region in the first hair mask is reduced to obtain the second hair mask to obtain the to-be-processed image. the position of the face area of the segmented character; and, also configured to determine the hair area of the character to be segmented from the to-be-processed image according to the location of the face area;
所述第二获取部分12，还被配置为：The second obtaining part 12 is further configured to:
缩小所述第一毛掩码中的头发区域，得到头发掩码；reducing the hair area in the first hair mask to obtain a hair mask;
依据所述头发掩码，确定所述待分割人物的头发区域中的头发像素点，得到所述第二毛掩码。According to the hair mask, hair pixels in the hair region of the person to be segmented are determined to obtain the second hair mask.
结合本申请任一实施方式，所述处理部分13，还被配置为：With reference to any embodiment of the present application, the processing part 13 is further configured to:
对所述待处理图像进行人脸检测处理，得到至少一个人脸框；performing face detection processing on the to-be-processed image to obtain at least one face frame;
选取所述至少一个人脸框中尺度最大的人脸框，作为所述待分割人物的人脸框；Select the face frame with the largest scale in the at least one face frame as the face frame of the character to be segmented;
将所述待分割人物的人脸框的位置和所述待分割人物的人脸框的尺寸，作为所述待分割人物的人脸区域的位置。The position of the face frame of the person to be divided and the size of the face frame of the person to be divided are used as the position of the face region of the person to be divided.
结合本申请任一实施方式，所述处理部分13，还被配置为：With reference to any embodiment of the present application, the processing part 13 is further configured to:
选取所述至少一个人脸框中面积最大的人脸框，作为所述待分割人物的人脸框。The face frame with the largest area in the at least one face frame is selected as the face frame of the person to be segmented.
结合本申请任一实施方式，所述处理部分13，还被配置为，在所述确定所述毛区域内颜色值处于所述毛颜色值范围之外的像素点，作为背景像素点之前，对所述待处理图像进行人像分割处理，得到第一人像掩码；With reference to any embodiment of the present application, the processing part 13 is further configured to, before determining that the pixel points in the hair area whose color values are outside the hair color value range are used as background pixels, Performing portrait segmentation processing on the to-be-processed image to obtain a first portrait mask;
以及依据所述第一人像掩码，确定所述待分割人物的头发区域中的人像像素点，得到第二人像掩码；And according to the first portrait mask, determine the portrait pixels in the hair region of the person to be segmented, and obtain the second portrait mask;
以及确定所述毛区域内颜色值处于所述毛颜色值范围之外的像素点，得到至少一个第一像素点；and determining the pixels whose color values in the hair area are outside the range of the hair color values to obtain at least one first pixel;
以及依据所述第二人像掩码，确定所述至少一个第一像素点中的人像像素点，得到至少一个第二像素点；and determining a portrait pixel in the at least one first pixel according to the second portrait mask to obtain at least one second pixel;
以及去除所述至少一个第一像素点中的所述至少一个第二像素点，得到所述背景像素点。and removing the at least one second pixel point in the at least one first pixel point to obtain the background pixel point.
结合本申请任一实施方式，所述第二获取部分12，还被配置为：With reference to any embodiment of the present application, the second obtaining part 12 is further configured as:
获取毛颜色值区间长度；Get the length of the hair color value interval;
统计所述第二毛掩码所对应的像素点的颜色直方图；Counting the color histogram of the pixel corresponding to the second hair mask;
依据所述颜色直方图，得到所对应的像素点的数量最多、且长度为所述毛颜色值区间长度的毛颜色值区间，作为所述毛颜色值范围。According to the color histogram, a hair color value interval with the largest number of corresponding pixel points and the length of which is the length of the hair color value interval is obtained as the hair color value range.
结合本申请任一实施方式，所述待处理图像还包括非待分割对象区域，所述非待分割对象区域为所述待处理图像中除所述待分割对象所覆盖的像素点区域之外的像素点区域；With reference to any embodiment of the present application, the image to be processed further includes a non-to-be-segmented object area, and the non-to-be-segmented object area is an area of the to-be-processed image other than the pixel area covered by the to-be-segmented object. pixel area;
所述处理部分13，还被配置为改变所述背景像素点的颜色和非待分割对象区域的颜色，得到所述待分割对象的留色图像；在所述留色图像中，所述非待分割对象区域的颜色和所述背景像素点的颜色相同。The processing part 13 is further configured to change the color of the background pixels and the color of the non-to-be-segmented object area to obtain a color-retained image of the to-be-segmented object; The color of the segmented object area is the same as the color of the background pixels.
结合本申请任一实施方式，所述处理部分13，还被配置为：With reference to any embodiment of the present application, the processing part 13 is further configured to:
对所述留色图像中的所述背景像素点和所述留色图像中的所述非待分割对象区域进行虚化处理，得到所述待分割对象的留色虚化图像。Perform a blurring process on the background pixels in the color-retained image and the non-to-be-segmented object area in the color-retained image to obtain a color-retained blurred image of the object to be segmented.
结合本申请任一实施方式，所述处理部分13，还被配置为：With reference to any embodiment of the present application, the processing part 13 is further configured to:
对所述背景像素点和所述非待分割对象区域进行灰度化处理，得到所述留色图像。Grayscale processing is performed on the background pixel points and the non-to-be-segmented object area to obtain the color-retained image.
结合本申请任一实施方式，所述处理部分13，还被配置为：With reference to any embodiment of the present application, the processing part 13 is further configured to:
对所述第二毛掩码与所述第二人像掩码进行融合处理，得到优化后的前景掩码；Perform fusion processing on the second hair mask and the second portrait mask to obtain an optimized foreground mask;
根据所述优化后的前景掩码，确定所述待处理图像中的优化背景区域；According to the optimized foreground mask, determine the optimized background area in the to-be-processed image;
改变所述优化背景区域的颜色，得到留色图像。Change the color of the optimized background area to obtain a color-retained image.
本实施例中，第一获取部分11和第二获取部分12可以是相同的数据接口，第一获取部分11和第二获取部分12也可以是不同的数据接口，处理部分13可以是处理器。In this embodiment, the first acquisition part 11 and the second acquisition part 12 may be the same data interface, the first acquisition part 11 and the second acquisition part 12 may also be different data interfaces, and the processing part 13 may be a processor.
在一些实施例中，本申请实施例提供的装置具有的功能或包含的模块可以用于执行上文方法实施例描述的方法，其具体实现可以参照上文方法实施例的描述。In some embodiments, the functions or modules included in the apparatus provided in the embodiments of the present application may be used to execute the methods described in the above method embodiments, and reference may be made to the descriptions in the above method embodiments for specific implementation.
在本申请实施例以及其他的实施例中，“部分”可以是部分电路、部分处理器、部分程序或软件等等，当然也可以是单元，还可以是模块也可以是非模块化的。In the embodiments of the present application and other embodiments, a "part" may be a part of a circuit, a part of a processor, a part of a program or software, etc., of course, a unit, a module or a non-modularity.
图10为本申请实施例提供的一种图像处理装置的硬件结构示意图。该图像处理装置2包括处理器21，存储器22，输入装置23，输出装置24。该处理器21、存储器22、输入装置23和输出装置24通过连接器相耦合，该连接器包括各类接口、传输线或总线等等，本申请实施例对此不作限定。应当理解，本申请的各个实施例中，耦合是指通过特定方式的相互联系，包括直接相连或者通过其他设备间接相连，例如可以通过各类接口、传输线、总线等相连。FIG. 10 is a schematic diagram of a hardware structure of an image processing apparatus according to an embodiment of the present application. The image processing device 2 includes a processor 21 , a memory 22 , an input device 23 , and an output device 24 . The processor 21, the memory 22, the input device 23, and the output device 24 are coupled through a connector, and the connector includes various types of interfaces, transmission lines, or buses, etc., which are not limited in this embodiment of the present application. It should be understood that, in various embodiments of the present application, coupling refers to mutual connection in a specific manner, including direct connection or indirect connection through other devices, such as various interfaces, transmission lines, and buses.
处理器21可以是一个或多个图形处理器(graphics processing unit，GPU)，在处理器21是一个GPU的情况下，该GPU可以是单核GPU，也可以是多核GPU。可选的，处理器21可以是多个GPU构成的处理器组，多个处理器之间通过一个或多个总线彼此耦合。可选的，该处理器还可以为其他类型的处理器等等，本申请实施例不作限定。The processor 21 may be one or more graphics processing units (graphics processing units, GPUs). In the case where the processor 21 is a GPU, the GPU may be a single-core GPU or a multi-core GPU. Optionally, the processor 21 may be a processor group composed of multiple GPUs, and the multiple processors are coupled to each other through one or more buses. Optionally, the processor may also be another type of processor, etc., which is not limited in this embodiment of the present application.
存储器22可被配置为存储计算机程序指令，以及被配置为执行本申请方案的程序代码在内的各类计算机程序代码。可选地，存储器包括但不限于是随机存储记忆体(random access memory，RAM)、只读存储器(read-only memory，ROM)、可擦除可编程只读存储器(erasable programmable read only memory，EPROM)、或便携式只读存储器(compact disc read-only memory，CD-ROM)，该存储器被配置为相关指令及数据。The memory 22 may be configured to store computer program instructions, as well as various types of computer program code, including program code configured to execute the aspects of the present application. Optionally, the memory includes, but is not limited to, random access memory (RAM), read-only memory (read-only memory, ROM), erasable programmable read-only memory (erasable programmable read only memory, EPROM) ), or a portable disc read-only memory (CD-ROM), which is configured for associated instructions and data.
输入装置23被配置为输入数据和/或信号，以及输出装置24被配置为输出数据和/或 信号。输入装置23和输出装置24可以是独立的器件，也可以是一个整体的器件。The input device 23 is configured to input data and/or signals, and the output device 24 is configured to output data and/or signals. The input device 23 and the output device 24 may be independent devices or may be an integral device.
可理解，本申请实施例中，存储器22不仅可被配置为存储相关指令，还可被配置为存储相关数据，如该存储器22可被配置为存储通过输入装置23获取的待处理图像，又或者该存储器22还可被配置为存储通过处理器21得到的背景像素点等等，本申请实施例对于该存储器中所存储的数据不作限定。It can be understood that in this embodiment of the present application, the memory 22 may be configured not only to store relevant instructions, but also to store relevant data. For example, the memory 22 may be configured to store images to be processed obtained through the input device 23 , or The memory 22 may also be configured to store background pixels and the like obtained by the processor 21, and the data stored in the memory is not limited in this embodiment of the present application.
可以理解的是，图10仅仅示出了一种图像处理装置的简化设计。在实际应用中，图像处理装置还可以分别包含必要的其他元件，包含但不限于任意数量的输入/输出装置、处理器、存储器等，而所有可以实现本申请实施例的图像处理装置都在本申请的保护范围之内。It can be understood that FIG. 10 only shows a simplified design of an image processing apparatus. In practical applications, the image processing apparatus may also include other necessary elements, including but not limited to any number of input/output devices, processors, memories, etc., and all image processing apparatuses that can implement the embodiments of the present application are included in this application. within the scope of protection of the application.
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的部分及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。Those of ordinary skill in the art can realize that the parts and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的***、装置和部分的具体工作过程，可以参考前述方法实施例中的对应过程。所属领域的技术人员还可以清楚地了解到，本申请各个实施例描述各有侧重，因此，在某一实施例未描述或未详细描述的部分可以参见其他实施例的记载。Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the system, device and part described above, reference may be made to the corresponding process in the foregoing method embodiments. Those skilled in the art can also clearly understand that the description of each embodiment of the present application has its own emphasis. Therefore, for parts that are not described or not described in detail in a certain embodiment, reference may be made to the records of other embodiments.
在本申请所提供的几个实施例中，应该理解到，所揭露的***、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述部分的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个部分或组件可以结合或者可以集成到另一个***，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或部分的间接耦合或通信连接，可以是电性，机械或其它的形式。In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the parts is only a logical function division. In actual implementation, there may be other division methods, for example, multiple parts or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or parts, and may be in electrical, mechanical or other forms.
所述作为分离部件说明的部分可以是或者也可以不是物理上分开的，作为部分显示的部件可以是或者也可以不是物理部分，即可以位于一个地方，或者也可以分布到多个网络部分上。可以根据实际的需要选择其中的部分或者全部部分来实现本实施例方案的目的。The parts described as separate parts may or may not be physically separated, and the parts shown as parts may or may not be physical parts, that is, they may be located in one place, or may be distributed to multiple network parts. Some or all of them may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
另外，在本申请各个实施例中的各功能部分可以集成在一个处理部分中，也可以是各个部分单独物理存在，也可以两个或两个以上部分集成在一个部分中。In addition, each functional part in each embodiment of the present application may be integrated into one processing part, or each part may exist physically alone, or two or more parts may be integrated into one part.
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者通过所述计算机可读存储介质进行传输。所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line，DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，数字通用光盘(digital versatile disc，DVD))、或者半导体介质(例如固态硬盘(solid state disk，SSD))等。In the above-mentioned embodiments, it may be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented in software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer may be a general purpose computer, special purpose computer, computer network, or other programmable device. The computer instructions may be stored in or transmitted over a computer-readable storage medium. The computer instructions can be sent from a website site, computer, server, or data center via wired (eg, coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.) another website site, computer, server or data center for transmission. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes an integration of one or more available media. The available media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, digital versatile discs (DVDs)), or semiconductor media (eg, solid state disks (SSDs)) )Wait.
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，该流程可以由计算机程序来指令相关的硬件完成，该程序可存储于计算机可读取存储介质中，该程序在执行时，可包括如上述各方法实施例的流程。而前述的存储介质包括：只读存储器(read-only memory，ROM)或随机存取存储器(random access memory，RAM)、磁碟或者光盘等各 种可存储程序代码的介质。Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented. The process can be completed by instructing the relevant hardware by a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed , which may include the processes of the foregoing method embodiments. The aforementioned storage medium includes: read-only memory (read-only memory, ROM) or random access memory (random access memory, RAM), magnetic disk or optical disk and other media that can store program codes.
工业实用性Industrial Applicability
本申请实施例公开了一种图像处理方法及装置、电子设备、计算机可读存储介质、计算机程序及程序产品。该方法包括：获取待处理图像；所述待处理图像包含待分割对象的毛区域；获取所述待分割对象的毛颜色值范围；确定所述毛区域内颜色值处于所述毛颜色值范围之外的像素点，作为背景像素点。由于毛颜色值范围可视为待分割对象的毛像素点的颜色值的参考范围，图像处理装置依据毛颜色值范围判断毛区域内的像素点是否为毛像素点，可确定毛区域内的背景像素点，从而提高待处理图像中待分割对象的分割准确度。The embodiments of the present application disclose an image processing method and apparatus, an electronic device, a computer-readable storage medium, a computer program, and a program product. The method includes: acquiring an image to be processed; the image to be processed includes a hair area of an object to be segmented; acquiring a range of hair color values of the object to be segmented; determining that the color value in the hair area is within the range of the hair color value The outer pixels are used as background pixels. Since the hair color value range can be regarded as the reference range of the color values of the hair pixels of the object to be segmented, the image processing device can determine whether the pixels in the hair area are hair pixels according to the hair color value range, and can determine the background in the hair area. pixel points, thereby improving the segmentation accuracy of the object to be segmented in the image to be processed.

Claims (16)

  1. 一种图像处理方法，所述方法包括：An image processing method, the method comprising:
    获取待处理图像；所述待处理图像包含待分割对象的毛区域；Obtain an image to be processed; the image to be processed includes the hair region of the object to be segmented;
    获取所述待分割对象的毛颜色值范围；Obtain the range of hair color values of the object to be segmented;
    确定所述毛区域内颜色值处于所述毛颜色值范围之外的像素点，作为背景像素点。Pixels whose color values in the hair region are outside the hair color value range are determined as background pixels.
  2. 根据权利要求1所述的方法，其中，所述获取所述待分割对象的毛颜色值范围，包括：The method according to claim 1, wherein the acquiring the range of hair color values of the object to be segmented comprises:
    对所述待处理图像进行毛分割处理，得到第一毛掩码；Perform hair segmentation processing on the to-be-processed image to obtain a first hair mask;
    缩小所述第一毛掩码中的毛区域，得到第二毛掩码；reducing the hair area in the first hair mask to obtain a second hair mask;
    依据所述第二毛掩码所对应的像素点的颜色值，得到所述毛颜色值范围。The hair color value range is obtained according to the color value of the pixel corresponding to the second hair mask.
  3. 根据权利要求2所述的方法，其中，所述待分割对象包括待分割人物，所述毛包括头发，在所述缩小所述第一毛掩码中的毛区域，得到第二毛掩码之前，所述方法还包括：The method according to claim 2, wherein the object to be segmented includes a person to be segmented, and the hair includes hair, before the reducing the hair area in the first hair mask to obtain the second hair mask , the method also includes:
    对所述待处理图像进行人脸检测处理，得到所述待分割人物的人脸区域的位置；Performing face detection processing on the to-be-processed image to obtain the position of the face region of the to-be-segmented character;
    依据所述人脸区域的位置，从所述待处理图像中确定所述待分割人物的头发区域；According to the position of the face area, determine the hair area of the person to be segmented from the image to be processed;
    所述缩小所述第一毛掩码中的毛区域，得到第二毛掩码，包括：The reducing the hair area in the first hair mask to obtain a second hair mask, including:
    缩小所述第一毛掩码中的头发区域，得到头发掩码；reducing the hair area in the first hair mask to obtain a hair mask;
    依据所述头发掩码，确定所述待分割人物的头发区域中的头发像素点，得到所述第二毛掩码。According to the hair mask, hair pixels in the hair region of the person to be segmented are determined to obtain the second hair mask.
  4. 根据权利要求3所述的方法，其中，所述对所述待处理图像进行人脸检测处理，得到所述待分割人物的人脸区域的位置，包括：The method according to claim 3, wherein the performing face detection processing on the to-be-processed image to obtain the position of the face region of the to-be-segmented character comprises:
    对所述待处理图像进行人脸检测处理，得到至少一个人脸框；performing face detection processing on the to-be-processed image to obtain at least one face frame;
    选取所述至少一个人脸框中尺度最大的人脸框，作为所述待分割人物的人脸框；Select the face frame with the largest scale in the at least one face frame as the face frame of the character to be segmented;
    将所述待分割人物的人脸框的位置和所述待分割人物的人脸框的尺寸，作为所述待分割人物的人脸区域的位置。The position of the face frame of the person to be divided and the size of the face frame of the person to be divided are used as the position of the face region of the person to be divided.
  5. 根据权利要求4所述的方法，其中，所述选取所述至少一个人脸框中尺度最大的人脸框，作为所述待分割人物的人脸框，包括：The method according to claim 4, wherein the selecting the face frame with the largest scale in the at least one face frame as the face frame of the character to be segmented comprises:
    选取所述至少一个人脸框中面积最大的人脸框，作为所述待分割人物的人脸框。The face frame with the largest area in the at least one face frame is selected as the face frame of the person to be segmented.
  6. 根据权利要求3至5中任意一项所述的方法，其中，在所述确定所述毛区域内颜色值处于所述毛颜色值范围之外的像素点，作为背景像素点之前，所述方法还包括：The method according to any one of claims 3 to 5, wherein before the pixel points whose color values in the hair region are outside the hair color value range are determined as background pixels, the method Also includes:
    对所述待处理图像进行人像分割处理，得到第一人像掩码；Performing portrait segmentation processing on the to-be-processed image to obtain a first portrait mask;
    依据所述第一人像掩码，确定所述待分割人物的头发区域中的人像像素点，得到第二人像掩码；According to the first portrait mask, determine the portrait pixels in the hair region of the person to be segmented, and obtain the second portrait mask;
    所述确定所述毛区域内颜色值处于所述毛颜色值范围之外的像素点，作为背景像素点，包括：The determining that the color value in the hair area is outside the range of the hair color value is used as the background pixel, including:
    确定所述毛区域内颜色值处于所述毛颜色值范围之外的像素点，得到至少一个第一像素点；Determining the pixels in the hair region whose color values are outside the hair color value range to obtain at least one first pixel;
    依据所述第二人像掩码，确定所述至少一个第一像素点中的人像像素点，得到至少一个第二像素点；According to the second portrait mask, determine the portrait pixels in the at least one first pixel to obtain at least one second pixel;
    去除所述至少一个第一像素点中的所述至少一个第二像素点，得到所述背景像素点。The at least one second pixel point in the at least one first pixel point is removed to obtain the background pixel point.
  7. 根据权利要求2至6中任意一项所述的方法，其中，所述依据所述第二毛掩码所对应的像素点的颜色值，得到所述毛颜色值范围，包括：The method according to any one of claims 2 to 6, wherein the obtaining the hair color value range according to the color value of the pixel corresponding to the second hair mask includes:
    获取毛颜色值区间长度；Get the length of the hair color value interval;
    统计所述第二毛掩码所对应的像素点的颜色直方图；Counting the color histogram of the pixel corresponding to the second hair mask;
    依据所述颜色直方图，得到所对应的像素点的数量最多、且长度为所述毛颜色值区间长度的毛颜色值区间，作为所述毛颜色值范围。According to the color histogram, a hair color value interval with the largest number of corresponding pixel points and the length of which is the length of the hair color value interval is obtained as the hair color value range.
  8. 根据权利要求1至7中任意一项所述的方法，其中，所述待处理图像还包括非待分割对象区域，所述非待分割对象区域为所述待处理图像中除所述待分割对象所覆盖的像素点区域之外的像素点区域；所述方法还包括：The method according to any one of claims 1 to 7, wherein the to-be-processed image further comprises a non-to-be-segmented object area, and the non-to-be-segmented object area is the non-to-be-segmented object area in the to-be-processed image except the to-be-segmented object The pixel point area outside the covered pixel point area; the method further includes:
    改变所述背景像素点的颜色和非待分割对象区域的颜色，得到所述待分割对象的留色图像；在所述留色图像中，所述非待分割对象区域的颜色和所述背景像素点的颜色相同。Change the color of the background pixels and the color of the non-to-be-segmented object area to obtain a color-retained image of the to-be-segmented object; in the color-retained image, the color of the non-to-be-segmented object area and the background pixels The dots are the same color.
  9. 根据权利要求8所述的方法，其中，所述方法还包括：The method of claim 8, wherein the method further comprises:
    对所述留色图像中的所述背景像素点和所述留色图像中的所述非待分割对象区域进行虚化处理，得到所述待分割对象的留色虚化图像。Perform a blurring process on the background pixels in the color-retained image and the non-to-be-segmented object area in the color-retained image to obtain a color-retained blurred image of the object to be segmented.
  10. 根据权利要求8或9所述的方法，其中，所述改变所述背景像素点的颜色和非待分割对象区域的颜色，得到所述待分割对象的留色图像，包括：The method according to claim 8 or 9, wherein the changing the color of the background pixels and the color of the non-to-be-segmented object area to obtain a color-retained image of the to-be-segmented object includes:
    对所述背景像素点和所述非待分割对象区域进行灰度化处理，得到所述留色图像。Grayscale processing is performed on the background pixel points and the non-to-be-segmented object area to obtain the color-retained image.
  11. 根据权利要求6所述的方法，其中，所述方法还包括：The method of claim 6, wherein the method further comprises:
    对所述第二毛掩码与所述第二人像掩码进行融合处理，得到优化后的前景掩码；Perform fusion processing on the second hair mask and the second portrait mask to obtain an optimized foreground mask;
    根据所述优化后的前景掩码，确定所述待处理图像中的优化背景区域；According to the optimized foreground mask, determine the optimized background area in the to-be-processed image;
    改变所述优化背景区域的颜色，得到留色图像。Change the color of the optimized background area to obtain a color-retained image.
  12. 一种图像处理装置，所述装置包括：An image processing device, the device comprising:
    第一获取部分，被配置为获取待处理图像；所述待处理图像包含待分割对象的毛区域；a first acquiring part, configured to acquire an image to be processed; the image to be processed includes the hair region of the object to be segmented;
    第二获取部分，被配置为获取所述待分割对象的毛颜色值范围；a second acquiring part, configured to acquire the range of hair color values of the object to be segmented;
    处理部分，被配置为确定所述毛区域内颜色值处于所述毛颜色值范围之外的像素点，作为背景像素点。The processing part is configured to determine the pixel points in the hair area whose color values are outside the hair color value range as background pixels.
  13. 一种电子设备，包括：处理器和存储器，所述存储器被配置为存储计算机程序代码，所述计算机程序代码包括计算机指令，在所述处理器执行所述计算机指令的情况下，所述电子设备执行如权利要求1至11中任意一项所述的方法。An electronic device comprising: a processor and a memory, the memory configured to store computer program code, the computer program code comprising computer instructions, the electronic device when the processor executes the computer instructions A method as claimed in any one of claims 1 to 11 is performed.
  14. 一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机程序，所述计算机程序包括程序指令，在所述程序指令被处理器执行的情况下，所述处理器执行权利要求1至11中任意一项所述的方法。A computer-readable storage medium storing a computer program in the computer-readable storage medium, the computer program comprising program instructions, and when the program instructions are executed by a processor, the processor executes claim 1 The method of any one of to 11.
  15. 一种计算机程序，所述计算机程序产品包括计算机程序或指令，在所述计算机程序或指令在计算机上运行的情况下，所述计算机执行权利要求1至11中任意一项所述的方法。A computer program, the computer program product comprising a computer program or instructions for performing the method of any one of claims 1 to 11 when the computer program or instructions are run on a computer.
  16. 一种计算机程序产品，所述计算机程序产品包括计算机程序或指令，在所述计算机程序或指令在计算机上运行的情况下，所述计算机执行权利要求1至11中任意一项所述的方法。A computer program product comprising a computer program or instructions for executing the method of any one of claims 1 to 11 when the computer program or instructions are run on a computer.
PCT/CN2021/107017 2021-01-22 2021-07-19 Image processing method and apparatus, and electronic device, computer-readable storage medium, computer program and program product WO2022156167A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110090411.4 2021-01-22
CN202110090411.4A CN112861661B (en) 2021-01-22 2021-01-22 Image processing method and device, electronic equipment and computer readable storage medium

Publications (1)

Publication Number Publication Date
WO2022156167A1 true WO2022156167A1 (en) 2022-07-28

Family

ID=76008125

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/107017 WO2022156167A1 (en) 2021-01-22 2021-07-19 Image processing method and apparatus, and electronic device, computer-readable storage medium, computer program and program product

Country Status (2)

Country Link
CN (1) CN112861661B (en)
WO (1) WO2022156167A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116957524A (en) * 2023-09-21 2023-10-27 青岛阿斯顿工程技术转移有限公司 Talent information intelligent management method and system in technology transfer process

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112861661B (en) * 2021-01-22 2022-11-08 深圳市慧鲤科技有限公司 Image processing method and device, electronic equipment and computer readable storage medium
CN113222973B (en) * 2021-05-31 2024-03-08 深圳市商汤科技有限公司 Image processing method and device, processor, electronic equipment and storage medium
WO2022261828A1 (en) * 2021-06-15 2022-12-22 Oppo广东移动通信有限公司 Image processing method and apparatus, electronic device, and computer-readable storage medium
CN113989395A (en) * 2021-10-29 2022-01-28 北京字节跳动网络技术有限公司 Fluff effect generation method, system, equipment, medium and product
CN117522760B (en) * 2023-11-13 2024-06-25 书行科技(北京)有限公司 Image processing method, device, electronic equipment, medium and product

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104484664A (en) * 2014-12-31 2015-04-01 小米科技有限责任公司 Human face image processing method and device
US20160154993A1 (en) * 2014-12-01 2016-06-02 Modiface Inc. Automatic segmentation of hair in images
CN110189340A (en) * 2019-06-03 2019-08-30 北京达佳互联信息技术有限公司 Image partition method, device, electronic equipment and storage medium
CN111179276A (en) * 2018-11-12 2020-05-19 北京京东尚科信息技术有限公司 Image processing method and device
CN112861661A (en) * 2021-01-22 2021-05-28 深圳市慧鲤科技有限公司 Image processing method and device, electronic equipment and computer readable storage medium

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113298845A (en) * 2018-10-15 2021-08-24 华为技术有限公司 Image processing method, device and equipment
CN111598903B (en) * 2020-05-21 2023-09-29 Oppo广东移动通信有限公司 Portrait segmentation method, device, storage medium and electronic equipment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160154993A1 (en) * 2014-12-01 2016-06-02 Modiface Inc. Automatic segmentation of hair in images
CN104484664A (en) * 2014-12-31 2015-04-01 小米科技有限责任公司 Human face image processing method and device
CN111179276A (en) * 2018-11-12 2020-05-19 北京京东尚科信息技术有限公司 Image processing method and device
CN110189340A (en) * 2019-06-03 2019-08-30 北京达佳互联信息技术有限公司 Image partition method, device, electronic equipment and storage medium
CN112861661A (en) * 2021-01-22 2021-05-28 深圳市慧鲤科技有限公司 Image processing method and device, electronic equipment and computer readable storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116957524A (en) * 2023-09-21 2023-10-27 青岛阿斯顿工程技术转移有限公司 Talent information intelligent management method and system in technology transfer process
CN116957524B (en) * 2023-09-21 2024-01-05 青岛阿斯顿工程技术转移有限公司 Talent information intelligent management method and system in technology transfer process

Also Published As

Publication number Publication date
CN112861661A (en) 2021-05-28
CN112861661B (en) 2022-11-08

Similar Documents

Publication Publication Date Title
WO2022156167A1 (en) Image processing method and apparatus, and electronic device, computer-readable storage medium, computer program and program product
CN111242881B (en) Method, device, storage medium and electronic equipment for displaying special effects
US10043308B2 (en) Image processing method and apparatus for three-dimensional reconstruction
CN109064390B (en) Image processing method, image processing device and mobile terminal
WO2019109801A1 (en) Method and device for adjusting photographing parameter, storage medium, and mobile terminal
US11398044B2 (en) Method for face modeling and related products
CN110100251B (en) Apparatus, method, and computer-readable storage medium for processing document
WO2020043155A1 (en) Multiple scale image fusion method and device, storage medium, and terminal
CN107771391B (en) Method and apparatus for determining exposure time of image frame
CN108965835B (en) Image processing method, image processing device and terminal equipment
CN110335216B (en) Image processing method, image processing apparatus, terminal device, and readable storage medium
CN110431563B (en) Method and device for correcting image
JP7378354B2 (en) Detecting finger presses from live video streams
WO2022179045A1 (en) Image processing method and apparatus, and storage medium, program and program product
WO2022135574A1 (en) Skin color detection method and apparatus, and mobile terminal and storage medium
CN110717452B (en) Image recognition method, device, terminal and computer readable storage medium
WO2022111461A1 (en) Recognition method and apparatus, and electronic device
WO2018027527A1 (en) Optical system imaging quality detection method and apparatus
CN111724421B (en) Image processing method and device, electronic equipment and storage medium
CN111222446B (en) Face recognition method, face recognition device and mobile terminal
CN111402391B (en) User face image display method, display device and corresponding storage medium
CN110047126B (en) Method, apparatus, electronic device, and computer-readable storage medium for rendering image
CN110610178A (en) Image recognition method, device, terminal and computer readable storage medium
CN111754411B (en) Image noise reduction method, image noise reduction device and terminal equipment
US20060010582A1 (en) Chin detecting method, chin detecting system and chin detecting program for a chin of a human face

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21920548

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 25.10.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 21920548

Country of ref document: EP

Kind code of ref document: A1