WO2024037287A1

WO2024037287A1 - Facial skin evaluation method and evaluation device

Info

Publication number: WO2024037287A1
Application number: PCT/CN2023/108738
Authority: WO
Inventors: 孙建美; 靳晨英; 刘汀科
Original assignee: 厦门松霖科技股份有限公司
Priority date: 2022-08-19
Filing date: 2023-07-21
Publication date: 2024-02-22
Also published as: WO2024037287A9; CN117617899A

Abstract

A facial skin evaluation method and evaluation device. The method comprises: controlling a first image acquisition assembly (30) to acquire a full facial image; identifying skin problems according to the full facial image and carrying out region marking; prompting to use a second image acquisition assembly (20) to acquire local images at the marked regions where there are skin problems, and obtaining a degree of severity and/or a score with respect to corresponding attributes of the skin problems at the marked regions on the basis of the acquired local images. In the present method, first regions where there are skin problems are quickly identified by means of the full facial image and marking is performed, and then magnified local granular images are acquired at the regions where there are skin problems by means of the second image acquisition assembly to obtain the degree of severity and/or the score with respect to corresponding attributes of the skin problems at the marked regions, thereby enabling a user to have a more accurate and comprehensive understanding of the health of their facial skin.

Description

一种脸部皮肤检测方法及检测装置Facial skin detection method and detection device

相关申请的交叉引用Cross-references to related applications

本公开要求于2022年08月19日提交的申请号为202210998003.3、名称为“一种脸部皮肤检测方法及检测装置”的中国专利申请的优先权，该中国专利申请的全部内容通过引用全部并入本文。This disclosure claims priority to the Chinese patent application with application number 202210998003.3 and titled "A Facial Skin Detection Method and Detection Device" submitted on August 19, 2022. The entire content of this Chinese patent application is incorporated by reference in its entirety. Enter this article.

技术领域Technical field

本公开涉及皮肤检测技术领域，特别是一种脸部皮肤检测方法及检测装置。The present disclosure relates to the technical field of skin detection, in particular to a facial skin detection method and detection device.

背景技术Background technique

随着现在生活水平的提高，大众越来越关注自身容貌与仪表，而美妆镜作为一种大众消费品越来受到喜爱，尤其是女性消费者几乎每天都会使用到美妆镜。镜子的主要功能是用于整理仪表与辅助化妆。而随着科技越来越发达，消费者对镜子功能要求越来越多样化、智能化。With the improvement of living standards, the public is paying more and more attention to their own appearance and appearance, and cosmetic mirrors are becoming more and more popular as a mass consumer product, especially female consumers who use cosmetic mirrors almost every day. The main function of the mirror is to organize your appearance and assist with makeup. As technology becomes more and more developed, consumers have increasingly diverse and intelligent requirements for mirror functions.

现有市场上美妆镜配置的第二图像采集组件基本是手持式，只能做局部检测，一般由用户自由在额头、脸颊、鼻翼、下巴各选取一个位置检测，由于用户对整个脸部健康不是很清楚，故在图像采集时，用户都是随意选择区域进行图像采集，或者，根据自己的感觉选择皮肤可能存在问题的区域进行图像采集，一方面，导致图像采集时间加长，另一方面可能无法采集到真正有问题的皮肤图像，最终分析出来的脸部皮肤数据无法反映脸部皮肤真实健康状态，导致皮肤问题不能得到及时处理。The second image acquisition component configured in the existing beauty mirror on the market is basically handheld and can only perform partial detection. Generally, the user can freely select a position on the forehead, cheeks, nose, and chin for detection. Since the user is concerned about the health of the entire face. It is not very clear, so when collecting images, users randomly select areas for image collection, or select areas with possible skin problems for image collection based on their own feelings. On the one hand, this results in longer image collection time, and on the other hand, it may It is impossible to collect real problematic skin images, and the final analyzed facial skin data cannot reflect the true health status of facial skin, resulting in skin problems that cannot be dealt with in a timely manner.

发明内容Contents of the invention

本公开的主要目的在于提出一种脸部皮肤检测方法及检测装置，其克服了现有技术的所存在的不足之处，先通过采集全脸图像快速识别出皮肤问题所在区域并进行区域标识，再通过第二图像采集组件(如测肤笔)对皮肤问题所在区域进行放大倍数的局部细微图像采集，获得标识区域在皮肤问题对应维度上的严重程度和/或分数，从而使得用户对脸部皮肤健康状况有更准确和全面的了解。The main purpose of the present disclosure is to propose a facial skin detection method and detection device, which overcome the shortcomings of the existing technology. First, by collecting full-face images, the area where the skin problem is located is quickly identified and the area is marked. Then, a second image acquisition component (such as a skin measuring pen) is used to collect partial and detailed images of the area where the skin problem is located to obtain the severity and/or score of the identified area in the corresponding dimension of the skin problem, thereby allowing the user to evaluate the face. Have a more accurate and comprehensive understanding of skin health.

本公开采用如下技术方案：This disclosure adopts the following technical solutions:

一方面，一种脸部皮肤检测方法，包括：In one aspect, a facial skin detection method includes:

控制第一图像采集组件采集全脸图像；Control the first image acquisition component to acquire a full face image;

根据全脸图像识别出皮肤问题并进行区域标识；Identify skin problems and identify areas based on full-face images;

提示使用第二图像采集组件对皮肤问题所在的标识区域采集局部图像，基于采集的局部图像，获得标识区域在皮肤问题对应维度上的严重程度和/或分数。Prompt to use the second image acquisition component to collect local images of the identified areas where skin problems are located. Based on the collected local images, partial image to obtain the severity and/or score of the identified area in the corresponding dimension of the skin problem.

在一些实施例中，控制第一图像采集组件采集全脸图像，具体包括：In some embodiments, controlling the first image acquisition component to acquire a full face image specifically includes:

当检测到整个脸部处于第一图像采集组件摄像区域内时，控制开启白光进行全脸拍照以采集全脸取像。When it is detected that the entire face is within the imaging area of the first image acquisition component, the white light is controlled to be turned on to take a full-face photo to collect a full-face image.

在一些实施例中，根据全脸图像识别出皮肤问题并进行区域标识，具体包括：In some embodiments, skin problems are identified and regional identification is performed based on the full face image, specifically including:

使用训练好的卷积神经网络模型识别出全脸图像中的皮肤问题，定位皮肤问题的图像坐标，并在全脸图像的对应位置区域进行标识。Use the trained convolutional neural network model to identify skin problems in the full-face image, locate the image coordinates of the skin problem, and mark the corresponding location area in the full-face image.

在一些实施例中，提示使用第二图像采集组件对皮肤问题所在的标识区域采集局部图像，具体包括：In some embodiments, it is prompted to use the second image acquisition component to collect a local image of the identified area where the skin problem is located, specifically including:

提示使用第二图像采集组件在自然光、UV光或偏振光下，对每个皮肤问题所在的标识区域分别采集预设张局部图像；所述预设张大于1张。It is prompted to use the second image acquisition component to collect preset partial images of the marked area where each skin problem is located under natural light, UV light or polarized light; the preset image is greater than 1 image.

在一些实施例中，提示使用第二图像采集组件在自然光、UV光或偏振光下，对每个皮肤问题所在的标识区域分别采集预设张局部图像，具体包括：In some embodiments, the second image acquisition component is used to collect preset partial images of the identified areas where each skin problem is located under natural light, UV light or polarized light, specifically including:

S501，提示用户使用第二图像采集组件在自然光、UV光或偏振光下，采集某一皮肤问题所在的标识区域的图像，并判定所拍摄的图像是否清晰，若不清晰则重复步骤S501；若清晰则进入步骤S502；S501, prompt the user to use the second image acquisition component to collect an image of the marked area where a certain skin problem is located under natural light, UV light or polarized light, and determine whether the captured image is clear. If not, repeat step S501; if If clear, proceed to step S502;

S502，提示用户对该标识区域在相同光源或不同光源下拍摄下一张图像；并重复步骤S501；S502, prompt the user to take the next image of the identified area under the same light source or a different light source; and repeat step S501;

S503，直至该区域清晰的图像数量达到预设张；S503, until the number of clear images in this area reaches the preset number;

S504，重复S501～S503，直至所有有皮肤问题的标识区域均采集到预设张局部图像。S504: Repeat S501 to S503 until all preset partial images are collected in all marked areas with skin problems.

在一些实施例中，控制第一图像采集组件采集全脸图像之前，还包括：In some embodiments, before controlling the first image acquisition component to acquire the full face image, the method further includes:

检测第二图像采集组件的开启状态，如果未开启，控制进入第一图像采集组件采集全脸图像步骤；如果开启，提示使用第二图像采集组件对划分的各个脸部区域采集局部图像，基于采集的局部图像，获得各脸部区域在皮肤问题对应维度上的严重程度和/或分数。Detect the opening status of the second image acquisition component. If it is not enabled, the control enters the step of acquiring the full face image by the first image acquisition component; if it is enabled, it is prompted to use the second image acquisition component to collect partial images of each divided facial area. Based on the acquisition The partial image is used to obtain the severity and/or score of each facial area in the corresponding dimension of the skin problem.

在一些实施例中，提示使用第二图像采集组件对划分的各个脸部区域采集局部图像，具体包括：In some embodiments, the prompt uses the second image acquisition component to collect partial images of each divided facial area, specifically including:

提示使用第二图像采集组件在自然光、UV光或偏振光下，对划分的各个脸部区域分别采集预设张局部图像；所述预设张大于1张。It is prompted to use the second image acquisition component to collect a preset partial image of each divided facial area under natural light, UV light or polarized light; the preset image is greater than 1.

在一些实施例中，提示使用第二图像采集组件在自然光、UV光或偏振光下，对划分的各个脸部区域分别采集预设张局部图像，具体包括：In some embodiments, the prompt uses the second image acquisition component to collect preset partial images of each divided facial area under natural light, UV light or polarized light, specifically including:

S801，提示用户使用第二图像采集组件在自然光、UV光或偏振光下，采集划分的某一脸部区域的图像，并判定所拍摄的图像是否清晰，若不清晰则重复步骤S801；若清晰则进入步骤S802；S801. Prompt the user to use the second image acquisition component to collect an image of a divided facial area under natural light, UV light or polarized light, and determine whether the captured image is clear. If it is not clear, repeat step S801; if it is clear, Then enter step S802;

S802，提示用户对该脸部区域在相同光源或不同光源下拍摄下一张图像；并重复步骤 S801；S802, prompt the user to take the next image of the facial area under the same light source or different light sources; and repeat the steps. S801;

S803，直至该区域清晰的图像数量达到预设张；S803, until the number of clear images in this area reaches the preset;

S804，重复S801～S803，直至所有划分的脸部区域均采集到预设张局部图像。S804: Repeat S801 to S803 until all divided facial areas have collected preset partial images.

在一些实施例中，所述脸部区域包括：额头、左脸颊、右脸颊、鼻部和下巴。In some embodiments, the facial areas include: forehead, left cheek, right cheek, nose, and chin.

在一些实施例中，所述皮肤问题包括：纹理、毛孔、痤疮、敏感、色斑、黑头或油脂中的至少一个；每个皮肤问题对应一个维度。In some embodiments, the skin problem includes: at least one of texture, pores, acne, sensitivity, stains, blackheads, or oil; each skin problem corresponds to one dimension.

在一些实施例中，所述严重程度包括：重度、中度或轻度。In some embodiments, the severity includes severe, moderate, or mild.

在一些实施例中，所述分数基于严重程度及对应的置信度获得，具体如下：

In some embodiments, the score is obtained based on severity and corresponding confidence, as follows:

其中，x表示置信度；f(x)表示分数。Among them, x represents the confidence level; f(x) represents the score.

另一方面，一种脸部皮肤检测装置，包括第一图像采集组件、美妆镜和第二图像采集组件；所述美妆镜和所述第一图像采集组件及第二图像采集组件分别进行数据交互；所述第一图像采集组件和所述美妆镜分立设置，或者所述第一图像采集组件设置在所述美妆镜上；所述美妆镜用于执行所述脸部皮肤检测方法的步骤。On the other hand, a facial skin detection device includes a first image acquisition component, a beauty mirror and a second image acquisition component; the beauty mirror, the first image acquisition component and the second image acquisition component respectively perform Data interaction; the first image acquisition component and the cosmetic mirror are provided separately, or the first image acquisition component is provided on the cosmetic mirror; the cosmetic mirror is used to perform the facial skin detection Method steps.

再一方面，一种脸部皮肤检测装置，包括第一图像采集组件、美妆镜、第二图像采集组件和外部终端设备；所述美妆镜和所述第一图像采集组件、第二图像采集组件及外部终端设备分别进行数据交互；所述第一图像采集组件和所述美妆镜分立设置，或者所述第一图像采集组件设置在所述美妆镜上；所述外部终端设备用于执行所述脸部皮肤检测方法的步骤。In yet another aspect, a facial skin detection device includes a first image acquisition component, a beauty mirror, a second image acquisition component and an external terminal device; the beauty mirror, the first image acquisition component, the second image acquisition component, and the external terminal device. The collection component and the external terminal device perform data interaction respectively; the first image collection component and the beauty mirror are set separately, or the first image collection component is set on the beauty mirror; the external terminal device is To perform the steps of the facial skin detection method.

再一方面，一种电子设备，包括：In yet another aspect, an electronic device includes:

处理器；以及processor; and

存储器，用于存储所述处理器的可执行指令；memory for storing executable instructions for the processor;

其中，所述处理器配置为经由执行所述可执行指令来执行上述任一项所述的脸部皮肤检测方法。Wherein, the processor is configured to execute any of the above facial skin detection methods by executing the executable instructions.

再一方面，一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现上述任一项所述的脸部皮肤检测方法。In yet another aspect, a computer-readable storage medium has a computer program stored thereon, and when the computer program is executed by a processor, the facial skin detection method described in any one of the above is implemented.

再一方面，一种计算机程序产品，包括计算机程序，所述计算机程序被处理器执行时实现上述任一项所述的脸部皮肤检测方法。In yet another aspect, a computer program product includes a computer program that implements any of the above facial skin detection methods when executed by a processor.

与现有技术相比，本公开的有益效果如下：Compared with the prior art, the beneficial effects of the present disclosure are as follows:

(1)本公开先通过采集全脸图像快速识别出脸部皮肤健康薄弱区域即皮肤问题所在区域并进行标识，使用户对整个脸部进行初步宏观的了解；再通过第二图像采集组件(如测肤笔)对标识的皮肤问题所在区域进行放大倍数的局部细微图像采集，获得标识区域在皮肤问题对应维度上的严重程度和/或分数，从而使得用户对脸部皮肤健康状况有更准确和全面的了解；(1) This disclosure first quickly identifies healthy and weak areas of facial skin, that is, skin problems, by collecting full-face images. The area is marked and marked, allowing the user to have a preliminary macro understanding of the entire face; and then a second image acquisition component (such as a skin measuring pen) is used to collect local and detailed images of the identified skin problem area with magnification, and the location of the marked area is obtained. The severity and/or score of skin problems correspond to dimensions, allowing users to have a more accurate and comprehensive understanding of facial skin health;

(2)本公开能够自动搜索有无第二图像采集组件WIFI热点等来判断第二图像采集组件是否开启，如果未开启，可以先进行全脸图像采集和分析，再引导用户开启第二图像采集组件对有皮肤问题的区域进行局部图像采集及分析；如果第二图像采集组件已开启，可以提示用户先进行全脸图像检测，如果用户不关闭第二图像采集组件，说明用户不想进行全脸图像采集及检测，则直接对用户选择区域进行局部图像采集及检测，或引导用户依次对额头、左脸颊、右脸颊、鼻部和下巴区域进行图像采集及检测，满足用户多方位的需求。(2) The present disclosure can automatically search for the presence of the second image collection component WIFI hotspot, etc. to determine whether the second image collection component is turned on. If it is not turned on, full face image collection and analysis can be performed first, and then the user can be guided to turn on the second image collection. The component collects and analyzes local images of areas with skin problems; if the second image collection component is turned on, the user can be prompted to perform full-face image detection first. If the user does not turn off the second image collection component, it means that the user does not want to perform full-face image detection. For collection and detection, it directly collects and detects partial images of the area selected by the user, or guides the user to sequentially collect and detect images of the forehead, left cheek, right cheek, nose and chin areas to meet the user's multi-faceted needs.

上述说明仅是本公开技术方案的概述，为了能够更清楚地了解本公开的技术手段，从而可依照说明书的内容予以实施，并且为了让本公开的上述和其他目的、特征和优点能够更明显易懂，以下列举本公开的具体实施方式。The above description is only an overview of the technical solution of the present disclosure. In order to understand the technical means of the present disclosure more clearly, so that they can be implemented according to the content of the specification, and in order to make the above and other objects, features and advantages of the present disclosure more obvious and easy. Understand, specific implementations of the present disclosure are listed below.

根据下文结合附图对本公开具体实施例的详细描述，本领域技术人员将会更加明了本公开的上述及其他目的、优点和特征。From the following detailed description of specific embodiments of the present disclosure in conjunction with the accompanying drawings, those skilled in the art will further understand the above and other objects, advantages and features of the present disclosure.

附图说明Description of drawings

图1为本公开实施例一的脸部皮肤检测方法的方法流程图；Figure 1 is a method flow chart of the facial skin detection method according to Embodiment 1 of the present disclosure;

图2为本公开实施例一的根据全脸图像识别出的皮肤问题示意图；Figure 2 is a schematic diagram of skin problems identified based on full face images according to Embodiment 1 of the present disclosure;

图3为本公开实施例一的第二图像采集组件未开启时的详细检测流程图；Figure 3 is a detailed detection flow chart when the second image acquisition component is not turned on according to Embodiment 1 of the present disclosure;

图4为本公开实施例一的第二图像采集组件开启时的详细检测流程图；Figure 4 is a detailed detection flow chart when the second image acquisition component is turned on according to Embodiment 1 of the present disclosure;

图5为本公开实施例二的脸部皮肤检测装置的结构框图；Figure 5 is a structural block diagram of a facial skin detection device according to Embodiment 2 of the present disclosure;

图6为本公开实施例二的脸部皮肤检测装置的主视图；Figure 6 is a front view of the facial skin detection device according to Embodiment 2 of the present disclosure;

图7为本公开实施例二的脸部皮肤检测装置的剖视图一；Figure 7 is a cross-sectional view of the facial skin detection device according to Embodiment 2 of the present disclosure;

图8为本公开实施例二的脸部皮肤检测装置的剖视图二；Figure 8 is a second cross-sectional view of the facial skin detection device according to Embodiment 2 of the present disclosure;

图9为本公开实施例三的脸部皮肤检测装置的结构框图。Figure 9 is a structural block diagram of a facial skin detection device according to Embodiment 3 of the present disclosure.

具体实施方式Detailed ways

下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述；显然，所描述的实施例仅仅是本公开一部分实施例，而不是全部的实施例，基于本公开中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure; obviously, the described embodiments are only some of the embodiments of the present disclosure, rather than all of the embodiments. Based on The embodiments in this disclosure and all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this disclosure.

在本公开的描述中，需要说明的是，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。In the description of the present disclosure, it should be noted that the terms "comprises,""comprises" or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus includes a series of elements. It includes not only those elements but also other elements not expressly listed or inherent in the process, method, article or equipment. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of additional identical elements in a process, method, article, or apparatus that includes the stated element.

在本公开的描述中，需要说明的是，术语“上”、“下”、“内”、“外”、“顶/底端”等指示的方位或位置关系为基于附图所示的方位或位置关系，仅是为了便于描述本公开和简化描述，而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作，因此不能理解为对本公开的限制。此外，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性。In the description of the present disclosure, it should be noted that the orientation or positional relationship indicated by the terms "upper", "lower", "inner", "outer", "top/bottom", etc. are based on the orientation shown in the drawings. or positional relationships are only for the convenience of describing the present disclosure and simplifying the description, but do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present disclosure. In addition, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance.

在本公开的描述中，需要说明的是，除非另有明确的规定和限定，术语“安装”、“设置有”、“套设/接”、“连接”等，应做广义理解，例如“连接”，可以是固定连接，也可以是可拆卸连接，或一体地连接，可以是机械连接，也可以是电连接，可以是直接相连，也可以通过中间媒介间接相连，可以是两个元件内部的连通，对于本领域的普通技术人员而言，可以具体情况理解上述术语在本公开中的具体含义。In the description of the present disclosure, it should be noted that, unless otherwise explicitly stated and limited, the terms "installed", "provided with", "set/connected", "connected", etc., should be understood in a broad sense, such as " "Connection" can be a fixed connection, a detachable connection, or an integral connection. It can be a mechanical connection or an electrical connection. It can be a direct connection or an indirect connection through an intermediate medium. It can be internal to two components. For those of ordinary skill in the art, the specific meanings of the above terms in this disclosure can be understood in specific situations.

在本公开的描述中，需要说明的是，除非另有明确的规定和限定，步骤标识S101、S102、S103等仅用于方便表述，并不表示执行顺序，相应的执行顺序可以进行调节。In the description of the present disclosure, it should be noted that, unless otherwise explicitly stipulated and limited, the step identifiers S101, S102, S103, etc. are only for convenience of expression and do not indicate the execution order, and the corresponding execution order can be adjusted.

实施例1Example 1

参见图1所示，一方面，一种脸部皮肤检测方法，包括：As shown in Figure 1, on the one hand, a facial skin detection method includes:

S101，控制第一图像采集组件采集全脸图像；S101, control the first image acquisition component to acquire full face images;

S102，根据全脸图像识别出皮肤问题并进行区域标识；S102, identify skin problems based on the full face image and identify the area;

S103，提示使用第二图像采集组件对皮肤问题所在的标识区域采集局部图像，基于采集的局部图像，获得标识区域在皮肤问题对应维度上的严重程度和/或分数。S103: Use the second image acquisition component to collect a partial image of the marked area where the skin problem is located, and obtain the severity and/or score of the marked area in the corresponding dimension of the skin problem based on the collected local image.

本实施例中，所述脸部皮肤检测方法可以通过安装在美妆镜中的控制主板实现，所述控制主板包括MCU控制器等，且与第一图像采集组件、第二图像采集组件相连接以进行数据交互。另一实施例中，所述脸部皮肤检测方法可以在外部终端实现，所述外部终端包括执行器等，且与第一图像采集组件、第二图像采集组件相连接以进行数据交互。其他实施例中，还可以是控制采集全脸图像及提示采集局部图像等在美妆镜中实现，根据全脸图像识别出皮肤问题并进行标识在外部终端设备实现，基于采集的局部图像获得标识区域在皮肤问题对应维度上的严重程度和/或分数第二图像采集组件或外部终端设备实现，各设备间相互连接以进行数据交互。具体采用何种方式实现可以根据需要进行布局，本公开不做限制。In this embodiment, the facial skin detection method can be implemented through a control mainboard installed in the beauty mirror. The control mainboard includes an MCU controller, etc., and is connected to the first image acquisition component and the second image acquisition component. for data interaction. In another embodiment, the facial skin detection method can be implemented in an external terminal. The external terminal includes an actuator, etc., and is connected to the first image acquisition component and the second image acquisition component for data interaction. In other embodiments, the method of controlling the collection of full-face images and prompting for the collection of partial images can also be implemented in a beauty mirror. The identification and identification of skin problems based on the full-face image can be implemented on an external terminal device, and the identification can be obtained based on the collected partial images. The severity and/or score of the area on the corresponding dimension of the skin problem is implemented by a second image acquisition component or an external terminal device, and each device is connected to each other for data interaction. The specific implementation method can be laid out as needed, and this disclosure does not limit it.

本公开先通过采集全脸图像快速识别出脸部皮肤健康薄弱区域即皮肤问题所在区域并进行区域标识，使用户对整个脸部进行初步宏观的了解；再通过第二图像采集组件对标识的皮肤问题所在区域进行放大倍数的局部细微图像采集，获得标识区域在皮肤问题对应维度上的严重程度和/或分数，从而使得用户对脸部皮肤健康状况有更准确和全面的了解。This disclosure first collects full-face images to quickly identify healthy weak areas of facial skin, that is, areas with skin problems, and identifies the areas, allowing users to have a preliminary macroscopic understanding of the entire face; and then uses the second image acquisition component to identify the identified skin areas. Collect local and detailed images of the problem area with magnification to obtain the location of the identified area. Skin problems correspond to severity and/or scores along dimensions, allowing users to have a more accurate and comprehensive understanding of facial skin health.

控制第一图像采集组件采集全脸图像，具体包括：Control the first image acquisition component to acquire full-face images, specifically including:

根据全脸图像识别出皮肤问题并进行区域标识，具体包括：Identify skin problems and identify areas based on full face images, including:

具体的，控制第一图像采集组件启动摄像头，先判定用户的全脸是否都在摄像区域内，若没有，则语音提示用户做相应移动；再对四周光线亮度进行判定，若环境较暗，则启动美妆镜镜体四周补妆灯的白色LED灯组；最后对全脸进行白光拍照，并基于训练号的卷积神经网络模型对皮肤问题进行识别(卷积神经网络模为事先设计的一个可行的网络模型，将大量的人脸训练数据集加载到网络模型中，然后进行自动训练，以得到很好的识别率)，最终识别人脸限定区域内的皮肤问题(纹理、毛孔、痤疮、敏感、色斑、黑头和油脂，并定位纹理、毛孔、痤疮、敏感、色斑、黑头和油脂等)的坐标，在全脸图像上的对应位置标识出来。Specifically, the first image acquisition component is controlled to start the camera, and first determines whether the user's entire face is within the camera area. If not, the user is prompted by voice to move accordingly; and then the surrounding light brightness is determined. If the environment is dark, then Start the white LED light set of makeup lights around the beauty mirror body; finally take a white light photo of the whole face, and identify skin problems based on the convolutional neural network model of the training number (the convolutional neural network model is a pre-designed Feasible network model, load a large number of face training data sets into the network model, and then perform automatic training to obtain a good recognition rate), and finally identify skin problems (texture, pores, acne, Sensitivity, stains, blackheads and oil, and locate the coordinates of texture, pores, acne, sensitivity, stains, blackheads and grease, etc., and mark the corresponding positions on the full face image.

参见图2所示，在全脸图像上对皮肤问题所在区域采用圆圈标示，同时在美妆镜或外部终端设备APP界面显示，并通过语音或界面提示用户是否立即局部检测。As shown in Figure 2, the skin problem area is marked with a circle on the full face image. It is also displayed on the beauty mirror or external terminal device APP interface, and the user is prompted through voice or the interface whether to perform local inspection immediately.

全脸图像检测主要是帮助用户快速找出面部肌肤健康薄弱的地方，通过对整个脸部进行初步宏观的检测，找出其存在皮肤问题的区域并进行标识。Full face image detection is mainly to help users quickly find out the healthy and weak areas of facial skin. Through preliminary macro inspection of the entire face, the areas with skin problems can be found and marked.

局部图像采集时，采用局部第二图像采集组件对准上述标识区域，在白光、偏振光、UV光等光源下，采用数十倍放大倍数的摄像头，依据图像清晰度自动拍照，对所有标识区域进行有针对性的图像采集。When collecting partial images, a partial second image acquisition component is used to align the above-mentioned marked areas. Under white light, polarized light, UV light and other light sources, a camera with dozens of times of magnification is used to automatically take pictures according to the image clarity, and capture all marked areas. Perform targeted image acquisition.

本实施例中，所述第二图像采集组件为测肤笔。In this embodiment, the second image collection component is a skin measuring pen.

由于后续计算该脸部区域在皮肤问题对应维度上的严重程度和/或分数依赖于局部图像是否清晰，如果用户拍摄的图像不清晰，则后续的步骤可能无法完成或者结果出现偏差。Since the subsequent calculation of the severity and/or score of the facial area in the corresponding dimension of the skin problem depends on whether the local image is clear, if the image taken by the user is not clear, subsequent steps may not be completed or the results may be biased.

具体的，提示使用第二图像采集组件在自然光、UV光或偏振光下，对每个皮肤问题所在的标识区域分别采集预设张局部图像，具体包括：Specifically, it is recommended to use the second image acquisition component to collect preset partial images of the marked area where each skin problem is located under natural light, UV light or polarized light, including:

S502，提示用户对该标识区域在相同光源或不同光源下拍摄下一张图像；并重复步骤S501； S502, prompt the user to take the next image of the identified area under the same light source or a different light source; and repeat step S501;

S503，直至该区域清晰的图像数量达到预设张；S503, until the number of clear images in this area reaches the preset;

对于清晰度的判断，可以使用现有技术中常见的图像识别和图像比较的算法。具体来说可以通过将第二图像采集组件拍摄的图像与预图像信息实时比对判定，当达到清晰度分值后，判定其为清晰，保存图像。还可以是通过图像识别算法如深度神经网络算法等对清晰度进行判断，当识别出为清晰图像时，保存图像。具体本实施例不做限制。For judgment of clarity, common image recognition and image comparison algorithms in the existing technology can be used. Specifically, the image captured by the second image acquisition component can be compared and determined in real time with the pre-image information. When the sharpness score is reached, the image is determined to be clear and the image is saved. The clarity can also be judged through an image recognition algorithm such as a deep neural network algorithm, and when a clear image is recognized, the image is saved. There are no specific limitations in this embodiment.

所述皮肤问题包括：纹理、毛孔、痤疮、敏感、色斑、黑头或油脂中的至少一个；每个皮肤问题对应一个维度。The skin problems include: at least one of texture, pores, acne, sensitivity, stains, blackheads or oil; each skin problem corresponds to one dimension.

所述严重程度包括：重度、中度或轻度。The severity includes: severe, moderate or mild.

所述分数基于严重程度及对应的置信度获得，具体如下：

The score is obtained based on the severity and corresponding confidence level, as follows:

对采集后的图像，利用预设算法进行数据运算，计算出皮肤问题对应维度上的严重程度和/或分数，这样更精确获悉皮肤的健康情况，并采取对应的护肤措施。具体的，严重程度的类别和对应的置信度可以根据现有的图像处理方法获取到，本实施例不做详细说明。For the collected images, a preset algorithm is used to perform data operations to calculate the severity and/or score of the skin problem in the corresponding dimension, so as to more accurately understand the health of the skin and take corresponding skin care measures. Specifically, the severity category and the corresponding confidence level can be obtained according to existing image processing methods, and will not be described in detail in this embodiment.

参见图3和图4所示，控制第一图像采集组件采集全脸图像之前，还包括：Referring to Figures 3 and 4, before controlling the first image acquisition component to acquire the full face image, it also includes:

本实施例中，所述脸部区域包括：额头、左脸颊、右脸颊、鼻部和下巴。In this embodiment, the facial area includes: forehead, left cheek, right cheek, nose and chin.

提示使用第二图像采集组件对划分的各个脸部区域采集局部图像，具体包括：Prompt to use the second image acquisition component to collect partial images of each divided facial area, including:

进一步的，提示使用第二图像采集组件在自然光、UV光或偏振光下，对划分的各个脸部区域分别采集预设张局部图像，具体包括：Further, it is suggested to use the second image acquisition component to collect preset partial images of each divided facial area under natural light, UV light or polarized light, specifically including:

S801，提示用户使用第二图像采集组件在自然光、UV光或偏振光下，采集划分的某一脸部区域的图像，并判定所拍摄的图像是否清晰，若不清晰则重复步骤S801；若清晰则进入步骤S802；S801, prompts the user to use the second image acquisition component to collect and divide the image under natural light, UV light or polarized light. Take an image of a certain facial area, and determine whether the captured image is clear. If it is not clear, repeat step S801; if it is clear, proceed to step S802;

S802，提示用户对该脸部区域在相同光源或不同光源下拍摄下一张图像；并重复步骤S801；S802, prompt the user to take the next image of the facial area under the same light source or different light sources; and repeat step S801;

本公开能够自动搜索有无第二图像采集组件WIFI热点等来判断第二图像采集组件是否开启，如果未开启，可以先进行全脸图像采集和分析，再引导用户开启第二图像采集组件对有皮肤问题的区域进行局部图像采集及分析；如果第二图像采集组件已开启，可以提示用户先进行全脸图像检测，如果用户不关闭第二图像采集组件，说明用户不想进行全脸图像采集及检测，则直接对用户选择区域进行局部图像采集及检测，或引导用户依次对额头、左脸颊、右脸颊、鼻部和下巴区域进行图像采集及检测，满足用户多方位的需求。The present disclosure can automatically search whether there is a second image collection component, WIFI hotspot, etc. to determine whether the second image collection component is turned on. If it is not turned on, full face image collection and analysis can be performed first, and then the user can be guided to turn on the second image collection component to detect whether the second image collection component is turned on. Local image collection and analysis are performed on areas with skin problems; if the second image collection component is turned on, the user can be prompted to perform full face image detection first. If the user does not turn off the second image collection component, it means that the user does not want to perform full face image collection and detection. , then directly collect and detect partial images of the area selected by the user, or guide the user to sequentially collect and detect images of the forehead, left cheek, right cheek, nose and chin areas to meet the user's multi-faceted needs.

实施例2Example 2

参见图5至图8所示，See Figure 5 to Figure 8,

一种脸部皮肤检测装置，包括第一图像采集组件30、美妆镜10和第二图像采集组件20；所述美妆镜10和所述第一图像采集组件30及第二图像采集组件20分别进行数据交互；所述第一图像采集组件30和所述美妆镜10分立设置，或者所述第一图像采集组件30设置在所述美妆镜10上。A facial skin detection device, including a first image collection component 30, a cosmetic mirror 10 and a second image collection component 20; the cosmetic mirror 10 and the first image collection component 30 and the second image collection component 20 Data interaction is performed respectively; the first image collection component 30 and the cosmetic mirror 10 are provided separately, or the first image collection component 30 is provided on the cosmetic mirror 10 .

本实施例的第一图像采集组件30包括设置在所述美妆镜10上方的主摄像头301。所述第二图像采集组件20放置在笔座201内。所述笔座201设置在美妆镜10旁且与所述美妆镜10一体化设计。The first image collection component 30 of this embodiment includes a main camera 301 disposed above the cosmetic mirror 10 . The second image collection component 20 is placed in the pen holder 201 . The pen holder 201 is arranged next to the cosmetic mirror 10 and is designed to be integrated with the cosmetic mirror 10 .

具体的，所述美妆镜10还包括镜体101、旋转组件102、支撑架103和底座104。所述镜体101通过支撑架103连接在底座104上。所述旋转组件102带动镜体101进行翻转，所述旋转组件102包括旋转轴1021。Specifically, the beauty mirror 10 also includes a mirror body 101, a rotating component 102, a support frame 103 and a base 104. The mirror body 101 is connected to the base 104 through a support frame 103 . The rotating component 102 drives the mirror body 101 to flip, and the rotating component 102 includes a rotating shaft 1021 .

进一步的，镜体101正面由触摸屏1011、显示屏1012、控制主板1013、补光灯组件1014等组成，其中主摄像头301位于镜体101上端中部。其触摸屏1011内侧对应丝印位置镂空黑色丝印，可以方便用户直接在镜面上操作各种功能。所述控制主板1013内存储有实现如上所述的脸部皮肤检测方法的软件程序，且所述软件程序能够在控制主板1013上执行，并在显示屏1012上显示执行结果。Further, the front of the scope 101 is composed of a touch screen 1011, a display screen 1012, a control motherboard 1013, a fill light assembly 1014, etc., in which the main camera 301 is located in the middle of the upper end of the scope 101. The inner side of the touch screen 1011 has a hollow black silk screen corresponding to the silk screen position, which allows users to operate various functions directly on the mirror surface. The control mainboard 1013 stores a software program that implements the facial skin detection method as described above, and the software program can be executed on the control mainboard 1013 and the execution results are displayed on the display screen 1012 .

本实施例的具体实施方式如下：The specific implementation of this embodiment is as follows:

A、首先美妆镜开机，中间显示屏启动，进入APP主界面，自动搜索有无测肤笔WIFI热点，并在APP界面上显示出来；A. First, turn on the beauty mirror, start the middle display screen, enter the main interface of the APP, automatically search for the WIFI hotspot of the skin test pen, and display it on the APP interface;

B、若有，参见图4所示，按照流程先进行局部皮肤检测，再进行全脸检测，然后出整体测试报告；B. If yes, please refer to Figure 4. According to the process, first perform local skin detection, then perform full face detection, and then The overall test report will be issued later;

C、若无，参见图3所示，则立即启动主摄像头，先判定用户的整个脸部是否都在摄像区域内，若没有，则语音提示用户做相应移动；再对四周光线亮度进行判定，若环境较暗，则启动镜体四周补妆灯的白色LED灯组；最后对全脸进行白光拍照，然后基于深度算法中卷积神经网络的目标检测，识别人脸限定区域内皮肤问题，并定位所检测肤况的图像坐标，在人脸上的对应位置区域标识出来；C. If not, as shown in Figure 3, the main camera will be started immediately to determine whether the user's entire face is within the camera area. If not, the user will be prompted by voice to move accordingly; and then the surrounding light brightness will be determined. If the environment is dark, the white LED light group of makeup lights around the mirror body is started; finally, the whole face is photographed with white light, and then based on the target detection of the convolutional neural network in the depth algorithm, skin problems in the limited area of the face are identified, and Locate the image coordinates of the detected skin condition and mark the corresponding location area on the face;

D、界面提示用户是否立即局部检测；D. The interface prompts the user whether to perform partial detection immediately;

E、有目的性引导用户，采用局部测肤笔对准上述标识区域，在白光、偏振光、UV光等光源下，采用数十倍放大倍数的摄像头，***依据图像清晰度自动拍照，对所有重度区域进行有针对性的图像采集，采集后图像，***利用自有算法进行数据运算，计算面部皮肤的关键维度：纹理、毛孔、痤疮、敏感、色斑、黑头、油脂等的分值，判定其重、中、轻度，这样更精确获悉皮肤的健康情况，更精准推荐解决方案。E. Purposefully guide users, use a local skin measuring pen to aim at the above-mentioned marked area, and use a camera with dozens of times of magnification under white light, polarized light, UV light and other light sources. The system will automatically take pictures based on the image clarity, and measure all Targeted image collection is carried out in severe areas. After collecting the images, the system uses its own algorithm to perform data operations to calculate the key dimensions of facial skin: texture, pores, acne, sensitivity, stains, blackheads, oil, etc., and determine the scores. It is divided into heavy, medium and light, so as to understand the health of the skin more accurately and recommend solutions more accurately.

实施例3Example 3

本实施例与实施例2的区别在于，实现如上所述的脸部皮肤检测方法的软件程序存储在外部终端设备中，且所述软件程序能够在外部终端设备上执行，并在外部终端设备的显示屏上显示执行结果。The difference between this embodiment and Embodiment 2 is that the software program that implements the facial skin detection method as described above is stored in an external terminal device, and the software program can be executed on the external terminal device and executed on the external terminal device. The execution results are shown on the display.

具体的，参见图9所示，一种脸部皮肤检测装置，包括第一图像采集组件901、美妆镜902、第二图像采集组件903和外部终端设备904；所述美妆镜902和所述第一图像采集组件901、第二图像采集组件903及外部终端设备904分别进行数据交互；所述第一图像采集组件901和所述美妆镜902分立设置，或者所述第一图像采集组件901设置在所述美妆镜902上。Specifically, as shown in Figure 9, a facial skin detection device includes a first image acquisition component 901, a cosmetic mirror 902, a second image acquisition component 903 and an external terminal device 904; the cosmetic mirror 902 and the cosmetic mirror 902. The first image acquisition component 901, the second image acquisition component 903 and the external terminal device 904 perform data interaction respectively; the first image acquisition component 901 and the cosmetic mirror 902 are set up separately, or the first image acquisition component 901 is provided on the cosmetic mirror 902.

本实施例中，所述第二图像采集组件903包括测肤笔。In this embodiment, the second image collection component 903 includes a skin measuring pen.

基于同一发明构思，本公开实施例还提供了一种电子设备，该电子设备以通用计算设备的形式表现。电子设备的组件可以包括但不限于：至少一个处理单元、至少一个存储单元、连接不同***组件(包括存储单元和处理单元)的总线。Based on the same inventive concept, embodiments of the present disclosure also provide an electronic device, which is embodied in the form of a general computing device. The components of the electronic device may include, but are not limited to: at least one processing unit, at least one storage unit, and a bus connecting different system components (including storage units and processing units).

其中，所述存储单元存储有程序代码，所述程序代码可以被所述处理单元执行，使得所述处理单元执行本说明书上述“示例性方法”部分中描述的根据本公开各种示例性实施方式的步骤。例如，所述处理单元可以执行上述方法实施例的如下步骤：控制第一图像采集组件采集全脸图像；根据全脸图像识别出皮肤问题并进行区域标识；提示使用第二图像采集组件对皮肤问题所在的标识区域采集局部图像，基于采集的局部图像，获得标识区域在皮肤问题对应维度上的严重程度和/或分数。Wherein, the storage unit stores program code, and the program code can be executed by the processing unit, so that the processing unit performs various exemplary implementations according to the present disclosure described in the "Example Method" section of this specification. A step of. For example, the processing unit can perform the following steps of the above method embodiment: control the first image acquisition component to collect a full-face image; identify skin problems based on the full-face image and perform area identification; prompt the use of the second image acquisition component to detect skin problems. Local images are collected in the marked area, and based on the collected local images, the severity and/or score of the marked area in the corresponding dimension of the skin problem is obtained.

存储单元可以包括易失性存储单元形式的可读介质，例如随机存取存储单元(RAM)和/或高速缓存存储单元，还可以进一步包括只读存储单元(ROM)。The storage unit may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) and/or a cache memory unit, and may further include a read-only memory unit (ROM).

存储单元还可以包括具有一组(至少一个)程序模块的程序/实用工具，这样的程序模块包括但不限于：操作***、一个或者多个应用程序、其它程序模块以及程序数据，这些示例中的每一个或某种组合中可能包括网络环境的实现。The storage unit may also include a program/utility having a set of (at least one) program modules such that the program Program modules include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each of these examples or some combination may include the implementation of a network environment.

总线可以为表示几类总线结构中的一种或多种，包括存储单元总线或者存储单元控制器、***总线、图形加速端口、处理单元或者使用多种总线结构中的任意总线结构的局域总线。A bus may be a local bus representing one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or using any of a variety of bus structures. .

电子设备也可以与一个或多个外部设备(例如键盘、指向设备、蓝牙设备等)通信，还可与一个或者多个使得用户能与该电子设备交互的设备通信，和/或与使得该电子设备能与一个或多个其它计算设备进行通信的任何设备(例如路由器、调制解调器等等)通信。这种通信可以通过输入/输出(I/O)接口进行。并且，电子设备还可以通过网络适配器与一个或者多个网络(例如局域网(LAN)，广域网(WAN)和/或公共网络，例如因特网)通信。如图所示，网络适配器通过总线与电子设备的其它模块通信。应当明白，尽管图中未示出，可以结合电子设备使用其它硬件和/或软件模块，包括但不限于：微代码、设备驱动器、冗余处理单元、外部磁盘驱动阵列、RAID***、磁带驱动器以及数据备份存储***等。An electronic device may also communicate with one or more external devices (e.g., keyboard, pointing device, Bluetooth device, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device, and/or with one or more devices that enable the electronic device to interact with the electronic device. A device can communicate with any device (eg, router, modem, etc.) that communicates with one or more other computing devices. This communication can occur through input/output (I/O) interfaces. Furthermore, the electronic device may also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN) and/or a public network such as the Internet) through a network adapter. As shown in the figure, the network adapter communicates with other modules of the electronic device through a bus. It will be understood that, although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and Data backup storage system, etc.

基于同一发明构思，本公开实施例还提供了一种计算机可读存储介质，该计算机可读存储介质可以是可读信号介质或者可读存储介质。其上存储有能够实现本公开上述方法的程序产品。在一些可能的实施方式中，本公开的各个方面还可以实现为一种程序产品的形式，其包括程序代码，当所述程序产品在终端设备上运行时，所述程序代码用于使所述终端设备执行本说明书上述“示例性方法”部分中描述的根据本公开各种示例性实施方式的步骤。Based on the same inventive concept, embodiments of the present disclosure also provide a computer-readable storage medium. The computer-readable storage medium may be a readable signal medium or a readable storage medium. Program products capable of implementing the above methods of the present disclosure are stored thereon. In some possible implementations, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to cause the The terminal device performs the steps according to various exemplary embodiments of the present disclosure described in the above "Example Method" section of this specification.

本公开实施例中的计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。More specific examples of computer-readable storage media in embodiments of the present disclosure may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM) ), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

在本公开中，计算机可读存储介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了可读程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。可读信号介质还可以是可读存储介质以外的任何可读介质，该可读介质可以发送、传播或者传输用于由指令执行***、装置或者器件使用或者与其结合使用的程序。In this disclosure, a computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave carrying readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium that can send, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or device.

可选地，计算机可读存储介质上包含的程序代码可以用任何适当的介质传输，包括但不限于无线、有线、光缆、RF等等，或者上述的任意合适的组合。Alternatively, program code embodied on a computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

在具体实施时，可以以一种或多种程序设计语言的任意组合来编写用于执行本公开操作的程序代码，所述程序设计语言包括面向对象的程序设计语言—诸如Java、C++ 等，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算设备上执行、部分地在用户设备上执行、作为一个独立的软件包执行、部分在用户计算设备上部分在远程计算设备上执行、或者完全在远程计算设备或服务器上执行。在涉及远程计算设备的情形中，远程计算设备可以通过任意种类的网络，包括局域网(LAN)或广域网(WAN)，连接到用户计算设备，或者，可以连接到外部计算设备(例如利用因特网服务提供商来通过因特网连接)。When implemented, program code for performing operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages—such as Java, C++ etc., also includes conventional procedural programming languages - such as "C" language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server execute on. In situations involving remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device, such as provided by an Internet service. (business comes via Internet connection).

基于同一发明构思，本公开实施例还提供了一种计算机程序产品，该计算机程序产品包括：计算机程序，所述计算机程序被处理器执行时实现上述脸部皮肤检测方法。Based on the same inventive concept, embodiments of the present disclosure also provide a computer program product. The computer program product includes: a computer program that implements the above facial skin detection method when executed by a processor.

以上所述，仅为本公开较佳的具体实施方式；但本公开的保护范围并不局限于此。任何熟悉本技术领域的技术人员在本公开揭露的技术范围内，根据本公开的技术方案及其改进构思加以等同替换或改变，都应涵盖在本公开的保护范围内。 The above are only preferred embodiments of the present disclosure; however, the protection scope of the present disclosure is not limited thereto. Any person familiar with the technical field who is familiar with the technical field shall make equivalent substitutions or changes based on the technical solutions and improvement concepts of the present disclosure within the technical scope disclosed by the present disclosure, and shall be covered by the protection scope of the present disclosure.

Claims

一种脸部皮肤检测方法，包括：A facial skin detection method, including:

控制第一图像采集组件采集全脸图像；Control the first image acquisition component to acquire a full face image;

根据全脸图像识别出皮肤问题并进行区域标识；Identify skin problems and identify areas based on full-face images;

提示使用第二图像采集组件对皮肤问题所在的标识区域采集局部图像，基于采集的局部图像，获得标识区域在皮肤问题对应维度上的严重程度和/或分数。It is prompted to use the second image acquisition component to collect a local image of the marked area where the skin problem is located, and based on the collected local image, obtain the severity and/or score of the marked area in the corresponding dimension of the skin problem.
根据权利要求1所述的脸部皮肤检测方法，其中，控制第一图像采集组件采集全脸图像，具体包括：The facial skin detection method according to claim 1, wherein controlling the first image acquisition component to acquire a full face image specifically includes:

当检测到整个脸部处于第一图像采集组件摄像区域内时，控制开启白光进行全脸拍照以采集全脸取像。When it is detected that the entire face is within the imaging area of the first image acquisition component, the white light is controlled to be turned on to take a full-face photo to collect a full-face image.
根据权利要求1所述的脸部皮肤检测方法，其中，根据全脸图像识别出皮肤问题并进行区域标识，具体包括：The facial skin detection method according to claim 1, wherein skin problems are identified based on the full face image and regional identification is performed, specifically including:

使用训练好的卷积神经网络模型识别出全脸图像中的皮肤问题，定位皮肤问题的图像坐标，并在全脸图像的对应位置区域进行标识。Use the trained convolutional neural network model to identify skin problems in the full-face image, locate the image coordinates of the skin problem, and mark the corresponding location area in the full-face image.
根据权利要求1所述的脸部皮肤检测方法，其中，提示使用第二图像采集组件对皮肤问题所在的标识区域采集局部图像，具体包括：The facial skin detection method according to claim 1, wherein the prompt is to use the second image acquisition component to collect a local image of the identified area where the skin problem is located, specifically including:

提示使用第二图像采集组件在自然光、UV光或偏振光下，对每个皮肤问题所在的标识区域分别采集预设张局部图像；所述预设张大于1张。It is prompted to use the second image acquisition component to collect preset partial images of the marked area where each skin problem is located under natural light, UV light or polarized light; the preset image is greater than 1 image.
根据权利要求4所述的脸部皮肤检测方法，其中，提示使用第二图像采集组件在自然光、UV光或偏振光下，对每个皮肤问题所在的标识区域分别采集预设张局部图像，具体包括：The facial skin detection method according to claim 4, wherein the prompt is to use the second image acquisition component to collect preset partial images of the identified area where each skin problem is located under natural light, UV light or polarized light. Specifically, include:

S501，提示用户使用第二图像采集组件在自然光、UV光或偏振光下，采集某一皮肤问题所在的标识区域的图像，并判定所拍摄的图像是否清晰，若不清晰则重复步骤S501；若清晰则进入步骤S502；S501, prompt the user to use the second image acquisition component to collect an image of the marked area where a certain skin problem is located under natural light, UV light or polarized light, and determine whether the captured image is clear. If not, repeat step S501; if If clear, proceed to step S502;

S502，提示用户对该标识区域在相同光源或不同光源下拍摄下一张图像；并重复步骤S501；S502, prompt the user to take the next image of the identified area under the same light source or a different light source; and repeat step S501;

S503，直至该区域清晰的图像数量达到预设张；S503, until the number of clear images in this area reaches the preset;

S504，重复S501～S503，直至所有有皮肤问题的标识区域均采集到预设张局部图像。S504: Repeat S501 to S503 until all preset partial images are collected in all marked areas with skin problems.
根据权利要求1所述的脸部皮肤检测方法，其中，控制第一图像采集组件采集全脸图像之前，还包括：The facial skin detection method according to claim 1, wherein before controlling the first image acquisition component to acquire the full face image, it further includes:

检测第二图像采集组件的开启状态，如果未开启，控制进入第一图像采集组件采集全脸图像步骤；如果开启，提示使用第二图像采集组件对划分的各个脸部区域采集局部图像，基于采集的局部图像，获得各脸部区域在皮肤问题对应维度上的严重程度和/或分数。Detect the open status of the second image acquisition component. If it is not open, the control enters the step of collecting the whole face image by the first image acquisition component; if it is open, it prompts to use the second image acquisition component to collect the divided face areas. Partial images: Based on the collected partial images, the severity and/or score of each facial area in the corresponding dimension of the skin problem is obtained.
根据权利要求6所述的脸部皮肤检测方法，其中，提示使用第二图像采集组件对划分的各个脸部区域采集局部图像，具体包括：The facial skin detection method according to claim 6, wherein prompting to use the second image acquisition component to collect partial images of each divided facial area specifically includes:

提示使用第二图像采集组件在自然光、UV光或偏振光下，对划分的各个脸部区域分别采集预设张局部图像；所述预设张大于1张。It is prompted to use the second image acquisition component to collect a preset partial image of each divided facial area under natural light, UV light or polarized light; the preset image is greater than 1.
根据权利要求7所述的脸部皮肤检测方法，其中，提示使用第二图像采集组件在自然光、UV光或偏振光下，对划分的各个脸部区域分别采集预设张局部图像，具体包括：The facial skin detection method according to claim 7, wherein the prompt is to use the second image acquisition component to collect preset partial images of each divided facial area under natural light, UV light or polarized light, specifically including:

S801，提示用户使用第二图像采集组件在自然光、UV光或偏振光下，采集划分的某一脸部区域的图像，并判定所拍摄的图像是否清晰，若不清晰则重复步骤S801；若清晰则进入步骤S802；S801. Prompt the user to use the second image acquisition component to collect an image of a divided facial area under natural light, UV light or polarized light, and determine whether the captured image is clear. If it is not clear, repeat step S801; if it is clear, Then enter step S802;

S802，提示用户对该脸部区域在相同光源或不同光源下拍摄下一张图像；并重复步骤S801；S802, prompt the user to take the next image of the facial area under the same light source or different light sources; and repeat step S801;

S803，直至该区域清晰的图像数量达到预设张；S803, until the number of clear images in this area reaches the preset;

S804，重复S801～S803，直至所有划分的脸部区域均采集到预设张局部图像。S804: Repeat S801 to S803 until all divided facial areas have collected preset partial images.
根据权利要求6所述的脸部皮肤检测方法，其中，所述脸部区域包括：额头、左脸颊、右脸颊、鼻部和下巴。The facial skin detection method according to claim 6, wherein the facial area includes: forehead, left cheek, right cheek, nose and chin.
根据权利要求1所述的脸部皮肤检测方法，其中，所述皮肤问题包括：纹理、毛孔、痤疮、敏感、色斑、黑头或油脂中的至少一个；每个皮肤问题对应一个维度。The facial skin detection method according to claim 1, wherein the skin problem includes: at least one of texture, pores, acne, sensitivity, stains, blackheads or oil; each skin problem corresponds to one dimension.
根据权利要求1所述的脸部皮肤检测方法，其中，所述严重程度包括：重度、中度或轻度。The facial skin detection method according to claim 1, wherein the severity includes: severe, moderate or mild.
根据权利要求11所述的脸部皮肤检测方法，其中，所述分数基于严重程度及对应的置信度获得，具体如下：

The facial skin detection method according to claim 11, wherein the score is obtained based on the severity and the corresponding confidence level, specifically as follows:

其中，x表示置信度；f(x)表示分数。Among them, x represents the confidence level; f(x) represents the score.
一种脸部皮肤检测装置，包括第一图像采集组件、美妆镜和第二图像采集组件；所述美妆镜和所述第一图像采集组件及第二图像采集组件分别进行数据交互；所述第一图像采集组件和所述美妆镜分立设置，或者所述第一图像采集组件设置在所述美妆镜上；所述美妆镜用于执行如权利要求1～12中任意一项所述的脸部皮肤检测方法的步骤。A facial skin detection device includes a first image acquisition component, a beauty mirror and a second image acquisition component; the beauty mirror interacts with the first image acquisition component and the second image acquisition component respectively; the The first image acquisition component and the cosmetic mirror are arranged separately, or the first image acquisition component is arranged on the cosmetic mirror; the cosmetic mirror is used to perform any one of claims 1 to 12 Described facial skin detection method steps of the law.
一种脸部皮肤检测装置，包括第一图像采集组件、美妆镜、第二图像采集组件和外部终端设备；所述美妆镜和所述第一图像采集组件、第二图像采集组件及外部终端设备分别进行数据交互；所述第一图像采集组件和所述美妆镜分立设置，或者所述第一图像采集组件设置在所述美妆镜上；所述外部终端设备用于执行如权利要求1～12中任意一项所述的脸部皮肤检测方法的步骤。A facial skin detection device, including a first image acquisition component, a beauty mirror, a second image acquisition component and an external terminal device; the beauty mirror, the first image acquisition component, the second image acquisition component and an external The terminal devices perform data interaction respectively; the first image acquisition component and the beauty mirror are set separately, or the first image acquisition component is set on the beauty mirror; the external terminal device is used to perform the following tasks: The steps of the facial skin detection method according to any one of claims 1 to 12.
一种电子设备，包括：An electronic device including:

处理器；以及processor; and

存储器，用于存储所述处理器的可执行指令；memory for storing executable instructions for the processor;

其中，所述处理器配置为经由执行所述可执行指令来执行权利要求1～12中任一项所述的脸部皮肤检测方法。Wherein, the processor is configured to execute the facial skin detection method according to any one of claims 1 to 12 by executing the executable instructions.
一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现权利要求1～12中任一项所述的脸部皮肤检测方法。A computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the facial skin detection method according to any one of claims 1 to 12 is implemented.
一种计算机程序产品，包括计算机程序，所述计算机程序被处理器执行时实现权利要求1～12中任一项所述的脸部皮肤检测方法。 A computer program product, including a computer program, which implements the facial skin detection method according to any one of claims 1 to 12 when executed by a processor.