TWI723123B - Image fusion method, device and equipment - Google Patents
Image fusion method, device and equipment Download PDFInfo
- Publication number
- TWI723123B TWI723123B TW106102466A TW106102466A TWI723123B TW I723123 B TWI723123 B TW I723123B TW 106102466 A TW106102466 A TW 106102466A TW 106102466 A TW106102466 A TW 106102466A TW I723123 B TWI723123 B TW I723123B
- Authority
- TW
- Taiwan
- Prior art keywords
- template
- area
- image
- fusion
- pixel
- Prior art date
Links
- 238000007500 overflow downdraw method Methods 0.000 title claims description 4
- 230000004927 fusion Effects 0.000 claims abstract description 149
- 238000009499 grossing Methods 0.000 claims abstract description 81
- 238000000034 method Methods 0.000 claims abstract description 65
- 238000005070 sampling Methods 0.000 claims abstract description 45
- 239000000463 material Substances 0.000 claims abstract description 23
- 238000010606 normalization Methods 0.000 claims abstract description 12
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 52
- 230000009466 transformation Effects 0.000 claims description 24
- 230000008569 process Effects 0.000 claims description 13
- 230000003796 beauty Effects 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 abstract description 18
- 238000007499 fusion processing Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 210000004709 eyebrow Anatomy 0.000 description 5
- 238000001914 filtration Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 230000002457 bidirectional effect Effects 0.000 description 2
- 230000002146 bilateral effect Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000011426 transformation method Methods 0.000 description 2
- 239000003086 colorant Substances 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Landscapes
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
本發明提供了一種圖像融合的方法、裝置和設備，其中方法包括：確定圖像中的融合區域，得到第一模板；對該第一模板進行降採樣，得到第二模板；對第二模板中的各像素點的像素值進行歸一化，得到第三模板；對該第三模板進行升採樣，得到第四模板，該第四模板的像素點個數等於該第一模板的像素點個數；將該第四模板中各像素點的像素值分別作為該圖像中對應像素點的權重，對該圖像中融合區域各像素點和融合素材進行加權融合。另外，在該歸一化之前還可以包括利用預定義的平滑模板對第二模板進行邊緣平滑以及基於圖像中融合區域的亮度，對第二模板進行亮度調整的步驟。本發明能夠降低圖像融合的計算量，降低時間成本和資源消耗。 The present invention provides a method, device and equipment for image fusion, wherein the method includes: determining a fusion area in an image to obtain a first template; down-sampling the first template to obtain a second template; Normalize the pixel value of each pixel in to obtain a third template; perform up-sampling on the third template to obtain a fourth template, the number of pixels in the fourth template is equal to the number of pixels in the first template Number; the pixel value of each pixel in the fourth template is used as the weight of the corresponding pixel in the image, and each pixel in the fusion area of the image and the fusion material are weighted and fused. In addition, before the normalization, it may also include the steps of using a predefined smoothing template to smooth the edges of the second template and adjusting the brightness of the second template based on the brightness of the fusion area in the image. The invention can reduce the calculation amount of image fusion, and reduce time cost and resource consumption.
Description
本發明涉及電腦影像處理技術領域，特別涉及一種圖像融合的方法、裝置和設備。 The present invention relates to the technical field of computer image processing, in particular to an image fusion method, device and equipment.
隨著智慧終端的不斷普及，人們利用智慧終端進行影像處理的需求越來越高，各類美顏類APP受到愛美人士的廣泛青睞。在這類APP中，常常會涉及到圖像融合處理，現有的圖像融合處理的複雜度較大，使得在圖像融合涉及到的像素面積較大時，計算量引起的時間成本很大，即時性難以保證，並且對系統資源的消耗和佔用都很大。 With the continuous popularization of smart terminals, people's demand for using smart terminals for image processing is increasing, and various beauty apps are widely favored by beauty lovers. In this type of APP, image fusion processing is often involved. The existing image fusion processing is more complicated, so that when the pixel area involved in image fusion is large, the time cost caused by the amount of calculation is very large. The immediacy is difficult to guarantee, and it consumes and occupies a lot of system resources.
有鑑於此，本發明提供了一種圖像融合的方法、裝置和設備，以便於降低圖像融合的計算量，降低時間成本和資源消耗。 In view of this, the present invention provides a method, device and equipment for image fusion, so as to reduce the calculation amount of image fusion, and reduce time cost and resource consumption.
具體技術方案如下：本發明提供了一種圖像融合的方法，該方法包括：確定圖像中的融合區域，得到第一模板； 對該第一模板進行降採樣，得到第二模板；對第二模板中的各像素點的像素值進行歸一化，得到第三模板；對該第三模板進行升採樣，得到第四模板，該第四模板的像素點個數等於該第一模板的像素點個數；將該第四模板中各像素點的像素值分別作為該圖像中對應像素點的權重，對該圖像中融合區域各像素點和融合素材進行加權融合。 The specific technical solutions are as follows: the present invention provides a method for image fusion, which includes: determining a fusion area in an image to obtain a first template; down-sampling the first template to obtain a second template; The pixel value of each pixel in the template is normalized to obtain a third template; the third template is up-sampled to obtain a fourth template, the number of pixels of the fourth template is equal to the number of pixels of the first template Number; the pixel value of each pixel in the fourth template is used as the weight of the corresponding pixel in the image, and each pixel in the fusion area in the image and the fusion material are weighted and fused.
根據本發明一優選實施方式，該確定圖像中的融合區域，得到第一模板包括：對圖像中的融合目標進行特徵點定位，該特徵點包括輪廓點；利用定位出的特徵點，去除圖像中除融合目標之外的區域，得到第一模板。 According to a preferred embodiment of the present invention, the determining the fusion area in the image to obtain the first template includes: locating feature points of the fusion target in the image, and the feature points include contour points; using the located feature points to remove In the image except for the fusion target area, the first template is obtained.
根據本發明一優選實施方式，對該第一模板進行降採樣，得到第二模板包括：採用仿射變換的方式，對該第一模板進行降採樣，使得到的第二模板的像素點個數為第一模板的
根據本發明一優選實施方式，在該對第二模板中的各像素點的像素值進行歸一化之前，該方法還包括：對第二模板的邊緣進行平滑處理。 According to a preferred embodiment of the present invention, before normalizing the pixel value of each pixel in the second template, the method further includes: smoothing the edge of the second template.
根據本發明一優選實施方式，該對第二模板的邊緣進行平滑處理包括：將該第二模板中融合區域的輪廓點分別向外和/或向內擴展M個像素點，該M為預設的正整數，將擴展的像素點所包圍的區域作為待平滑區域；將預定義的平滑模板仿射到該待平滑區域，得到平滑後的第二模板。 According to a preferred embodiment of the present invention, the smoothing of the edge of the second template includes: extending the contour points of the fusion area in the second template outward and/or inward by M pixels, where M is a preset A positive integer of, the area surrounded by the extended pixels is taken as the area to be smoothed; the predefined smoothing template is affine to the area to be smoothed to obtain a smoothed second template.
根據本發明一優選實施方式，該將預定義的平滑模板仿射到該待平滑區域包括：將該平滑模板中像素點的像素值，仿射為該待平滑區域中對應位置像素點的像素值。 According to a preferred embodiment of the present invention, the affine the predefined smoothing template to the area to be smoothed includes: affine the pixel value of the pixel in the smoothing template to the pixel value of the pixel at the corresponding position in the area to be smoothed .
根據本發明一優選實施方式，該將預定義的平滑模板仿射到該待平滑區域包括：在該平滑模板上的平滑區域和該第二模板上的待平滑區域分別採用相同的方式進行三角剖分，得到相同個數的三角區域；將該平滑模板中各三角區域分別仿射到第二模板中對應位置的三角區域。 According to a preferred embodiment of the present invention, the affine the predefined smoothing template to the area to be smoothed includes: the smoothing area on the smoothing template and the area to be smoothed on the second template are respectively triangulated in the same way And obtain the same number of triangular areas; each triangular area in the smooth template is affine respectively to the triangular area at the corresponding position in the second template.
根據本發明一優選實施方式，在該對第二模板中的各像素點的像素值進行歸一化之前，該方法還包括：對該圖像中的融合區域進行亮度統計；依據亮度統計結果，對平滑處理後的第二模板進行亮度調整。 According to a preferred embodiment of the present invention, before normalizing the pixel value of each pixel in the second template, the method further includes: performing brightness statistics on the fusion area in the image; and according to the brightness statistics result, Adjust the brightness of the smoothed second template.
根據本發明一優選實施方式，該依據亮度統計結果， 對平滑處理後的第二模板進行亮度調整包括：確定該圖像中的融合區域的亮度均值和平滑處理後的第二模板的亮度均值之間的差值；將平滑處理後的第二模板中各像素點的亮度值分別加上該差值。 According to a preferred embodiment of the present invention, the brightness adjustment of the smoothed second template based on the brightness statistics includes: determining the average brightness of the fusion area in the image and the average brightness of the smoothed second template. The difference between the two; the brightness value of each pixel in the second template after the smoothing process is respectively added to the difference.
根據本發明一優選實施方式，對該圖像中融合區域各像素點和融合素材進行加權融合包括：利用Imagei_new=weight_maski * Imagei_old+(1-weight_maski)*Colori，確定融合後得到的各像素點的像素值；其中Imagei_new為該圖像中融合區域融合後得到的第i個像素點的像素值，weight_maski為該第四模板中第i個像素點的像素值，Imagei_old為該圖像中融合區域第i個像素點的像素值，Colori為融合素材提供的第i個像素點的像素值。 According to a preferred embodiment of the present invention, performing weighted fusion of each pixel in the fusion area and the fusion material in the image includes: using Image i_new =weight_mask i * Image i_old +(1-weight_mask i )*Color i to determine the result after fusion The pixel value of each pixel; where Image i_new is the pixel value of the i-th pixel obtained after the fusion area in the image is fused, weight_mask i is the pixel value of the i-th pixel in the fourth template, Image i_old Is the pixel value of the i-th pixel in the fusion area of the image, and Color i is the pixel value of the i-th pixel provided by the fusion material.
根據本發明一優選實施方式，該方法應用於美顏類APP；該融合區域為人臉區域；該融合素材為粉底色。 According to a preferred embodiment of the present invention, the method is applied to a beauty APP; the fusion area is a face area; and the fusion material is a foundation color.
本發明還提供了一種圖像融合的裝置，該裝置包括：模板確定單元，用於確定圖像中的融合區域，得到第一模板；降採樣單元，用於對該第一模板進行降採樣，得到第二模板；歸一化單元，用於對第二模板中的各像素點的像素值 進行歸一化，得到第三模板；升採樣單元，用於對該第三模板進行升採樣，得到第四模板，該第四模板的像素點個數等於該第一模板的像素點個數；加權融合單元，用於將該第四模板中各像素點的像素值分別作為該該圖像中對應像素點的權重，對該圖像中融合區域各像素點和融合素材進行加權融合。 The present invention also provides an image fusion device, which includes: a template determining unit for determining a fusion area in the image to obtain a first template; a down-sampling unit for down-sampling the first template, Obtain the second template; a normalization unit for normalizing the pixel value of each pixel in the second template to obtain the third template; up-sampling unit for up-sampling the third template to obtain The fourth template, the number of pixels in the fourth template is equal to the number of pixels in the first template; a weighted fusion unit is used for the pixel value of each pixel in the fourth template as the corresponding pixel in the image The weight of the pixel points, and each pixel point and the fusion material in the fusion area in the image are weighted and fused.
根據本發明一優選實施方式，該模板確定單元，具體用於：對圖像中的融合目標進行特徵點定位，該特徵點包括輪廓點；利用定位出的特徵點，去除圖像中除融合目標之外的區域，得到第一模板。根據本發明一優選實施方式，該降採樣單元，具體用於採用仿射變換的方式，對該第一模板進行降採樣，使得到的第二模板的像素點個數為第一模板的
根據本發明一優選實施方式，該裝置還包括：邊緣平滑單元，用於對該第二模板的邊緣進行平滑處理，將平滑處理後的第二模板輸出給該歸一化單元。 According to a preferred embodiment of the present invention, the device further includes: an edge smoothing unit for smoothing the edges of the second template, and outputting the smoothed second template to the normalization unit.
根據本發明一優選實施方式，該邊緣平滑單元，具體 用於：將該第二模板中融合區域的輪廓點分別向外和/或向內擴展M個像素點，該M為預設的正整數，將擴展的像素點所包圍的區域作為待平滑區域；將預定義的平滑模板仿射到該待平滑區域，得到平滑後的第二模板。 According to a preferred embodiment of the present invention, the edge smoothing unit is specifically configured to extend the contour points of the fusion area in the second template outward and/or inward respectively by M pixels, where M is a preset positive integer , Taking the area surrounded by the expanded pixels as the area to be smoothed; affine a predefined smoothing template to the area to be smoothed to obtain a smoothed second template.
根據本發明一優選實施方式，該邊緣平滑單元在將預定義的平滑模板仿射到該待平滑區域時，具體執行：將該平滑模板中像素點的像素值，仿射為該待平滑區域中對應位置像素點的像素值。 According to a preferred embodiment of the present invention, when the edge smoothing unit affines the predefined smoothing template to the area to be smoothed, it specifically executes: affine the pixel values of the pixels in the smoothing template to be in the area to be smoothed The pixel value of the pixel at the corresponding position.
根據本發明一優選實施方式，該邊緣平滑單元在將預定義的平滑模板仿射到該待平滑區域時，具體執行：在該平滑模板上的平滑區域和該第二模板上的待平滑區域分別採用相同的方式進行三角剖分，得到相同個數的三角區域；將該平滑模板中各三角區域分別仿射到第二模板中對應位置的三角區域。 According to a preferred embodiment of the present invention, when the edge smoothing unit affines the predefined smoothing template to the area to be smoothed, it specifically executes: the smoothing area on the smoothing template and the area to be smoothed on the second template respectively Triangulation is performed in the same way to obtain the same number of triangular areas; each triangular area in the smooth template is affineed to the triangular area at the corresponding position in the second template.
根據本發明一優選實施方式，該裝置還包括：亮度調整單元，用於獲取該邊緣平滑單元輸出的第二模板，對該圖像中的融合區域進行亮度統計，依據亮度統計結果，對獲取的第二模板進行亮度調整，將亮度調整後的第二模板輸出給該歸一化單元。 According to a preferred embodiment of the present invention, the device further includes: a brightness adjustment unit, configured to obtain a second template output by the edge smoothing unit, perform brightness statistics on the fusion region in the image, and perform brightness statistics on the obtained The second template performs brightness adjustment, and the second template after the brightness adjustment is output to the normalization unit.
根據本發明一優選實施方式，該亮度調整單元在依據亮度統計結果，對平滑處理後的第二模板進行亮度調整 時，具體執行：確定該圖像中的融合區域的亮度均值和平滑處理後的第二模板的亮度均值之間的差值；將平滑處理後的第二模板中各像素點的亮度值分別加上該差值。 According to a preferred embodiment of the present invention, when the brightness adjustment unit adjusts the brightness of the smoothed second template according to the brightness statistics, it specifically executes: determining the average brightness of the fusion area in the image and the smoothed The difference between the mean brightness values of the second template; the brightness value of each pixel in the second template after the smoothing process is respectively added to the difference.
根據本發明一優選實施方式，該加權融合單元，具體用於：利用Imagei_new=weight_maski * Imagei_old+(1-weight_maski)*Colori，確定融合後得到的各像素點的像素值；其中Imagei_new為該圖像中融合區域融合後得到的第i個像素點的像素值，weight_maski為該第四模板中第i個像素點的像素值，Imagei_old為該圖像中融合區域第i個像素點的像素值，Colori為融合素材提供的第i個像素點的像素值。 According to a preferred embodiment of the present invention, the weighted fusion unit is specifically used to: use Image i_new = weight_mask i * Image i_old +(1-weight_mask i )*Color i to determine the pixel value of each pixel point obtained after fusion; where Image i_new is the pixel value of the i-th pixel obtained after the fusion area in the image is fused, weight_mask i is the pixel value of the i-th pixel in the fourth template, Image i_old is the i-th pixel in the fusion area in the image The pixel value of each pixel, Color i is the pixel value of the i-th pixel provided by the fusion material.
根據本發明一優選實施方式，該裝置應用於美顏類APP；該融合區域為人臉區域；該融合素材為粉底色。 According to a preferred embodiment of the present invention, the device is applied to a beauty APP; the fusion area is a face area; and the fusion material is a foundation color.
本發明還提供了一種設備，包括一個或者多個處理器；記憶體；一個或者多個程式，該一個或者多個程式儲存在該記憶體中，被該一個或者多個處理器執行以實現如下操作：確定圖像中的融合區域，得到第一模板； 對該第一模板進行降採樣，得到第二模板；對第二模板中的各像素點的像素值進行歸一化，得到第三模板；對該第三模板進行升採樣，得到第四模板，該第四模板的像素點個數等於該第一模板的像素點個數；將該第四模板中各像素點的像素值分別作為該圖像中對應像素點的權重，對該圖像中融合區域各像素點和融合素材進行加權融合。 The present invention also provides a device, including one or more processors; memory; one or more programs, the one or more programs are stored in the memory and executed by the one or more processors to achieve the following Operation: Determine the fusion area in the image to obtain the first template; down-sample the first template to obtain the second template; normalize the pixel value of each pixel in the second template to obtain the third template ; Up-sampling the third template to obtain a fourth template, the number of pixels in the fourth template is equal to the number of pixels in the first template; the pixel value of each pixel in the fourth template is used as the The weight of the corresponding pixel in the image is weighted and fused with each pixel in the fusion area of the image and the fusion material.
由以上技術方案可以看出，本發明採用對圖像進行降採樣的方式，對降採樣後的融合區域進行權重計算，然後再升採樣回原始圖像大小，得到原始圖像中融合區域各像素點在融合時對應的權重，大大降低了因權重計算所帶來的計算量，降低了時間成本和資源消耗。 It can be seen from the above technical solutions that the present invention adopts the method of down-sampling the image to calculate the weight of the down-sampled fusion area, and then up-sample it back to the original image size to obtain the pixels of the fusion area in the original image. The weights corresponding to the points during fusion greatly reduce the amount of calculation caused by the weight calculation, and reduce the time cost and resource consumption.
01‧‧‧模板確定單元 01‧‧‧Template determination unit
02‧‧‧降採樣單元 02‧‧‧Downsampling unit
03‧‧‧歸一化單元 03‧‧‧Normalization unit
04‧‧‧升採樣單元 04‧‧‧Upsampling unit
05‧‧‧加權融合單元 05‧‧‧Weighted Fusion Unit
06‧‧‧邊緣平滑單元 06‧‧‧Edge smoothing unit
07‧‧‧亮度調整單元 07‧‧‧Brightness adjustment unit
圖1為本發明實施例提供的主要方法流程圖；圖2為本發明實施例提供的一種詳細方法流程圖；圖3a為本發明實施例提供的人臉圖像的示意圖；圖3b為對圖3a進行特徵點定位的示意圖；圖3c為基於圖3b得到的第一模板區域的示意圖；圖3d為圖3c中基於人臉輪廓產生的待平滑區域；圖3e為利用三角剖分法結合平滑模板進行平滑的示意圖；圖4為本發明實施例提供的裝置結構圖； 圖5為本發明實施例提供的設備結構圖。 Fig. 1 is a flowchart of the main method provided by an embodiment of the present invention; Fig. 2 is a flowchart of a detailed method provided by an embodiment of the present invention; Fig. 3a is a schematic diagram of a face image provided by an embodiment of the present invention; Fig. 3a is a schematic diagram of feature point positioning; Fig. 3c is a schematic diagram of the first template area obtained based on Fig. 3b; Fig. 3d is the area to be smoothed generated based on the face contour in Fig. 3c; Fig. 3e is the use of triangulation combined with a smoothing template A schematic diagram of smoothing; FIG. 4 is a structure diagram of an apparatus provided by an embodiment of the present invention; FIG. 5 is a structure diagram of an apparatus provided by an embodiment of the present invention.
為了使本發明的目的、技術方案和優點更加清楚，下面結合圖式和具體實施例對本發明進行詳細描述。 In order to make the objectives, technical solutions, and advantages of the present invention clearer, the following describes the present invention in detail with reference to the drawings and specific embodiments.
在本發明實施例中使用的術語是僅出於描述特定實施例的目的，而非旨在限制本發明。在本發明實施例和所附請求項書中所使用的單數形式的“一種”、“所述”和“該”也旨在包括多數形式，除非上下文清楚地表示其他含義。 The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms of "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.
應當理解，本文中使用的術語“和/或”僅是一種描述關聯物件的關聯關係，表示可以存在三種關係，例如，A和/或B，可以表示：單獨存在A，同時存在A和B，單獨存在B這三種情況。另外，本文中字元“/”，一般表示前後關聯物件是一種“或”的關係。 It should be understood that the term "and/or" used in this text is only an association relationship describing related objects, which means that there can be three relationships, for example, A and/or B can mean that A alone exists, and A and B exist at the same time. There are three cases of B alone. In addition, the character "/" in this text generally indicates that the associated objects before and after are in an "or" relationship.
取決於語境，如在此所使用的詞語“如果”可以被解釋成為“在……時”或“當……時”或“回應於確定”或“回應於檢測”。類似地，取決於語境，短語“如果確定”或“如果檢測(陳述的條件或事件)”可以被解釋成為“當確定時”或“回應於確定”或“當檢測(陳述的條件或事件)時”或“回應於檢測(陳述的條件或事件)”。 Depending on the context, the word "if" as used herein can be interpreted as "when" or "when" or "in response to determination" or "in response to detection". Similarly, depending on the context, the phrase "if determined" or "if detected (stated condition or event)" can be interpreted as "when determined" or "response to determination" or "when detected (stated condition or event)" Event)" or "response to detection (statement or event)".
圖1為本發明實施例提供的主要方法流程圖，如圖1中所示，該方法主要包括以下步驟： Fig. 1 is a flowchart of the main method provided by an embodiment of the present invention. As shown in Fig. 1, the method mainly includes the following steps:
在101中，確定圖像中的融合區域，得到第一模板。 In 101, the fusion area in the image is determined, and the first template is obtained.
本步驟中涉及的圖像是需要進行融合處理的圖像，圖像中的融合區域指的是需要進行融合處理的區域。融合區域可以是指定的目的地區域，也可以是通過特徵點定位的方式確定的目的地區域，具體將在後續實施例中詳述。本步驟實際上是將圖像中的融合區域截取出來，得到第一模板，可以通過將圖像中融合目標之外的區域去除的方式得到。 The image involved in this step is the image that needs to be fused, and the fusion area in the image refers to the area that needs to be fused. The fusion area may be a designated destination area, or a destination area determined through feature point positioning, which will be detailed in subsequent embodiments. This step is actually to cut out the fusion area in the image to obtain the first template, which can be obtained by removing the area outside the fusion target in the image.
在102中，對第一模板進行降採樣，得到第二模板。 In 102, the first template is down-sampled to obtain the second template.
為了降低對融合區域進行融合處理所產生的計算量，在本步驟中，可以對第一模板進行降採樣處理，即減少第一模板中的像素個數。 In order to reduce the amount of calculation generated by the fusion processing on the fusion region, in this step, the first template may be down-sampling processing, that is, the number of pixels in the first template can be reduced.
圖像降採樣的方式有很多，例如最近鄰降採樣法、B樣條降採樣法等。在本發明實施例中可以採用仿射變換的方式，例如採用對第一模板進行縮放變換的方式，設置仿射參數，使得到的第二模板的像素點個數為第一模板的
由於圖像融合是否自然通常體現在融合區域的邊緣處，因此優選地，為了使得圖像融合更加的自然，可以進一步對第二模板的邊緣進行平滑處理。對圖像邊緣進行平滑處理的方式也有很多，例如簡單模糊方式、高斯模糊方式、中值濾波方式、高斯濾波方式等。在本發明實施例中可以利用預定義的平滑模板對第二模板的邊緣進行平滑處理，具體將在後續實施例中詳述。 Since whether the image fusion is natural or not is usually reflected at the edge of the fusion area, preferably, in order to make the image fusion more natural, the edge of the second template may be further smoothed. There are also many ways to smooth the edges of an image, such as simple blur, Gaussian blur, median filter, and Gaussian filter. In the embodiment of the present invention, a predefined smoothing template may be used to smooth the edge of the second template, which will be described in detail in subsequent embodiments.
另外，為了降低原始圖像亮度與平滑處理後第二模板的亮度之間的差異所產生的影響，可以對原始的圖像中融合區域進行亮度統計，依據亮度統計結果對平滑處理後的第二模板進行亮度調整，具體的調整方式將在後續實施例中詳述。 In addition, in order to reduce the impact of the difference between the brightness of the original image and the brightness of the smoothed second template, brightness statistics can be performed on the fusion area in the original image, and the smoothed second template can be calculated according to the brightness statistics results. The template performs brightness adjustment, and the specific adjustment method will be described in detail in subsequent embodiments.
在103中，對第二模板中的各像素點的像素值進行歸一化，得到第三模板。 In 103, the pixel value of each pixel in the second template is normalized to obtain the third template.
確定第三模板實際上就是確定各像素點在後續進行圖像融合時，所採用的融合權重。在進行融合時，為了體現圖像各像素點的特徵，權重係數由第二模板中各像素點的像素值體現，在本步驟中採用對各像素點的像素值進行歸一化的方式。 Determining the third template is actually determining the fusion weight used by each pixel in subsequent image fusion. During the fusion, in order to reflect the characteristics of each pixel of the image, the weight coefficient is embodied by the pixel value of each pixel in the second template. In this step, the pixel value of each pixel is normalized.
在104中，對第三模板進行升採樣，得到第四模板，第四模板的像素點個數等於第一模板的像素點個數。 In 104, the third template is up-sampled to obtain the fourth template, and the number of pixels of the fourth template is equal to the number of pixels of the first template.
在圖像融合過程中，計算量主要體現在確定融合權重的過程，進行降採樣得到權重後，需要對權重進行升採樣，得到原始圖像中融合區域對應各像素點的權重。因此在本步驟中，對包含權重資訊的第三模板進行升採樣，得到第四模板。 In the image fusion process, the amount of calculation is mainly reflected in the process of determining the fusion weight. After the weight is obtained by downsampling, the weight needs to be up-sampled to obtain the weight of each pixel corresponding to the fusion area in the original image. Therefore, in this step, the third template containing weight information is up-sampled to obtain the fourth template.
同樣升採樣的方式也存在多種，可以採用諸如雙邊濾波、引導濾波、雙向插值等方式。在本發明實施例中，可以採用逆仿射變換的方式，即利用步驟102中仿射變換所採用的仿射參數，對第三模板進行逆仿射變換，得到像素點個數與第一模板相同的第四模板。 There are also many ways of upsampling, such as bilateral filtering, guided filtering, and bidirectional interpolation. In the embodiment of the present invention, inverse affine transformation can be used, that is, the third template is inversely affine transformed by using the affine parameters used in the affine transformation in
在105中，將第四模板中各像素點的像素值分別作為圖像中對應像素點的權重，對圖像中融合區域各像素點和融合素材進行加權融合。 In 105, the pixel value of each pixel in the fourth template is used as the weight of the corresponding pixel in the image, and each pixel in the fusion area in the image and the fusion material are weighted and fused.
本發明實施例中採用的是加權融合的方式，可以利用如下公式確定融合後得到的各像素點的像素值：利用Imagei_new=weight_maski * Imagei_old+(1-weight_maski)*Colori (1) In the embodiment of the present invention, a weighted fusion method is adopted. The following formula can be used to determine the pixel value of each pixel obtained after fusion: Image i_new =weight_mask i * Image i_old +(1-weight_mask i )*Color i (1 )
其中Imagei_new為圖像中融合區域融合後得到的第i個像素點的像素值，weight_maski為第四模板中第i個像素點的像素值，Imagei_old為圖像中融合區域第i個像素點的像素值，Colori為融合素材提供的第i個像素點的像素值。在本發明實施例中，融合素材可以是一副圖像、色彩集中的一種或多種等。 Where Image i_new is the pixel value of the i-th pixel obtained after the fusion area in the image is fused, weight_mask i is the pixel value of the i-th pixel in the fourth template, Image i_old is the i-th pixel in the fusion area in the image The pixel value of the point, Color i is the pixel value of the i-th pixel provided by the fusion material. In the embodiment of the present invention, the fusion material may be an image, one or more of a set of colors, and so on.
本發明所提供的方法可以應用於對靜態圖像的融合處理，由於大大降低了計算量，能夠保證即時性，因此也可以應用於對影片圖像的融合處理。另外，本發明提供的上述方法的執行主體可以為使用者終端中的應用，也可以為使用者終端應用中的外掛程式或軟體開發套件(Software Development Kit，SDK)等功能單元，或者，還可以位於伺服器端，本發明實施例對此不進行特別限定。上述應用可以是諸如影像處理類應用、美顏類應用等。下面結合圖2，以美顏類應用中對人臉進行粉底試妝為例，對上述方法進行詳細描述。 The method provided by the present invention can be applied to the fusion processing of static images. Since the calculation amount is greatly reduced and the immediacy can be ensured, it can also be applied to the fusion processing of film images. In addition, the execution subject of the above-mentioned method provided by the present invention can be an application in a user terminal, a plug-in program or a software development kit (SDK) and other functional units in a user terminal application, or it can be It is located on the server side, which is not particularly limited in the embodiment of the present invention. The above-mentioned applications may be, for example, image processing applications, beauty applications, and so on. The above method will be described in detail below with reference to FIG. 2, taking the foundation test on a face in a beauty application as an example.
圖2為本發明實施例提供的一種詳細方法流程圖，在 本實施例中實現對圖像中人臉進行粉底試妝，即將圖像中的人臉區域與粉底色進行融合。如圖2中所示，該流程可以具體包括以下步驟： FIG. 2 is a detailed flow chart of a method provided by an embodiment of the present invention. In this embodiment, a foundation test on the face in the image is implemented, that is, the face area in the image and the foundation color are merged. As shown in Figure 2, the process may specifically include the following steps:
在201中，對圖像中的人臉區域進行特徵點定位，得到人臉的輪廓點以及預設的器官的輪廓點。 In 201, feature point positioning is performed on the face area in the image to obtain contour points of the face and preset contour points of the organs.
在本發明實施例中，並不對特徵點定位的具體方式進行限制，可以採用諸如基於SDM(Supervised Descent Method，監督下降方法)模型的定位、id-exp模型定位等任意特徵點定位方式，最終可以得到上述特徵點的位置資訊。 In the embodiment of the present invention, the specific method of feature point positioning is not limited. Any feature point positioning method such as SDM (Supervised Descent Method) model-based positioning, id-exp model positioning, etc. can be used. Obtain the position information of the above-mentioned feature points.
假設對圖3a中的人臉區域進行特徵點定位，可以得到如圖3b中所示的特徵點，即人臉的輪廓點以及眼睛、眉毛、嘴巴的輪廓點。需要說明的是，圖3b中僅是示意性的，為了方便查看誇大了特徵點的效果，在實際的特徵點定位時，定位出的特徵點數量和細微性可能與圖3b中不一致。 Assuming that the feature points of the face region in Figure 3a are located, the feature points shown in Figure 3b can be obtained, namely the contour points of the face and the contour points of the eyes, eyebrows, and mouth. It should be noted that FIG. 3b is only schematic. In order to facilitate viewing the effect of exaggerating the feature points, in actual feature point positioning, the number and fineness of the located feature points may be inconsistent with those in FIG. 3b.
在202中，利用定位出的特徵點，去除圖像中除人臉區域之外的區域，得到第一模板。 In 202, using the located feature points, remove the area except the face area in the image to obtain the first template.
在本實施例中，實際上是去除人臉的輪廓點之外的區域，以及去除眼睛、眉毛、嘴巴的輪廓點所包圍的區域。得到的第一模板區域示意性的如圖3c中白色區域所示，圖3c中未體現出各像素點的實際像素值。 In this embodiment, the area outside the contour points of the human face is actually removed, as well as the area surrounded by the contour points of the eyes, eyebrows, and mouth. The obtained first template area is schematically shown as the white area in FIG. 3c, and the actual pixel value of each pixel is not shown in FIG. 3c.
在203中，採用仿射變換的方式，對第一模板進行降採樣，使得到的第二模板的像素點個數為第一模板的
所謂仿射變換是一種二維座標到二維座標之間的線性變換。仿射變換可以通過一系列的原子變換來實現，包括：平移(Translation)、縮放(Scale)、翻轉(Flip)、旋轉(Rotation)和剪切(Shear)。其中本發明實施例涉及的仿射變換是其中的縮放變換，設置合適的縮放參數(即仿射參數)，將第一模板縮小至原來的
其中，N值越大，所產生的計算量越小，N值越小，對圖片處理的品質越高，具體可以根據實際的需求進行衡量和選取。 Among them, the larger the value of N, the smaller the amount of calculation generated, and the smaller the value of N, the higher the quality of image processing, which can be measured and selected according to actual needs.
在204中，將第二模板中的輪廓點分別向外和向內擴展M個像素點，得到待平滑區域。 In 204, the contour points in the second template are respectively expanded outward and inward by M pixel points to obtain the area to be smoothed.
本步驟實際上是為圖像的邊緣平滑做準備，確定待平滑區域。在本步驟中，可以將第二模板中人臉的輪廓點、眼睛的輪廓點、眉毛的輪廓點以及嘴巴的輪廓點分別向內和/或向外擴展M個像素點，M為預設的正整數，例如取3，分別得到帶狀的待平滑區域。所謂向內和向外擴展可以是沿著輪廓點連線的法線方向進行的兩個方向的擴展。 This step actually prepares for the smoothing of the edges of the image and determines the area to be smoothed. In this step, the contour points of the face, the contour points of the eyes, the contour points of the eyebrows, and the contour points of the mouth in the second template can be expanded inward and/or outward respectively by M pixels, where M is a preset A positive integer, such as 3, respectively obtains the strip-shaped area to be smoothed. The so-called inward and outward expansion can be the expansion in two directions along the normal line of the line connecting the contour points.
示意性的如圖3d所示，圖3d中僅示出了人臉輪廓產生的待平滑區域，眼睛、眉毛、嘴巴輪廓產生的待平滑區域類似。 As shown schematically in Fig. 3d, Fig. 3d only shows the area to be smoothed generated by the contour of the face, and the contours of the eye, eyebrows, and mouth have similar areas to be smoothed.
在205中，採用三角剖分方式，將預定義的人臉平滑模板仿射到待平滑區域，得到平滑後的第二模板。 In 205, a triangulation method is used to affine a predefined face smoothing template to the area to be smoothed to obtain a smoothed second template.
對人臉邊緣進行平滑處理，目的是使圖像中人臉亮度平緩漸變，減小突變梯度，使得人臉邊緣更加柔和自然，從而改善圖像品質。 The purpose of smoothing the edges of the face is to smoothly change the brightness of the face in the image, reduce the sudden change gradient, and make the edges of the face softer and more natural, thereby improving the image quality.
為了加快平滑處理的速度，在本實施例中採用平滑模板的方式。由於人臉的大致形狀是基本相同的，因此可以預先利用人臉的形狀形成一個邊緣已經平滑處理的模板，在本模板中，經平滑處理的邊緣區域可以適當大一點。在對第二模板進行平滑處理時，可以將預先定義好的平滑模板仿射到待平滑區域，從而完成對待平滑區域的平滑。這種方式，可以快速實現邊緣平滑，避免了即時進行模糊平滑所帶來的計算量和時間開銷。 In order to speed up the smoothing process, a smoothing template method is adopted in this embodiment. Since the general shape of the human face is basically the same, the shape of the human face can be used in advance to form a template with smoothed edges. In this template, the smoothed edge area can be appropriately larger. When the second template is smoothed, the predefined smoothing template can be affine to the area to be smoothed, so as to complete the smoothing of the area to be smoothed. In this way, edge smoothing can be quickly achieved, and the amount of calculation and time overhead caused by instant blur smoothing is avoided.
在本步驟中採用的三角剖分法，是在平滑模板上的平滑區域和第二模板上的待平滑區域採用相同的方式進行三角剖分，得到相同的三角區域個數。以待平滑區域為例，將內外邊緣均分成m個點，然後待平滑區域被分成2m個三角，m為預設的正整數，如圖3e中所示。平滑模板上的平滑區域採用相同的剖分方式，將平滑模板中各三角區域分別仿射到第二模板中對應位置的三角區域。 In the triangulation method used in this step, the smooth area on the smoothing template and the area to be smoothed on the second template are triangulated in the same way to obtain the same number of triangular areas. Taking the area to be smoothed as an example, the inner and outer edges are equally divided into m points, and then the area to be smoothed is divided into 2m triangles, where m is a preset positive integer, as shown in Figure 3e. The smooth area on the smooth template adopts the same division method, and each triangle area in the smooth template is respectively affine to the triangle area at the corresponding position in the second template.
其中，本步驟中涉及的仿射指的是對應位置像素點的像素值的仿射，即將平滑模板上像素點的像素值仿射到第二模板上待平滑區域中對應位置的像素點，例如將平滑模板上平滑區域的像素點A仿射到第二模板上待平滑區域的 像素點a，則像素點a取像素點A的像素值，若像素點A為剖分得到的某三角形的頂角，那麼像素點a為第二模板上對應位置的三角形的頂角。 Among them, the affine involved in this step refers to the affine of the pixel value of the pixel at the corresponding position, that is, the pixel value of the pixel on the smoothing template is affine to the pixel at the corresponding position in the area to be smoothed on the second template, for example Affine the pixel point A of the smooth area on the smoothing template to the pixel point a of the area to be smoothed on the second template, then the pixel point a takes the pixel value of the pixel point A, if the pixel point A is the vertex of a certain triangle obtained by splitting Angle, then the pixel point a is the apex angle of the triangle at the corresponding position on the second template.
在206中，對圖像中的人臉區域進行亮度統計，依據亮度統計結果，對平滑處理後的第二模板進行亮度調整。 In 206, brightness statistics are performed on the face area in the image, and brightness adjustment is performed on the smoothed second template according to the brightness statistics result.
本步驟的目的是，使得平滑後的第二模板的亮度能夠儘量的與原始圖像中人臉實際亮度匹配。例如，可以統計原始圖像中人臉區域亮度的均值以及平滑後的第二模板的亮度均值，確定兩者的差值；然後將各平滑後的第二模板中各像素點的亮度值均加上該差值。當然，還可以採用其他亮度調整的具體方式，在此不再一一列舉。 The purpose of this step is to make the brightness of the smoothed second template match the actual brightness of the face in the original image as much as possible. For example, you can count the average value of the brightness of the face area in the original image and the average value of the brightness of the smoothed second template to determine the difference; then add the brightness value of each pixel in each smoothed second template Above the difference. Of course, other specific methods of brightness adjustment can also be used, which will not be listed here.
在207中，對亮度調整後的第二模板中的各像素值進行歸一化，得到第三模板。 In 207, the pixel values in the second template after the brightness adjustment are normalized to obtain the third template.
為了在圖像融合時，體現圖像各像素點的特徵，權重係數由第二模板中各像素點的像素值體現，在本步驟中採用對各像素點的像素值進行歸一化處理後，得到的第三模板中各像素點的像素值就可以作為該像素點在後續融合時採用的權重。 In order to reflect the characteristics of each pixel in the image during image fusion, the weight coefficient is reflected by the pixel value of each pixel in the second template. In this step, the pixel value of each pixel is normalized. The pixel value of each pixel in the third template obtained can be used as the weight of the pixel in the subsequent fusion.
權重的計算是圖像融合中帶來較大計算量的其中一個重要處理，在本發明實施例中，首先通過降採樣的方式，基於像素點較少的模板進行權重計算後，在升採樣回去，得到所有像素點的權重，相比較直接基於原始圖像大小進行權重計算，大大降低了計算消耗和時間成本。 The calculation of weights is one of the important processes that bring about a large amount of calculation in image fusion. In the embodiment of the present invention, firstly, by down-sampling, the weight is calculated based on a template with fewer pixels, and then the up-sampling is performed. , Get the weights of all pixels, compared to directly calculate the weights based on the original image size, which greatly reduces the computational consumption and time cost.
在208中，利用203中採用的仿射參數，將第三模板 進行逆仿射變換，得到第四模板。 In 208, the third template is subjected to inverse affine transformation using the affine parameters used in 203 to obtain the fourth template.
在203中採用的是縮放變換，在本步驟中同樣採用縮放變換，但本步驟中需要依據在203中設置的仿射參數，設置對應的仿射參數，從而實現逆變換。將逆仿射變換後得到的第四模板的像素點個數升至與第一模板相同。也就是說，本步驟實際上是要升採樣回第一模板的像素點個數，提高解析度。在逆仿射變換過程中，由於像素點個數增多，對於增加的像素點，其像素值可以採用插值的方式得到。 The scaling transformation is adopted in 203, and the scaling transformation is also adopted in this step. However, in this step, the corresponding affine parameters need to be set according to the affine parameters set in 203, so as to realize the inverse transformation. The number of pixels of the fourth template obtained after the inverse affine transformation is increased to the same as that of the first template. In other words, this step is actually to upsample the number of pixels back to the first template to improve the resolution. In the process of inverse affine transformation, as the number of pixels increases, the pixel value of the increased pixel can be obtained by interpolation.
在209中，將第四模板中各像素點的像素值分別作為圖像中對應像素點的權重，對圖像中人臉區域各像素點和粉底色進行加權融合。 In 209, the pixel value of each pixel in the fourth template is used as the weight of the corresponding pixel in the image, and each pixel in the face area in the image and the foundation color are weighted and merged.
本步驟在進行加權融合時，可以採用上述的公式(1)，其中Imagei_new為圖像中人臉區域融合後得到的第i個像素點的像素值，Imagei_old為圖像中人臉區域(不包含眉毛、眼睛和嘴巴的區域)第i個像素點的像素值，Colori為粉底色的像素值，在本實施例中，各像素粉底色的像素值可以取相同的值。 When performing weighted fusion in this step, the above formula (1) can be used, where Image i_new is the pixel value of the i-th pixel obtained after the fusion of the face area in the image, and Image i_old is the face area in the image ( The area that does not include eyebrows, eyes, and mouth) is the pixel value of the i-th pixel, and Color i is the pixel value of the foundation color. In this embodiment, the pixel value of the foundation color of each pixel can take the same value.
本發明實施例中涉及的像素點的像素值涉及到R、G、B三個通道的值，在進行上述仿射以及融合處理時，需要分別對各像素點R、G、B三個通道的值分別進行處理，這是較為公知的內容，在此僅作簡單說明。 The pixel value of the pixel involved in the embodiment of the present invention involves the values of the three channels of R, G, and B. When performing the above-mentioned affine and fusion processing, it is necessary to separately analyze the values of the three channels of R, G, and B of each pixel. The values are processed separately, which is a relatively well-known content and is only briefly described here.
以上是對本發明所提供方法進行的詳細描述，下面結合圖4對本發明提供的裝置進行詳細描述。如圖4所示， 該裝置可以包括：模板確定單元01、降採樣單元02、歸一化單元03、升採樣單元04以及加權融合單元05，還可以進一步包括邊緣平滑單元06和亮度調整單元07。各組成單元的主要功能如下： The above is a detailed description of the method provided by the present invention, and the device provided by the present invention will be described in detail below with reference to FIG. 4. As shown in FIG. 4, the device may include: a
模板確定單元01負責確定圖像中的融合區域，得到第一模板。具體地，模板確定單元01可以首先對圖像中的融合目標進行特徵點定位，特徵點包括輪廓點；然後利用定位出的特徵點，去除圖像中除融合目標之外的區域，得到第一模板。 The
降採樣單元02負責對第一模板進行降採樣，得到第二模板。圖像降採樣的方式有很多，例如最近鄰降採樣法、B樣條降採樣法等。在本發明實施例中降採樣單元02採用仿射變換的方式，對第一模板進行降採樣，使得到的第二模板的像素點個數為第一模板的
歸一化單元03負責對第二模板中的各像素點的像素值進行歸一化，得到第三模板。 The
升採樣單元04負責對第三模板進行升採樣，得到第四模板，第四模板的像素點個數等於第一模板的像素點個數。同樣升採樣的方式也存在多種，可以採用諸如雙邊濾波、引導濾波、雙向插值等方式。在本發明實施例中，升採樣單元04可以採用逆仿射變換的方式，對第三模板進行升採樣，使得到的第四模板的像素點個數為第三模板的N倍。 The up-
加權融合單元05負責將第四模板中各像素點的像素值分別作為圖像中對應像素點的權重，對圖像中融合區域各像素點和融合素材進行加權融合。 The
具體可以採用Imagei_new=weight_maski * Imagei_old+(1-weight_maski)*Colori，確定融合後得到的各像素點的像素值；其中Imagei_new為圖像中融合區域融合後得到的第i個像素點的像素值，weight_maski為第四模板中第i個像素點的像素值，Imagei_old為圖像中融合區域第i個像素點的像素值，Colori為融合素材提供的第i個像素點的像素值。 Specifically, Image i_new =weight_mask i * Image i_old +(1-weight_mask i )*Color i can be used to determine the pixel value of each pixel obtained after fusion; where Image i_new is the i-th one obtained after fusion of the fused area in the image The pixel value of the pixel, weight_mask i is the pixel value of the i-th pixel in the fourth template, Image i_old is the pixel value of the i-th pixel in the fusion area of the image, and Color i is the i-th pixel provided by the fusion material The pixel value of the point.
為了使得融合區域邊緣平緩漸變，減小突變梯度，更加融合和自然，邊緣平滑單元06負責對第二模板的邊緣進行平滑處理，將平滑處理後的第二模板輸出給歸一化單元03。 In order to make the edges of the fusion region gradually gradually change, reduce the abrupt gradient, and be more integrated and natural, the
具體地，邊緣平滑單元06可以將第二模板中融合區域的輪廓點分別向外和/或向內擴展M個像素點，M為預設的正整數，將擴展的像素點所包圍的區域作為待平滑區域；將預定義的平滑模板仿射到待平滑區域，得到平滑後的第二模板。在進行仿射時，可以將平滑模板中像素點的像素值，仿射為待平滑區域中對應位置像素點的像素值。 Specifically, the
其中可以預先利用融合區域的形狀形成一個邊緣已經平滑處理的模板，該模板就是平滑模板。這種將預先定義好的平滑模板仿射到待平滑區域，從而完成對待平滑區域 的平滑的方式，可以快速實現邊緣平滑，避免了即時進行模糊平滑所帶來的計算量和時間開銷。 The shape of the fusion area can be used in advance to form a template whose edges have been smoothed, and the template is a smooth template. This way of affine a pre-defined smoothing template to the area to be smoothed, thereby completing the smoothing of the area to be smoothed, can quickly realize edge smoothing, and avoid the amount of calculation and time overhead caused by instant blur smoothing.
更進一步地，邊緣平滑單元06在將預定義的平滑模板仿射到待平滑區域時，可以採用三角剖分的方式，即在平滑模板上的平滑區域和第二模板上的待平滑區域分別採用相同的方式進行三角剖分，得到相同個數的三角區域；然後將平滑模板中各三角區域分別仿射到第二模板中對應位置的三角區域。 Furthermore, when the
亮度調整單元07負責獲取邊緣平滑單元06輸出的第二模板，對圖像中的融合區域進行亮度統計，依據亮度統計結果，對獲取的第二模板進行亮度調整，將亮度調整後的第二模板輸出給歸一化單元03。亮度調整單元07可以使得平滑後的第二模板的亮度能夠儘量的與原始圖像中人臉實際亮度匹配。在進行亮度調整時，可以確定圖像中的融合區域的亮度均值和平滑處理後的第二模板的亮度均值之間的差值；然後將平滑處理後的第二模板中各像素點的亮度值分別加上差值。 The
該裝置可以應用於影像處理類APP，也可以應用於美顏類APP等。該裝置可以體現為一個應用的形式，可以是運行於設備本地的應用程式(nativeAPP)，也可以是設備瀏覽器上的一個網頁程式(webApp)。除此之外，也可以體現為應用中的外掛程式或SDK的形式。 The device can be applied to image processing apps, and can also be applied to beauty apps, etc. The device can be embodied in the form of an application, which can be an application (nativeAPP) running locally on the device, or a webpage application (webApp) on the device browser. In addition, it can also be embodied in the form of a plug-in or SDK in the application.
除了應用於上述美顏類APP中諸如粉底試妝的場景之外，本發明還可以應用於其他圖像融合的場景，例如將 一副圖像中紅色的蘋果與另一幅圖像中黃色的蘋果進行融合。 In addition to being applied to the above-mentioned beauty APPs such as foundation test makeup scenes, the present invention can also be applied to other image fusion scenes, such as combining a red apple in one image with a yellow one in another image. Apple performs fusion.
本發明實施例提供的上述方法和裝置可以以設置並運行於設備中的電腦程式體現。該設備可以包括一個或多個處理器，還包括記憶體和一個或多個程式，如圖5中所示。其中該一個或多個程式儲存於記憶體中，被上述一個或多個處理器執行以實現本發明上述實施例中所示的方法流程和/或裝置操作。例如，被上述一個或多個處理器執行的方法流程，可以包括：確定圖像中的融合區域，得到第一模板；對第一模板進行降採樣，得到第二模板；對第二模板中的各像素點的像素值進行歸一化，得到第三模板；對第三模板進行升採樣，得到第四模板，第四模板的像素點個數等於第一模板的像素點個數；將第四模板中各像素點的像素值分別作為圖像中對應像素點的權重，對該圖像中融合區域各像素點和融合素材進行加權融合。 The above-mentioned method and device provided by the embodiment of the present invention may be embodied by a computer program set up and running in the device. The device may include one or more processors, as well as memory and one or more programs, as shown in FIG. 5. The one or more programs are stored in the memory and executed by the one or more processors to implement the method flow and/or the device operation shown in the above embodiments of the present invention. For example, the process of the method executed by the one or more processors may include: determining the fusion area in the image to obtain the first template; down-sampling the first template to obtain the second template; The pixel value of each pixel is normalized to obtain the third template; the third template is up-sampled to obtain the fourth template, and the number of pixels in the fourth template is equal to the number of pixels in the first template; The pixel value of each pixel in the template is used as the weight of the corresponding pixel in the image, and each pixel in the fusion area in the image and the fusion material are weighted and fused.
由以上描述可以看出，本發明提供的方法和裝置可以具備以下優點： It can be seen from the above description that the method and device provided by the present invention can have the following advantages:
1)本發明採用對圖像進行降採樣的方式，對降採樣後的融合區域進行權重計算，然後再升採樣回原始圖像大小，得到原始圖像中融合區域各像素點在融合時對應的權重，大大降低了因權重計算所帶來的計算量，降低了時間 成本和資源消耗。 1) The present invention adopts the method of down-sampling the image to calculate the weight of the down-sampled fusion area, and then up-sample it back to the original image size to obtain the corresponding pixel points of the fusion area in the original image during fusion The weight greatly reduces the amount of calculation caused by the weight calculation, and reduces the time cost and resource consumption.
2)在權重計算過程中，對降採樣後的融合區域進行邊緣平滑和/或亮度調整，更進一步降低了計算量，降低了時間成本和資源消耗。 2) In the weight calculation process, edge smoothing and/or brightness adjustment are performed on the down-sampled fusion area, which further reduces the amount of calculation and reduces time cost and resource consumption.
3)在進行邊緣平滑時，採用預定義的平滑模板，還可以進一步結合三角剖分的方式，可以快速實現邊緣平滑，避免了即時進行模糊平滑所帶來的計算量和時間開銷。 3) When performing edge smoothing, a predefined smoothing template is used, and the triangulation method can be further combined to quickly achieve edge smoothing, avoiding the amount of calculation and time overhead caused by instant blur smoothing.
4)可以在高解析度的畫質下，仍保持即時性。不僅可以應用於靜態圖像，也可以應用於影片圖像。 4) It can still maintain real-time performance under high-resolution image quality. It can be applied not only to static images, but also to movie images.
5)當應用於粉底試妝時，由於僅對人臉區域的邊緣進行平滑處理，能夠有效保留人臉自身的紋理資訊。無需使用者手工操作，能夠根據人臉膚色的亮度自動調整融合權重，得到更加真實的試妝體驗。 5) When applied to foundation makeup, since only the edges of the face area are smoothed, the texture information of the face itself can be effectively preserved. There is no need for manual operation by the user, and the fusion weight can be automatically adjusted according to the brightness of the skin color of the face, and a more realistic makeup trial experience can be obtained.
在本發明所提供的幾個實施例中，應該理解到，所揭露的裝置和方法，可以通過其它的方式實現。例如，以上所描述的裝置實施例僅是示意性的，例如，該單元的劃分，僅為一種邏輯功能劃分，實際實現時可以有另外的劃分方式。 In the several embodiments provided by the present invention, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiment described above is only illustrative. For example, the division of the unit is only a logical function division, and there may be other division methods in actual implementation.
該作為分離部件說明的單元可以是或者也可以不是物理上分開的，作為單元顯示的部件可以是或者也可以不是物理單元，即可以位於一個地方，或者也可以分佈到多個網路單元上。可以根據實際的需要選擇其中的部分或者全部單元來實現本實施例方案的目的。 The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
另外，在本發明各個實施例中的各功能單元可以集成在一個處理單元中，也可以是各個單元單獨物理存在，也可以兩個或兩個以上單元集成在一個單元中。上述集成的單元既可以採用硬體的形式實現，也可以採用硬體加軟體功能單元的形式實現。 In addition, the functional units in the various embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be realized either in the form of hardware, or in the form of hardware plus software functional units.
上述以軟體功能單元的形式實現的集成的單元，可以儲存在一個電腦可讀取儲存媒介中。上述軟體功能單元儲存在一個儲存媒介中，包括若干指令用以使得一台電腦設備(可以是個人電腦，伺服器，或者網路設備等)或處理器(processor)執行本發明各個實施例該方法的部分步驟。而前述的儲存媒介包括：隨身碟、行動硬碟、唯讀記憶體(Read-Only Memory，ROM)、隨機存取記憶體(Random Access Memory，RAM)、磁碟或者光碟等各種可以儲存程式碼的媒介。 The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer readable storage medium. The above-mentioned software functional unit is stored in a storage medium, and includes a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute the method of the various embodiments of the present invention Part of the steps. The aforementioned storage media include: flash drives, mobile hard drives, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disks or optical disks, etc., which can store code Medium.
以上該僅為本發明的較佳實施例而已，並不用以限制本發明，凡在本發明的精神和原則之內，所做的任何修改、等同替換、改進等，均應包含在本發明保護的範圍之內。 The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection of the present invention. Within the range.
Claims (23)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW106102466A TWI723123B (en) | 2017-01-23 | 2017-01-23 | Image fusion method, device and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW106102466A TWI723123B (en) | 2017-01-23 | 2017-01-23 | Image fusion method, device and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| TW201828247A TW201828247A (en) | 2018-08-01 |
| TWI723123B true TWI723123B (en) | 2021-04-01 |
Family
ID=63960446
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| TW106102466A TWI723123B (en) | 2017-01-23 | 2017-01-23 | Image fusion method, device and equipment |
Country Status (1)
| Country | Link |
|---|---|
| TW (1) | TWI723123B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW201032592A (en) * | 2009-02-27 | 2010-09-01 | Arcsoft Hangzhou Co Ltd | Method for upscaling images and videos and associated image processing device |
| CN101853504A (en) * | 2010-05-07 | 2010-10-06 | 厦门大学 | Image quality evaluating method based on visual character and structural similarity (SSIM) |
| TW201347542A (en) * | 2012-01-03 | 2013-11-16 | Dolby Lab Licensing Corp | Specifying visual dynamic range coding operations and parameters |
| CN103839244A (en) * | 2014-02-26 | 2014-06-04 | 南京第五十五所技术开发有限公司 | Real-time image fusion method and device |
| US20150030242A1 (en) * | 2013-07-26 | 2015-01-29 | Rui Shen | Method and system for fusing multiple images |
-
2017
- 2017-01-23 TW TW106102466A patent/TWI723123B/en not_active IP Right Cessation
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW201032592A (en) * | 2009-02-27 | 2010-09-01 | Arcsoft Hangzhou Co Ltd | Method for upscaling images and videos and associated image processing device |
| CN101853504A (en) * | 2010-05-07 | 2010-10-06 | 厦门大学 | Image quality evaluating method based on visual character and structural similarity (SSIM) |
| TW201347542A (en) * | 2012-01-03 | 2013-11-16 | Dolby Lab Licensing Corp | Specifying visual dynamic range coding operations and parameters |
| US20150030242A1 (en) * | 2013-07-26 | 2015-01-29 | Rui Shen | Method and system for fusing multiple images |
| CN103839244A (en) * | 2014-02-26 | 2014-06-04 | 南京第五十五所技术开发有限公司 | Real-time image fusion method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| TW201828247A (en) | 2018-08-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106846241B (en) | Image fusion method, device and equipment | |
| US9547908B1 (en) | Feature mask determination for images | |
| US9864901B2 (en) | Feature detection and masking in images based on color distributions | |
| WO2016161553A1 (en) | Avatar generation and animations | |
| CN107610202B (en) | Face image replacement method, device and storage medium | |
| JP4461789B2 (en) | Image processing device | |
| CN107507217B (en) | Method and device for making certificate photo and storage medium | |
| CN107507216B (en) | Method and device for replacing local area in image and storage medium | |
| CN107564080B (en) | Face image replacement system | |
| WO2022078041A1 (en) | Occlusion detection model training method and facial image beautification method | |
| US20160314619A1 (en) | 3-Dimensional Portrait Reconstruction From a Single Photo | |
| EP3859681A1 (en) | Method for generating facial animation from single image | |
| WO2013189101A1 (en) | Hair modeling and portrait editing method based on single image | |
| CN110503704B (en) | Method and device for constructing three-dimensional graph and electronic equipment | |
| Chen et al. | Improved seam carving combining with 3D saliency for image retargeting | |
| WO2023066120A1 (en) | Image processing method and apparatus, electronic device, and storage medium | |
| WO2017173578A1 (en) | Image enhancement method and device | |
| US20170011552A1 (en) | 3D Model Enhancement | |
| Kim et al. | Layered non-photorealistic rendering with anisotropic depth-of-field filtering | |
| CN114429518A (en) | Face model reconstruction method, device, equipment and storage medium | |
| CN114581979A (en) | Image processing method and device | |
| WO2023207741A1 (en) | Modeling method for metaverse scene material and related device | |
| TWI723123B (en) | Image fusion method, device and equipment | |
| CN111275610A (en) | Method and system for processing face aging image | |
| JP6244885B2 (en) | Image processing apparatus, image processing method, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| MM4A | Annulment or lapse of patent due to non-payment of fees |