CN107977605A - Ocular Boundary characteristic extraction method and device based on deep learning - Google Patents

Ocular Boundary characteristic extraction method and device based on deep learning Download PDF

Info

Publication number
CN107977605A
CN107977605A CN201711091696.3A CN201711091696A CN107977605A CN 107977605 A CN107977605 A CN 107977605A CN 201711091696 A CN201711091696 A CN 201711091696A CN 107977605 A CN107977605 A CN 107977605A
Authority
CN
China
Prior art keywords
mrow
network
msub
boundary
deep learning
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201711091696.3A
Other languages
Chinese (zh)
Other versions
CN107977605B (en
Inventor
徐枫
温佺
雍俊海
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tsinghua University
Original Assignee
Tsinghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tsinghua University filed Critical Tsinghua University
Priority to CN201711091696.3A priority Critical patent/CN107977605B/en
Publication of CN107977605A publication Critical patent/CN107977605A/en
Application granted granted Critical
Publication of CN107977605B publication Critical patent/CN107977605B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/193—Preprocessing; Feature extraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Ophthalmology & Optometry (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a kind of ocular Boundary characteristic extraction method and device based on deep learning, wherein, method includes：Training dataset is obtained, by manually demarcating four borders containing semantic information on eye image, and is stored in the boundary graph of four-way and obtains training dataset；A set of overall nested deep neural network for Boundary Extraction is improved, the loss function and network structure of deep neural network are improved, so that the semantic information comprising four edges circle can simultaneously export the boundary graph of corresponding four-way；By the method for deep learning, Training is carried out using the embedding deep neural network of the improved entirety of training data set pair, obtains the optimized parameter of network, to use four edges circle in neural network forecast eye image.This method can extract four kinds of borders in eye image by the method for deep learning, so as to provide good basis for the real-time three-dimensional reconstruction of eyelid, and can provide the ocular boundary characteristic containing semantic information.

Description

Ocular Boundary characteristic extraction method and device based on deep learning
Technical field
It is more particularly to a kind of based on deep learning the present invention relates to computer vision, computer graphics techniques field Ocular Boundary characteristic extraction method and device.
Background technology
In computer graphics and computer vision field, it is always a weight to generate lively real mask animation Will and it is challenging the problem of.Face is a most important part on human body, and eyes are more known as the window of soul, eye The slight movement of portion's area skin expresses the emotion that the mankind enrich.Therefore, the real-time three-dimensional of eyelid is rebuild and can greatly be carried The sense of reality of mask animation is risen, there is high scientific research and application value.
However, the area of ocular is very small, and contained in the movement of eyelid skin block and fold, for Ocular shape and the Direct Modeling of movement are typically extremely difficult.Although for the people of different sex, races, age The shape of group's ocular and movement are had nothing in common with each other, but there are feature same as below for the eyes of the mankind：Upper eyelid and lower eyelid, Double-edged eyelid and sleeping silkworm (groups of people's presence), these features show as specific border in eye image, can be good at reflecting The shape of eyelid and movement.
The content of the invention
It is contemplated that solve at least some of the technical problems in related technologies.
For this reason, an object of the present invention is to provide a kind of ocular Boundary characteristic extraction side based on deep learning Method, this method can rebuild for the real-time three-dimensional of eyelid and provide good basis, and can provide the eye containing semantic information Portion zone boundary feature.
It is another object of the present invention to propose a kind of ocular Boundary characteristic extraction device based on deep learning.
To reach above-mentioned purpose, one aspect of the present invention embodiment proposes a kind of ocular border based on deep learning Feature extracting method, comprises the following steps：Training dataset is obtained, wherein, contained by four manually demarcated on eye image The border of semantic information, and be stored in the boundary graph of four-way and obtain the training dataset；To a set of overall nested use It is improved in the deep neural network of Boundary Extraction, wherein, improve the loss function and network knot of the deep neural network Structure, so that the semantic information comprising four edges circle can simultaneously export the boundary graph of the corresponding four-way；Pass through deep learning Method, carries out Training using the embedding deep neural network of the improved entirety of training data set pair, obtains network Optimized parameter, to use four edges circle in neural network forecast eye image.
The ocular Boundary characteristic extraction method based on deep learning of the embodiment of the present invention, can pass through deep learning Method extraction eye image in four kinds of borders so that for eyelid real-time three-dimensional rebuild provide good basis, and Ocular boundary characteristic containing semantic information can be provided.
In addition, the ocular Boundary characteristic extraction method according to the above embodiment of the present invention based on deep learning may be used also With with following additional technical characteristic：
Further, in one embodiment of the invention, it is described to contain language by manually demarcating four on eye image The border of adopted information, further comprises：Control point is demarcated by hand in a line circle in office；Determined according to the control point corresponding Spline curve；The spline curve of predetermined width is drawn in the boundary graph passage belonging to any bar border.
Further, in one embodiment of the invention, it is described to a set of overall nested depth for Boundary Extraction Degree neutral net is improved, and is further comprised：Definition includes the loss function of the four edges circle semantic information, will be described whole The convolution kernel depth for the convolutional layer being connected in the deep neural network of body nesting with side output and final output is changed to 4 by 1.
Further, in one embodiment of the invention, it is described embedding using the improved entirety of training data set pair Deep neural network carry out Training, further comprise：Using the network of pre-training as initial value, the instruction is used Practice the original image in data set as network inputs, with reference to deep learning platform using the boundary graph of the four-way of calibration to institute The training that deep neural network carries out having supervision is stated, obtains the optimized parameter of the network.
Further, in one embodiment of the invention, the loss function is：
Wherein, I and G is respectively input picture and four-way boundary graph, ψfWithThe respectively final output and net of network The side output of network, αfWithThe weight of respectively final loss and side loss, L (ψf(I, θ), G) damaged for the final of network Lose, ψfThe final output of network in the case that (I, θ) is I and θ, θ is network parameter, and L is cross entropy loss function,Lost for side,The side output of network in the case of for I and θ.
To reach above-mentioned purpose, another aspect of the present invention embodiment proposes a kind of ocular side based on deep learning Boundary's feature deriving means, including：Acquisition module, for obtaining training dataset, wherein, by manually demarcating on eye image Four borders containing semantic information, and be stored in the boundary graph of four-way and obtain the training dataset；Module is improved, is used for A set of overall nested deep neural network for Boundary Extraction is improved, wherein, improve the deep neural network Loss function and network structure so that the semantic information comprising four edges circle can simultaneously export the border of the corresponding four-way Figure；Training module, for the method by deep learning, utilizes the embedding depth nerve of the improved entirety of training data set pair Network carries out Training, obtains the optimized parameter of network, to use four edges circle in neural network forecast eye image.
The ocular Boundary characteristic extraction device based on deep learning of the embodiment of the present invention, can pass through deep learning Method extraction eye image in four kinds of borders so that for eyelid real-time three-dimensional rebuild provide good basis, and Ocular boundary characteristic containing semantic information can be provided.
In addition, the ocular Boundary characteristic extraction device according to the above embodiment of the present invention based on deep learning may be used also With with following additional technical characteristic：
Further, in one embodiment of the invention, the acquisition module, further comprises：Unit is demarcated, is used for Control point is demarcated by hand in a line circle in office；Acquiring unit, for determining corresponding spline curve according to the control point；Paint Unit processed, for drawing the spline curve of predetermined width in the boundary graph passage belonging to any bar border.
Further, in one embodiment of the invention, the improvement module, further comprises：Definition is comprising described The loss function of four edges circle semantic information, will in the overall nested deep neural network with side output and final output The convolution kernel depth of connected convolutional layer is changed to 4 by 1.
Further, in one embodiment of the invention, the training module, further comprises：Use pre-training Network is as initial value, using the original image that the training data is concentrated as network inputs, with reference to deep learning platform profit The training that has supervision is carried out to the deep neural network with the boundary graph of the four-way of calibration, obtains the optimal ginseng of the network Number.
Further, in one embodiment of the invention, the loss function is：
Wherein, I and G is respectively input picture and four-way boundary graph, ψfWithThe respectively final output and net of network The side output of network, αfWithThe weight of respectively final loss and side loss, L (ψf(I, θ), G) damaged for the final of network Lose, ψfThe final output of network in the case that (I, θ) is I and θ, θ is network parameter, and L is cross entropy loss function,Lost for side,The side output of network in the case of for I and θ.
The additional aspect of the present invention and advantage will be set forth in part in the description, and will partly become from the following description Obtain substantially, or recognized by the practice of the present invention.
Brief description of the drawings
Of the invention above-mentioned and/or additional aspect and advantage will become from the following description of the accompanying drawings of embodiments Substantially and it is readily appreciated that, wherein：
Fig. 1 is the flow according to the ocular Boundary characteristic extraction method based on deep learning of the embodiment of the present invention Figure；
Fig. 2 is the stream according to the ocular Boundary characteristic extraction method based on deep learning of one embodiment of the invention Cheng Tu；
Fig. 3 is to predict four-way according to the improved overall nested networks with optimized parameter of one embodiment of the invention The result schematic diagram of boundary graph；
Fig. 4 is to predict four-way according to the improved overall nested networks with optimized parameter of one embodiment of the invention The result schematic diagram of the part calibration of boundary graph；
Fig. 5 is to be shown according to the structure of the ocular Boundary characteristic extraction device based on deep learning of the embodiment of the present invention It is intended to.
Embodiment
The embodiment of the present invention is described below in detail, the example of the embodiment is shown in the drawings, wherein from beginning to end Same or similar label represents same or similar element or has the function of same or like element.Below with reference to attached The embodiment of figure description is exemplary, it is intended to for explaining the present invention, and is not considered as limiting the invention.
The ocular boundary characteristic based on deep learning proposed according to embodiments of the present invention is described with reference to the accompanying drawings Extracting method and device, describe the ocular based on deep learning proposed according to embodiments of the present invention with reference to the accompanying drawings first Boundary characteristic extraction method.
Fig. 1 is the flow chart of the ocular Boundary characteristic extraction method based on deep learning of the embodiment of the present invention.
As shown in Figure 1, the ocular Boundary characteristic extraction method based on deep learning of being somebody's turn to do comprises the following steps：
In step S101, training dataset is obtained, wherein, by manually demarcating four on eye image containing semantic letter The border of breath, and be stored in the boundary graph of four-way and obtain training dataset.
Further, in one embodiment of the invention, by manually demarcating four on eye image containing semantic letter The border of breath, further comprises：Control point is demarcated by hand in a line circle in office；Determine that corresponding batten is bent according to control point Line；The spline curve of predetermined width is drawn in boundary graph passage belonging to a line circle in office.
It is understood that with reference to shown in Fig. 1 and Fig. 2, the four-way side of training dataset, i.e. eye areas is manually demarcated Boundary's figure.First, the embodiment of the present invention demarcates control point by hand on certain border, then determines its corresponding sample according to control point Bar curve, finally draws the spline curve of one fixed width in the boundary graph passage belonging to the border, wherein, in boundary graph The probability that the higher expression pixel of pixel value belongs to border is bigger, and therefore, above-mentioned line transect is plotted as in an all pixels It is worth on the gray level image for 0 and draws the line transect that gray value is 255, represents the position on this border.Particularly, double-edged eyelid In partial eye area image and it is not present with the lower boundary of sleeping silkworm, then the side corresponding to this two borders in this case All image pixel values of boundary's figure passage are 0.
In step s 102, a set of overall nested deep neural network for Boundary Extraction is improved, wherein, The loss function and network structure of deep neural network are improved, so that the semantic information comprising four edges circle can simultaneously export accordingly The boundary graph of four-way.
Further, in one embodiment of the invention, to a set of overall nested depth god for Boundary Extraction It is improved, further comprises through network：Definition includes the loss function of four edges circle semantic information, by overall nested depth The convolution kernel depth for the convolutional layer being connected in neutral net with side output and final output is changed to 4 by 1.
Further, in one embodiment of the invention, loss function is：
Wherein, I and G is respectively input picture and four-way boundary graph, ψfWithThe respectively final output and net of network The side output of network, αfWithThe weight of respectively final loss and side loss, L (ψf(I, θ), G) damaged for the final of network Lose, ψfThe final output of network in the case that (I, θ) is I and θ, θ is network parameter, and L is cross entropy loss function,Lost for side,The side output of network in the case of for I and θ.
It is understood that the embodiment of the present invention can be to a set of overall nested depth nerve net for Boundary Extraction Network is improved.Wherein, which is generally used for all borders in extraction image, when for extracting ocular Border when, on the one hand its extraction border in include noise information, the border of such as iris and sclera, eyebrow；On the other hand Four edges circle that it is extracted are in same boundary graph, the semantic information not comprising four edges circle.Therefore, the embodiment of the present invention pair The entirety nested networks are optimized as follows, and the semantic information that can include four edges circle can simultaneously export corresponding four-way Boundary graph.For loss function, whether the embodiment of the present invention belongs to pixel in four edges circle border judgement by considering Mistake and the punishment produced, the loss function being defined as follows：
Wherein
Wherein, I and G is respectively input picture and four-way boundary graph, ψfWithThe respectively final output of network and side Side exports, αfWithThe weight of respectively final loss and side loss.Loss function L calculate Pixel-level network output and Cross entropy between legitimate reading,To belong to the set of the pixel on border in i-th border in legitimate reading, β is to belong to border Pixel account for the ratios of all pixels,Belong to the probability of border i, Pr (g for pixel jj=0 | I；θ) it is picture Plain j belongs to the probability of non-border pixel.
In step s 103, by the method for deep learning, the embedding depth god of the improved entirety of training data set pair is utilized Training is carried out through network, obtains the optimized parameter of network, to use four edges circle in neural network forecast eye image.
Further, in one embodiment of the invention, the embedding depth god of the improved entirety of training data set pair is utilized Training is carried out through network, is further comprised：Using the network of pre-training as initial value, concentrated using training data Original image as network inputs, with reference to deep learning platform using the four-way of calibration boundary graph to deep neural network into Row has the training of supervision, obtains the optimized parameter of network.
It is understood that the embodiment of the present invention can be trained to obtain optimized parameter to above-mentioned network, for predicting Four edges circle of ocular.The embodiment of the present invention is put down using the training dataset manually demarcated with reference to the deep learning increased income Platform, carries out arameter optimization on the network model of pre-training, obtains optimal network parameter, has optimal ginseng so as to use Several network models predicts the four-way boundary graph containing semantic information of any eye image.
In one particular embodiment of the present invention, as shown in figure 3, the embodiment of the present invention includes one group of eye image conduct Training set, one group of eye image are implemented as test set and a set of overall depth of nesting neutral net being not optimised, the present invention Example illustrate using improved overall nested networks prediction four-way boundary graph with optimized parameter as a result, specific steps such as Under：
First, training dataset is manually demarcated.I.e. for each eye image in training set, four edges are demarcated respectively The position on boundary.For each edge circle, 6 control points are demarcated by hand first, then determine that its corresponding batten is bent according to control point Line, finally draws the spline curve of one fixed width in the boundary graph passage belonging to the border.Part calibration is illustrated in Fig. 4 As a result, wherein showing for convenience, each edge circle represents with different colours and be covered in (double-edged eyelid on original eye image： Blueness 1, upper eyelid：Green 2, lower eyelid：Red 3, the lower boundary for the silkworm that crouches：Purple 4).In addition, the situation that double-edged eyelid are not present is such as Shown in Fig. 4 (c), shown in the situation that sleeping silkworm is not present such as Fig. 4 (d), respectively not under double-edged eyelid and sleeping silkworm under both of these case Border is demarcated.
Secondly, the embodiment of the present invention is improved a set of overall nested deep neural network for Boundary Extraction. A convolutional layer is all connected with before the output of primitive network side and final output, its convolution kernel depth is 1, i.e., can only export single-pass The boundary graph in road, and do not include semantic information.The convolution kernel depth of convolutional layer is improved to 4 in the present embodiment, with four-way The boundary graph containing semantic information it is corresponding.
Finally, the embodiment of the present invention is trained to obtain optimized parameter to above-mentioned network, for predicting the four of ocular Bar border.The present embodiment uses the model of a pre-training of above-mentioned overall nested networks as initial value, on its basis into Row arameter optimization.The present embodiment uses input of the original eye image of training set as network, and uses the four-way of calibration Boundary graph carries out network the training for having supervision, when iterations reaches 2000 times training restrained and by network at this time Optimized parameter of the parameter as network.The present embodiment so using the overall nested networks prediction with optimized parameter containing semantic letter The four-way ocular boundary graph of breath, prediction result are as shown in Figure 3.
The ocular Boundary characteristic extraction method based on deep learning proposed according to embodiments of the present invention, can pass through Four kinds of borders in the method extraction eye image of deep learning, so as to provide good base for the real-time three-dimensional reconstruction of eyelid Plinth, and the ocular boundary characteristic containing semantic information can be provided, the real-time three-dimensional reconstruction to eyelid has greater significance.
The ocular boundary characteristic based on deep learning proposed according to embodiments of the present invention referring next to attached drawing description Extraction element.
Fig. 5 is that the structure of the ocular Boundary characteristic extraction device based on deep learning of one embodiment of the invention is shown It is intended to.
As shown in figure 5, being somebody's turn to do the ocular Boundary characteristic extraction device 10 based on deep learning includes：Acquisition module 100, Improve module 200 and training module 300.
Wherein, acquisition module 100 is used to obtain training dataset, wherein, by manually demarcating four on eye image Border containing semantic information, and be stored in the boundary graph of four-way and obtain training dataset.Module 200 is improved to be used for a set of The overall nested deep neural network for Boundary Extraction is improved, wherein, improve the loss function of deep neural network And network structure, so that the semantic information comprising four edges circle can simultaneously export the boundary graph of corresponding four-way.Training module 300 For the method by deep learning, supervision instruction has been carried out using the embedding deep neural network of the improved entirety of training data set pair Practice, obtain the optimized parameter of network, to use four edges circle in neural network forecast eye image.The device of the embodiment of the present invention 10 can extract four kinds of borders in eye image by the method for deep learning, so as to be provided for the real-time three-dimensional reconstruction of eyelid Good basis, and the ocular boundary characteristic containing semantic information can be provided.
Further, in one embodiment of the invention, acquisition module 100, further comprise：Demarcate unit, obtain Unit and drawing unit.Wherein, unit is demarcated to be used to demarcate control point by hand in a line circle in office.Acquiring unit is used for basis Control point determines corresponding spline curve.Drawing unit is used to draw default width in the boundary graph passage belonging to a line circle in office The spline curve of degree.
Further, in one embodiment of the invention, module 200 is improved, is further comprised：Definition includes four edges The loss function of boundary's semantic information, the convolution being connected in overall nested deep neural network with side output and final output The convolution kernel depth of layer is changed to 4 by 1.
Further, in one embodiment of the invention, training module 300, further comprise：Use the net of pre-training Network, using the original image that training data is concentrated as network inputs, calibration is utilized with reference to deep learning platform as initial value The boundary graph of four-way the training for having supervision is carried out to deep neural network, obtain the optimized parameter of network.
Further, in one embodiment of the invention, loss function is：
Wherein, I and G is respectively input picture and four-way boundary graph, ψfWithThe respectively final output and net of network The side output of network, αfWithThe weight of respectively final loss and side loss, L (ψf(I, θ), G) damaged for the final of network Lose, ψfThe final output of network in the case that (I, θ) is I and θ, θ is network parameter, and L is cross entropy loss function,Lost for side,The side output of network in the case of for I and θ.
It should be noted that the foregoing explanation to the ocular Boundary characteristic extraction embodiment of the method based on deep learning Illustrate the ocular Boundary characteristic extraction device based on deep learning for being also applied for the embodiment, details are not described herein again.
The ocular Boundary characteristic extraction device based on deep learning proposed according to embodiments of the present invention, can pass through Four kinds of borders in the method extraction eye image of deep learning, so as to provide good base for the real-time three-dimensional reconstruction of eyelid Plinth, and the ocular boundary characteristic containing semantic information can be provided, the real-time three-dimensional reconstruction to eyelid has greater significance.
In the description of the present invention, it is to be understood that term " " center ", " longitudinal direction ", " transverse direction ", " length ", " width ", " thickness ", " on ", " under ", "front", "rear", "left", "right", " vertical ", " level ", " top ", " bottom " " interior ", " outer ", " up time The orientation or position relationship of the instruction such as pin ", " counterclockwise ", " axial direction ", " radial direction ", " circumferential direction " be based on orientation shown in the drawings or Position relationship, is for only for ease of and describes the present invention and simplify description, rather than indicates or imply that signified device or element must There must be specific orientation, with specific azimuth configuration and operation, therefore be not considered as limiting the invention.
In addition, term " first ", " second " are only used for description purpose, and it is not intended that instruction or hint relative importance Or the implicit quantity for indicating indicated technical characteristic.Thus, define " first ", the feature of " second " can be expressed or Implicitly include at least one this feature.In the description of the present invention, " multiple " are meant that at least two, such as two, three It is a etc., unless otherwise specifically defined.
In the present invention, unless otherwise clearly defined and limited, term " installation ", " connected ", " connection ", " fixation " etc. Term should be interpreted broadly, for example, it may be fixedly connected or be detachably connected, or integrally；Can be that machinery connects Connect or be electrically connected；It can be directly connected, can also be indirectly connected by intermediary, can be in two elements The connection in portion or the interaction relationship of two elements, unless otherwise restricted clearly.For those of ordinary skill in the art For, the concrete meaning of above-mentioned term in the present invention can be understood as the case may be.
In the present invention, unless otherwise clearly defined and limited, fisrt feature can be with "above" or "below" second feature It is that the first and second features directly contact, or the first and second features pass through intermediary mediate contact.Moreover, fisrt feature exists Second feature " on ", " top " and " above " but fisrt feature are directly over second feature or oblique upper, or be merely representative of Fisrt feature level height is higher than second feature.Fisrt feature second feature " under ", " lower section " and " below " can be One feature is immediately below second feature or obliquely downward, or is merely representative of fisrt feature level height and is less than second feature.
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show The description of example " or " some examples " etc. means specific features, structure, material or the spy for combining the embodiment or example description Point is contained at least one embodiment of the present invention or example.In the present specification, schematic expression of the above terms is not It must be directed to identical embodiment or example.Moreover, particular features, structures, materials, or characteristics described can be in office Combined in an appropriate manner in one or more embodiments or example.In addition, without conflicting with each other, the skill of this area Art personnel can be tied the different embodiments or example described in this specification and different embodiments or exemplary feature Close and combine.
Although the embodiment of the present invention has been shown and described above, it is to be understood that above-described embodiment is example Property, it is impossible to limitation of the present invention is interpreted as, those of ordinary skill in the art within the scope of the invention can be to above-mentioned Embodiment is changed, changes, replacing and modification.

Claims (10)

  1. A kind of 1. ocular Boundary characteristic extraction method based on deep learning, it is characterised in that comprise the following steps：
    Training dataset is obtained, wherein, by manually demarcating four borders containing semantic information on eye image, and it is stored in The training dataset is obtained in the boundary graph of four-way；
    A set of overall nested deep neural network for Boundary Extraction is improved, wherein, improve the depth nerve The loss function and network structure of network, so that the semantic information comprising four edges circle can simultaneously export the corresponding four-way Boundary graph；And
    By the method for deep learning, prison has been carried out using the embedding deep neural network of the improved entirety of training data set pair Supervise and instruct white silk, obtain the optimized parameter of network, to use four edges circle in neural network forecast eye image.
  2. 2. the ocular Boundary characteristic extraction method according to claim 1 based on deep learning, it is characterised in that institute State by manually demarcating four borders containing semantic information on eye image, further comprise：
    Control point is demarcated by hand in a line circle in office；
    Corresponding spline curve is determined according to the control point；
    The spline curve of predetermined width is drawn in the boundary graph passage belonging to any bar border.
  3. 3. the ocular Boundary characteristic extraction method according to claim 1 based on deep learning, it is characterised in that institute State and a set of overall nested deep neural network for Boundary Extraction is improved, further comprise：
    Definition include the loss function of the four edges circle semantic information, by the deep neural network of the overall nesting with side The convolution kernel depth for the convolutional layer that side output is connected with final output is changed to 4 by 1.
  4. 4. the ocular Boundary characteristic extraction method according to claim 1 based on deep learning, it is characterised in that institute State and carry out Training using the embedding deep neural network of the improved entirety of training data set pair, further comprise：
    Using the network of pre-training as initial value, using the original image that the training data is concentrated as network inputs, knot The training that deep learning platform carries out the deep neural network to have supervision using the boundary graph of the four-way of calibration is closed, is obtained The optimized parameter of the network.
  5. 5. according to ocular Boundary characteristic extraction method of the claim 1-4 any one of them based on deep learning, it is special Sign is that the loss function is：
    <mrow> <mi>arg</mi> <mi> </mi> <msub> <mi>min</mi> <mi>&amp;theta;</mi> </msub> <mrow> <mo>(</mo> <msub> <mi>&amp;alpha;</mi> <mi>f</mi> </msub> <mi>L</mi> <mo>(</mo> <mrow> <msub> <mi>&amp;psi;</mi> <mi>f</mi> </msub> <mrow> <mo>(</mo> <mrow> <mi>I</mi> <mo>,</mo> <mi>&amp;theta;</mi> </mrow> <mo>)</mo> </mrow> <mo>,</mo> <mi>G</mi> </mrow> <mo>)</mo> <mo>+</mo> <msubsup> <mi>&amp;Sigma;</mi> <mrow> <mi>k</mi> <mo>=</mo> <mn>1</mn> </mrow> <mn>5</mn> </msubsup> <msub> <mi>&amp;alpha;</mi> <msub> <mi>s</mi> <mi>k</mi> </msub> </msub> <mi>L</mi> <mo>(</mo> <mrow> <msub> <mi>&amp;psi;</mi> <msub> <mi>s</mi> <mi>k</mi> </msub> </msub> <mrow> <mo>(</mo> <mrow> <mi>I</mi> <mo>,</mo> <mi>&amp;theta;</mi> </mrow> <mo>)</mo> </mrow> <mo>,</mo> <mi>G</mi> </mrow> <mo>)</mo> <mo>)</mo> </mrow> <mo>,</mo> </mrow>
    Wherein, I and G is respectively input picture and four-way boundary graph, ψfWithThe respectively final output of network and network Side exports, αfWithThe weight for weight and the side loss respectively finally lost, L (ψf(I, θ), G) it is the final of network Loss, ψfThe final output of network in the case that (I, θ) is I and θ, θ is network parameter, and L is cross entropy loss function,Lost for side,The side output of network in the case of for I and θ.
  6. A kind of 6. ocular Boundary characteristic extraction device based on deep learning, it is characterised in that including：
    Acquisition module, for obtaining training dataset, wherein, by manually demarcating four on eye image containing semantic information Border, and be stored in the boundary graph of four-way and obtain the training dataset；
    Module is improved, for being improved to a set of overall nested deep neural network for Boundary Extraction, wherein, improve The loss function and network structure of the deep neural network, so that the semantic information comprising four edges circle can simultaneously export accordingly The boundary graph of the four-way；And
    Training module, for the method by deep learning, utilizes the embedding depth god of the improved entirety of training data set pair Training is carried out through network, obtains the optimized parameter of network, to use four edges circle in neural network forecast eye image.
  7. 7. the ocular Boundary characteristic extraction device according to claim 6 based on deep learning, it is characterised in that institute Acquisition module is stated, is further comprised：
    Unit is demarcated, control point is demarcated by hand in a line circle in office；
    Acquiring unit, for determining corresponding spline curve according to the control point；
    Drawing unit, for drawing the spline curve of predetermined width in the boundary graph passage belonging to any bar border.
  8. 8. the ocular Boundary characteristic extraction device according to claim 6 based on deep learning, it is characterised in that institute Improvement module is stated, is further comprised：
    Definition include the loss function of the four edges circle semantic information, by the deep neural network of the overall nesting with side The convolution kernel depth for the convolutional layer that side output is connected with final output is changed to 4 by 1.
  9. 9. the ocular Boundary characteristic extraction device according to claim 6 based on deep learning, it is characterised in that institute Training module is stated, is further comprised：
    Using the network of pre-training as initial value, using the original image that the training data is concentrated as network inputs, knot The training that deep learning platform carries out the deep neural network to have supervision using the boundary graph of the four-way of calibration is closed, is obtained The optimized parameter of the network.
  10. 10. according to ocular Boundary characteristic extraction device of the claim 6-9 any one of them based on deep learning, it is special Sign is that the loss function is：
    <mrow> <mi>arg</mi> <mi> </mi> <msub> <mi>min</mi> <mi>&amp;theta;</mi> </msub> <mrow> <mo>(</mo> <msub> <mi>&amp;alpha;</mi> <mi>f</mi> </msub> <mi>L</mi> <mo>(</mo> <mrow> <msub> <mi>&amp;psi;</mi> <mi>f</mi> </msub> <mrow> <mo>(</mo> <mrow> <mi>I</mi> <mo>,</mo> <mi>&amp;theta;</mi> </mrow> <mo>)</mo> </mrow> <mo>,</mo> <mi>G</mi> </mrow> <mo>)</mo> <mo>+</mo> <msubsup> <mi>&amp;Sigma;</mi> <mrow> <mi>k</mi> <mo>=</mo> <mn>1</mn> </mrow> <mn>5</mn> </msubsup> <msub> <mi>&amp;alpha;</mi> <msub> <mi>s</mi> <mi>k</mi> </msub> </msub> <mi>L</mi> <mo>(</mo> <mrow> <msub> <mi>&amp;psi;</mi> <msub> <mi>s</mi> <mi>k</mi> </msub> </msub> <mrow> <mo>(</mo> <mrow> <mi>I</mi> <mo>,</mo> <mi>&amp;theta;</mi> </mrow> <mo>)</mo> </mrow> <mo>,</mo> <mi>G</mi> </mrow> <mo>)</mo> <mo>)</mo> </mrow> <mo>,</mo> </mrow>
    Wherein, I and G is respectively input picture and four-way boundary graph, ψfWithThe respectively final output of network and network Side exports, αfWithThe weight of respectively final loss and side loss, L (ψf(I, θ), G) lost for the final of network, ψf The final output of network in the case that (I, θ) is I and θ, θ is network parameter, and L is cross entropy loss function, Lost for side,The side output of network in the case of for I and θ.
CN201711091696.3A 2017-11-08 2017-11-08 Eye region boundary feature extraction method and device based on deep learning Active CN107977605B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711091696.3A CN107977605B (en) 2017-11-08 2017-11-08 Eye region boundary feature extraction method and device based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711091696.3A CN107977605B (en) 2017-11-08 2017-11-08 Eye region boundary feature extraction method and device based on deep learning

Publications (2)

Publication Number Publication Date
CN107977605A true CN107977605A (en) 2018-05-01
CN107977605B CN107977605B (en) 2020-04-24

Family

ID=62013116

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711091696.3A Active CN107977605B (en) 2017-11-08 2017-11-08 Eye region boundary feature extraction method and device based on deep learning

Country Status (1)

Country Link
CN (1) CN107977605B (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108830901A (en) * 2018-06-22 2018-11-16 维沃移动通信有限公司 A kind of image processing method and electronic equipment
CN109389085A (en) * 2018-10-09 2019-02-26 清华大学 Lip reading identification model training method and device based on parametric curve
CN110969191A (en) * 2019-11-07 2020-04-07 吉林大学 Glaucoma prevalence probability prediction method based on similarity maintenance metric learning method
CN112329546A (en) * 2020-10-15 2021-02-05 杭州电子科技大学 Eye height measuring method based on deep learning
CN112799510A (en) * 2021-01-27 2021-05-14 清华大学 Automatic generation method and device for multi-style natural eyeball movement
WO2021191692A1 (en) * 2020-03-27 2021-09-30 International Business Machines Corporation Annotation of digital images for machine learning

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105718878A (en) * 2016-01-19 2016-06-29 华南理工大学 Egocentric vision in-the-air hand-writing and in-the-air interaction method based on cascade convolution nerve network
CN105769120A (en) * 2016-01-27 2016-07-20 深圳地平线机器人科技有限公司 Fatigue driving detection method and device
CN106650688A (en) * 2016-12-30 2017-05-10 公安海警学院 Eye feature detection method, device and recognition system based on convolutional neural network
CN106960456A (en) * 2017-03-28 2017-07-18 长沙全度影像科技有限公司 A kind of method that fisheye camera calibration algorithm is evaluated
CN107133616A (en) * 2017-04-02 2017-09-05 南京汇川图像视觉技术有限公司 A kind of non-division character locating and recognition methods based on deep learning
CN107168527A (en) * 2017-04-25 2017-09-15 华南理工大学 The first visual angle gesture identification and exchange method based on region convolutional neural networks

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105718878A (en) * 2016-01-19 2016-06-29 华南理工大学 Egocentric vision in-the-air hand-writing and in-the-air interaction method based on cascade convolution nerve network
CN105769120A (en) * 2016-01-27 2016-07-20 深圳地平线机器人科技有限公司 Fatigue driving detection method and device
CN106650688A (en) * 2016-12-30 2017-05-10 公安海警学院 Eye feature detection method, device and recognition system based on convolutional neural network
CN106960456A (en) * 2017-03-28 2017-07-18 长沙全度影像科技有限公司 A kind of method that fisheye camera calibration algorithm is evaluated
CN107133616A (en) * 2017-04-02 2017-09-05 南京汇川图像视觉技术有限公司 A kind of non-division character locating and recognition methods based on deep learning
CN107168527A (en) * 2017-04-25 2017-09-15 华南理工大学 The first visual angle gesture identification and exchange method based on region convolutional neural networks

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108830901A (en) * 2018-06-22 2018-11-16 维沃移动通信有限公司 A kind of image processing method and electronic equipment
CN108830901B (en) * 2018-06-22 2020-09-25 维沃移动通信有限公司 Image processing method and electronic equipment
CN109389085A (en) * 2018-10-09 2019-02-26 清华大学 Lip reading identification model training method and device based on parametric curve
CN109389085B (en) * 2018-10-09 2020-09-25 清华大学 Lip language recognition model training method and device based on parameterized curve
CN110969191A (en) * 2019-11-07 2020-04-07 吉林大学 Glaucoma prevalence probability prediction method based on similarity maintenance metric learning method
CN110969191B (en) * 2019-11-07 2022-10-25 吉林大学 Glaucoma prevalence probability prediction method based on similarity maintenance metric learning method
WO2021191692A1 (en) * 2020-03-27 2021-09-30 International Business Machines Corporation Annotation of digital images for machine learning
US11205287B2 (en) 2020-03-27 2021-12-21 International Business Machines Corporation Annotation of digital images for machine learning
CN112329546A (en) * 2020-10-15 2021-02-05 杭州电子科技大学 Eye height measuring method based on deep learning
CN112799510A (en) * 2021-01-27 2021-05-14 清华大学 Automatic generation method and device for multi-style natural eyeball movement
CN112799510B (en) * 2021-01-27 2022-09-09 清华大学 Automatic generation method and device for multi-style natural eyeball movement

Also Published As

Publication number Publication date
CN107977605B (en) 2020-04-24

Similar Documents

Publication Publication Date Title
CN107977605A (en) Ocular Boundary characteristic extraction method and device based on deep learning
CN110378985B (en) Animation drawing auxiliary creation method based on GAN
CN112102283B (en) Retina fundus blood vessel segmentation method based on depth multi-scale attention convolution neural network
CN106920227B (en) The Segmentation Method of Retinal Blood Vessels combined based on deep learning with conventional method
CN112132817B (en) Retina blood vessel segmentation method for fundus image based on mixed attention mechanism
CN109919830B (en) Method for restoring image with reference eye based on aesthetic evaluation
CN109829855A (en) A kind of super resolution ratio reconstruction method based on fusion multi-level features figure
CN110197493A (en) Eye fundus image blood vessel segmentation method
CN109671018A (en) A kind of image conversion method and system based on production confrontation network and ResNets technology
CN107977671A (en) A kind of tongue picture sorting technique based on multitask convolutional neural networks
CN107437092A (en) The sorting algorithm of retina OCT image based on Three dimensional convolution neutral net
CN106408562A (en) Fundus image retinal vessel segmentation method and system based on deep learning
CN109544457A (en) Image super-resolution method, storage medium and terminal based on fine and close link neural network
CN111931908B (en) Face image automatic generation method based on face contour
CN110473142A (en) Single image super resolution ratio reconstruction method based on deep learning
CN108921169B (en) A kind of eye fundus image blood vessel segmentation method
CN113724354B (en) Gray image coloring method based on reference picture color style
CN112633288B (en) Face sketch generation method based on painting brush touch guidance
CN102147921A (en) Graph theory-based Chinese medicinal tongue nature and tongue coat separation algorithm
CN110334719A (en) The method and system of object image are built in a kind of extraction remote sensing image
CN113486944A (en) Face fusion method, device, equipment and storage medium
CN114581434A (en) Pathological image processing method based on deep learning segmentation model and electronic equipment
CN107633196A (en) A kind of eyeball moving projection scheme based on convolutional neural networks
CN113763406A (en) Infant brain MRI segmentation method based on semi-supervised learning
WO2021223042A1 (en) Method for implementing machine intelligence similar to human intelligence

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant