WO2013144184A2 - Hand held device and method for capturing images of skin portions - Google Patents

Hand held device and method for capturing images of skin portions Download PDF

Info

Publication number
WO2013144184A2
WO2013144184A2 PCT/EP2013/056479 EP2013056479W WO2013144184A2 WO 2013144184 A2 WO2013144184 A2 WO 2013144184A2 EP 2013056479 W EP2013056479 W EP 2013056479W WO 2013144184 A2 WO2013144184 A2 WO 2013144184A2
Authority
WO
WIPO (PCT)
Prior art keywords
image
skin
liquid
contact
gel
Prior art date
Application number
PCT/EP2013/056479
Other languages
French (fr)
Other versions
WO2013144184A3 (en
Inventor
Philippe Held
Original Assignee
Dermosafe Sa
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dermosafe Sa filed Critical Dermosafe Sa
Publication of WO2013144184A2 publication Critical patent/WO2013144184A2/en
Publication of WO2013144184A3 publication Critical patent/WO2013144184A3/en

Links

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B08CLEANING
    • B08BCLEANING IN GENERAL; PREVENTION OF FOULING IN GENERAL
    • B08B7/00Cleaning by methods not provided for in a single other subclass or a single group in this subclass
    • B08B7/0007Cleaning by methods not provided for in a single other subclass or a single group in this subclass by explosions
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/44Detecting, measuring or recording for evaluating the integumentary system, e.g. skin, hair or nails
    • A61B5/441Skin evaluation, e.g. for skin disorder diagnosis
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0012Biomedical image inspection
    • G06T7/0014Biomedical image inspection using an image reference approach
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/74Details of notification to user or communication with user or patient ; user input means
    • A61B5/7475User input or interface means, e.g. keyboard, pointing device, joystick
    • A61B5/748Selection of a region of interest, e.g. using a graphics tablet
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/74Details of notification to user or communication with user or patient ; user input means
    • A61B5/7475User input or interface means, e.g. keyboard, pointing device, joystick
    • A61B5/7495User input or interface means, e.g. keyboard, pointing device, joystick using a reader or scanner device, e.g. barcode scanner
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10024Color image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30088Skin; Dermal
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30092Stomach; Gastric
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00ICT specially adapted for the handling or processing of medical images
    • G16H30/20ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00ICT specially adapted for the handling or processing of medical images
    • G16H30/40ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/70ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients

Definitions

  • the present invention concerns a method and a device for capturing images of skin portions, in particular for detection of skin lesions and skin cancer, such as melanomas. Description of related art
  • the number of skin cancer has more than double for the last 20 years. This is a cancer that has 95% chance to recover if detected at early stage. Melanoma and skin cancer are the only cancer that can be detected visually and with non invasive methods.
  • the ABCD rules (Asymmetry, Border, Colour, Diameter) using naked eyes or macro photography has a low sensitivity (-80%) and a low specificity (-75%) depending of the various studies done on this subject. Also this method is even much harder to use for the very early stage detection. For early stage melanoma ABCD gives only -50% sensitivity and -60% specificity.
  • Dermatoscopy is a rather a recent method (really used at large scale only after the year 2000) used to detect skin cancer at early stage. This method can reach a sensitivity of detection of > 95 %in sensitivity with a well trained dermatologist. This method is used more and more often by the specialist but requires some good training to be efficient. But the sensitivity level of detection is such higher versus the other method that this shouldbe much broadly used.
  • One way to improve the quality of the images and the quantity of information that can be retrieved from each image is to spread o gel or liquid on the captured skin portion.
  • a gel such as a dermoscopic gels or liquids (water, alcohol 70%, etc)
  • This technology is known as epiluminescence microscopy.
  • the gel or liquid must be carefully spread over the skin, in an adequate height, and without any bubbles. This is difficult to achieve for non specialists, and especially time-consuming even for specialists, especially when a large number of moles or skin lesions need to be examined.
  • US2004092802 discloses a hand held device and a method for determining the variation in concentration of chromophores within an epithelial tissue.
  • the proposed device comprises lights sources, image sensor, data processing and displaying means.
  • the device could comprise a patient-contacting film that is coated with index matching oil.
  • Main drawbacks of this solution derive from the limited quantity of index matching oil that could be coated on the film. Actually, this quantity limits the number of manipulations on the patient's skin during both the preparation and the images acquisition phases.
  • US20120172685 discloses a skin analysis system.
  • This system is in form of additional equipment for consumer electronic devices.
  • the proposed solution allows periodically image acquisitions of moles, images comparisons and calculation of moles development rates.
  • the device could generate alerts, inviting user to an urging consultation to medical specialists.
  • the system provides consumables in form of a sheet with self- adhesive elements to enter information in the image, in particular ID numbers, dates, names and image calibration elements. These consumables are to be stuck upon the skin around the region of interest.
  • the location of the skin lesions images is provided by taking a large body portion image where multiple skin lesions are captured together with said consumables.
  • WO02080771 discloses a disposable calibrated camera for skin lesion examination.
  • This camera assembly is equipped with a transparent window that, during the use, could be coated with an index matching liquid.
  • a liquid reservoir equips the camera assembly.
  • camera assembly focuses on a single-use mode.
  • camera assembly components are specifically designed for a single use, limiting the employ of skin anomalies evolutions analysis and survey methods that are indispensable to support skin lesions and melanomas detections.
  • the invention is related to a hand held device for capturing images of skin portions for dermatoscopy analysis, comprising:
  • an image sensor for capturing said image
  • a lens arranged for projecting a focused image of the skin on the sensor when the device is in direct contact with the skin
  • a memory for storing the captured digital image
  • a tank for storing a plurality of portions of contact gel and/or liquid.
  • the tank thus stores a plurality of portions of contact gel and/or liquid that can be used for capturing a plurality of images of successive skin portions. Therefore, the application of dermatoscopic gel or liquid is easier since the gel or liquid used for a plurality of successive image captures may be provided directly from a tank comprised in the device, without needing an external recipient for the gel or liquid.
  • the device further includes a distribution nozzle able to spread it on the skin.
  • the type of distribution nozzle could be a jet nozzle, a spray nozzle and/or flat stream nozzle in order to control the flow and quantity of contact gel and/or liquid that is to spread.
  • the invention thus provides a new simple type of dermatoscopy device, that anyone could use at home and send good quality dermatoscopy pictures to a medical service centre for advice, could open a revolution in the early skin cancer detection. Health cost reduction and decease of death is the final outcome of such solution.
  • the device allows patients and physicists to take quickly and many dermatoscopy pictures, ideally in less than few seconds, preferably less than 3 seconds per spot.
  • the device could be a full electronic autonomous offline dermatoscope.
  • the inspection process could be done offline, meaning without any direct view on a computer screen.
  • the device could also work with a connected computer when required.
  • the tank for storing a plurality of portions of contact gel and/or liquid is located above or under the surface of the body device.
  • This tank could be entirely or partially made of elastic materials so that a portion of contact gel and/or liquid could be delivered by a simple pressure with a finger or with a mechanical pressure element on the tank to deliver the gel or liquid through an aperture.
  • a portion of contact gel and/or liquid could be delivered by actuating a lever acting against the surface of the tank for expelling a controlled amount or gel or liquid from the tank when the lever is actuated.
  • the device includes a tank located within the device body.
  • a portion of contact gel and/or liquid could be delivered by actuating a mechanically or an electro-mechanically system, controlled by an external switch, button, roller and/or levers.
  • the tank is made of a disposable gel and/or liquid cartridge that can be inserted within the device
  • the device further includes a disposable contact glass.
  • This contact glass could be intended for direct contact with the skin.
  • the contact glass could be made of glass or any other transparent material, including plastic.
  • the single use contact lens could be a single use contact lens.
  • the single use contact lens could be delivered with a contact gel and/or liquid already on the contact lens.
  • One purpose of the contact glass is to maintain flat and thin the liquid or gel on the skin for wet dermoscopy.
  • the contact glass could also be a lens that refocuses the ray light onto the sensor.
  • the device includes a distribution nozzle able to spread contact gel and/or liquid onto the outside surface of this contact glass and/or onto the skin.
  • the type of distribution nozzle could be a jet nozzle, a spray nozzle and/or flat stream nozzle in order to control the flow and quantity of contact gel and/or liquid that is to spread. Multiple nozzle of the same type or combination of various types could also be employed to improve the spreading efficacy.
  • a small canal and/or a groove is provided in the contact glass that will help to bring the gel or liquid to the skin portion. This embodiment is more comfortable for the patient because of less liquid spreading and flowing all over.
  • the hand held device is further equipped with a small pressure sensor that will indicate to the user by a beep and / or small led and / or other visual indicator that the pressure is right or too high or too low. Those indications can also be used in combination with an interactive web base platform to drive the user to the best and optimum image capture process.
  • the device further includes at least one reference marker on the contact glass, or on any other glass or lens in the optical path, in order to include a reference mark in the captured digital image.
  • the marker could be a colour marker.
  • the device also includes software for calibrating the colour of the captured image, based on the known colour of the markers.
  • the device further comprises a lighting system for lighting the portion of skin during the capture of the image.
  • the device further comprises a light pointer for projecting a light beam on the centre of portion of skin from which an image should be captured. The device is then equipped with means for automatically switching this light off during the capture of the image.
  • the device further comprises data selection means for indicating the body area from which an image is taken. The data selection means could be arranged for allowing a user selection of one body area among predefined body areas.
  • the device further comprises a computer product having a software code arranged for causing a microprocessor that runs said software code to include an indication of a body area as metadata in corresponding captured images.
  • the device further comprises software means for automatic identification of image area containing skin lesions.
  • the device further includes computer vision modules for recognizing features in the capture images, for example features representatives of particular lesions or melanomas.
  • the device could also include software modules for recognizing changes between successive images of a same spot, in particular changes representatives of a melanoma.
  • the device further comprises a computer in charge of the execution of at least one of the calibration, image processing and/or analysis software modules.
  • the device could be connected to this computer by one or multiple communication means, as, for example, an USB cable.
  • Fig. 1 shows a view over a hand held device.
  • Fig. 2 shows markers on the skin contact glass.
  • Fig. 3 shows a single use contact glass with the contact gel already on the contact glass.
  • Fig. 4 shows a hand held device with contact gel and/or liquid tank according to the invention.
  • Fig. 5 shows details of a hand held device contact glass with contact gel and/or liquid distribution canal or groove according to the invention.
  • Fig. 6 shows a device equipped with contact glass provided by contact gel already on the contact glass.
  • Fig. 7 shows a hand held device with a light pointer.
  • Fig. 8 shows a view of a spot with markers clearly visible.
  • Fig. 9 shows successive views of the same spot taken at different time.
  • Fig. 10 shows two successive pictures of a same spot where similar areas are circled.
  • Fig. 1 1 shows an example of a reference picture of a skin portion used for automatic or manual determination of skin type.
  • Fig. 12 shows a skin image capturing system comprising a hand held device with contact gel and/or liquid tank and a connected computer.
  • Fig. 13 shows a skin image capturing system comprising a hand held device and a computer.
  • Figure 1 shows a device, or dermoscope, comprising an image sensor and a contact glass 2 that will be in contact with the skin for dermatoscopy picture. It is also possible to have a device with a lens or glass maintained at a predefined distance of the skin but without direct contact.
  • the captured images of skin portions can be associated with metadata which can be used by a software in a remote computer 66 for analysing the pictures as showed in Figs.12-13.
  • the focus of the device is set at factory so that a lens 70 projects a focused image of the skin portion on the image sensor. The user has not to worry about adjusting the focus.
  • the skin contact glass 2 may have some markers 6 on it (Fig. 2) that will be used for some software treatment. There might be some white and black reference dots. There might be some reference colour dots.
  • the markers are clearly visible on each image captured with the device, as shown on Figs. 8-1 1 .
  • the colour and/or brightness of those reference dots are known and might be used for any image treatment during inspection by the user or the dermatologist, for calibrating the colour and brightness and saturation of the stored and captured image, and/or the colour of the displayed and/or printed image.
  • the expected colour and brightness of the reference marks can be compared with their colours in the captured images, to adjust the colour and brightness of the whole image.
  • the orientation of successive images of a same spot can be adjusted by software, by comparing matching patterns in different orientations, as shown on Figure 9.
  • a software module in the device 1 , or in a computer 66 receiving the images captured with the device can be used for adapting the orientation of various images of a same portion of skin and facilitating their comparison.
  • This is a way to calibrate the colours and to have systematically the same results independently to the screen or computer platform used. This allows making sure that comparing pictures of the same lesion at different time, a reference is available for a true comparison in order to detect correctly any colour changes.
  • the software used to visualize the lesion will automatically adjust the colour and brightness based on the reference dot. This might also be used for automatic or human detection of significant patterns.
  • the device 1 may include a single use contact glass 2 with the contact gel and or Iiquid8 already on the glass.
  • An example of single-use glass 2 with gel 8 on it is illustrated on Figure 3.
  • a thin protection foil 1 1 is used to protect this contact gel and/or liquid layer 8 during transportation and storage of this contact glass in a packaging blister 10.
  • the contact glass can be removed from the blister and mounted at the extremity of the device 1 , as illustrated on Figures 6, 12 and 13.
  • the contact glass with the gel on it may be replaced after each image, or after each series of images captured from a patient. If the amount of liquid or gel on the contact glass is not sufficient, it might be possible to add more gel from a tank 22 comprised in the device.
  • the device 1 might comprise a contact glass that will be used substantially during the whole life of the device 1 , without need to be replaced after each use.
  • the device is equipped with a tank 22 for storing a plurality of portions of contact gel and/or liquid as shown on Fig. 4.
  • this tank could be actuated in order to dispense the right quantity of gel and/or water through a distribution canal 23 equipped with an aperture to make the dermatoscopy picture.
  • This tank 22 could be located on the external surface of the device body 5 in order to be manipulated by the device's user.
  • the tank could be, for example, entirely or partially made of elastic materials. A simple pressure of the tank with at least one finger will distribute the required portion of contact gel and/or liquid from the tank on the skin portion.
  • the device could comprise a lever (not shown) whose movement actuates in a mechanical way the tank, delivering a more precisely defined portion of contact gel and/or liquid onto the skin and/or onto the contact glass.
  • the device could comprise an electro-mechanical actuator activated by a switch, a button and/or a roller to deliver a precise portion of gel or liquid onto the skin and/or onto the contact glass.
  • the tank could be located within the device body 5.
  • a part of distribution canal 23 could be located on or within the hand held device body 5. [0053] A part of the distribution canal 23 could be located on or within the hand held device extender 3.
  • the tank may be provided with an overture and a lid for contact gel and/or liquid recharge purpose.
  • the tank could be made of a disposable contact gel and/or liquid cartridge for a safe, clean and rapid recharge.
  • the device could be equipped with distribution nozzle 21 to control the direction, the flow and the quantity of contact gel and/or liquid that is to spread on the skin and/or on the contact glass 2.
  • distribution nozzle 21 to control the direction, the flow and the quantity of contact gel and/or liquid that is to spread on the skin and/or on the contact glass 2.
  • Various types of nozzle could be employed, as for example, jet nozzle, spray nozzle and flat stream nozzle. Multiple nozzles of the same type or combination of various types could be used to improve the efficacy and regularity of the spreading operation.
  • the gel or the liquid is applied first on the skin or on the contact glass of the dermatoscope, and then the device is applied and pressed on the lesion. This requires some practice and fast handling operation when the gel or liquid is applied on vertical skin portion.
  • a small canal is provided in the glass, or a groove 24 along the body of the device, that will help to bring the gel or liquid to the skin portion 33 (Fig. 5).
  • the device is pressed on the skin lesion and then we push the liquid through.
  • This embodiment is also more comfortable for the patient because of less liquid spreading and flowing all over.
  • Using wet dermoscopy requires to apply the right pressure. If the pressure is too low, the epiluminescence is not appearing and more air bubbles might be seen. If the pressure is too high, the liquid might be faster flowing outside the area. If the pressure is really above a normal and reasonable pressure it can influence the blood vessel close by and change some aspect of the lesion.
  • one feature of the invention is a small pressure sensor 69 that will indicate to the user by a beep and / or small led and / or other visual indicator that the pressure is right or too high or too low.
  • the small pressure sensor could be mounted in order to measure the pressure between the contact glass 2 and the extender 3, or
  • the device may comprise a light pointer 31 shown on Figure7that projects a focused beam on the skin 30 where the centre of the image is. This help to centre small lesions 32 of few millimetres diameter only.
  • the ray of light pointer can be with LED or laser. The power must be very low and should not risk any injuries to the eyes or the skin.
  • the device 1 automatically switches off the ray of light pointer 32 and turns on a white LED, or a plurality of LEDS producing in combination a white light.
  • the LED remains on only few seconds.
  • the power of the LED is automatically adjusted to have the reference spots on the skin contact glass correctly lighting.
  • the device further comprises indication retrieving means of body area from which the acquired image is or will be captured.
  • the purpose is to code the picture with the body area to make easier some offline software treatment at later stage.
  • the indication retrieving means could comprise means for automatically matching the captured image with a previously captured image stored in the memory 68, and automatically or manually
  • a signature from the captured image could be retrieved and compared with signatures already available and associated with a body area. This signature could depend on skin natural markers such as nevus 36, skin pores, skin lesions 33, skin grains, hairs34, buttons and/or veins (Fig. 9).
  • the indication retrieving means could further comprise means for stitching the acquired image with other stored images, and building an image of a larger body area.
  • the indication retrieving means could further comprise an inertial sensor in the hand-held device for determining the position of said hand-held device.
  • the patient Before to start shooting on some skin lesion, the patient could furthermore select manually the body area on which he or she has the lesion to be controlled.
  • the device could further comprise a code bar reader exploiting the device sensor 61 .
  • the body area could so manually selected capturing a code bar image indicating a particular body area.
  • the code-bar could be a conventional code-bar, a QR-code or any other type of visual coding that can be used as identification with image analyzes.
  • the body area can be identified such as right forearm, left forearm, neck, top hand right, etc.
  • the localisation can be much more precisely identified (for instance with a number, such as for example ten, different zone on the left arm)
  • All the pictures captured are saved on an onboard memory 68, which could be a FRAM, and/or a removable SD cards for instance.
  • Pictures stored in the memory 68 may be downloaded to a computer when the device 1 or the portable memory (SD cards) is plugged into a computer 66.
  • a USB interface 63 could be used for this purpose.
  • Pictures can also be transferred over a wireless interface, for example over wifi or Bluetooth or any wireless technology.
  • the device contains at least one battery64 that is rechargeable via USB port 63 or via a battery loader or loading station.
  • the device contains a real time clock 67 that allows to clearly identifying the time of capture of each picture.
  • the device 1 contains all the necessary code to automatically open and connect to the web application, in order to transfer the images to a remote server for storage, comparison with previously captured images of the same skin portion, and/or analysis by a dermatologist or
  • a coding of each picture is used throughout the web application.
  • the device associates a unique code similar to what is described below to the picture:
  • This code could be transferred along with the picture to a remote computer 66 or server where is can be used for classifying and retrieving the picture, and for facilitating the comparison with other images of the same spot at different times.
  • the user identification part of the code can be an anonymous identification or an alias.
  • the images captured with the device can be processed locally in the device, for example in order to adjust their colour, contrast and brightness, possibly based on the reference marks.
  • a software module in the device might carry out computer vision steps for
  • An indication might be displayed to the user, for example a visual or audible indication, when a suspicious spot has been found on the skin.
  • the images might then be transferred to a further computer, for example over a wireless interface, a USB cable, a memory card and so on.
  • the images might also be transferred to a remote computer over an
  • the computer receives the images with associated metadata, such as a picture code, and carries out further methods for processing the images, classifying them and for assisting the user or the dermatologist in detecting skin lesions and melanomas and/or automatic detection of suspicious images.
  • metadata such as a picture code
  • the device can be used with a real time interactive web base application for guidance, real time information and training and feedback.
  • the software in the device and/or in the computer might carry out an automatic detection of skin cancer with digital image analysis.
  • the software might also assist the user or dermatologist and help him in the fast and reliable detection of melanomas.
  • the specialist operator while checking and scrolling through hundreds of pictures for a pre diagnostics, is thus driven through all the steps with software assistance to highlight some area or characteristics when required, and to help him in verifying the images and detecting more quickly and in a more reliable way the most critical images.
  • the software executed on the computer can detect that several pictures taken at different time correspond to a same spot.
  • the comparison might be used on the date - it is unlikely that several pictures of same spot taken on the same day need to be compared -, on the body part identification, and on computer vision algorithms for matching successive views of a same spot when those views include similar group of pixels, or possible transformation from one view to the next.
  • Fig. 10 shows two views of a same spot taken at two months intervals; the circled area 35 corresponds to similar group of pixels which can be used for automatic matching of views.
  • the software can also be used for facilitating manual matching of successive images, for example by presenting to the user a selection of images likely to correspond and that the user can manually match.
  • the software can also be used for automatic detection of areas that have changed between successive pictures, and automatic marking of those areas, or automatic marking of series of pictures in which significant changes were observed. [0088]
  • the software might detect patterns or image elements
  • Skin cancer such as melanomas can have various levels of the development characterized by changing aspects and various patterns.
  • the software might comprise routines or modules for detecting image portions representative of some of those levels. For example, a module can be executed for detecting and
  • the process might also comprise a step of indicating of detecting the skin type of the patient.
  • a skin type might be associated to a series of pictures taken during a session.
  • the skin type might be included in the image coding.
  • the skin type might be determined from a picture taken at a specific location, for example a location without melanomas.
  • An example of image which can be used for automatic and/or manual determination of skin type is shown on Figure 10.
  • the skin type might be indicated as a skin index type and might depend on the skin colour.
  • USB cable Personal Computer: Real time clock: Memory

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • Animal Behavior & Ethology (AREA)
  • Veterinary Medicine (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Pathology (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Biophysics (AREA)
  • Dermatology (AREA)
  • Public Health (AREA)
  • Biomedical Technology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)

Abstract

A hand held device (20) for capturing images of skin portions for dermatoscopy analysis, comprising: an image sensor for capturing said image; a lens (70) arranged for projecting a focused image of the skin on the sensor when the device is in direct contact with the skin; a memory for storing the captured digital image; a tank (22) for storing a plurality of portions of contact gel and/or liquid.

Description

Hand held device and method for capturing images of skin portions
Field of the invention
[0001] The present invention concerns a method and a device for capturing images of skin portions, in particular for detection of skin lesions and skin cancer, such as melanomas. Description of related art
[0001] The number of skin cancer has more than double for the last 20 years. This is a cancer that has 95% chance to recover if detected at early stage. Melanoma and skin cancer are the only cancer that can be detected visually and with non invasive methods. [0002] The ABCD rules (Asymmetry, Border, Colour, Diameter) using naked eyes or macro photography has a low sensitivity (-80%) and a low specificity (-75%) depending of the various studies done on this subject. Also this method is even much harder to use for the very early stage detection. For early stage melanoma ABCD gives only -50% sensitivity and -60% specificity.
[0003] Dermatoscopy is a rather a recent method (really used at large scale only after the year 2000) used to detect skin cancer at early stage. This method can reach a sensitivity of detection of > 95 %in sensitivity with a well trained dermatologist. This method is used more and more often by the specialist but requires some good training to be efficient. But the sensitivity level of detection is such higher versus the other method that this shouldbe much broadly used.
[0004] Today, electronic dermatoscopes are targeting the professional market and would be in average very expensive for a non professional. [0005] Thanks to the new technologies of information (tele-video, web etc) that have so much improved for the lastl 0 years that tele-medicine starts to be a real subject of actuality. However the implementation of such system tends to be really not as common as it could be expected. Many doctors are reticent even though there is a clear market need from the patient. [0006] For the detection of skin cancer there are today few solutions in the market to get a first level of information to get a skin lesion controlled, but usually it is only by using standard macro photography (with a smartphone, a digital camera, i.e.). Even if this can be used for an auto control and observe variation of the lesion over time, this is definitively not sensitive enough to have a secure pre diagnostic based on a picture only. Those solutions are definitively not suitable for regular and full screening check having many spots to control.
[0007] One way to improve the quality of the images and the quantity of information that can be retrieved from each image is to spread o gel or liquid on the captured skin portion. For example it is known to apply a gel, such as a dermoscopic gels or liquids (water, alcohol 70%, etc) , that will reveal structures under the surface of the skin that won't be visible without the gel. This technology is known as epiluminescence microscopy. However, the gel or liquid must be carefully spread over the skin, in an adequate height, and without any bubbles. This is difficult to achieve for non specialists, and especially time-consuming even for specialists, especially when a large number of moles or skin lesions need to be examined. If the amount of gel or liquid varies, the quality of the images decreases, and a comparison between successive images of the same skin portion is difficult. [0008] Another technique is based on the use of polarized light to achieve similar type of images. Both techniques can provide complementary information, but the trend and according to some studies is to use wet dermoscopy to cover more type of pattern recognition while and better for final diagnosis, while polarized can be enough for first level of screening. [0009] US2004092802 discloses a hand held device and a method for determining the variation in concentration of chromophores within an epithelial tissue. The proposed device comprises lights sources, image sensor, data processing and displaying means. The device could comprise a patient-contacting film that is coated with index matching oil. Main drawbacks of this solution derive from the limited quantity of index matching oil that could be coated on the film. Actually, this quantity limits the number of manipulations on the patient's skin during both the preparation and the images acquisition phases.
[0010] US20120172685 discloses a skin analysis system. This system is in form of additional equipment for consumer electronic devices. The proposed solution allows periodically image acquisitions of moles, images comparisons and calculation of moles development rates. The device could generate alerts, inviting user to an urging consultation to medical specialists. The system provides consumables in form of a sheet with self- adhesive elements to enter information in the image, in particular ID numbers, dates, names and image calibration elements. These consumables are to be stuck upon the skin around the region of interest. The location of the skin lesions images is provided by taking a large body portion image where multiple skin lesions are captured together with said consumables.
[0011] WO02080771 discloses a disposable calibrated camera for skin lesion examination. This camera assembly is equipped with a transparent window that, during the use, could be coated with an index matching liquid. In one embodiment, a liquid reservoir equips the camera assembly.
However, this camera assembly focuses on a single-use mode. Actually, camera assembly components are specifically designed for a single use, limiting the employ of skin anomalies evolutions analysis and survey methods that are indispensable to support skin lesions and melanomas detections.
[0012] It is an aim of the present invention to provide a new and improved method and a new and improved device for the detection of skin lesions and melanomas. Brief summary of the invention
[0013] According to the invention, these aims are achieved by means of methods and devices comprising the claimed features. [0014] In one aspect, the invention is related to a hand held device for capturing images of skin portions for dermatoscopy analysis, comprising:
an image sensor for capturing said image;
a lens arranged for projecting a focused image of the skin on the sensor when the device is in direct contact with the skin;
a memory for storing the captured digital image;
a tank for storing a plurality of portions of contact gel and/or liquid.
[0015] The tank thus stores a plurality of portions of contact gel and/or liquid that can be used for capturing a plurality of images of successive skin portions. Therefore, the application of dermatoscopic gel or liquid is easier since the gel or liquid used for a plurality of successive image captures may be provided directly from a tank comprised in the device, without needing an external recipient for the gel or liquid.
[0016] In one embodiment, the device further includes a distribution nozzle able to spread it on the skin. The type of distribution nozzle could be a jet nozzle, a spray nozzle and/or flat stream nozzle in order to control the flow and quantity of contact gel and/or liquid that is to spread.
Multiple nozzle of the same type or combination of various types could also be employed to improve the spreading efficacy and regularity. Therefore, since the gel or liquid is distributed from a tank and with a nozzle comprised in the device, a regular, controlled amount of gel or liquid can be spread over various portions of skin.
[0017] The invention thus provides a new simple type of dermatoscopy device, that anyone could use at home and send good quality dermatoscopy pictures to a medical service centre for advice, could open a revolution in the early skin cancer detection. Health cost reduction and decease of death is the final outcome of such solution.
[0018] The device allows patients and physicists to take quickly and many dermatoscopy pictures, ideally in less than few seconds, preferably less than 3 seconds per spot.
[0019] The device could be a full electronic autonomous offline dermatoscope. The inspection process could be done offline, meaning without any direct view on a computer screen. Alternatively, the device could also work with a connected computer when required.
[0020] In one embodiment, the tank for storing a plurality of portions of contact gel and/or liquid is located above or under the surface of the body device. This tank could be entirely or partially made of elastic materials so that a portion of contact gel and/or liquid could be delivered by a simple pressure with a finger or with a mechanical pressure element on the tank to deliver the gel or liquid through an aperture.
[0021] In another embodiment, a portion of contact gel and/or liquid could be delivered by actuating a lever acting against the surface of the tank for expelling a controlled amount or gel or liquid from the tank when the lever is actuated.
[0022] In one embodiment, the device includes a tank located within the device body. A portion of contact gel and/or liquid could be delivered by actuating a mechanically or an electro-mechanically system, controlled by an external switch, button, roller and/or levers. [0023] In one embodiment, the tank is made of a disposable gel and/or liquid cartridge that can be inserted within the device
[0024] In one embodiment, the device further includes a disposable contact glass. This contact glass could be intended for direct contact with the skin. The contact glass could be made of glass or any other transparent material, including plastic. The single use contact lens could be a single use contact lens. The single use contact lens could be delivered with a contact gel and/or liquid already on the contact lens. One purpose of the contact glass is to maintain flat and thin the liquid or gel on the skin for wet dermoscopy. In an embodiment, the contact glass could also be a lens that refocuses the ray light onto the sensor.
[0025] In one embodiment, the device includes a distribution nozzle able to spread contact gel and/or liquid onto the outside surface of this contact glass and/or onto the skin. The type of distribution nozzle could be a jet nozzle, a spray nozzle and/or flat stream nozzle in order to control the flow and quantity of contact gel and/or liquid that is to spread. Multiple nozzle of the same type or combination of various types could also be employed to improve the spreading efficacy. [0026] In one embodiment, a small canal and/or a groove is provided in the contact glass that will help to bring the gel or liquid to the skin portion. This embodiment is more comfortable for the patient because of less liquid spreading and flowing all over.
[0027] Wet dermoscopy requires to apply the right pressure. In one embodiment, the hand held device is further equipped with a small pressure sensor that will indicate to the user by a beep and / or small led and / or other visual indicator that the pressure is right or too high or too low. Those indications can also be used in combination with an interactive web base platform to drive the user to the best and optimum image capture process.
[0028] In one embodiment, the device further includes at least one reference marker on the contact glass, or on any other glass or lens in the optical path, in order to include a reference mark in the captured digital image. The marker could be a colour marker. [0029] In one embodiment, the device also includes software for calibrating the colour of the captured image, based on the known colour of the markers.
[0030] In one embodiment, the device further comprises a lighting system for lighting the portion of skin during the capture of the image.
[0031] In one embodiment, the device further comprises a light pointer for projecting a light beam on the centre of portion of skin from which an image should be captured. The device is then equipped with means for automatically switching this light off during the capture of the image. [0032] In one embodiment, the device further comprises data selection means for indicating the body area from which an image is taken. The data selection means could be arranged for allowing a user selection of one body area among predefined body areas.
[0033] In one embodiment, the device further comprises a computer product having a software code arranged for causing a microprocessor that runs said software code to include an indication of a body area as metadata in corresponding captured images.
[0034] In another embodiment, the device further comprises software means for automatic identification of image area containing skin lesions. [0035] In one embodiment, the device further includes computer vision modules for recognizing features in the capture images, for example features representatives of particular lesions or melanomas. The device could also include software modules for recognizing changes between successive images of a same spot, in particular changes representatives of a melanoma.
[0036] In one embodiment, the device further comprises a computer in charge of the execution of at least one of the calibration, image processing and/or analysis software modules. The device could be connected to this computer by one or multiple communication means, as, for example, an USB cable.
Brief Description of the Drawings
[0037] The invention will be better understood with the aid of the description of an embodiment given by way of example and illustrated by the figures, in which:
Fig. 1 shows a view over a hand held device.
Fig. 2 shows markers on the skin contact glass.
Fig. 3 shows a single use contact glass with the contact gel already on the contact glass.
Fig. 4 shows a hand held device with contact gel and/or liquid tank according to the invention.
Fig. 5 shows details of a hand held device contact glass with contact gel and/or liquid distribution canal or groove according to the invention.
Fig. 6shows a device equipped with contact glass provided by contact gel already on the contact glass.
Fig. 7shows a hand held device with a light pointer.
Fig. 8 shows a view of a spot with markers clearly visible.
Fig. 9 shows successive views of the same spot taken at different time.
Fig. 10 shows two successive pictures of a same spot where similar areas are circled. Fig. 1 1 shows an example of a reference picture of a skin portion used for automatic or manual determination of skin type.
Fig. 12shows a skin image capturing system comprising a hand held device with contact gel and/or liquid tank and a connected computer.
Fig. 13shows a skin image capturing system comprising a hand held device and a computer.
Detailed Description of possible embodiments of the Invention
[0038] Figure 1 shows a device, or dermoscope, comprising an image sensor and a contact glass 2 that will be in contact with the skin for dermatoscopy picture. It is also possible to have a device with a lens or glass maintained at a predefined distance of the skin but without direct contact. The captured images of skin portions can be associated with metadata which can be used by a software in a remote computer 66 for analysing the pictures as showed in Figs.12-13.
[0039] The focus of the device is set at factory so that a lens 70 projects a focused image of the skin portion on the image sensor. The user has not to worry about adjusting the focus.
[0040] The skin contact glass 2 may have some markers 6 on it (Fig. 2) that will be used for some software treatment. There might be some white and black reference dots. There might be some reference colour dots.
[0041] The markers are clearly visible on each image captured with the device, as shown on Figs. 8-1 1 . The colour and/or brightness of those reference dots are known and might be used for any image treatment during inspection by the user or the dermatologist, for calibrating the colour and brightness and saturation of the stored and captured image, and/or the colour of the displayed and/or printed image. The expected colour and brightness of the reference marks can be compared with their colours in the captured images, to adjust the colour and brightness of the whole image.
[0042] Furthermore, the orientation of successive images of a same spot, for example successive views taken at different times can be adjusted by software, by comparing matching patterns in different orientations, as shown on Figure 9. A software module in the device 1 , or in a computer 66 receiving the images captured with the device, can be used for adapting the orientation of various images of a same portion of skin and facilitating their comparison. [0043] This is a way to calibrate the colours and to have systematically the same results independently to the screen or computer platform used. This allows making sure that comparing pictures of the same lesion at different time, a reference is available for a true comparison in order to detect correctly any colour changes. [0044] The software used to visualize the lesion will automatically adjust the colour and brightness based on the reference dot. This might also be used for automatic or human detection of significant patterns.
[0045] The device 1 may include a single use contact glass 2 with the contact gel and or Iiquid8 already on the glass. An example of single-use glass 2 with gel 8 on it is illustrated on Figure 3. In the example, a thin protection foil 1 1 is used to protect this contact gel and/or liquid layer 8 during transportation and storage of this contact glass in a packaging blister 10. The contact glass can be removed from the blister and mounted at the extremity of the device 1 , as illustrated on Figures 6, 12 and 13. The contact glass with the gel on it may be replaced after each image, or after each series of images captured from a patient. If the amount of liquid or gel on the contact glass is not sufficient, it might be possible to add more gel from a tank 22 comprised in the device. [0046] Alternatively, the device 1 might comprise a contact glass that will be used substantially during the whole life of the device 1 , without need to be replaced after each use.
[0047] In both embodiments, some gel or water needs to be applied on the skin for every new spot to check. In order to make the process faster, easy to use, and more reproducible, the device is equipped with a tank 22 for storing a plurality of portions of contact gel and/or liquid as shown on Fig. 4. When the device 20 is on contact with the skin, this tank could be actuated in order to dispense the right quantity of gel and/or water through a distribution canal 23 equipped with an aperture to make the dermatoscopy picture.
[0048] This tank 22 could be located on the external surface of the device body 5 in order to be manipulated by the device's user. The tank could be, for example, entirely or partially made of elastic materials. A simple pressure of the tank with at least one finger will distribute the required portion of contact gel and/or liquid from the tank on the skin portion.
[0049] Alternatively, the device could comprise a lever (not shown) whose movement actuates in a mechanical way the tank, delivering a more precisely defined portion of contact gel and/or liquid onto the skin and/or onto the contact glass.
[0050] Alternatively, the device could comprise an electro-mechanical actuator activated by a switch, a button and/or a roller to deliver a precise portion of gel or liquid onto the skin and/or onto the contact glass. [0051] The tank could be located within the device body 5.
[0052] A part of distribution canal 23 could be located on or within the hand held device body 5. [0053] A part of the distribution canal 23 could be located on or within the hand held device extender 3.
[0054] The tank may be provided with an overture and a lid for contact gel and/or liquid recharge purpose. [0055] In case of need, the tank could be made of a disposable contact gel and/or liquid cartridge for a safe, clean and rapid recharge.
[0056] The device could be equipped with distribution nozzle 21 to control the direction, the flow and the quantity of contact gel and/or liquid that is to spread on the skin and/or on the contact glass 2. [0057] Various types of nozzle could be employed, as for example, jet nozzle, spray nozzle and flat stream nozzle. Multiple nozzles of the same type or combination of various types could be used to improve the efficacy and regularity of the spreading operation.
[0058] In one embodiment, the gel or the liquid is applied first on the skin or on the contact glass of the dermatoscope, and then the device is applied and pressed on the lesion. This requires some practice and fast handling operation when the gel or liquid is applied on vertical skin portion.
[0059] Therefore, in another embodiment, a small canal is provided in the glass, or a groove 24 along the body of the device, that will help to bring the gel or liquid to the skin portion 33 (Fig. 5). In such case, the device is pressed on the skin lesion and then we push the liquid through. This embodiment is also more comfortable for the patient because of less liquid spreading and flowing all over. [0060] Using wet dermoscopy requires to apply the right pressure. If the pressure is too low, the epiluminescence is not appearing and more air bubbles might be seen. If the pressure is too high, the liquid might be faster flowing outside the area. If the pressure is really above a normal and reasonable pressure it can influence the blood vessel close by and change some aspect of the lesion. But usually after few shots only, it is easy to get it right. Nevertheless, one feature of the invention is a small pressure sensor 69 that will indicate to the user by a beep and / or small led and / or other visual indicator that the pressure is right or too high or too low.
[0061] Those indications can also be used in combination with an interactive web base platform to drive the user to the best and optimum image capture process.
[0062] The small pressure sensor could be mounted in order to measure the pressure between the contact glass 2 and the extender 3, or
alternatively, between the extender 3 and the body 5 of the hand held device.
[0063] The device may comprise a light pointer 31 shown on Figure7that projects a focused beam on the skin 30 where the centre of the image is. This help to centre small lesions 32 of few millimetres diameter only. The ray of light pointer can be with LED or laser. The power must be very low and should not risk any injuries to the eyes or the skin.
[0064] Just before the snap shot, the device 1 automatically switches off the ray of light pointer 32 and turns on a white LED, or a plurality of LEDS producing in combination a white light. The LED remains on only few seconds. The power of the LED is automatically adjusted to have the reference spots on the skin contact glass correctly lighting.
[0065] The device further comprises indication retrieving means of body area from which the acquired image is or will be captured. The purpose is to code the picture with the body area to make easier some offline software treatment at later stage.
[0066] This means could be automatically executed and/or manually driven. [0067] The indication retrieving means could comprise means for automatically matching the captured image with a previously captured image stored in the memory 68, and automatically or manually
determining the body area to which said previously captured image belongs. A signature from the captured image could be retrieved and compared with signatures already available and associated with a body area. This signature could depend on skin natural markers such as nevus 36, skin pores, skin lesions 33, skin grains, hairs34, buttons and/or veins (Fig. 9).
[0068] The indication retrieving means could further comprise means for stitching the acquired image with other stored images, and building an image of a larger body area.
[0069] The indication retrieving means could further comprise an inertial sensor in the hand-held device for determining the position of said hand-held device. [0070] Before to start shooting on some skin lesion, the patient could furthermore select manually the body area on which he or she has the lesion to be controlled.
[0071] An example of device with a menu based selection, body area selection roller 4 and display 40 is shown on Figures 4, 6, 7, 12 and 13. [0072] The device could further comprise a code bar reader exploiting the device sensor 61 . The body area could so manually selected capturing a code bar image indicating a particular body area. The code-bar could be a conventional code-bar, a QR-code or any other type of visual coding that can be used as identification with image analyzes. [0073] The body area can be identified such as right forearm, left forearm, neck, top hand right, etc. [0074] With detailed body map and multiple code bar or QR code or other, the localisation can be much more precisely identified (for instance with a number, such as for example ten, different zone on the left arm)
[0075] All the pictures captured are saved on an onboard memory 68, which could be a FRAM, and/or a removable SD cards for instance. Pictures stored in the memory 68 may be downloaded to a computer when the device 1 or the portable memory (SD cards) is plugged into a computer 66.A USB interface 63 could be used for this purpose. Pictures can also be transferred over a wireless interface, for example over wifi or Bluetooth or any wireless technology.
[0076] The device contains at least one battery64 that is rechargeable via USB port 63 or via a battery loader or loading station.
[0077] The device contains a real time clock 67 that allows to clearly identifying the time of capture of each picture. [0078] The device 1 contains all the necessary code to automatically open and connect to the web application, in order to transfer the images to a remote server for storage, comparison with previously captured images of the same skin portion, and/or analysis by a dermatologist or
automatically with a computer vision system for example. [0079] A coding of each picture is used throughout the web application. The device associates a unique code similar to what is described below to the picture:
ID: 1 12395657824 - 001 - 00002 - 0000001 - 2501 122102 - xxxxxx - xxxxxx - xxxxxx (Device - user - location - Picture number - date / time picture - flag diagnose - date diagnose - Visa). [0080] This code could be transferred along with the picture to a remote computer 66 or server where is can be used for classifying and retrieving the picture, and for facilitating the comparison with other images of the same spot at different times. The user identification part of the code can be an anonymous identification or an alias.
[0081] The images captured with the device can be processed locally in the device, for example in order to adjust their colour, contrast and brightness, possibly based on the reference marks. Moreover, a software module in the device might carry out computer vision steps for
automatically identifying images areas which might with some probability match a skin cancer or another skin lesion. An indication might be displayed to the user, for example a visual or audible indication, when a suspicious spot has been found on the skin.
[0082] The images might then be transferred to a further computer, for example over a wireless interface, a USB cable, a memory card and so on. The images might also be transferred to a remote computer over an
Internet connection. The computer receives the images with associated metadata, such as a picture code, and carries out further methods for processing the images, classifying them and for assisting the user or the dermatologist in detecting skin lesions and melanomas and/or automatic detection of suspicious images.
[0083] The device can be used with a real time interactive web base application for guidance, real time information and training and feedback.
[0084] The software in the device and/or in the computer might carry out an automatic detection of skin cancer with digital image analysis. The software might also assist the user or dermatologist and help him in the fast and reliable detection of melanomas. The specialist operator while checking and scrolling through hundreds of pictures for a pre diagnostics, is thus driven through all the steps with software assistance to highlight some area or characteristics when required, and to help him in verifying the images and detecting more quickly and in a more reliable way the most critical images.
[0085] For example, the software executed on the computer can detect that several pictures taken at different time correspond to a same spot. The comparison might be used on the date - it is unlikely that several pictures of same spot taken on the same day need to be compared -, on the body part identification, and on computer vision algorithms for matching successive views of a same spot when those views include similar group of pixels, or possible transformation from one view to the next. Fig. 10 shows two views of a same spot taken at two months intervals; the circled area 35 corresponds to similar group of pixels which can be used for automatic matching of views.
[0086] The software can also be used for facilitating manual matching of successive images, for example by presenting to the user a selection of images likely to correspond and that the user can manually match.
[0087] The software can also be used for automatic detection of areas that have changed between successive pictures, and automatic marking of those areas, or automatic marking of series of pictures in which significant changes were observed. [0088] The software might detect patterns or image elements
representative of different types of lesions. Skin cancer such as melanomas can have various levels of the development characterized by changing aspects and various patterns. The software might comprise routines or modules for detecting image portions representative of some of those levels. For example, a module can be executed for detecting and
highlighting pictures having blue gray globules which are sometimes hard to see but very specific to carcinoma at level 2. The software can highlight the pictures having those areas, and in those pictures highlight the area that matches the characteristics of the blue gray globules. At level 6, the software might highlight the group of pixels that have similar colour of blood vessels and organized like blood vessel. [0089] The process might also comprise a step of indicating of detecting the skin type of the patient. A skin type might be associated to a series of pictures taken during a session. The skin type might be included in the image coding. The skin type might be determined from a picture taken at a specific location, for example a location without melanomas. An example of image which can be used for automatic and/or manual determination of skin type is shown on Figure 10. The skin type might be indicated as a skin index type and might depend on the skin colour.
List of elements used in the drawings
1 : Hand held device for capturing images of skin portion
2: Contact glass
3: Extender
4: Roller for data selection
5: Body of the hand held device
6: Image calibration marker
7: Acquiring button
8: Contact gel and/or liquid
9: Field-of-view of the image sensor
10: Contact glass packaging blister
1 1 : Thin protection foil
20: Hand held device for capturing images of skin portion
21 : Contact gel and/or liquid distribution nozzle
22: Contact gel and/or liquid tank
23: Contact gel and/or liquid distribution canal
24: Contact gel and/or liquid distribution canal or groove within the contact glass
30: Skin surface
31 : Ray light focusing point
32: Skin portion of interest
33: Skin anomalies
34: Hair
35: Similar areas between two successive pictures of a same spot
36: Nevus
40: LCD display
60: Lighting system
61 : Image sensor
62: Microprocessor
63: USB interface
64: Battery pack
65: USB cable : Personal Computer: Real time clock: Memory
: Pressure sensor: Lens

Claims

Claims
1 . A hand held device (20) for capturing images of skin portions (32) for dermatoscopy analysis, comprising:
an image sensor (61 ) for capturing said image;
a lens (70) arranged for projecting a focused image of the skin on the sensor when the device is in direct contact with the skin;
a memory (68) for storing the captured digital image;
characterized by
a tank (22) for storing a plurality of portions of contact gel and/or liquid.
2. The device of claim 1 , wherein said hand held device comprises at least one distribution canal (23) and at least one contact gel and/or liquid distribution nozzle (21 ).
3. The device of claim 2, wherein said distribution nozzle is a jet nozzle, a spray nozzle and/or flat stream nozzle.
4. The device of one of the claims 1 to 3, wherein said hand held device comprises at least one distribution canal and/or a groove (23) for said contact gel or liquid.
5. The device of one of the claims 1 to 4, wherein said contact glass (2) comprises a canal and/or a groove (24) for said contact gel or liquid.
6. The device of one of the claims 1 to 5, wherein said tank is located on the body (5) external surface of said hand held device in order to be manipulated by the device's user.
7. The device of the claim 6, wherein said tank is entirely or partially made of elastic materials and wherein a press of said tank distributes a portion of contact gel and/or liquid that is required to take a dermatoscopy image.
8. The device of the claim 6, wherein said tank comprises a lever whose movement actuates mechanically said tank so that a portion of contact gel and/or liquid that is distributed as required for taking a dermatoscopy image.
9. The device of one of the claims 2 to 5, further comprising a switch, a button and/or a roller to electro-mechanically actuate said tank for distributing a portion of contact gel and/or liquid that is required to take a dermatoscopy image.
10. The device of claim9, wherein said tank is located within said hand held device.
1 1 .The device of one of the claims 1 to 10, wherein said tank comprises a disposable gel and/or liquid cartridge.
12. The device of one the claims 1 to1 1 , further comprising a contact glass (2) for direct contact with the skin.
13. The device of one of the claims 1 to12, wherein said contact glass is a disposable single-use contact glass. .
14. The device of claim13, wherein said single-use contact glass is associated with a contact gel and/or liquid already on the contact glass (8).
1 5. The device of one of the claims 13 to14, wherein said distribution nozzle (21 ) and/or distribution canal (23)is arranged for spreading said contact gel and/or liquid onto the outside surface of said contact glass.
16. The device of one of the claims 1 to 1 5, wherein said hand held device comprises at least one pressure sensor (69).
17. The device of one of the claims 1 to 16, wherein said hand held device comprises at least one reference marker on a glass for including a reference mark in said captured digital image.
18. The device of claim 17, said marker being a colour marker (6).
19. The device of claim 18, comprising colour adjusting software for automatically adjusting the colour of the captured image on the basis of the colour of said marker.
20. The device of one of the claims 1 to 19, further comprising a lighting system (60) for lighting the portion of skin during the capture of the image.
21 .The device of one of the claims 1 to 20, further comprising a light pointer (31 ) for projecting a light beam on the centre of portion of skin from which an image should be captured.
22. The device of claim 21 , further comprising means for automatically switching said light off during the capture of the image.
23. The device of one of the claims 1 to 22, further comprising indication retrieving means of body area from which said captured image is captured.
24. The device of claim 23, wherein said indication retrieving means comprises means for automatically matching said image with a previously captured image, and automatically or manually determining the body area to which said previously captured image belongs.
25. The device of claim 24, wherein said indication retrieving means comprises means for retrieving a signature from said captured image and comparing said signature with already available signatures associated with a body area.
26. The device of claim 25, wherein said signature depends on skin natural markers such as nevus, skin pores, skin lesions, skin grains, hairs, buttons and/or veins.
27. The device of one of the claims 23 to 26, wherein said indication retrieving means comprises means for stitching said image with other images, and building an image of a larger body area.
28. The device of one of the claims 23 to 27, wherein said indication retrieving means comprises an inertial sensor in said hand-held device for determining the position of said hand-held device.
29. The device of one of the claims 23 to 28, further comprising data selection means for indicating the body area from which an image is taken.
30. The device of claim 29, wherein the selection is made on the hand-held device using a roller (4) and a display (40).
31 .The device of one of the claims 23 to 30, said sensor (61 ) being arranged to work as a code-bar reader for determining said body area from a captured code-bar image.
32. The device of one of the claims 24 to31 , further comprising software means for including an indication of body area as metadata in the corresponding captured image.
33. The device of one of the claims 24 to 32, further comprising software means for automatic identification of image area containing skin lesions.
34. A system for capturing and analysing images of skin portions for dermatoscopy analysis, comprising
a hand held device (20) according to one of the claims 1 to 33;
a computer (66) for receiving images captured with said hand held device, said computer having program means for analysing said images.
35. The system of claim 34, said program means comprising colour adjusting software for automatically adjusting the colour of the displayed image on the basis of the colour of said marker (6).
36. The system of one of the claims34or 35, said program means comprising image orientation software for automatically adjusting the orientation of successive images of a same sport on the basis of the position of said markers.
37. The system of one of the claims34to 36, said program means comprising computer vision means for analysing said images and assisting in the detection of skin lesion or skin cancer such as melanomas.
38. The system of one of the claims 34to 37, said program means comprising computer vision means for identifying successive images of a same portion of the skin.
PCT/EP2013/056479 2012-02-11 2013-03-27 Hand held device and method for capturing images of skin portions WO2013144184A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CH4292012 2012-02-11
CH0429/12 2012-03-27

Publications (2)

Publication Number Publication Date
WO2013144184A2 true WO2013144184A2 (en) 2013-10-03
WO2013144184A3 WO2013144184A3 (en) 2013-12-19

Family

ID=47721882

Family Applications (2)

Application Number Title Priority Date Filing Date
PCT/EP2013/056479 WO2013144184A2 (en) 2012-02-11 2013-03-27 Hand held device and method for capturing images of skin portions
PCT/EP2013/056481 WO2013144186A1 (en) 2012-02-11 2013-03-27 Hand held device and method for capturing images of skin portions

Family Applications After (1)

Application Number Title Priority Date Filing Date
PCT/EP2013/056481 WO2013144186A1 (en) 2012-02-11 2013-03-27 Hand held device and method for capturing images of skin portions

Country Status (1)

Country Link
WO (2) WO2013144184A2 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104490378A (en) * 2015-01-10 2015-04-08 钛脉科技(北京)有限公司 Remote medical care system with skin recognition function based on smart television
CN104490377A (en) * 2015-01-10 2015-04-08 钛脉科技(北京)有限公司 Remote medical care system with skin recognition function based on tablet computer
CN104523255A (en) * 2015-01-10 2015-04-22 钛脉科技(北京)有限公司 Tablet-computer-based long-distance medical care system with behavior monitoring function
US10674953B2 (en) 2016-04-20 2020-06-09 Welch Allyn, Inc. Skin feature imaging system
WO2024042451A1 (en) * 2022-08-22 2024-02-29 Lumenis Be Ltd. Apparatus and method for sensing and analyzing skin condition

Families Citing this family (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10460076B2 (en) * 2013-08-16 2019-10-29 Sohi, Llc Superimposed verified photo with order or request
US9924875B2 (en) 2014-06-13 2018-03-27 The Procter & Gamble Company Apparatus and methods for modifying keratinous surfaces
KR101988846B1 (en) 2014-06-13 2019-06-13 더 프록터 앤드 갬블 캄파니 Cartridges for the deposition of treatment compositions on keratinous surfaces
CA3024544C (en) 2014-06-13 2020-06-23 The Procter & Gamble Company Apparatus and methods for modifying keratinous surfaces
CA2949129C (en) 2014-06-13 2020-06-30 The Procter & Gamble Company Apparatus and methods for modifying keratinous surfaces
US10188193B2 (en) 2014-07-25 2019-01-29 The Procter & Gamble Company Applicator heads for handheld treatment apparatus for modifying keratinous surfaces
US9949552B2 (en) * 2014-07-25 2018-04-24 The Procter & Gamble Company Handheld treatment apparatus for modifying keratinous surfaces
US20160022009A1 (en) * 2014-07-25 2016-01-28 The Procter & Gamble Company Handheld treatment apparatus for modifying keratinous surfaces
US9955769B2 (en) 2014-07-25 2018-05-01 The Procter & Gamble Company Applicator heads for handheld treatment apparatus for modifying keratinous surfaces
US11083880B2 (en) 2014-07-25 2021-08-10 The Procter & Gamble Company Angled cartridge assembly for a dispensing device
US10314378B2 (en) 2014-07-25 2019-06-11 The Procter & Gamble Company Cartridge assembly for a dispensing device
DE102014116442A1 (en) * 2014-11-11 2016-05-12 Rico Unger Imaging system for analytical purposes
EP3286695A1 (en) * 2015-04-24 2018-02-28 Canfield Scientific, Incorporated Dermatological feature tracking over multiple images
PL3307142T3 (en) * 2015-06-10 2021-10-04 Tyto Care Ltd. Apparatus and method for inspecting skin lesions
US11116302B2 (en) 2015-06-11 2021-09-14 The Procter & Gamble Company Apparatus and methods for modifying keratinous surfaces
CN106994002B (en) * 2017-04-12 2024-05-24 广州创弘医疗科技有限公司 Electronic skin mirror device, operation interface for same and switching method
EP3685786A1 (en) * 2019-01-24 2020-07-29 Koninklijke Philips N.V. A method of determining a position and/or orientation of a hand-held device with respect to a subject, a corresponding apparatus and a computer program product

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002080771A1 (en) 2001-04-03 2002-10-17 Polartechnics Limited Disposable calibrated camera for skin lesion examination
US20040092802A1 (en) 2000-07-07 2004-05-13 Cane Michael Roger Epithelial diagnostic aid
US20120172685A1 (en) 2011-01-04 2012-07-05 Ergylink System for analysing the skin and associated method

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2364376A (en) * 2000-07-05 2002-01-23 Astron Clinica Ltd Skin illumination and examination apparatus
AUPR509801A0 (en) * 2001-05-18 2001-06-14 Polartechnics Limited Boundary finding in dermatological examination
NZ556655A (en) * 2005-01-19 2010-10-29 Dermaspect Llc Devices and methods for identifying and monitoring changes of a suspect area on a patient
US8622904B2 (en) * 2009-07-28 2014-01-07 Lisa Jernigan Use of heat sensitive color changing formula to detect and prevent the onset of decubitus ulcers

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040092802A1 (en) 2000-07-07 2004-05-13 Cane Michael Roger Epithelial diagnostic aid
WO2002080771A1 (en) 2001-04-03 2002-10-17 Polartechnics Limited Disposable calibrated camera for skin lesion examination
US20120172685A1 (en) 2011-01-04 2012-07-05 Ergylink System for analysing the skin and associated method

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104490378A (en) * 2015-01-10 2015-04-08 钛脉科技(北京)有限公司 Remote medical care system with skin recognition function based on smart television
CN104490377A (en) * 2015-01-10 2015-04-08 钛脉科技(北京)有限公司 Remote medical care system with skin recognition function based on tablet computer
CN104523255A (en) * 2015-01-10 2015-04-22 钛脉科技(北京)有限公司 Tablet-computer-based long-distance medical care system with behavior monitoring function
US10674953B2 (en) 2016-04-20 2020-06-09 Welch Allyn, Inc. Skin feature imaging system
US11382558B2 (en) 2016-04-20 2022-07-12 Welch Allyn, Inc. Skin feature imaging system
WO2024042451A1 (en) * 2022-08-22 2024-02-29 Lumenis Be Ltd. Apparatus and method for sensing and analyzing skin condition

Also Published As

Publication number Publication date
WO2013144184A3 (en) 2013-12-19
WO2013144186A1 (en) 2013-10-03

Similar Documents

Publication Publication Date Title
WO2013144184A2 (en) Hand held device and method for capturing images of skin portions
US9782078B2 (en) Device, system and method for blood vessel imaging and marking
JP4455216B2 (en) Detection device
US9696897B2 (en) Image-based measurement tools
US20190273890A1 (en) Devices and methods for identifying and monitoring changes of a suspect area of a patient
US20220084659A1 (en) Diagnostic test kits and methods of analyzing the same
EP2957223B1 (en) Skin and wound assessment tool
ES2555992T3 (en) System and procedure to analyze at least one characteristic of the skin
US20120206587A1 (en) System and method for scanning a human body
US20070038118A1 (en) Subcutaneous tissue imager
US20150182757A1 (en) A method and system for cosmetic skin procedures for home use
CN106061386A (en) Wearable electronic device for enhancing visualization during insertion of an invasive device
CN101442932A (en) Systems and methods for wound area management
WO2012051394A1 (en) Methods and apparatus for imaging detecting, and monitoring surficial and subdermal inflammation
CN101442970A (en) Systems and methods for wound area management
CN107920739A (en) Apparatus and method for checking skin injury
KR102036040B1 (en) Diagnosis Device of optical skin disease
AU2005281546B2 (en) Miniature device for analysing physico-chemical properties of the skin
JP5320111B2 (en) Blood component measuring device
JP2000201894A (en) Optometric instrument and iris information obtaining system using this
KR20240017895A (en) In vitro diagnostic device for biometric information analysis based on imaging
KR20230161304A (en) In vitro diagnostic device for biometric information analysis based on imaging
CN117651517A (en) Processing device, processing program, processing method, and processing system
KR20200058764A (en) Automatic data storage system for breast measurement data

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13713412

Country of ref document: EP

Kind code of ref document: A2

122 Ep: pct application non-entry in european phase

Ref document number: 13713412

Country of ref document: EP

Kind code of ref document: A2