CN112288781A - Image registration method, apparatus and computer program product - Google Patents

Image registration method, apparatus and computer program product Download PDF

Info

Publication number
CN112288781A
CN112288781A CN201810963040.4A CN201810963040A CN112288781A CN 112288781 A CN112288781 A CN 112288781A CN 201810963040 A CN201810963040 A CN 201810963040A CN 112288781 A CN112288781 A CN 112288781A
Authority
CN
China
Prior art keywords
image
pixel
registered
candidate
value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201810963040.4A
Other languages
Chinese (zh)
Other versions
CN112288781B (en
Inventor
李林森
徐伟彬
金欢
姜泽飞
周志良
颜钦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Genemind Biosciences Co Ltd
Original Assignee
Genemind Biosciences Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Genemind Biosciences Co Ltd filed Critical Genemind Biosciences Co Ltd
Priority to CN201810963040.4A priority Critical patent/CN112288781B/en
Publication of CN112288781A publication Critical patent/CN112288781A/en
Application granted granted Critical
Publication of CN112288781B publication Critical patent/CN112288781B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses an image registration method, an image registration device and a computer program product. The image registration method comprises: performing first registration on an image to be registered based on a reference image, wherein the first registration comprises the steps of determining a first offset of a preset area on the image to be registered and a corresponding preset area on the reference image, and moving all bright spots on the image to be registered based on the first offset; and performing second registration on the first registered image to be registered based on the reference image, wherein the second registration comprises merging the first registered image to be registered and the reference image, calculating the offset of all overlapped bright spots of a preset area on the merged image to determine a second offset, taking two or more bright spots with the distance smaller than a preset pixel as one overlapped bright spot, and moving all the bright spots on the first registered image to be registered based on the second offset to realize the registration of the image to be registered. The method can realize high-precision image registration and meet the situation of high-precision requirement on image rectification.

Description

Image registration method, apparatus and computer program product
Technical Field
The present invention relates to the field of image processing, and in particular, to an image registration method, an image registration apparatus, and a computer program product having an image registration function.
Background
In applications involving multiple acquisitions of images of the same object or objects at different times, it is generally necessary to de-skew/register the acquired images so that the change information of the object can be accurately obtained based on the de-skewed images.
In a platform including a nucleic acid sequence measurement using an image obtained from a nucleic acid molecule, it is generally necessary to acquire images of the nucleic acid molecule in the same field of view at different times while moving hardware, and from a plurality of images taken at different times including information in an identification image, sequence information of the nucleic acid molecule in the field of view can be specified. In actual image acquisition, since hardware movement has a certain precision, that is, a certain error exists between the specified movement amount and the actual movement amount, and/or due to changes in the form of the nucleic acid molecule and the like caused by changes in the environment/system in which the nucleic acid molecule is located, positional information of the immobilized nucleic acid molecule in the obtained images of the field of view at a plurality of times may be different, making it difficult to accurately identify and determine the sequence of the nucleic acid molecule directly using the obtained image information.
Therefore, the method for rectifying deviation of a plurality of images of the same object or a plurality of images of the same object acquired at different times is yet to be further developed or improved.
Disclosure of Invention
Embodiments of the present invention are directed to solving at least one of the technical problems occurring in the related art or at least providing an alternative practical solution.
According to an embodiment of the present invention, there is provided an image registration method including: performing first registration on an image to be registered based on a reference image, wherein the reference image and the image to be registered correspond to the same object, the reference image and the image to be registered both comprise a plurality of bright spots, the first registration comprises determining a first offset of a predetermined area on the image to be registered and a corresponding predetermined area on the reference image, and moving all the bright spots on the image to be registered based on the first offset to obtain a first registered image to be registered; and carrying out second registration on the first registered image to be registered based on the reference image, wherein the second registration comprises merging the first registered image to be registered and the reference image to obtain a merged image, calculating the offset of all overlapped bright spots of a preset area on the merged image to determine a second offset, taking two or more bright spots with the distance smaller than a preset pixel as one overlapped bright spot, and moving all the bright spots on the first registered image to be registered based on the second offset to realize the registration of the image to be registered.
According to another embodiment of the present invention, there is provided an image registration apparatus for implementing the image registration method in the above-described embodiment of the present invention, the apparatus including: the first registration module is used for performing first registration on the image to be registered based on the reference image, and comprises the steps of determining a first offset of a preset area on the image to be registered and a corresponding preset area on the reference image, moving all bright spots on the image to be registered based on the first offset, and obtaining the image to be registered after the first registration, wherein the reference image and the image to be registered correspond to the same object, and both the reference image and the image to be registered comprise a plurality of bright spots; and the second registration module is used for carrying out second registration on the first registered image to be registered from the first registration module based on the reference image, and comprises the steps of merging the first registered image to be registered and the reference image to obtain a merged image, calculating the offset of all overlapped bright spots of a preset area on the merged image to determine a second offset, defining two or more bright spots with the distance smaller than a preset pixel as one overlapped bright spot, and moving all the bright spots on the first registered image to be registered based on the second offset to realize the registration of the image to be registered.
According to still another embodiment of the present invention, there is provided a computer-readable storage medium storing a program for execution by a computer, the execution of the program including performing the image registration method in any of the above embodiments. Computer-readable storage media include, but are not limited to, read-only memory, random-access memory, magnetic or optical disks, and the like.
There is also provided, in accordance with an embodiment of the present invention, a terminal, a computer program product, including instructions which, when the program is executed by a computer, cause the computer to perform all or part of the steps of the image registration method in the above-described embodiment of the present invention.
By using the image registration method and device and/or the terminal/computer program product for realizing image registration in the embodiment of the invention, high-precision image rectification can be realized, and the method and device are particularly suitable for scenes with high-precision image rectification requirements.
Additional aspects and advantages of embodiments of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of embodiments of the invention.
Drawings
Fig. 1 is a flowchart illustrating an image registration method according to an embodiment of the present invention.
FIG. 2 is a schematic diagram of an image rectification process and a rectification result according to an embodiment of the present invention.
Fig. 3 is a flowchart illustrating an image registration method according to an embodiment of the present invention.
FIG. 4 is a diagram of a matrix corresponding to candidate hot spots and associated pixels in accordance with an embodiment of the present invention.
Fig. 5 is a flowchart illustrating an image registration method according to an embodiment of the present invention.
Fig. 6 is a schematic diagram of pixel values in a range of m1 × m2 centered on a central pixel point of the pixel point matrix according to the embodiment of the present invention.
Fig. 7 is a schematic diagram illustrating comparison between bright spot detection results before and after the determination according to the second bright spot detection threshold in the embodiment of the present invention.
Fig. 8 is a schematic diagram of an image registration apparatus in an embodiment of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.
In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any order or number of technical features indicated. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.
The embodiment of the invention provides an image registration method, namely an image rectification method, as shown in fig. 1, comprising: s10, performing first registration on the image to be registered based on the reference image, wherein the reference image and the image to be registered correspond to the same object, both the reference image and the image to be registered contain a plurality of bright spots, the first registration comprises determining a first offset of a predetermined area on the image to be registered and a corresponding predetermined area on the reference image, and moving all the bright spots on the image to be registered based on the first offset to obtain a first registered image to be registered; s20, performing second registration on the first registered image to be registered based on the reference image, including merging the first registered image to be registered and the reference image to obtain a merged image, calculating the offset of all overlapped bright spots of a predetermined area on the merged image to determine a second offset, wherein two or more bright spots with the distance smaller than a predetermined pixel are one overlapped bright spot, and moving all the bright spots on the first registered image to be registered based on the second offset to realize the registration of the image to be registered. The method can be relatively called coarse registration and fine registration through two times of associated registration, comprises the step of performing fine registration by using bright spots on an image, can quickly realize high-precision deviation correction of the image based on a small amount of data information, and is particularly suitable for scenes with high-precision image deviation correction requirements. For example, single molecule-level image detection, such as images of sequencing reactions from third generation sequencing platforms. The term single molecule scale refers to a size with resolution of a single or a few molecules, e.g. 10, 8, 5, 4 or less than 3 molecules.
The term "bright spots" (also referred to as "spots", spots or peaks) refers to light-emitting points on an image, one light-emitting point occupying at least one pixel. So called "pixel point" is the same as "pixel".
In certain embodiments, the image to be registered is from a sequencing platform that utilizes optical imaging principles for sequence determination. The term sequencing, also known as sequencing, refers to nucleic acid sequencing, including DNA sequencing and/or RNA sequencing, including long-fragment sequencing and/or short-fragment sequencing, and sequencing a biochemical reaction including extension of a base. Sequencing can be carried out by a sequencing platform, and the sequencing platform can be selected from but not limited to Hisq/Miseq/Nextseq sequencing platform of Illumina, Ion Torrent platform of Thermo Fisher/Life Technologies, BGISEQ platform of Huada gene and single-molecule sequencing platform; the sequencing mode can select single-ended sequencing or double-ended sequencing; the sequencing results/data obtained, i.e.the fragments read by the assay, are called reads (reads), the length of which is called read length. The so-called "bright spots" correspond to the optical signal of an extended base or base cluster.
The predetermined area on the image may be the entire image or a part of the image. In one example, the predetermined region on the image is a portion of the image, such as a 512 x 512 region in the center of the image. The center of the image is the center of the field of view, the intersection point of the optical axis of the imaging system and the imaging plane can be referred to as the image center point, and the region centered on the center point can be regarded as the image center region.
In some embodiments, the image to be registered is from a nucleic acid sequencing platform, which includes an imaging system and a nucleic acid sample carrying system, the nucleic acid molecules to be detected with optical detection marks are fixed in a reactor, which is also called a chip, and the chip is loaded on a movable stage, and the movable stage drives the chip to move to realize image acquisition of the nucleic acid molecules to be detected at different positions (different fields of view) of the chip. Generally, there are precision limitations on the movement of the optical system and/or the mobile station, for example, there are deviations between the position to which the command specifies the movement and the position to which the actual movement of the mechanical structure is to be made, especially in application scenarios with high precision requirements, whereby, in the process of moving hardware according to the command to perform multiple image acquisitions of the same position (field of view) at different time points, it is difficult to completely align the multiple images of the same field of view acquired at different time points, and the images are aligned in a de-skewing manner, which is advantageous for accurately determining the nucleotide sequence of the nucleic acid molecule based on the change of information in the multiple images acquired at the multiple time points.
In some embodiments, the reference image is obtained by construction, and the reference image may be constructed during registration of the image to be registered, or may be pre-constructed and recalled as needed for storage.
In some examples, constructing the reference image includes: acquiring a first image and a second image, wherein the first image and the second image correspond to the same object as the image to be registered; performing coarse registration on the second image based on the first image, wherein the coarse registration comprises the steps of determining the offset between the second image and the first image, and moving the second image based on the offset to obtain a second image after the coarse registration; and combining the first image and the second image after coarse registration to obtain a reference image, wherein the first image and the second image both comprise a plurality of bright spots. Therefore, the image containing more or relatively more complete information is obtained by construction, and the image is used as the deviation rectifying reference, so that more accurate image registration is favorably realized. For the image obtained by the nucleic acid sequence determination, a plurality of images are utilized to construct a reference image, which is beneficial to enabling the reference image to obtain complete speckle information of corresponding nucleic acid molecules and is beneficial to image rectification based on the speckle.
In some embodiments, the first image and the second image are from the same field of view at different times of a nucleic acid sequencing reaction (sequencing reaction). Here, one extension to achieve the four types of bases A/U, T, G and C is defined as one round of sequencing reaction. In one example, a round of sequencing reaction includes multiple base extension reactions, such as monochromatic sequencing, using reaction substrates (nucleotide analogs) corresponding to four types of bases all with the same fluorescent dye, and a round of sequencing reaction includes four base extension reactions (4repeats), where for one field of view, one base extension reaction includes one image capture, and the first image and the second image are the same field of view of different base extension reactions. Therefore, the reference image obtained by processing and collecting the information of the first image and the second image is used as the basis for deviation correction, and more accurate image deviation correction is facilitated.
In another example, a single-molecule two-color sequencing reaction uses two of the reaction substrates (nucleotide analogs) corresponding to four types of bases with one fluorescent dye and two fluorescent dyes with different excitation wavelengths, one round of sequencing reaction includes two base extension reactions, two types of base reaction substrates with different dyes perform a binding reaction in one base extension reaction, one base extension reaction includes two image acquisitions at different excitation wavelengths for one field of view, and the first image and the second image are respectively from different base extension reactions or the same field of view at different excitation wavelengths in the same base extension reaction. Therefore, the reference image obtained by processing and collecting the information of the first image and the second image is used as the basis for deviation correction, and more accurate image deviation correction is facilitated.
In yet another example, a round of sequencing reactions includes a single base extension reaction, such as a two-color sequencing reaction of a second generation sequencing platform, with four types of base reaction substrates (e.g., nucleotide analogs) with dye a, dye b, dye a and dye b, and without any dye, respectively, the excitation wavelengths of dye a and dye b being different; the four types of reaction substrates realize one round of sequencing reaction in the same base extension reaction, and the first image and the second image are respectively from different rounds of sequencing reactions or the same field of view under different excitation wavelengths in the same round of sequencing reactions. Therefore, the reference image obtained by processing and collecting the information of the first image and the second image is used as the basis for deviation correction, and more accurate image deviation correction is facilitated.
The first image and/or the second image may be one image or a plurality of images. Further, in some embodiments, the method further includes constructing a reference image by using the third image and the fourth image, the image to be aligned, the first image, the second image, the third image and the fourth image are from the same field of view of the sequencing reaction, the first image, the second image, the third image and the fourth image respectively correspond to fields of view of the four types of base extension reactions, a plurality of nucleic acid molecules with optically detectable labels exist in the fields of view during the base extension reactions, at least a part of the nucleic acid molecules are represented as bright spots on the images, and constructing the reference image further includes: performing coarse registration on the third image based on the first image, wherein the coarse registration comprises determining the offset of the third image and the first image, and moving the third image based on the offset to obtain a coarsely registered third image; performing coarse registration on the fourth image based on the first image, wherein the coarse registration comprises determining the offset of the fourth image and the first image, and moving the fourth image based on the offset to obtain a coarsely registered fourth image; and merging the first image and the second image after coarse registration, the third image after coarse registration and the fourth image after coarse registration to obtain a reference image.
In S10, the implementation of the first registration is not limited, and the first offset may be determined using frequency domain registration using fourier transform, for example. Specifically, the first shift amount, the shift amount of the second image and the first image, the shift amount of the third image and the first image, and/or the shift amount of the fourth image and the first image may be determined by, for example, two-dimensional discrete fourier transform in a pure Phase Correlation Function (Phase-Only Correlation Function) in Kenji TAKITA et al, ice trans. The first registration/coarse registration may achieve a 1-pixel (1pixel) accuracy. In this way, the first offset can be determined quickly and accurately and/or a reference image favorable for accurate rectification can be constructed.
In some embodiments, the reference image and the image to be registered are binarized images. Therefore, the method is favorable for reducing the calculation amount and quickly rectifying the deviation.
In one example, both the image to be rectified and the reference image are binarized images, that is, each pixel in the image is not a, that is, b, for example, a is 1, b is 0, and the pixel mark is 1 and is brighter than the pixel mark is 0, or has greater intensity; here, it is defined that the extension of a nucleic acid molecule to be tested by one base or one base in the nucleic acid sequencing process is called one cycle (cycle), the reference image is constructed by using the images of cycles 1-4 of rounds 1-4, and the first image and the second image are selected from any one, two or three of the images of cycles 1-4.
In one example, the first image is an image cycle1, the image cycle2-4 is a second image, and the image cycle2-4 is subjected to coarse registration in sequence based on the image cycle1 to respectively obtain a coarsely registered image cycle 2-4; the image cycle1 and the coarsely registered image cycle2-4 are merged to obtain a reference image. The merged image is referred to as a coincident bright spot in the merged image. Two bright spots on two images that are not more than 1.5 pixels apart are set as coincident bright spots in one example, based primarily on the size of the bright spots of the corresponding nucleic acid molecules and the imaging system resolution. The central area of the synthesized image with 4 cycles is used as a reference image, so that the reference image has a sufficient amount of bright spots and subsequent registration is facilitated, the bright spots in the central area of the image are detected and positioned, the bright spot information is relatively more accurate, and accurate registration is facilitated.
In one example, the following steps are performed to deskew an image: 1) roughly rectifying the deviation of a certain view field of an image ring 5 collected from a fifth round of reaction, wherein the ring 5 is a binarized image, taking a center 512 × 512 region of the image, for example, and a center image (the center 512 × 512 region of a corresponding reference image) synthesized with rings 1-4, carrying out two-dimensional discrete Fourier transform, and obtaining an offset (x0, y0) by using frequency domain registration, namely, realizing rough image registration, wherein x0 and y0 can reach the precision of 1 pixel; 2) combining (merge) the roughly registered image and the reference image based on the bright spots on the image, including calculating an offset (x1, y1) of the overlapped bright spots in the central area of the loop 5 image and the corresponding area of the reference image, which is the coordinate position of the bright spot of the image to be rectified-the coordinate position of the corresponding bright spot on the reference image, which can be expressed as offset (x1, y1) which is currcyclepoints-basePoints; the average offset of all the superimposed patches is found to give a fine offset in the range of [0,0] to [1,1 ]. In one example, two bright spots on two images with a distance of no more than 1.5 pixels are set as coincident bright spots; 3) in summary, the offsets (x0, y0) - (x1, y1) of different cycles of a visual field image (fov) are obtained, and can be expressed as: currcyclepoints + (x0, y0) - (x1, y1) which represent the original coordinates of the bright spot, i.e. the coordinates in the image before rectification. The deviation rectifying result obtained by the image deviation rectifying has higher accuracy, and the deviation rectifying precision is less than or equal to 0.1 pixel. Fig. 2 illustrates a rectification process and a result, in fig. 2, an image C is rectified based on an image a, circles in the image a and the image C represent bright spots, bright spots marked by the same number are overlapped bright spots, and an image C- > a represents a rectification result, that is, a result of aligning the image C to the image a.
In some embodiments, referring to fig. 3, the image registration method further includes S01 identifying a hot spot, including performing hot spot detection on the image by using a k1 x k2 matrix, determining that a matrix with a central pixel value not less than any non-central pixel value of the matrix corresponds to a candidate hot spot, and determining whether the candidate hot spot is a hot spot, where both k1 and k2 are odd numbers greater than 1, and the k1 x k2 matrix includes k1 x k2 pixels. The image is selected from at least one of the images to be registered, the images constituting the reference image. By using the method to detect the bright spots on the image, the detection of the bright spots (spots or peaks) on the image can be quickly and effectively realized, and particularly the image collected from the nucleic acid sequence determination reaction can be detected. The method has no special limitation on the image to be detected, namely the original input data, is suitable for processing and analyzing the image generated by any platform for carrying out nucleic acid sequence determination by using the optical detection principle, including but not limited to second generation and third generation sequencing, has the characteristics of high accuracy and high efficiency, and can acquire more information representing the sequence from the image. Especially for random images and signal recognition with high accuracy requirements.
In some embodiments, the image is from a nucleic acid sequencing reaction, the nucleic acid molecule has an optically detectable label, such as a fluorescent label, and the fluorescent molecule is capable of being excited to fluoresce when illuminated with a laser of a particular wavelength, and the image is acquired by an imaging system. The acquired image includes a spot of light/bright spot that may correspond to the location of the fluorescent molecule. Understandably, when the image is at the focal plane position, the size of the bright spot corresponding to the position of the fluorescent molecule in the acquired image is small and the brightness is high; when the fluorescent light source is located at the non-focal surface position, the size of a bright spot corresponding to the position of the fluorescent molecules in the acquired image is larger and the brightness is lower. In addition, other non-target or subsequently difficult to utilize substances/information may be present in the field of view, such as impurities and the like; further, in photographing a single-molecule field of view, a large amount of molecular aggregation (cluster) and the like may also interfere with the target single-molecule information acquisition. A single molecule is said to be a few molecules, for example no more than 10 molecules, for example one, two, three, four, five, six, eight or ten molecules.
In some examples, a center pixel value of the matrix is greater than a first preset value, any pixel value not in the center of the matrix is greater than a second preset value, and the first preset value and the second preset value are related to an average pixel value of the image.
In some embodiments, the image may be subjected to traversal detection using a k1 × k2 matrix, the set of first and/or second preset values being related to the average pixel value of the image. For a grayscale image, the pixel values are the same as the grayscale values. k1 × k2 matrix, k1 and k2 may be equal or unequal. In one example, the imaging system related parameters are: the objective lens is 60 times, the size of the electronic sensor is 6.5 μm, the minimum size of the image formed by the microscope is 0.1 μm, the obtained image or the input image can be a 16-bit gray scale or color image of 512 × 512, 1024 × 1024 or 2048 × 2048, and the value ranges of k1 and k2 are both more than 1 and less than 10. In one example, k1 ═ k2 ═ 3; in another example, k 1-k 2-5. If the image is a color image, one pixel point of the color image has three pixel values, the color image can be converted into a gray image, and then bright spot detection is carried out, so that the calculated amount and the complexity of the image detection process are reduced. The non-grayscale image may be optionally, but not limited to, converted to a grayscale image using a floating-point algorithm, an integer method, a shift method, or an average value method, etc.
In one example, the inventors can obtain the bright spot detection result from the optical detection mark by counting a large amount of image processing, and taking the first preset value as 1.4 times and the second preset value as 1.1 times as large as the average pixel value of the image, so as to eliminate interference.
The size, the similarity degree and/or the strength with the ideal bright spots can be used for further screening judgment of the candidate bright spots. In some embodiments, the size of the candidate bright spots on the comparison image is quantitatively reflected by the size of the connected domain corresponding to the candidate bright spots, so as to screen and judge whether the candidate bright spots are the wanted bright spots.
In one example, determining whether the candidate hot spot is a hot spot comprises: and calculating the size Area of the connected domain corresponding to one candidate bright spot, wherein the size Area of the corresponding connected domain is larger than a third preset value, judging that the candidate bright spot corresponding to the connected domain with the size larger than the third preset value is one bright spot, A represents the size of the connected pixels/connected pixels of the row where the center of the matrix corresponding to the candidate bright spot is located, B represents the size of the connected pixels/connected pixels of the column where the center of the matrix corresponding to the candidate bright spot is located, and defining the connected pixels which are larger than the average pixel value in a k1 k2 matrix as the connected domain corresponding to the candidate bright spot. Therefore, the bright spots corresponding to the marker molecules and conforming to the subsequent sequence identification can be effectively obtained, and the nucleic acid sequence information can be obtained.
In one example, with the average pixel value of the image as a reference, two or more adjacent pixels not smaller than the average pixel value are called connected pixels/connected pixels (pixel connectivity), as shown in fig. 4, the two or more adjacent pixels are enlarged to indicate the center of the matrix corresponding to the candidate bright spot, the bold frame indicates the 3 × 3 matrix corresponding to the candidate bright spot, the pixel marked with 1 is a pixel not smaller than the average pixel value of the image, the pixel marked with 0 is a pixel smaller than the average pixel value, a is 3, B is 6, and the size of the connected component corresponding to the candidate bright spot is a B is 3 — 6.
The third preset value can be determined according to the information of the sizes of the connected components corresponding to all the candidate bright spots on the image. For example, the size of the connected domain corresponding to each candidate bright spot on the image is calculated, and the average value of the sizes of the connected domains of the bright spots is taken as a third preset value to represent one characteristic of the image; for another example, the sizes of the connected components corresponding to the candidate bright spots on the image may be sorted from small to large, and the size of the 50 th, 60 th, 70 th, 80 th or 90 th quantile connected component may be taken as the third preset value. Therefore, the speckle information can be effectively obtained, and the subsequent identification of the nucleic acid sequence is facilitated.
In some examples, candidate blobs are screened by statistically setting parameters to quantitatively reflect the intensity characteristics of the comparative candidate blobs. In one example, determining whether the candidate hot spot is a hot spot comprises: calculating Score of one candidate spot ((k1 × k2-1) CV-EV)/((CV + EV)/(k1 × k2)), and determining that the candidate spot with the Score larger than the fourth preset value is one spot, CV represents a central pixel value of a matrix corresponding to the candidate spot, and EV represents a sum of non-central pixel values of the matrix corresponding to the spot. Therefore, the bright spots corresponding to the marker molecules and conforming to the subsequent sequence identification can be effectively obtained, and the nucleic acid sequence information can be obtained.
The fourth predetermined value may be determined according to the information of the scores of all candidate bright spots on the image. For example, when the number of the candidate bright spots on the image is greater than a certain number, which meets the requirement of statistical quantitative requirements, for example, the number of the candidate bright spots on the image is greater than 30, the Score values of all the candidate bright spots of the image can be calculated and sorted in ascending order, and the fourth preset value can be set as the Score value of the 50 th, 60 th, 70 th, 80 th or 90 th quantile, so that the candidate bright spots smaller than the Score value of the 50 th, 60 th, 70 th, 80 th or 90 th quantile can be excluded, which is beneficial to effectively obtaining the target bright spot and is beneficial to accurately identifying the subsequent base sequence. The basis for this processing or screening setting is that, in general, the bright spots that have a large difference in central and edge intensities/pixel values and that converge are considered to be the bright spots corresponding to the positions of the molecules to be detected. Typically, the number of candidate bright spots on the image is greater than 50, greater than 100, or greater than 1000.
In some examples, candidate bright spots are screened in combination with morphology and intensity/brightness. In one example, determining whether the candidate hot spot is a hot spot comprises: calculating the size Area of a connected domain corresponding to a candidate bright spot, and calculating the Score of the candidate bright spot, wherein the Score is ((k1 k2-1) CV-EV)/((CV + EV)/(k1 k2)), A represents the size of connected pixels/connected pixels of a row where the center of a matrix corresponding to the candidate bright spot is located, B represents the size of connected pixels/connected pixels of a column where the center of the matrix corresponding to the candidate bright spot is located, a connected pixel which is larger than the average pixel value in a k1 k2 matrix is defined as a connected domain corresponding to the candidate bright spot, CV represents the center pixel value of the matrix corresponding to the candidate bright spot, and EV represents the sum of non-center pixel values of the matrix corresponding to the candidate bright spot; and judging the candidate bright spots of which the size of the corresponding connected domain is larger than the third preset value and the score is larger than the fourth preset value as one bright spot. Thus, the speckle information corresponding to the nucleic acid molecule and beneficial to the subsequent sequence recognition can be effectively obtained. The third preset value and/or the fourth preset value may be considered and set with reference to the previous embodiments.
In some embodiments, referring to fig. 5, the image registration method further includes S03 identifying a bright spot, the image to be registered and/or the reference image being from a field in which a base extension reaction occurs, a plurality of nucleic acid molecules with optically detectable labels being present in the field in which the base extension reaction occurs, at least a portion of the nucleic acid molecules appearing as a bright spot on the image, and S03 including: preprocessing an image to obtain a preprocessed image, wherein the preprocessed image is selected from at least one of an image to be registered and an image for constructing a reference image; determining a critical value to simplify the preprocessed image, wherein assignment of pixel values of pixel points on the preprocessed image smaller than the critical value to a first preset value and assignment of pixel values of pixel points on the preprocessed image not smaller than the critical value to a second preset value is carried out to obtain a simplified image; determining a first speckle detection threshold c1 based on the pre-processed image; identifying candidate bright spots on the image based on the preprocessed image and the simplified image, including judging a pixel matrix meeting at least two conditions in a) -c) as a candidate bright spot, a) in the preprocessed image, the pixel value of the central pixel of the pixel matrix is maximum, the pixel matrix can be represented as r1 r2, r1 and r2 are both odd numbers larger than 1, r1 r2 pixel matrix comprises r1 r2 pixels, b) in the simplified imageIn the image, the pixel value of the central pixel point of the pixel point matrix is a second preset value, and the connected pixels of the pixel point matrix are larger than
Figure BDA0001774280060000071
And c) the pixel value of the central pixel of the pixel matrix in the preprocessed image is greater than a third preset value and meets the requirement of g1 × g2>c1, g1 is a correlation coefficient of two-dimensional Gaussian distribution in a range of m1 × m2 by taking a central pixel point of the pixel point matrix as a center, g2 is a pixel in a range of m1 × m2, m1 and m2 are both odd numbers larger than 1, and a range of m1 × m2 contains m1 × m2 pixel points; and determining whether the candidate hot spot is a hot spot. The method for detecting the bright spots on the image comprises the step of training the judgment condition or the combination of the judgment conditions determined by the inventor through a large amount of data, and can quickly and effectively realize the detection of the bright spots on the image, particularly the image collected from the nucleic acid sequence determination reaction. The method has no special limitation on the image to be detected, namely the original input data, is suitable for processing and analyzing the image generated by any platform for carrying out nucleic acid sequence determination by using the optical detection principle, including but not limited to second generation and third generation sequencing, has the characteristics of high accuracy and high efficiency, and can acquire more information representing the sequence from the image. Especially for random images and signal recognition with high accuracy requirements.
For a grayscale image, the pixel values are the same as the grayscale values. If the image is a color image, one pixel point of the color image has three pixel values, the color image can be converted into a gray image, and then bright spot detection is carried out, so that the calculated amount and the complexity of the image detection process are reduced. The non-grayscale image may be optionally, but not limited to, converted to a grayscale image using a floating-point algorithm, an integer method, a shift method, or an average value method, etc.
In some embodiments, pre-processing the image comprises: determining the background of the image by utilizing an opening operation; converting the image into a first image by utilizing top hat operation based on the background; performing Gaussian blur processing on the first image to obtain a second image; the second image is sharpened to obtain what is referred to as a pre-processed image. Therefore, the method can effectively reduce noise of the image or improve the signal to noise ratio of the image, and is favorable for accurate detection of the bright spots.
The opening operation is a morphological treatment, namely, a process of expanding firstly and then corroding, wherein the corrosion operation can make the foreground (the interested part) smaller, and the expanding can make the foreground larger; the on operation can be used to eliminate small objects, separate objects at fine points, and smooth the boundaries of larger objects without significantly changing their area. The size of the structural element p1 × p2 (basic template for processing an image) for performing an open operation on an image in this embodiment is not particularly limited, and p1 and p2 are odd numbers. In one example, the structural elements p 1p 2 may be 15 x 15, 31 x 31, etc., which ultimately enable a pre-processed image to be obtained that facilitates subsequent processing analysis.
Top hat operations are often used to separate patches that are brighter than nearby points (bright spots/bright spots), and in the case where an image has a large background and tiny objects are regular, top hat operations can be used to extract the background. In one example, top-hat transforming the image includes performing an open operation on the image, and subtracting the open operation result from the original image to obtain a first image, i.e., a top-hat transformed image. The mathematical expression of top-hat transformation is dst tophat (src, element) ═ src-open (src, element). The inventor considers that the result of the opening operation enlarges the crack or the local low-brightness area, so that the image obtained by subtracting the image after the opening operation from the original image highlights the area brighter than the area around the outline of the original image, the operation is related to the size of the selected nucleus, and can be considered to be related to the expected size of the bright point/bright spot, if the bright point is not the expected size, the effect after the processing can cause the whole image to generate a plurality of small bulges, and particularly, the bright point/bright spot can be stained in a lump by referring to the virtual focus image. In one example, the expected size of the bright spot, i.e., the size of the selected kernel, is 3 × 3, and the resulting top-hat transformed image is favorable for further denoising processing.
Gaussian Blur (also called Gaussian filtering) is a linear smoothing filter, is suitable for eliminating Gaussian noise, and is widely applied to a noise reduction process of image processing. Generally speaking, gaussian filtering is a process of performing weighted average on the whole image, and the value of each pixel point is obtained by performing weighted average on the value of each pixel point and other pixel values in the neighborhood. The specific operation of gaussian filtering is: each pixel in the image is scanned using a template (or convolution, mask), and the weighted average gray value of the pixels in the neighborhood determined by the template is used to replace the value of the pixel in the center of the template. In one example, the first image is subjected to gaussian blurring, which is performed in OpenCV using a gaussian filtering gaussian blur function, the gaussian distribution parameter Sigma takes 0.9, the two-dimensional filter matrix (convolution kernel) used is 3 × 3, and after the gaussian blurring from the image perspective, the small protrusions on the first image are smoothed and the image edges are smooth. Further, the second image, i.e., the gaussian filtered image, is sharpened, for example, by performing a two-dimensional laplacian sharpening, and after the image is processed from the viewpoint of the image, the edge is sharpened, and the image after the gaussian blur is restored.
In some embodiments, simplifying the pre-processed image comprises: determining a critical value based on the background and the preprocessed image; and comparing the pixel value of the pixel point on the preprocessed image with the critical value, assigning the pixel value of the pixel point on the preprocessed image smaller than the critical value as a first preset value, and assigning the pixel value of the pixel point on the preprocessed image not smaller than the critical value as a second preset value to obtain the simplified image. Therefore, according to the critical value determining mode and the determined critical value summarized by a large amount of test data of the inventor, the preprocessed image is simplified, such as binaryzation, so that the method is beneficial to accurate detection of subsequent bright spots, accurate identification of subsequent bases, acquisition of high-quality data and the like.
Specifically, in some examples, obtaining the simplified image includes: dividing the sharpened result obtained after preprocessing by an open operation result to obtain a group of numerical values corresponding to the image pixel points; and determining the critical value of the image after the binarization preprocessing through the set of values. For example, the set of values may be sorted in ascending order of magnitude, and the value corresponding to the 20 th, 30 th or 40 th percentile of the set of values may be used as the binarization critical value/threshold value. Therefore, the obtained binary image is beneficial to accurate detection and identification of subsequent bright spots.
In one example, the structural element of the open operation during image preprocessing is p1 × p2, so called dividing the preprocessed image (sharpened result) by the open operation result to obtain a group of arrays/matrices p1 × p2 with the same size as the structural element, in each array, arranging the p1 × p2 values contained in the array in ascending order of size, and taking the value corresponding to the thirty-th percentile in the array as the binarization critical value/threshold value of the region (value matrix), so as to determine to binarize each region on the threshold image respectively, and the finally obtained binarization result emphasizes the required information while denoising, which is favorable for accurate detection of subsequent bright spots.
In some examples, the determination of the first speckle detection threshold is made using the Otsu method. Otsu's method (OTSU algorithm) can also be called maximum inter-class variance method, and it utilizes the maximum inter-class variance to segment images, meaning that the probability of misclassification is small and the accuracy is high. Assuming that the segmentation threshold of the foreground and the background of the preprocessed image is T (c1), the proportion of the number of pixels belonging to the foreground in the whole image is w0Average gray of μ0(ii) a The proportion of the number of pixels belonging to the background to the whole image is w1Average gray of μ1. And (3) recording the total average gray level of the image to be processed as mu and the between-class variance as var, and then:
μ＝ω0*μ0+ω1*μ1；var＝ω0(μ0-μ)2+ω1(μ1-μ)2substituting the latter into the former to obtain an equivalent formula: var ═ ω0ω1(μ1-μ0)2. And obtaining a segmentation threshold T which enables the inter-class variance to be maximum by adopting a traversal method, namely obtaining the first speckle detection threshold c 1.
In some embodiments, identifying the candidate hot spot on the image based on the preprocessed image and the simplified image includes determining a pixel matrix satisfying all of the conditions a) -c) as a candidate hot spot. Therefore, the accuracy of the subsequent determination of the nucleic acid sequence based on the speckle information and the quality of the off-line data can be effectively improved.
Specifically, in one example, the conditions that need to be satisfied by the determination of the candidate bright spots include a), k1, and k2 may be equal or unequal. In one example, the imaging system related parameters are: the objective lens is 60 times, the size of the electronic sensor is 6.5 μm, the minimum size of the image formed by the microscope is 0.1 μm, the obtained image or the input image can be a 16-bit gray scale or color image of 512 × 512, 1024 × 1024 or 2048 × 2048, and the value ranges of k1 and k2 are both more than 1 and less than 10. In one example, in a pre-processed image, k 1-k 2-3 is set according to the expected size of the bright spot; in another example, k 1-k 2-5 is set.
In one example, the condition that the candidate bright spot needs to be determined includes b), in the simplified image, the pixel value of the central pixel of the pixel matrix is a second preset value, and the connected pixels of the pixel matrix are larger than the connected pixels of the pixel matrix
Figure BDA0001774280060000091
That is, the pixel value of the central pixel is greater than the threshold value and the connected pixels are greater than two-thirds of the matrix. Here, two or more pixels whose adjacent pixel values are all the second preset value are called connected pixels/connected pixels (pixel connectivity), for example, the simplified image is a binarized image, the first preset value is 0, the second preset value is 1, as shown in fig. 4, the bold and enlarged representation indicates the center of the called pixel matrix, the thick frame indicates a pixel matrix 3 × 3, that is, k1 ═ k2 ═ 3, the pixel value of the center pixel of the matrix is 1, the connected pixels are 4, and smaller than the connected pixels (pixel connectivity)
Figure BDA0001774280060000092
The pixel point matrix does not meet the condition b), and the pixel point matrix is not a candidate bright spot.
In one example, the condition that needs to be satisfied for the determination of the candidate bright spot includes c), in the preprocessed image, g2 is the modified m1 m2 range of pixels, i.e., the modified m1 m2 range of pixel sums. In an example, the correction is performed according to the proportion of the pixels having the pixel values of the second preset value in the range of m1 × m2 corresponding to the simplified image, for example, as shown in fig. 6, m1 is m2 is 5, the proportion of the pixels having the pixel values of the second preset value in the range of m1 × m2 corresponding to the simplified image is 13/25(13 pieces of "1"), and g2 after the correction is 13/25. Therefore, the method is beneficial to more accurately detecting and identifying the bright spots and is beneficial to analyzing and reading the subsequent bright spot information.
In some examples, the determining whether the candidate hot spot is a hot spot further comprises: determining a second hot spot detection threshold value based on the preprocessed image, and judging the candidate hot spots with the pixel values not less than the second hot spot detection threshold value as hot spots. In a specific example, the pixel value of the pixel point where the coordinate of the candidate hot spot is located is taken as the pixel value of the candidate hot spot. Through further screening of the candidate bright spots by using the second bright spot detection threshold determined based on the preprocessed image, at least one part of the bright spots which are more likely to be the image background and have brightness (intensity) and/or shape of 'bright spots' can be excluded, so that accurate identification of a subsequent sequence based on the bright spots is facilitated, and the quality of off-line data is improved.
In one example, the coordinates of the candidate bright spots, including sub-pixel level coordinates, may be obtained using a barycentric method. And calculating the gray value of the coordinate position of the candidate bright spot by using a bilinear interpolation method.
In some specific examples, determining whether the candidate hot spot is a hot spot includes: dividing the preprocessed image into a group of regions (blocks) with a preset size, and sequencing pixel values of pixel points in the regions to determine a second bright spot detection threshold corresponding to the regions; and judging the candidate bright spots with the pixel values not less than the second bright spot detection threshold value corresponding to the area as the bright spots. Therefore, the difference of different areas of the image, such as the integral fall of light intensity, is distinguished, the further detection and identification of the bright spots are separately carried out, the accurate identification of the bright spots is facilitated, and more bright spots are obtained.
The preprocessed image is said to be divided into a set of regions (blocks) of a predetermined size, with or without overlap between the blocks. In one example, there is no overlap between blocks. In some embodiments, the size of the pre-processed image is not less than 512 × 512, such as 512 × 512, 1024 × 1024, 1800 × 1800, or 2056 × 2056, and the region of the predetermined size may be set to 200 × 200. Therefore, the method is beneficial to quickly calculating, judging and identifying the bright spots.
In some embodiments, when the second bright spot detection threshold corresponding to the region is determined, the pixel values of the pixels in each block are arranged in an ascending order according to the size, p10+ (p10-p1) × 4.1 is taken as the second bright spot detection threshold corresponding to the block, that is, the background of the block, p1 represents the pixel value of the tenth percentile, and p10 represents the pixel value of the tenth percentile. The threshold is a stable threshold obtained by a large amount of data training tests of the inventor, and can eliminate a large amount of bright spots on the background. It will be appreciated that this threshold may need to be adjusted appropriately when the optical system is adjusted and the overall pixel distribution of the image changes. Fig. 7 is a schematic diagram showing comparison between the bright spot detection results before and after the processing, that is, a schematic diagram showing the bright spot detection results before and after the background of the area is eliminated, the upper half of fig. 7 is the bright spot detection result after the processing, the lower half is the bright spot detection result without the processing, and the cross mark is the candidate bright spot or the bright spot.
The logic and/or steps represented in the flowcharts or otherwise described herein, such as a sequence listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable storage medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable storage medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable storage medium may even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
Referring to fig. 8, an embodiment of the present invention provides an image registration apparatus 100, which is configured to implement all or part of the steps of the image registration method in any of the above embodiments, where the apparatus 100 includes: the first registration module 110 is configured to perform first registration on an image to be registered based on a reference image, where the first registration includes determining a first offset between a predetermined region on the image to be registered and a corresponding predetermined region on the reference image, and moving all bright spots on the image to be registered based on the first offset to obtain the first registered image to be registered, where the reference image and the image to be registered correspond to the same object, and both the reference image and the image to be registered include multiple bright spots; the second registration module 120 is configured to perform second registration on the first registered image to be registered from the first registration module based on the reference image, including merging the first registered image to be registered and the reference image to obtain a merged image, calculating offsets of all the overlapping bright spots of the predetermined area on the merged image to determine a second offset, where two or more bright spots with distances smaller than a predetermined pixel are one overlapping bright spot, and moving all the bright spots on the first registered image to be registered based on the second offset to realize registration of the image to be registered.
The above description of the technical features and advantages of the image registration method in any embodiment of the present invention is also applicable to the image registration apparatus, and will not be described herein again. It will be appreciated that additional technical features of the image registration method in any of the above embodiments, including sub-steps, additional steps, alternative or preferred settings or processes, etc., may be implemented by making the apparatus or modules of the apparatus further comprise units/modules or sub-units/sub-modules.
For example, in some examples, the image registration apparatus 100 further includes a reference image construction module 109 for constructing a reference image, including: acquiring a first image and a second image, wherein the first image and the second image correspond to the same object as the image to be registered; performing coarse registration on the second image based on the first image, wherein the coarse registration comprises the steps of determining the offset between the second image and the first image, and moving the second image based on the offset to obtain a second image after the coarse registration; and combining the first image and the second image after coarse registration to obtain a reference image, wherein the first image and the second image both comprise a plurality of bright spots.
In some examples, constructing the reference image in the reference image construction module 109 further includes using a third image and a fourth image, the image to be registered, the first image, the second image, the third image, and the fourth image being from the same field of view of the sequencing reaction, the first image, the second image, the third image, and the fourth image corresponding to fields of view of four types of base extension reactions, a/U, T, G and C, respectively, in which a plurality of nucleic acid molecules with optically detectable labels are present, at least a portion of the nucleic acid molecules appearing as bright spots on the images, and constructing the reference image further includes: performing coarse registration on the third image based on the first image, wherein the coarse registration comprises determining the offset of the third image and the first image, and moving the third image based on the offset to obtain a coarsely registered third image; performing coarse registration on the fourth image based on the first image, wherein the coarse registration comprises determining the offset of the fourth image and the first image, and moving the fourth image based on the offset to obtain a coarsely registered fourth image; and merging the first image and the second image after coarse registration, the third image after coarse registration and the fourth image after coarse registration to obtain a reference image.
In some examples, the reference image and the image to be registered are binarized images.
In some examples, the first offset, the offset of the second image and the first image, the offset of the third image and the first image, and/or the offset of the fourth image and the first image are determined using a two-dimensional discrete fourier transform.
In some examples, the image registration apparatus 100 further includes a first hot spot detection module 107, configured to perform hot spot detection on the image by using a k1 × k2 matrix, determine that a matrix with a central pixel value not less than any pixel value other than the center of the matrix corresponds to a candidate hot spot, and determine whether the candidate hot spot is a hot spot, where k1 and k2 are both odd numbers greater than 1, and the k1 × k2 matrix includes k1 × k2 pixels.
In some examples, the central pixel value of the matrix is greater than a first predetermined value, any pixel value not in the center of the matrix is greater than a second predetermined value, and the first predetermined value and/or the second predetermined value is related to an average pixel value of the image.
In some examples, in the first speckle detection module 107, determining whether the candidate speckle is a speckle includes: calculating the size Area of a connected domain corresponding to a candidate bright spot, wherein the size Area is A B, judging that the candidate bright spot of which the size of the corresponding connected domain is larger than a third preset value is a bright spot, A represents the connected size of a row where the center of a matrix corresponding to the candidate bright spot is located, B represents the connected size of a column where the center of the matrix corresponding to the candidate bright spot is located, defining a connected pixel point larger than an average pixel value as a connected domain, and/or calculating the Score of the candidate bright spot ((k1 k2-1) CV-I)/(CV + EV)/(k1 k2)), judging that the candidate bright spot of which the Score is larger than a fourth preset value is a bright spot, CV represents the central pixel value of the matrix corresponding to the candidate bright spot, and EV represents the sum of non-central pixel values of the matrix corresponding to the bright spot.
In some examples, the image registration apparatus 100 further includes a second bright spot detection module 105, where the image to be registered or any image constituting the reference image is acquired from a field where a base extension reaction occurs, and a plurality of nucleic acid molecules with optically detectable labels are present on the field where the base extension reaction occurs, and at least a part of the nucleic acid molecules appear as bright spots on the image, and the second bright spot detection module 105 is configured to: preprocessing the image to obtain a preprocessed image; determining a critical value to simplify the pre-processed image, wherein the step of assigning the pixel value of the pixel point on the pre-processed image smaller than the critical value to be a first preset value and assigning the pixel value of the pixel point on the pre-processed image not smaller than the critical value to be a second preset valueValues to obtain a reduced image; determining a first speckle detection threshold c1 based on the pre-processed image; identifying candidate bright spots on the image based on the preprocessed image and the simplified image, including judging a pixel point matrix satisfying at least two conditions in a) -c) as a candidate bright spot, a) in the preprocessed image, the pixel value of the central pixel point of the pixel point matrix is maximum, the pixel point matrix can be represented as r1 r2, r1 and r2 are both odd numbers larger than 1, the r1 r2 pixel point matrix comprises r1 r2 pixel points, b) in the simplified image, the pixel value of the central pixel point of the pixel point matrix is a second preset value, and the connected pixels of the pixel point matrix are larger than the connected pixels of the pixel point matrix
Figure BDA0001774280060000121
And c) the pixel value of the central pixel of the pixel matrix in the preprocessed image is greater than a third preset value and meets the requirement of g1 × g2>c1, g1 is a correlation coefficient of two-dimensional Gaussian distribution in a range of m1 × m2 by taking a central pixel point of the pixel point matrix as a center, g2 is a pixel in a range of m1 × m2, m1 and m2 are both odd numbers larger than 1, and a range of m1 × m2 contains m1 × m2 pixel points; and determining whether the candidate hot spot is a hot spot.
In some examples, in the second speckle detection module 105, determining whether the candidate speckle is a speckle includes: determining a second hot spot detection threshold value based on the pre-processed image, and judging the candidate hot spot of which the pixel value is not less than the second hot spot detection threshold value as the hot spot.
In some examples, the pixel value of the candidate hot spot is the pixel value of the pixel point where the coordinates of the candidate hot spot are located.
In some examples, in the second speckle detection module 105, determining a second speckle detection threshold based on the preprocessed image, and determining candidate speckle having a pixel value not less than the second speckle detection threshold as speckle includes: dividing the preprocessed image into a group of regions with preset sizes, sequencing pixel values of pixel points in the regions to determine second bright spot detection thresholds corresponding to the regions, and judging candidate bright spots with pixel values not smaller than the second bright spot detection thresholds corresponding to the regions as the bright spots for the candidate bright spots in the regions.
In some examples, in the second bright spot detection module 105, pre-processing the image includes: determining the background of the image by using an opening operation, converting the image into a first image by using a top hat operation based on the background, performing Gaussian blur processing on the first image to obtain a second image, and sharpening the second image to obtain a preprocessed image.
In some examples, in the second speckle detection module 105, determining a critical value to simplify the pre-processed image, obtaining the simplified image includes: and determining a critical value based on the background and the preprocessed image, and comparing the pixel value of the pixel point on the preprocessed image with the critical value to obtain a simplified image.
In some examples, in the second hot spot detection module 105, g2 is the corrected pixels in the range of m1 × m2, and is corrected according to the proportion of the pixels with the pixel values in the corresponding range of m1 × m2 of the simplified image as the second preset value.
In some examples, in the first speckle detection module 107 and/or the second speckle detection module 105, determining whether the candidate speckle is a speckle further comprises: and if the candidate bright spots are judged to be the bright spots, calculating the center coordinates of the sub-pixels of the bright spots and/or the intensity values of the center coordinates of the sub-pixels, and if the candidate bright spots are not judged to be the bright spots, discarding the candidate bright spots.
Embodiments of the present invention also provide a computer program product including instructions which, when executed by a computer, cause the computer to perform all or part of the steps of the image registration method in any of the above embodiments.
Those skilled in the art will appreciate that, in addition to implementing the controller/processor in purely computer readable program code means, the same functionality can be implemented entirely by logically transforming method steps into logic such that the controller takes the form of logic gates, switches, application specific integrated circuits, editable logic controllers, embedded microcontrollers and the like. Thus, such a controller/processor may be considered a hardware component, and the means included therein for performing the various functions may also be considered as an arrangement within the hardware component. Or even means for performing the functions may be regarded as being both a software module for performing the method and a structure within a hardware component.
In the description of the present specification, a description of one embodiment, some embodiments, one or some specific embodiments, one or some examples, etc. means that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one example or example of the present invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, etc. described may be combined in any suitable manner in any one or more embodiments or examples.
While embodiments of the invention have been shown and described, it will be understood by those of ordinary skill in the art that: various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims (9)

1. An image registration method, comprising:
performing first registration on an image to be registered based on a reference image, wherein the reference image and the image to be registered correspond to the same object, the reference image and the image to be registered both comprise a plurality of bright spots, and the method comprises the following steps of,
determining a first offset of a preset area on the image to be registered and a corresponding preset area on the reference image, and moving all bright spots on the image to be registered based on the first offset to obtain a first registered image to be registered;
performing second registration on the image to be registered after the first registration based on the reference image, including,
merging the first registered image to be registered and the reference image to obtain a merged image,
calculating the offset of all the overlapping bright spots of the predetermined area on the merged image to determine a second offset, two or more bright spots having a distance smaller than a predetermined pixel being one of the overlapping bright spots,
and moving all the bright spots on the image to be registered after the first registration based on the second offset so as to realize the registration of the image to be registered.
2. The method of claim 1, wherein the reference image is obtained by a construction comprising:
acquiring a first image and a second image, wherein the first image and the second image correspond to the same object as the image to be registered;
performing coarse registration on a second image based on a first image, wherein the coarse registration comprises determining the offset of the second image and the first image, and moving the second image based on the offset to obtain a second image after coarse registration;
merging the first image and the coarsely registered second image to obtain the reference image, wherein the first image and the second image both comprise a plurality of bright spots;
optionally, constructing the reference image further comprises using a third image and a fourth image, the image to be registered, the first image, the second image, the third image and the fourth image are from the same field of view of the sequencing reaction, the first image, the second image, the third image and the fourth image correspond to the field of view of the four types of base extension reactions a/U, T, G and C, respectively, in which a plurality of nucleic acid molecules with optically detectable labels are present, at least a portion of the nucleic acid molecules appear as the bright spots on the images, and constructing the reference image further comprises:
carrying out coarse registration on a third image based on a first image, wherein the coarse registration comprises the steps of determining the offset of the third image and the first image, and moving the third image based on the offset to obtain a coarsely registered third image;
performing coarse registration on a fourth image based on the first image, wherein the coarse registration comprises determining the offset of the fourth image and the first image, and moving the fourth image based on the offset to obtain a coarsely registered fourth image;
merging the first image and the coarsely registered second image, the coarsely registered third image and the coarsely registered fourth image to obtain the reference image;
optionally, the reference image and the image to be registered are binary images;
optionally, the first offset, the offset of the second image and the first image, the offset of the third image and the first image, and/or the offset of the fourth image and the first image are determined using a two-dimensional discrete fourier transform.
3. The method of any of claims 1-2, further comprising identifying the hot spots, including examining the image using a k1 x k2 matrix, determining that a matrix having a center pixel value that is not less than any pixel value that is not the center of the matrix corresponds to a candidate hot spot, and determining whether the candidate hot spot is the hot spot, wherein k1 and k2 are both odd numbers greater than 1, and wherein the k1 k2 matrix comprises k1 x k2 pixels;
optionally, a central pixel value of the matrix is greater than a first preset value, any pixel value of a non-central pixel of the matrix is greater than a second preset value, and the first preset value and the second preset value are related to an average pixel value of the image;
optionally, determining whether the candidate hot spot is the hot spot comprises:
calculating the size Area of a connected domain corresponding to one candidate bright spot, namely A, B, judging that the candidate bright spot of which the size of the corresponding connected domain is larger than a third preset value is one bright spot, wherein A represents the connected size of a row in which the center of a matrix corresponding to the candidate bright spot is positioned, B represents the connected size of a column in which the center of the matrix corresponding to the candidate bright spot is positioned, and a connected pixel point of which the value is larger than the average pixel value is defined as a connected domain, and/or
Calculating Score of one of the candidate patches ((k1 × k2-1) CV-EV)/((CV + EV)/(k1 × k2)), and determining that the candidate patch having a Score greater than a fourth preset value is one of the patches, CV represents a central pixel value of a matrix corresponding to the candidate patch, and EV represents a sum of non-central pixel values of the matrix corresponding to the patch.
4. The method of any one of claims 1-2, further comprising identifying the bright spots, wherein the image is taken of a field in which a base extension reaction occurs, wherein a plurality of nucleic acid molecules with optically detectable labels are present in the field in which the base extension reaction occurs, and wherein at least a portion of the nucleic acid molecules appear as bright spots on the image, and wherein the method comprises:
preprocessing the image to obtain a preprocessed image;
determining a critical value to simplify the preprocessed image, wherein the step of assigning the pixel value of the pixel point on the preprocessed image smaller than the critical value to be a first preset value and assigning the pixel value of the pixel point on the preprocessed image not smaller than the critical value to be a second preset value is included to obtain a simplified image;
determining a first speckle detection threshold c1 based on the pre-processed image;
identifying a candidate hot spot on the image based on the preprocessed image and the simplified image, including determining a matrix of pixels that satisfies at least two of the following conditions a) -c) as one of the candidate hot spots,
a) in the preprocessed image, the pixel value of the central pixel point of the pixel point matrix is maximum, the pixel point matrix can be represented as r1 r2, r1 and r2 are both odd numbers larger than 1, the r1 r2 pixel point matrix comprises r1 r2 pixel points,
b) in the simplified image, the pixel value of the central pixel point of the pixel point matrix is a second preset value, and the connected pixels of the pixel point matrix are larger than
Figure FDA0001774280050000021
And
c) the pixel value of the central pixel point of the pixel point matrix in the preprocessed image is larger than a third preset value and meets g1 × g2> c1, g1 is a two-dimensional Gaussian distribution correlation coefficient in a range of m1 × m2 with the central pixel point of the pixel point matrix as the center, g2 is the pixel in the range of m1 × m2, m1 and m2 are both odd numbers larger than 1, and m1 × m2 includes m1 × m2 pixel points; and
determining whether the candidate hot spot is the hot spot;
optionally, the determining whether the candidate hot spot is a hot spot comprises:
determining a second speckle detection threshold based on the pre-processed image, an
Judging the candidate bright speckles with the pixel values not less than the second bright speckle detection threshold value as the bright speckles;
optionally, the pixel value of the candidate bright spot is the pixel value of a pixel point where the coordinate of the candidate bright spot is located;
optionally, the determining a second hot spot detection threshold based on the preprocessed image, and determining a candidate hot spot having a pixel value not less than the second hot spot detection threshold as the hot spot includes:
dividing the pre-processed image into a set of regions of a predetermined size,
sorting the pixel values of the pixel points in the area to determine a second hot spot detection threshold corresponding to the area,
for the candidate bright spots positioned in the area, judging the candidate bright spots with the pixel values not smaller than the second bright spot detection threshold value corresponding to the area as the bright spots;
optionally, pre-processing the image, comprising:
determining a background of the image using an on operation,
converting the image into a first image using a top hat operation based on the background,
performing Gaussian blur processing on the first image to obtain a second image,
sharpening the second image to obtain the preprocessed image;
optionally, the determining a critical value to simplify the preprocessed image to obtain a simplified image includes:
determining the critical value based on the background and the pre-processed image,
comparing the pixel value of the pixel point on the preprocessed image with the critical value to obtain the simplified image;
optionally, g2 is pixels in the range of m1 m2 after correction, and the correction is carried out according to the proportion of pixels with pixel values in the range of m1 m2 corresponding to the simplified image as a second preset value;
optionally, determining whether the candidate hot spot is the hot spot further comprises:
if the candidate bright spots are judged to be the bright spots, calculating the sub-pixel center coordinates of the bright spots and/or the intensity values of the sub-pixel center coordinates,
and if the candidate bright spots are judged not to be the bright spots, discarding the candidate bright spots.
5. An image registration apparatus, comprising:
a first registration module for performing a first registration of the image to be registered based on the reference image, comprising,
determining a first offset of a predetermined area on the image to be registered and a corresponding predetermined area on the reference image, and moving all bright spots on the image to be registered based on the first offset to obtain a first registered image to be registered, wherein the reference image and the image to be registered correspond to the same object, and both the reference image and the image to be registered contain a plurality of bright spots;
a second registration module for performing a second registration on the first registered image to be registered from the first registration module based on the reference image, including,
merging the first registered image to be registered and the reference image to obtain a merged image,
calculating the offset of all the overlapping bright spots of the predetermined area on the merged image to determine a second offset, two or more bright spots having a distance smaller than a predetermined pixel being one of the overlapping bright spots,
moving all the bright spots on the image to be registered after the first registration based on the second offset so as to realize the registration of the image to be registered;
optionally, the method further includes a reference image construction module, configured to construct the reference image, including:
acquiring a first image and a second image, wherein the first image and the second image correspond to the same object as the image to be registered;
performing coarse registration on a second image based on a first image, wherein the coarse registration comprises determining the offset of the second image and the first image, and moving the second image based on the offset to obtain a second image after coarse registration;
merging the first image and the coarsely registered second image to obtain the reference image, wherein the first image and the second image both comprise a plurality of bright spots;
optionally, constructing the reference image in the reference image construction module further comprises using a third image and a fourth image, the image to be registered, the first image, the second image, the third image and the fourth image are from the same field of view of a sequencing reaction, the first image, the second image, the third image and the fourth image respectively correspond to the field of view of four types of base extension reactions, a plurality of nucleic acid molecules with optically detectable labels are present in the field of view during the base extension reaction, at least a portion of the nucleic acid molecules appear as the bright spots on the images, and constructing the reference image further comprises:
carrying out coarse registration on a third image based on a first image, wherein the coarse registration comprises the steps of determining the offset of the third image and the first image, and moving the third image based on the offset to obtain a coarsely registered third image;
performing coarse registration on a fourth image based on the first image, wherein the coarse registration comprises determining the offset of the fourth image and the first image, and moving the fourth image based on the offset to obtain a coarsely registered fourth image;
merging the first image and the coarsely registered second image, the coarsely registered third image and the coarsely registered fourth image to obtain the reference image;
optionally, the reference image and the image to be registered are binarized images.
6. The apparatus of claim 5, characterized in that the first offset, the offset of the second image and the first image, the offset of the third image and the first image and/or the offset of the fourth image and the first image are determined using a two-dimensional discrete Fourier transform.
7. The apparatus of any of claims 5-6, further comprising a first hot spot detection module, said first hot spot detection module configured to perform hot spot detection on the image using a k1 x k2 matrix, determine that a matrix having a center pixel value that is not less than any pixel value that is not in the center of said matrix corresponds to a candidate hot spot, and determine whether said candidate hot spot is said hot spot, wherein both k1 and k2 are odd numbers greater than 1, and wherein the k1 x k2 matrix comprises k1 k2 pixels;
optionally, the central pixel value of the matrix is greater than a first preset value, any pixel value of the non-central pixel of the matrix is greater than a second preset value, and the first preset value and/or the second preset value are/is related to the average pixel value of the image;
optionally, in the first speckle detection module, determining whether the candidate speckle is the speckle comprises:
calculating the size Area of a connected domain corresponding to one candidate bright spot, namely A, B, judging that the candidate bright spot of which the size of the corresponding connected domain is larger than a third preset value is one bright spot, wherein A represents the connected size of a row in which the center of a matrix corresponding to the candidate bright spot is positioned, B represents the connected size of a column in which the center of the matrix corresponding to the candidate bright spot is positioned, and a connected pixel point of which the value is larger than the average pixel value is defined as a connected domain, and/or
Calculating Score of one of the candidate patches ((k1 × k2-1) CV-EV)/((CV + EV)/(k1 × k2)), and determining that the candidate patch having a Score greater than a fourth preset value is one of the patches, CV represents a central pixel value of a matrix corresponding to the candidate patch, and EV represents a sum of non-central pixel values of the matrix corresponding to the patch.
8. The device according to any one of claims 5 to 6, further comprising a second speckle detection module, wherein the image is acquired from a field in which the base extension reaction occurs, and a plurality of nucleic acid molecules with optically detectable labels are present in the field in which the base extension reaction occurs, and at least a part of the nucleic acid molecules appear as speckles on the image, and wherein the second speckle detection module is configured to:
preprocessing the image to obtain a preprocessed image;
determining a critical value to simplify the preprocessed image, wherein the step of assigning the pixel value of the pixel point on the preprocessed image smaller than the critical value to be a first preset value and assigning the pixel value of the pixel point on the preprocessed image not smaller than the critical value to be a second preset value is included to obtain a simplified image;
determining a first speckle detection threshold c1 based on the pre-processed image;
identifying a candidate hot spot on the image based on the preprocessed image and the simplified image, including determining a matrix of pixels that satisfies at least two of the following conditions a) -c) as one of the candidate hot spots,
a) in the preprocessed image, the pixel value of the central pixel point of the pixel point matrix is maximum, the pixel point matrix can be represented as r1 r2, r1 and r2 are both odd numbers larger than 1, the r1 r2 pixel point matrix comprises r1 r2 pixel points,
b) in the simplified image, the pixel value of the central pixel point of the pixel point matrix is a second preset value, and the connected pixels of the pixel point matrix are larger than
Figure FDA0001774280050000051
And
c) the pixel value of the central pixel point of the pixel point matrix in the preprocessed image is larger than a third preset value and meets g1 × g2> c1, g1 is a two-dimensional Gaussian distribution correlation coefficient in a range of m1 × m2 with the central pixel point of the pixel point matrix as the center, g2 is the pixel in the range of m1 × m2, m1 and m2 are both odd numbers larger than 1, and m1 × m2 includes m1 × m2 pixel points; and
determining whether the candidate hot spot is the hot spot;
optionally, in the second speckle detection module, the determining whether the candidate speckle is a speckle comprises:
determining a second speckle detection threshold based on the pre-processed image, an
Judging the candidate bright speckles with the pixel values not less than the second bright speckle detection threshold value as the bright speckles;
optionally, the pixel value of the candidate bright spot is the pixel value of a pixel point where the coordinate of the candidate bright spot is located;
optionally, in the second speckle detection module, the determining a second speckle detection threshold based on the preprocessed image, and determining a candidate speckle having a pixel value not less than the second speckle detection threshold as the speckle includes:
dividing the pre-processed image into a set of regions of a predetermined size,
sorting the pixel values of the pixel points in the area to determine a second hot spot detection threshold corresponding to the area,
for the candidate bright spots positioned in the area, judging the candidate bright spots with the pixel values not smaller than the second bright spot detection threshold value corresponding to the area as the bright spots;
optionally, in the second bright spot detection module, preprocessing the image comprises:
determining a background of the image using an on operation,
converting the image into a first image using a top hat operation based on the background,
performing Gaussian blur processing on the first image to obtain a second image,
sharpening the second image to obtain the preprocessed image;
optionally, in the second speckle detection module, the determining a critical value to simplify the preprocessed image includes:
determining the critical value based on the background and the pre-processed image,
comparing the pixel value of the pixel point on the preprocessed image with the critical value to obtain the simplified image;
optionally, in the second hot spot detection module, g2 is a pixel in a range of m1 × m2 after correction, and the correction is performed according to a proportion of a pixel point in a corresponding range of m1 × m2 of the simplified image, where a pixel value of the pixel point is a second preset value;
optionally, in the first speckle detection module and/or the second speckle detection module, determining whether the candidate speckle is the speckle further includes:
if the candidate bright spots are judged to be the bright spots, calculating the sub-pixel center coordinates of the bright spots and/or the intensity values of the sub-pixel center coordinates,
and if the candidate bright spots are judged not to be the bright spots, discarding the candidate bright spots.
9. A computer program product comprising instructions which, when said program is executed by said computer, cause said computer to carry out all or part of the steps of the method according to any one of claims 1 to 4.
CN201810963040.4A 2018-08-22 2018-08-22 Image registration method, apparatus and computer program product Active CN112288781B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810963040.4A CN112288781B (en) 2018-08-22 2018-08-22 Image registration method, apparatus and computer program product

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810963040.4A CN112288781B (en) 2018-08-22 2018-08-22 Image registration method, apparatus and computer program product

Publications (2)

Publication Number Publication Date
CN112288781A true CN112288781A (en) 2021-01-29
CN112288781B CN112288781B (en) 2024-07-05

Family

ID=74418974

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810963040.4A Active CN112288781B (en) 2018-08-22 2018-08-22 Image registration method, apparatus and computer program product

Country Status (1)

Country Link
CN (1) CN112288781B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024119327A1 (en) * 2022-12-05 2024-06-13 深圳华大生命科学研究院 Image processing method and apparatus, device, and medium
WO2024140739A1 (en) * 2022-12-30 2024-07-04 广东润鹏生物技术有限公司 Light spot fusion method and apparatus for sequencing image, storage medium, and computer device

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105427327A (en) * 2015-12-10 2016-03-23 北京中科紫鑫科技有限责任公司 Image registration method of DNA sequencing and apparatus thereof
CN105550990A (en) * 2015-12-10 2016-05-04 北京中科紫鑫科技有限责任公司 DNA sequence Fourier registering method and device
CN107730541A (en) * 2016-08-12 2018-02-23 广州康昕瑞基因健康科技有限公司 Method for registering images and system and image taking alignment method and system
EP3306566A1 (en) * 2016-10-10 2018-04-11 Direct Genomics Co., Ltd. Method and system for processing image
WO2018068600A1 (en) * 2016-10-10 2018-04-19 深圳市瀚海基因生物科技有限公司 Image processing method and system
CN108022259A (en) * 2016-11-04 2018-05-11 北京建筑大学 Interference SAR complex image corregistration method and system

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105427327A (en) * 2015-12-10 2016-03-23 北京中科紫鑫科技有限责任公司 Image registration method of DNA sequencing and apparatus thereof
CN105550990A (en) * 2015-12-10 2016-05-04 北京中科紫鑫科技有限责任公司 DNA sequence Fourier registering method and device
CN107730541A (en) * 2016-08-12 2018-02-23 广州康昕瑞基因健康科技有限公司 Method for registering images and system and image taking alignment method and system
EP3306566A1 (en) * 2016-10-10 2018-04-11 Direct Genomics Co., Ltd. Method and system for processing image
WO2018068600A1 (en) * 2016-10-10 2018-04-19 深圳市瀚海基因生物科技有限公司 Image processing method and system
CN108022259A (en) * 2016-11-04 2018-05-11 北京建筑大学 Interference SAR complex image corregistration method and system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
吴伟;丁香乾;闫明;: "基于异常区域感知的多时相高分辨率遥感图像配准", 计算机应用, no. 10, 10 October 2016 (2016-10-10) *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024119327A1 (en) * 2022-12-05 2024-06-13 深圳华大生命科学研究院 Image processing method and apparatus, device, and medium
WO2024140739A1 (en) * 2022-12-30 2024-07-04 广东润鹏生物技术有限公司 Light spot fusion method and apparatus for sequencing image, storage medium, and computer device

Also Published As

Publication number Publication date
CN112288781B (en) 2024-07-05

Similar Documents

Publication Publication Date Title
CN112823352B (en) Base recognition method, system and sequencing system
CN103543277B (en) A kind of blood group result recognizer based on gray analysis and category identification
CN107945150B (en) Image processing method and system for gene sequencing and computer readable storage medium
US8073233B2 (en) Image processor, microscope system, and area specifying program
US11847766B2 (en) Method and device for detecting bright spots on image, and computer program product
WO2020037572A1 (en) Method and device for detecting bright spot on image, and image registration method and device
JP2013526717A5 (en)
CN113012757B (en) Method and system for identifying bases in nucleic acids
CN112204615A (en) Fluorescent image registration method, gene sequencer system and storage medium
CN112289377B (en) Method, apparatus and computer program product for detecting bright spots on an image
US12008775B2 (en) Method and device for image registration, and computer program product
CN113158895B (en) Bill identification method and device, electronic equipment and storage medium
CN112289381B (en) Method, device and computer product for constructing sequencing template based on image
CN112288781B (en) Image registration method, apparatus and computer program product
Marcuzzo et al. Automated Arabidopsis plant root cell segmentation based on SVM classification and region merging
CN107274349B (en) Method and device for determining inclination angle of fluorescence image of biochip
WO2020037571A1 (en) Method and apparatus for building sequencing template on basis of images, and computer program product
CN112285070B (en) Method and device for detecting bright spots on image and image registration method and device
US11170506B2 (en) Method for constructing sequencing template based on image, and base recognition method and device
CN110031471B (en) Method, system and device for analyzing surface defect growth of large-caliber optical element
CN112288783B (en) Method for constructing sequencing template based on image, base identification method and device
CN115546145A (en) Defect detection method and device based on machine vision and electronic equipment
CN113128500A (en) Mask-RCNN-based non-motor vehicle license plate recognition method and system
Yang et al. A novel binarization approach for license plate
CN108369735B (en) Method for determining the position of a plurality of objects in a digital image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40035905

Country of ref document: HK

GR01 Patent grant
GR01 Patent grant
CP03 Change of name, title or address

Address after: 518000 podium 502A and 502B, podium 602, Luohu investment holding building, No. 112, Qingshuihe 1st Road, Qingshuihe community, Qingshuihe street, Luohu District, Shenzhen, Guangdong

Patentee after: Shenzhen Zhenmai Biotechnology Co.,Ltd.

Country or region after: China

Address before: 518000 5th and 6th floors, block 2, Shenye Jinyuan Building, No.116, Qingshuihe 1st Road, Qingshuihe street, Luohu District, Shenzhen City, Guangdong Province

Patentee before: Shenzhen Zhenmai Biotechnology Co.,Ltd.

Country or region before: China