CA2603389C - Identifying statistically linear data - Google Patents
Identifying statistically linear data Download PDFInfo
- Publication number
- CA2603389C CA2603389C CA2603389A CA2603389A CA2603389C CA 2603389 C CA2603389 C CA 2603389C CA 2603389 A CA2603389 A CA 2603389A CA 2603389 A CA2603389 A CA 2603389A CA 2603389 C CA2603389 C CA 2603389C
- Authority
- CA
- Canada
- Prior art keywords
- data set
- measure
- calculating
- smoothed
- original data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
-
- C—CHEMISTRY; METALLURGY
- C12—BIOCHEMISTRY; BEER; SPIRITS; WINE; VINEGAR; MICROBIOLOGY; ENZYMOLOGY; MUTATION OR GENETIC ENGINEERING
- C12Q—MEASURING OR TESTING PROCESSES INVOLVING ENZYMES, NUCLEIC ACIDS OR MICROORGANISMS; COMPOSITIONS OR TEST PAPERS THEREFOR; PROCESSES OF PREPARING SUCH COMPOSITIONS; CONDITION-RESPONSIVE CONTROL IN MICROBIOLOGICAL OR ENZYMOLOGICAL PROCESSES
- C12Q1/00—Measuring or testing processes involving enzymes, nucleic acids or microorganisms; Compositions therefor; Processes of preparing such compositions
- C12Q1/68—Measuring or testing processes involving enzymes, nucleic acids or microorganisms; Compositions therefor; Processes of preparing such compositions involving nucleic acids
- C12Q1/6844—Nucleic acid amplification reactions
- C12Q1/6851—Quantitative amplification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/08—Feature extraction
- G06F2218/10—Feature extraction by analysing the shape of a waveform, e.g. extracting parameters relating to peaks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Complex Calculations (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Apparatus Associated With Microorganisms And Enzymes (AREA)
- Measuring Or Testing Involving Enzymes Or Micro-Organisms (AREA)
- Measuring Pulse, Heart Rate, Blood Pressure Or Blood Flow (AREA)
- Radar Systems Or Details Thereof (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
- Investigating Or Analysing Biological Materials (AREA)
Abstract
Methods, apparatus, and systems are provided for processing a data set having noise to determine whether the data set exhibits statistically linear behavior. A true data signal is calculated based on local properties of the data, and an estimate of the noise in the data is calculated from the true data signal. A measure of the estimated noise is then compared to properties of a linear fit to the data set.
Description
IDENTIFYING STATISTICALLY LINEAR DATA
BACKGROUND OF THE INVENTION
[0001] The present invention relates generally to data processing systems and methods, and more particularly to systems and methods for identifying statistically linear data in a data set of an amplification process, such as polymerase chain reaction (PCR).
BACKGROUND OF THE INVENTION
[0001] The present invention relates generally to data processing systems and methods, and more particularly to systems and methods for identifying statistically linear data in a data set of an amplification process, such as polymerase chain reaction (PCR).
[0002] Many experimental processes exhibit amplification of a quantity. For example, in PCR, the quantity may correspond to the number of parts of a DNA strand that have been replicated, which dramatically increases during an amplification stage or region. Other experimental processes exhibiting amplification include bacterial growth processes. The quantity is detected from an experimental device via a data signal, whose data points are analyzed to determine information about the amplification. As part of the data analysis, it is important to know if amplification has potentially occurred; otherwise, effort might be wasted on analyzing non-amplifying data. If the data is statistically linear, then amplification has not occurred.
[0003] Ideally, the data from the amplification detection device would be a monotonic and continuous signal, thus one could easily identify whether the data, or portions thereof, has statistically linear behavior. However, the signal from the amplification device typically contains noise, thus making identifying a behavior of the signal difficult. The noise manifests itself in each data point in the signal from the device having random fluctuations that occur on top of the true signal, e.g. the actual number of DNA strands. Thus, the data requires processing to allow for identifying of linear behavior.
[0004] A typical prior method for processing data to determine if it is statistically linear is with a linear least squares (LSQ) fit. The correlation value of the LSQ fit can be used to determine whether there is an adequate fit. By standard convention, a correlation value of 0 is related to a bad fit, thus the data is not linear, and a value of 1 suggests a good fit for linearity.
The problem is that in the presence of noise, the correlation value can be close to 0 or 1 for data that looks statistically linear.
Additionally, the correlation value does not correspond to a physical value that may provide additional insight and efficacy. Thus, the correlation value is not an acceptable criterion, particularly for data that can be extremely noisy.
The problem is that in the presence of noise, the correlation value can be close to 0 or 1 for data that looks statistically linear.
Additionally, the correlation value does not correspond to a physical value that may provide additional insight and efficacy. Thus, the correlation value is not an acceptable criterion, particularly for data that can be extremely noisy.
[0005] Therefore it is desirable to provide systems and methods for processing a data set having noise, and for identifying whether the dataset is statistically linear, that overcome the above and other problems.
BRIEF SUMMARY OF THE INVENTION
BRIEF SUMMARY OF THE INVENTION
[0006] Accordingly, embodiments of the present invention provide methods and systems directed to processing data to determine whether the data exhibits statistically linear behavior. Statistically linear data means that the data generally does not curve downward or upward or otherwise display amplification. Such data typically appears to be roughly linear with a large noise signal superposed upon it. The data may be received from real-time PCR processes or other processes exhibiting amplification or growth.
[0007] According to one exemplary embodiment of the present invention, a method of processing data is provided. The method typically includes receiving an original set of data points having a signal component and a noise component. The original data set is fit to a linear function. In one aspect, the fit is accomplished by calculating a linear least squares fit to the data set. The method also includes calculating a residual between the original data set and the linear fit, and calculating a measure of the residual between the original data set and the linear fit. In one aspect, the measure is a standard deviation.
[0008] The method also typically includes estimating the noise component present in the data set by calculating a smoothed data set and calculating the residual between the smoothed data set and the original data set. A smoothed data point is based on values of original data points that are local to that smoothed data point. In one aspect, a low pass filter is used to calculate the smoothed data set.
Exemplary low pass filters include a Savitzy-Golay filter, a digital filter, or digital smoothing polynomial filter. In another aspect, a value of a smoothed data point is an average of original data points within a window around the smoothed data point.
Exemplary low pass filters include a Savitzy-Golay filter, a digital filter, or digital smoothing polynomial filter. In another aspect, a value of a smoothed data point is an average of original data points within a window around the smoothed data point.
[009] The method also typically includes calculating a measure of the residual of the estimated noise, and comparing the measures to determine whether the original data set exhibits statistically linear behavior. The comparing may include calculating a ratio of the first and second measure to determine if the ratio is smaller or greater than a pre defined value. In one aspect, the pre-defined value is of order 1.
[0010] In preferred aspects, the method is implemented in a processor, such as a processor in a stand-alone computer, a network attached computer or a data acquisition device such as a real-time PCR machine. One example of a real-time PCR machine is the iCycler iQ System provided by Bio-Rad Laboratories.
[0011] According to another exemplary embodiment of the present invention, an information storage medium having a plurality of instructions adapted to direct an information processing device to perform an operation of processing data to determine whether the curve exhibits linear behavior is provided. In one aspect, the information storage medium is a RAM or ROM unit, hard drive, CD, DVD or other portable medium.
[0012] According to another exemplary embodiment of the present invention, a PCR
detection system is provided. The PCR detection system includes a detector for producing an original set of data points having a signal component and a noise component and includes logic for processing data to determine whether the data exhibits linear behavior.
detection system is provided. The PCR detection system includes a detector for producing an original set of data points having a signal component and a noise component and includes logic for processing data to determine whether the data exhibits linear behavior.
[0013] In accordance with another exemplary embodiment, a method of processing a data set to determine whether the data set exhibits statistically linear behavior includes receiving an original set of data points having a signal component and a noise component.
Each data point represents a physical quantity of a substance during an amplification process.
The method further includes fitting the original data set to a linear function, calculating a residual between the original data set and the fitted linear function, and calculating a first measure of the residual between the original data set and the fitted linear function. The method further includes estimating the noise component present in the original data set, by calculating a smoothed data set by determining a set of smoothed data points wherein a value of a smoothed data point is based on values of a plurality of original data points that are local to that smoothed data point, and by calculating a residual between the smoothed data set and the original data set. The method further includes calculating a second measure of the residual between the smoothed data set and the original data set, and comparing the first measure to the second measure to determine whether the original data set exhibits statistically linear behavior. The method is implemented using a processor.
[0013a] In accordance with another illustrative embodiment, a computer-readable information storage medium stores a plurality of instructions adapted to direct an information processing device to perform an operation of processing data to determine whether the data exhibits linear behavior. The operation includes receiving an original set of data points having a signal component and a noise component. Each data point represents a physical quantity of a substance during an amplification process. The operation further includes fitting the original data set to a linear function, calculating a residual between the original data set and the fitted linear function, and calculating a first measure of the residual between the original data set and the fitted linear function.
The operation further includes estimating the noise component present in the original data set, by calculating a smoothed data set by determining a set of smoothed data points wherein a value of a smoothed data point is based on values of a plurality of original data points that are local to that smoothed data point, and by calculating a residual between the smoothed data set and the original data set. The operation further includes calculating a second measure of the residual between the smoothed data set and the original data set, and comparing the first measure to the second measure to determine whether the original data set exhibits statistically linear behavior.
[0013b] In accordance with another illustrative embodiment, a polymerase chain reaction (PCR) detection system includes a detector for producing an original set of data points having a signal component and a noise component, wherein the data represents a PCR
amplification curve.
The system further includes means for processing data to determine whether the data exhibits linear behavior, by fitting the original data set to a linear function, calculating a residual between the original data set and the fitted linear function, calculating a first measure of the residual between the original data set and the fitted linear function, and estimating the noise component present in the original data set. Estimating the noise component includes calculating a smoothed data set by determining a set of smoothed data points, wherein a value of a smoothed data point is based on values of a plurality of original data points that are local to that smoothed data point, and calculating a residual between the smoothed data set and the original data set.
Determining further includes calculating a second measure of the residual between the smoothed data set and the original data set, and comparing the first measure to the second measure to determine whether the original data set exhibits statistically linear behavior.
Each data point represents a physical quantity of a substance during an amplification process.
The method further includes fitting the original data set to a linear function, calculating a residual between the original data set and the fitted linear function, and calculating a first measure of the residual between the original data set and the fitted linear function. The method further includes estimating the noise component present in the original data set, by calculating a smoothed data set by determining a set of smoothed data points wherein a value of a smoothed data point is based on values of a plurality of original data points that are local to that smoothed data point, and by calculating a residual between the smoothed data set and the original data set. The method further includes calculating a second measure of the residual between the smoothed data set and the original data set, and comparing the first measure to the second measure to determine whether the original data set exhibits statistically linear behavior. The method is implemented using a processor.
[0013a] In accordance with another illustrative embodiment, a computer-readable information storage medium stores a plurality of instructions adapted to direct an information processing device to perform an operation of processing data to determine whether the data exhibits linear behavior. The operation includes receiving an original set of data points having a signal component and a noise component. Each data point represents a physical quantity of a substance during an amplification process. The operation further includes fitting the original data set to a linear function, calculating a residual between the original data set and the fitted linear function, and calculating a first measure of the residual between the original data set and the fitted linear function.
The operation further includes estimating the noise component present in the original data set, by calculating a smoothed data set by determining a set of smoothed data points wherein a value of a smoothed data point is based on values of a plurality of original data points that are local to that smoothed data point, and by calculating a residual between the smoothed data set and the original data set. The operation further includes calculating a second measure of the residual between the smoothed data set and the original data set, and comparing the first measure to the second measure to determine whether the original data set exhibits statistically linear behavior.
[0013b] In accordance with another illustrative embodiment, a polymerase chain reaction (PCR) detection system includes a detector for producing an original set of data points having a signal component and a noise component, wherein the data represents a PCR
amplification curve.
The system further includes means for processing data to determine whether the data exhibits linear behavior, by fitting the original data set to a linear function, calculating a residual between the original data set and the fitted linear function, calculating a first measure of the residual between the original data set and the fitted linear function, and estimating the noise component present in the original data set. Estimating the noise component includes calculating a smoothed data set by determining a set of smoothed data points, wherein a value of a smoothed data point is based on values of a plurality of original data points that are local to that smoothed data point, and calculating a residual between the smoothed data set and the original data set.
Determining further includes calculating a second measure of the residual between the smoothed data set and the original data set, and comparing the first measure to the second measure to determine whether the original data set exhibits statistically linear behavior.
[0014] Reference to the remaining portions of the specification, including the drawings and claims, will realize other features and advantages of embodiments of the present invention. Further features and advantages of embodiments of the present invention, as well as the structure and operation of various embodiments are described in detail below with respect to the accompanying drawings. In the drawings, like reference numbers indicate identical or functionally similar elements.
BRIEF DESCRIPTION OF THE DRAWINGS
BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 illustrates an example of a PCR amplification curve.
[0016] Figure 2 illustrates a real-time PCR data set exhibiting noise and statistically linear behavior.
[0017] Figure 3 illustrates a real-time PCR data set exhibiting noise and amplification.
[0018] Figure 4 illustrates a method of processing a data set to determine whether the data set exhibits statistically linear behavior according to an embodiment of the present invention.
[0019] Figure 5A illustrates a linear fit to a data set exhibiting statistically linear behavior.
[0020] Figure 5B illustrates a linear fit to a data set exhibiting amplifying behavior.
[0021] Figure 6A illustrates a smoothed data set of real-time PCR data according to an embodiment of the present invention.
[0022] Figure 6B illustrates an estimated noise of real-time PCR data according to an embodiment of the present invention.
[0023] Figure 7 illustrates a system that processes real-time PCR data according to an embodiment of the present invention.
DETAILED DESCRIPTION OF THE INVENTION
DETAILED DESCRIPTION OF THE INVENTION
[0024] The present invention provides techniques for processing a data set and identifying whether the data set is statistically linear, as well as distinguishing such a linear data set from a data set containing an amplification signal. In preferred aspects, the present invention is particularly useful for processing data from PCR growth or amplification processes to identify and remove statistically linear data prior to further analysis of the data. It should be appreciated, however, that the teachings of the present invention are applicable to processing any data set or curve that may include noise, and particularly curves that should otherwise exhibit growth or amplification such as a bacterial growth process.
[0025] Figure 1 shows an example of a PCR curve 100, where intensity values 110 vs.
cycle number 120 are plotted for a typical PCR process. The values 110 may be any physical quantity of interest, and the cycle number may be any unit associated with time or number of steps in the process. Such amplification curves typically have a linear region 130 followed by an amplification region 140 and then by an asymptotic region 150, as shown in FIG. 1.
There also might be additional types of behavior such as downward curving data. An amplification region may have exponential, sigmoidal, high order polynomial, or other type of logistic function or logistic curve that models growth.
cycle number 120 are plotted for a typical PCR process. The values 110 may be any physical quantity of interest, and the cycle number may be any unit associated with time or number of steps in the process. Such amplification curves typically have a linear region 130 followed by an amplification region 140 and then by an asymptotic region 150, as shown in FIG. 1.
There also might be additional types of behavior such as downward curving data. An amplification region may have exponential, sigmoidal, high order polynomial, or other type of logistic function or logistic curve that models growth.
[0026] To understand the experimental process involved, it is important to identify the position and shape of amplification region 140. For example, in a PCR process, it may be desirable to identify the onset of amplification, which occurs at the end of the baseline region (linear region 130). A step in identifying the location is to identify if a possible amplification region even exists, as a PCR process may not show any amplification. However, since real-time PCR data has noise, the identification of whether the data set might exhibit amplification, or equivalently that it is not statistically linear, can be difficult.
[0027] For example, Figure 2 illustrates a linear region 230 of a real-time PCR curve 200 made from a data set with data points 240 that include a signal and noise.
Note that even for devices that produce a constant signal, this data must be broken into data points for analysis.
The noise causes the fluctuations in the data points. Overall, the data is generally moving upward (i.e. positive slope) in a linear fashion. However, as curve 200 is very non-linear from point to point, the generally linear behavior cannot be determined by directly analyzing curve 200 at any one point along the curve. A direct analysis of curve 200 would falsely determine that the data does not exhibit statistically linear behavior.
Embodiments of the present invention effectively determine whether data exhibits statistically linear behavior.
Note that even for devices that produce a constant signal, this data must be broken into data points for analysis.
The noise causes the fluctuations in the data points. Overall, the data is generally moving upward (i.e. positive slope) in a linear fashion. However, as curve 200 is very non-linear from point to point, the generally linear behavior cannot be determined by directly analyzing curve 200 at any one point along the curve. A direct analysis of curve 200 would falsely determine that the data does not exhibit statistically linear behavior.
Embodiments of the present invention effectively determine whether data exhibits statistically linear behavior.
[0028] Additionally, it is important to differentiate data curves having linear behavior and data curves having amplifying behavior. Figure 3 illustrates a real-time PCR
curve 300 that exhibits amplification. Initially, the data exhibits linear behavior in region 330 and in later cycles there is amplification in region 340. Embodiments of the present invention robustly and with. consistent accuracy differentiate between PCR curve 200 having only linear behavior and PCR curves possibly having an amplifying region, such as PCR
curve 300.
curve 300 that exhibits amplification. Initially, the data exhibits linear behavior in region 330 and in later cycles there is amplification in region 340. Embodiments of the present invention robustly and with. consistent accuracy differentiate between PCR curve 200 having only linear behavior and PCR curves possibly having an amplifying region, such as PCR
curve 300.
[0029] Figure 4 illustrates a method 400 of processing data to determine whether the data exhibits statistically linear behavior according to an embodiment of the present invention.
The data set is composed of data points and represents a curve having a signal component and a noise component.
The data set is composed of data points and represents a curve having a signal component and a noise component.
[0030] In step 405, the data set is first collected or received. The data set may be received through many mechanisms. For example, the data set may be acquired by a processor (executing instructions) resident in a PCR data acquiring device such as an iCycler iQ device or similar PCR analysis device. The data set may be provided to the processor in real time as the data is being collected, or it may be stored in a memory unit or buffer and provided to the processor after the experiment has been completed. Similarly, the data set may be provided to a separate system such as a desktop computer system via a network connection (e.g., LAN, VPN, intranet, Internet, etc.) or direct connection (e.g., USB or other direct wired or wireless connection) to the acquiring device, or provided on a portable medium such as a CD, DVD, floppy disk or the like to a stand-alone computer system. After the data set has been received or acquired, the data may be analyzed.
[0031] In step 410, a linear fit to the data set is calculated. Typically, a fit defines a merit function S that measures the agreement or difference between the data set and the fit, where small values of the merit function typically represent better parameters for the fit. For example, in a linear least squares fit, the merit function is the squares of the difference between the data values Y and the fit function f (xi), where for N data N
points S = (Y; - f (Xi)) 2. In a PCR process, Y is the data intensity and x is the cycle number. Figure 5A shows a linear fit 510 of the PCR curve 200. Figure 5B shows a linear fit 550 of PCR curve 300.
[00321 Merit functions may include different weight contributions or normalization factors to the merit function for different data points. Merit functions may also scale data point values or take a function of data points before a difference is taken. The difference may be taken between the data at one x value and f (x) at a different x value. For example, a term in the merit function may represent the length of a line from data curve to the linear fit, such that the line is perpendicular to the linear fit. This occurs at a difference cycle number unless the linear fit has a slope of 0. One skilled in the art will recognize the many different merit functions that could be used.
[0033] In step 415, a residual R between the data and the linear fit is calculated. The residual R is a set of values corresponding to an error in the data points from the linear fit.
For example, the residual may be the difference in the linear fit value and the actual data point for each cycle number, giving R. = Y; - f (xi), which is a standard form of the residual.
In some embodiments, the residual is related to the values used to determine the merit function of the linear fit. In other embodiments, the residual is a different value. In Figure 5A, errors 520 are used to calculate values of the residual R between curve 200 and linear fit 510. In Figure 5B, errors 560 are used to calculate values of the residual R
between curve 300 and linear fit 550.
[0034] In step 420, a measure cs, of the residual between the data and the linear fit is calculated. The measure is a single value made from the set of values that are the residual.
N
In one embodiment, the residual is a standard deviation, giving o'1 = 1 ~R? .
Some N i_1 embodiments may have a weighting value for each value of the residual, and other embodiments may subject each residual value or all residual values to additional or other functions. One skilled in the art will recognize the many different measures that could be used.
[0035] In step 425, an estimated noise component present in the data set is calculated. The data is presumed to consist of two components, a true signal and noise. Thus, the noise is the difference between the true signal and the actual data point. However, the true signal can never be directly measured as noise is always added or present when a signal is detected.
[0036] The true signal is estimated as a smoothed data set composed of smoothed data points. Figure 6A shows a smoothed data set 670 of PCR curve 300. A value of a smoothed data point is based on a function G of a plurality of original data points that are local to that smoothed data point. The term local relates to how far away the x value of the data points are from the data point being calculated. For example, a point may be local to another point if they differ by a preset number (window) of cycles. A window of three and five cycles has proved adequate, but other windows may be used, such as 10 or 20 cycles or more. A
variable window value may also be used, i.e. each smoothed data point may be calculated with a different window. Additionally, a window having fractions of a cycle may be used, for example where fractional data points are interpolated. A window may also not be symmetric around a data point, i.e. one point before and three points after that data point may be used. A point ceases to be local once the difference in the x value approaches the total scale used, i.e. total number of cycles.
[0037] In one embodiment, the function G is a moving average or low pass filter. For example, the function G may take an average of the original data points within a prescribed L+K
number of cycles, e.g. a centered mean. Thus in one embodiment, G(xL) = 1 IY2 2K + 1;_L-K
where L is the index of the smoothed data point being calculated and K is the window used.
[0038] Also, in step 425, a residual between the smoothed data and the original data is calculated. This residual is defined to be the estimated noise. The residual between the smoothed data and the original data may be defined in the same manner as the residual between the original data and the linear fit, or the residuals may be defined in a different manner. Figure 6B shows an estimated noise component 680 associated with PCR
curve 300 and smoothed data set 670. A superposition of noise component 680 on signal 670 gives the data curve 300.
[0039] In step 430, a measure 62 of the residual between the smoothed data and the original data is calculated. The 62 value is used as a measure of the amplitude of intrinsic noise. In one embodiment 62 is a standard deviation. The measures 61 and G2 may be defined in a similar or different fashion.
[0040] In step 435, the first measure 61 is compared to the second measure 62 to determine whether the data set exhibits linear behavior. In one embodiment, a ratio of 61 and 62 is taken. If the ratio is smaller or greater than a pre-defined value then the data is determined to exhibit linear behavior. For example, if 61/62 is less than a value of order one, e.g. 1.5, the data is determined to be linear. Equivalently, the expression 61< co* 62 may be used. This expression states that the measure of the difference between the data and a linear fit must be less than a constant times the measure of the estimated noise present in the data. In some embodiments, the value of co may vary.
[0041] The constant co is related to the fact that the definition of noise, as well as other values, is not unique. The value for co may be obtained by examining large numbers of data sets to obtain a reasonable value for this number. Studies have indicated a value of 1.5 works well for the constant (co), when a standard deviation of a standard residual is used. When other residuals and measures of the residual are used, other values might be more suitable. In general, a value of co on the order of 1 should work well.
[0042] Once the data has been identified as statistically flat (linear), e.g.
not curving downward or upward or otherwise displaying amplification, the data may be discarded from further analysis.
[0043] In certain aspects, code and instructions for controlling a processor to implement the data processing techniques of the present invention are stored on a computer-readable or information storage medium such as a RAM or ROM unit, hard drive, CD, DVD or other portable medium.
[0065] Figure 7 illustrates a system 700 according to one embodiment of the present invention. The system as shown includes a sample 705, such as bacteria or DNA, within a sample holder 710. A physical characteristic 715, such as a fluorescence intensity value, from the sample is detected by detector 720. A signal 725, including a noise component, is sent from detector 720 to logic system 730. The data from signal 725 may be stored in a local memory 735 or an external memory 740 or storage device 745. In one embodiment, an analog to digital converter converts an analog signal to digital form.
[0066] Logic system 730 may be, or may include, a computer system, ASIC, microprocessor, etc. It may also include or be coupled with a display (e.g., monitor, LED
display, etc.) and a user input device (e.g., mouse, keyboard, buttons, etc.).
Logic system 730 and the other components may be part of a stand alone or network connected computer system, or they may be directly attached to or incorporated in a thermal cycler device. Logic system 730 may also include optimization software that executes in a processor 750.
[0067] According to one embodiment, logic system 730 includes instructions for processing data and identifying statistically flat data. The instructions are preferably downloaded and stored in a memory modules 735, 740, or 745 (e.g., hard drive or other memory such as a local or attached RAM or ROM), although the instructions can be provided on any software storage medium such as a floppy disk, CD, DVD, etc. It should be understood that computer code for implementing aspects of the present invention can be implemented in a variety of coding languages such as C, C++, Java, Visual Basic, and others, or any scripting language, such as VBScript, JavaScript, Perl or markup languages such as XML. In addition, a variety of languages and protocols can be used in the external and internal storage and transmission of data and commands according to aspects of the present invention.
[0044] It will be appreciated that the process described herein is illustrative and that variations and modifications are possible. Steps described as sequential may be executed in parallel, order of steps may be varied, and steps may be modified or combined.
[0045] While the invention has been described by way of example and in terms of the specific embodiments, it is to be understood that the invention is not limited to the disclosed embodiments. To the contrary, it is intended to cover various modifications and similar arrangements as would be apparent to those skilled in the art. Therefore, the scope of the appended claims should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements.
points S = (Y; - f (Xi)) 2. In a PCR process, Y is the data intensity and x is the cycle number. Figure 5A shows a linear fit 510 of the PCR curve 200. Figure 5B shows a linear fit 550 of PCR curve 300.
[00321 Merit functions may include different weight contributions or normalization factors to the merit function for different data points. Merit functions may also scale data point values or take a function of data points before a difference is taken. The difference may be taken between the data at one x value and f (x) at a different x value. For example, a term in the merit function may represent the length of a line from data curve to the linear fit, such that the line is perpendicular to the linear fit. This occurs at a difference cycle number unless the linear fit has a slope of 0. One skilled in the art will recognize the many different merit functions that could be used.
[0033] In step 415, a residual R between the data and the linear fit is calculated. The residual R is a set of values corresponding to an error in the data points from the linear fit.
For example, the residual may be the difference in the linear fit value and the actual data point for each cycle number, giving R. = Y; - f (xi), which is a standard form of the residual.
In some embodiments, the residual is related to the values used to determine the merit function of the linear fit. In other embodiments, the residual is a different value. In Figure 5A, errors 520 are used to calculate values of the residual R between curve 200 and linear fit 510. In Figure 5B, errors 560 are used to calculate values of the residual R
between curve 300 and linear fit 550.
[0034] In step 420, a measure cs, of the residual between the data and the linear fit is calculated. The measure is a single value made from the set of values that are the residual.
N
In one embodiment, the residual is a standard deviation, giving o'1 = 1 ~R? .
Some N i_1 embodiments may have a weighting value for each value of the residual, and other embodiments may subject each residual value or all residual values to additional or other functions. One skilled in the art will recognize the many different measures that could be used.
[0035] In step 425, an estimated noise component present in the data set is calculated. The data is presumed to consist of two components, a true signal and noise. Thus, the noise is the difference between the true signal and the actual data point. However, the true signal can never be directly measured as noise is always added or present when a signal is detected.
[0036] The true signal is estimated as a smoothed data set composed of smoothed data points. Figure 6A shows a smoothed data set 670 of PCR curve 300. A value of a smoothed data point is based on a function G of a plurality of original data points that are local to that smoothed data point. The term local relates to how far away the x value of the data points are from the data point being calculated. For example, a point may be local to another point if they differ by a preset number (window) of cycles. A window of three and five cycles has proved adequate, but other windows may be used, such as 10 or 20 cycles or more. A
variable window value may also be used, i.e. each smoothed data point may be calculated with a different window. Additionally, a window having fractions of a cycle may be used, for example where fractional data points are interpolated. A window may also not be symmetric around a data point, i.e. one point before and three points after that data point may be used. A point ceases to be local once the difference in the x value approaches the total scale used, i.e. total number of cycles.
[0037] In one embodiment, the function G is a moving average or low pass filter. For example, the function G may take an average of the original data points within a prescribed L+K
number of cycles, e.g. a centered mean. Thus in one embodiment, G(xL) = 1 IY2 2K + 1;_L-K
where L is the index of the smoothed data point being calculated and K is the window used.
[0038] Also, in step 425, a residual between the smoothed data and the original data is calculated. This residual is defined to be the estimated noise. The residual between the smoothed data and the original data may be defined in the same manner as the residual between the original data and the linear fit, or the residuals may be defined in a different manner. Figure 6B shows an estimated noise component 680 associated with PCR
curve 300 and smoothed data set 670. A superposition of noise component 680 on signal 670 gives the data curve 300.
[0039] In step 430, a measure 62 of the residual between the smoothed data and the original data is calculated. The 62 value is used as a measure of the amplitude of intrinsic noise. In one embodiment 62 is a standard deviation. The measures 61 and G2 may be defined in a similar or different fashion.
[0040] In step 435, the first measure 61 is compared to the second measure 62 to determine whether the data set exhibits linear behavior. In one embodiment, a ratio of 61 and 62 is taken. If the ratio is smaller or greater than a pre-defined value then the data is determined to exhibit linear behavior. For example, if 61/62 is less than a value of order one, e.g. 1.5, the data is determined to be linear. Equivalently, the expression 61< co* 62 may be used. This expression states that the measure of the difference between the data and a linear fit must be less than a constant times the measure of the estimated noise present in the data. In some embodiments, the value of co may vary.
[0041] The constant co is related to the fact that the definition of noise, as well as other values, is not unique. The value for co may be obtained by examining large numbers of data sets to obtain a reasonable value for this number. Studies have indicated a value of 1.5 works well for the constant (co), when a standard deviation of a standard residual is used. When other residuals and measures of the residual are used, other values might be more suitable. In general, a value of co on the order of 1 should work well.
[0042] Once the data has been identified as statistically flat (linear), e.g.
not curving downward or upward or otherwise displaying amplification, the data may be discarded from further analysis.
[0043] In certain aspects, code and instructions for controlling a processor to implement the data processing techniques of the present invention are stored on a computer-readable or information storage medium such as a RAM or ROM unit, hard drive, CD, DVD or other portable medium.
[0065] Figure 7 illustrates a system 700 according to one embodiment of the present invention. The system as shown includes a sample 705, such as bacteria or DNA, within a sample holder 710. A physical characteristic 715, such as a fluorescence intensity value, from the sample is detected by detector 720. A signal 725, including a noise component, is sent from detector 720 to logic system 730. The data from signal 725 may be stored in a local memory 735 or an external memory 740 or storage device 745. In one embodiment, an analog to digital converter converts an analog signal to digital form.
[0066] Logic system 730 may be, or may include, a computer system, ASIC, microprocessor, etc. It may also include or be coupled with a display (e.g., monitor, LED
display, etc.) and a user input device (e.g., mouse, keyboard, buttons, etc.).
Logic system 730 and the other components may be part of a stand alone or network connected computer system, or they may be directly attached to or incorporated in a thermal cycler device. Logic system 730 may also include optimization software that executes in a processor 750.
[0067] According to one embodiment, logic system 730 includes instructions for processing data and identifying statistically flat data. The instructions are preferably downloaded and stored in a memory modules 735, 740, or 745 (e.g., hard drive or other memory such as a local or attached RAM or ROM), although the instructions can be provided on any software storage medium such as a floppy disk, CD, DVD, etc. It should be understood that computer code for implementing aspects of the present invention can be implemented in a variety of coding languages such as C, C++, Java, Visual Basic, and others, or any scripting language, such as VBScript, JavaScript, Perl or markup languages such as XML. In addition, a variety of languages and protocols can be used in the external and internal storage and transmission of data and commands according to aspects of the present invention.
[0044] It will be appreciated that the process described herein is illustrative and that variations and modifications are possible. Steps described as sequential may be executed in parallel, order of steps may be varied, and steps may be modified or combined.
[0045] While the invention has been described by way of example and in terms of the specific embodiments, it is to be understood that the invention is not limited to the disclosed embodiments. To the contrary, it is intended to cover various modifications and similar arrangements as would be apparent to those skilled in the art. Therefore, the scope of the appended claims should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements.
Claims (23)
1. A method of processing a data set to determine whether the data set exhibits statistically linear behavior, the method comprising:
receiving an original set of data points having a signal component and a noise component, each data point representing a physical quantity of a substance during an amplification process;
fitting the original data set to a linear function;
calculating a residual between the original data set and the fitted linear function;
calculating a first measure of the residual between the original data set and the fitted linear function;
estimating the noise component present in the original data set, by:
i) calculating a smoothed data set by determining a set of smoothed data points, wherein a value of a smoothed data point is based on values of a plurality of original data points that are local to that smoothed data point; and ii) calculating a residual between the smoothed data set and the original data set;
calculating a second measure of the residual between the smoothed data set and the original data set; and comparing the first measure to the second measure to determine whether the original data set exhibits statistically linear behavior, wherein the method is implemented using a processor.
receiving an original set of data points having a signal component and a noise component, each data point representing a physical quantity of a substance during an amplification process;
fitting the original data set to a linear function;
calculating a residual between the original data set and the fitted linear function;
calculating a first measure of the residual between the original data set and the fitted linear function;
estimating the noise component present in the original data set, by:
i) calculating a smoothed data set by determining a set of smoothed data points, wherein a value of a smoothed data point is based on values of a plurality of original data points that are local to that smoothed data point; and ii) calculating a residual between the smoothed data set and the original data set;
calculating a second measure of the residual between the smoothed data set and the original data set; and comparing the first measure to the second measure to determine whether the original data set exhibits statistically linear behavior, wherein the method is implemented using a processor.
2. The method of claim 1, wherein fitting the original data set includes using a least squares fit.
3. The method of claim 1, wherein calculating the smoothed data set comprises using a low pass filter.
4. The method of claim 1, wherein a value of a smoothed data point is an average of the original data points within a window around the smoothed data point.
5. The method of claim 4, wherein the window is five units.
6. The method of claim 1, wherein the first measure and the second measure are each a standard deviation.
7. The method of claim 1, wherein comparing includes calculating a ratio of the first and second measure to determine if the ratio is smaller or greater than a pre-defined value.
8. The method of claim 7, wherein the pre-defined value is of order 1.
9. The method of claim 1, wherein the data represents a polymerase chain reaction (PCR) amplification curve.
10. The method of claim 1, wherein the processor is integrated in one of a stand alone computer system, a networked computer system or a real-time polymerase chain reaction (PCR) machine.
11. A computer-readable information storage medium storing a plurality of instructions adapted to direct an information processing device to perform an operation of processing data to determine whether the data exhibits linear behavior, the operation comprising the steps of:
receiving an original set of data points having a signal component and a noise component, each data point representing a physical quantity of a substance during an amplification process;
fitting the original data set to a linear function;
calculating a residual between the original data set and the fitted linear function;
calculating a first measure of the residual between the original data set and the fitted linear function;
estimating the noise component present in the original data. set, by:
i) calculating a smoothed data set by determining a set of smoothed data points, wherein a value of a smoothed data point is based on values of a plurality of original data points that are local to that smoothed data point; and
receiving an original set of data points having a signal component and a noise component, each data point representing a physical quantity of a substance during an amplification process;
fitting the original data set to a linear function;
calculating a residual between the original data set and the fitted linear function;
calculating a first measure of the residual between the original data set and the fitted linear function;
estimating the noise component present in the original data. set, by:
i) calculating a smoothed data set by determining a set of smoothed data points, wherein a value of a smoothed data point is based on values of a plurality of original data points that are local to that smoothed data point; and
12 ii) calculating a residual between the smoothed data set and the original data set;
calculating a second measure of the residual between the smoothed data set and the original data set; and comparing the first measure to the second measure to determine whether the original data set exhibits statistically linear behavior.
12. The information storage medium of claim 11, wherein fitting the original data set includes using a least squares fit.
calculating a second measure of the residual between the smoothed data set and the original data set; and comparing the first measure to the second measure to determine whether the original data set exhibits statistically linear behavior.
12. The information storage medium of claim 11, wherein fitting the original data set includes using a least squares fit.
13. The information storage medium of claim 11, wherein calculating the smoothed data set comprises using a low pass filter.
14. The information storage medium of claim 11, wherein a value of a smoothed data point is an average of the original data points within a window around the smoothed data point.
15. The information storage medium of claim 11, wherein the first measure and the second measure are each a standard deviation.
16. The information storage medium of claim 11, wherein the comparing includes calculating a ratio of the first and second measure to determine if the ratio is smaller or greater than a pre-defined value.
17. The information storage medium of claim 11, wherein the data represents a polymerase chain reaction (PCR) amplification curve.
18. A polymerase chain reaction (PCR) detection system comprising:
a detector for producing an original set of data points having a signal component and a noise component, wherein the data represents a PCR amplification curve; and means for processing data to determine whether the data exhibits linear behavior, by:
fitting the original data set to a linear function;
calculating a residual between the original data set and the fitted linear function;
calculating a first measure of the residual between the original data set and the fitted linear function;
estimating the noise component present in the original data set, by:
i) calculating a smoothed data set by determining a set of smoothed data points, wherein a value of a smoothed data point is based on values of a plurality of original data points that are local to that smoothed data point; and ii) calculating a residual between the smoothed data set and the original data set;
calculating a second measure of the residual between the smoothed data set and the original data set; and comparing the first measure to the second measure to determine whether the original data set exhibits statistically linear behavior.
a detector for producing an original set of data points having a signal component and a noise component, wherein the data represents a PCR amplification curve; and means for processing data to determine whether the data exhibits linear behavior, by:
fitting the original data set to a linear function;
calculating a residual between the original data set and the fitted linear function;
calculating a first measure of the residual between the original data set and the fitted linear function;
estimating the noise component present in the original data set, by:
i) calculating a smoothed data set by determining a set of smoothed data points, wherein a value of a smoothed data point is based on values of a plurality of original data points that are local to that smoothed data point; and ii) calculating a residual between the smoothed data set and the original data set;
calculating a second measure of the residual between the smoothed data set and the original data set; and comparing the first measure to the second measure to determine whether the original data set exhibits statistically linear behavior.
19. The PCR detection system of claim 18, wherein fitting the original data set includes using a least squares fit.
20. The PCR detection system of claim 18, wherein calculating the smoothed data set comprises using a low pass filter.
21. The PCR detection system of claim 18, wherein a value of a smoothed data point is an average of the original data points within a window around the smoothed data point.
22. The PCR detection system of claim 18, wherein the first measure and the second measure are each a standard deviation.
23. The PCR detection system of claim 18, wherein the comparing includes calculating a ratio of the first and second measure to determine if the ratio is smaller or greater than a pre-defined value.
Applications Claiming Priority (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US68118205P | 2005-05-13 | 2005-05-13 | |
US60/681,182 | 2005-05-13 | ||
US11/432,856 | 2006-05-11 | ||
US11/432,856 US7647187B2 (en) | 2005-05-13 | 2006-05-11 | Identifying statistically linear data |
PCT/US2006/018549 WO2006124673A2 (en) | 2005-05-13 | 2006-05-12 | Identifying statistically linear data |
Publications (2)
Publication Number | Publication Date |
---|---|
CA2603389A1 CA2603389A1 (en) | 2006-11-23 |
CA2603389C true CA2603389C (en) | 2012-07-10 |
Family
ID=37431939
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CA2603389A Expired - Fee Related CA2603389C (en) | 2005-05-13 | 2006-05-12 | Identifying statistically linear data |
Country Status (6)
Country | Link |
---|---|
EP (1) | EP1880334A4 (en) |
JP (1) | JP5091122B2 (en) |
CN (1) | CN101292245B (en) |
AU (1) | AU2006247597B2 (en) |
CA (1) | CA2603389C (en) |
WO (1) | WO2006124673A2 (en) |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090119020A1 (en) * | 2007-09-25 | 2009-05-07 | Roche Molecular Systems, Inc. | Pcr elbow determination using quadratic test for curvature analysis of a double sigmoid |
JPWO2009054474A1 (en) * | 2007-10-26 | 2011-03-10 | 凸版印刷株式会社 | Allele determination apparatus and method, and computer program |
JP5166608B2 (en) * | 2008-09-09 | 2013-03-21 | バイオ−ラッド ラボラトリーズ,インコーポレイティド | Multistage regression-based PCR analysis system |
US8219366B2 (en) * | 2009-08-26 | 2012-07-10 | Roche Molecular Sytems, Inc. | Determination of elbow values for PCR for parabolic shaped curves |
JP5379044B2 (en) * | 2010-02-25 | 2013-12-25 | 株式会社日立ハイテクノロジーズ | Automatic analyzer |
US10176293B2 (en) | 2012-10-02 | 2019-01-08 | Roche Molecular Systems, Inc. | Universal method to determine real-time PCR cycle threshold values |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CA2139070C (en) * | 1994-12-23 | 2010-03-30 | Burton W. Blais | Method for enhancing detection ability of nucleic acid assays employing polymerase chain reaction |
US6175602B1 (en) * | 1998-05-27 | 2001-01-16 | Telefonaktiebolaget Lm Ericsson (Publ) | Signal noise reduction by spectral subtraction using linear convolution and casual filtering |
WO2002034948A2 (en) * | 2000-10-25 | 2002-05-02 | City Of Hope | Candidate region mismatch scanning for genotyping and mutation detection |
US7228237B2 (en) * | 2002-02-07 | 2007-06-05 | Applera Corporation | Automatic threshold setting and baseline determination for real-time PCR |
US20050069904A1 (en) * | 2003-09-29 | 2005-03-31 | Stuart Peirson | Analysis of real-time PCR data |
-
2006
- 2006-05-12 CN CN2006800162194A patent/CN101292245B/en not_active Expired - Fee Related
- 2006-05-12 JP JP2008511434A patent/JP5091122B2/en not_active Expired - Fee Related
- 2006-05-12 AU AU2006247597A patent/AU2006247597B2/en not_active Ceased
- 2006-05-12 EP EP06752532A patent/EP1880334A4/en not_active Withdrawn
- 2006-05-12 CA CA2603389A patent/CA2603389C/en not_active Expired - Fee Related
- 2006-05-12 WO PCT/US2006/018549 patent/WO2006124673A2/en active Application Filing
Also Published As
Publication number | Publication date |
---|---|
CA2603389A1 (en) | 2006-11-23 |
EP1880334A4 (en) | 2010-01-20 |
WO2006124673A2 (en) | 2006-11-23 |
AU2006247597A1 (en) | 2006-11-23 |
CN101292245B (en) | 2013-04-17 |
JP2008546048A (en) | 2008-12-18 |
JP5091122B2 (en) | 2012-12-05 |
CN101292245A (en) | 2008-10-22 |
WO2006124673A3 (en) | 2008-04-17 |
AU2006247597B2 (en) | 2010-02-11 |
EP1880334A2 (en) | 2008-01-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7720611B2 (en) | Baselining amplification data | |
JP6388627B2 (en) | Method and system for analyzing reaction using information system | |
CA2603389C (en) | Identifying statistically linear data | |
Matsunaga | How to Factor-Analyze Your Data Right: Do's, Don'ts, and How-To's. | |
JP4610196B2 (en) | Automatic threshold setting and baseline determination for real-time PCR | |
JP5166608B2 (en) | Multistage regression-based PCR analysis system | |
US7647187B2 (en) | Identifying statistically linear data | |
Rotello et al. | Analysis of RT distributions in the remember—know paradigm | |
JP6441812B2 (en) | Method for analyzing body fluid samples | |
JP6907388B2 (en) | Analytical methods and systems for analyzing nucleic acid amplification reactions | |
EP2439536A1 (en) | New classification method for spectral data | |
US8700381B2 (en) | Methods for nucleic acid quantification | |
CN107067034B (en) | Method and system for rapidly identifying infrared spectrum data classification | |
JP5313663B2 (en) | Baseline amplified data | |
WO2006124571A2 (en) | Baselining amplification data | |
JPS63229350A (en) | Analysis of state in spectrum analyser | |
JP2005108111A (en) | Method for determining discriminating item between groups and program for executing the method | |
JPH05173852A (en) | Method and device for estimating reliability of software |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
EEER | Examination request | ||
MKLA | Lapsed |
Effective date: 20160512 |