CN115309814A - Internet of things data reconstruction method based on structured low-rank tensor completion - Google Patents
Internet of things data reconstruction method based on structured low-rank tensor completion Download PDFInfo
- Publication number
- CN115309814A CN115309814A CN202210943271.5A CN202210943271A CN115309814A CN 115309814 A CN115309814 A CN 115309814A CN 202210943271 A CN202210943271 A CN 202210943271A CN 115309814 A CN115309814 A CN 115309814A
- Authority
- CN
- China
- Prior art keywords
- data
- matrix
- tensor
- order
- low
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/26—Visual data mining; Browsing structured data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/11—Complex mathematical operations for solving equations, e.g. nonlinear equations, general mathematical optimization problems
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Optimization (AREA)
- Mathematical Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Computational Mathematics (AREA)
- Operations Research (AREA)
- Algebra (AREA)
- Software Systems (AREA)
- Arrangements For Transmission Of Measured Signals (AREA)
Abstract
The invention relates to an Internet of things data reconstruction method based on structured low-rank tensor completion, which comprises the steps of firstly dispersing a monitoring area into a plurality of grid points, and deploying a sensor node in each grid point; assuming that a sensor node senses data once every other time slot, the data received by a base station in time T form a third-order tensor; secondly, data reconstruction is converted into a basic low-rank tensor completion problem, and a low-rank tensor completion model is constructed; and finally, carrying out block Hankel matrix change on the expansion matrix of each mode i of the third-order tensor, improving the basic low-order tensor completion model into a structured low-order tensor completion model, solving the augmented Lagrangian function of the structured low-order tensor completion model to obtain the third-order tensor, and completing data reconstruction of the Internet of things. Arranging data acquired at continuous moments by three-order tensors, and fully utilizing the spatial correlation of the data; the block Hankel matrix change is carried out on each mode i expansion matrix of the third-order tensor, data reconstruction is carried out through the combination of structuralization and low-rank tensor completion, the space-time correlation of the data is further mined and utilized, the influence of basic mismatch on reconstruction performance in a sparse constraint method is relieved, and the data reconstruction precision is improved.
Description
Technical Field
The invention belongs to the technical field of data processing of the Internet of things, and particularly relates to a data reconstruction method of the Internet of things based on structured low-rank tensor completion.
Background
Since the advent of the Internet of Things (IoT), as a computer and the Internet, the third wave of information industry development has spread its application across many areas. The IoT equipment layer is composed of a large number of sensor nodes with sensing and communication capabilities, the sensor nodes are deployed in a monitoring area in a random mode, and each sensor node has certain computing, storage and communication capabilities and can continuously monitor sensing environment information. Due to the influence of factors such as hardware condition limitation, unstable network communication, severe environment and the like, the data loss problem of the internet of things is inevitable. Data missing will cause the data of the internet of things to be unable to be normally used for subsequent analysis and application, so reconstructing the complete IoT data with high precision has become a research hotspot in the field.
IoT data reconstruction methods based on sparse constraints can be divided into three categories: a reconstruction method based on Compressed Sensing (CS for short), a reconstruction method based on Matrix Completion (MC for short), and a reconstruction method based on tensor Completion. In particular, CS-based methods use measurement matrices, compress sampled signals from IoT data that has sparsity in certain transform domains, and reconstruct the sampled signals using optimization methods. The data of the Internet of things can be arranged into a two-dimensional matrix according to the sensor nodes and the acquisition time, and the data space-time correlation enables the data matrix of the Internet of things to have low rank and accords with the premise of matrix completion application, so that missing data of the Internet of things can be reconstructed based on the matrix completion. The CS and MC based methods generally arrange data collected by spatially distributed sensor nodes in a vector form, thereby ignoring spatial correlation between sensor nodes. As the high-order expansion of the matrix, the tensor can express high-dimensional data in a natural and compact mode, so that the internet of things data completion method naturally expands from low-rank matrix completion to low-rank tensor completion. According to different tensor rank definitions, the internet of things data reconstruction method based on various low-rank tensor completions is proposed successively. Compared with reconstruction methods based on CS and MC, the reconstruction method based on tensor completion can further mine the spatiotemporal correlation of IoT data.
The first-order, second-order and high-order sparse constraint methods have very outstanding results in the data reconstruction research of the internet of things, but the methods all assume that signals are sparse a priori. However, since the real frequency of the data of the internet of things is actually specified in the continuous domain, when the base mismatch inevitably exists between the real frequency and the assumed base, the reconstruction performance of the sparse feature constraint-based method is reduced. Therefore, the method for reconstructing the data of the internet of things based on the structured low-rank tensor completion is provided, and the influence of the base mismatch on the reconstruction accuracy is further relieved by utilizing the time-space correlation among the data of the internet of things, so that the data reconstruction accuracy is improved.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to solve the technical problem of providing a method for reconstructing data of the Internet of things based on structured low-rank tensor completion.
The technical scheme adopted by the invention for solving the technical problems is as follows:
an Internet of things data reconstruction method based on structured low-rank tensor completion comprises the following steps:
due to data loss, only D environment information measured values are transmitted to the base station in time T, D < M multiplied by N multiplied by L, and the data received by the base station passesIt is shown that,a random sampling operator is represented that is,representing tensor from third orderThe data obtained by the middle random sampling comprises D environment information measurement values, and the positions of the non-sampling points are filled with zeros; Ω represents an observation set;
wherein alpha is i Tensor representing the third orderI =1,2,3, satisfying α, the weight of the kernel norm of the expanded matrix of (a) i > 0 andX (i) tensor representing the third orderThe mode i expands the matrix, | · | | non-woven phosphor * And | · | non-conducting phosphor F Respectively representing a matrix nuclear norm and an F-norm, wherein lambda is a regularization parameter;
wherein the content of the first and second substances,an operator for converting the matrix into a block hankel matrix;
converting the low-rank tensor completion model of the formula (3) into an equivalent constraint optimization problem of the formula (6) by introducing variable splitting;
solving the constraint equation (6) by using an alternating direction multiplier method, and firstly obtaining an augmented Lagrange function of an original objective function, so the augmented Lagrange function of the equation (6) is expressed as:
in the formula, beta represents a penalty coefficient, beta > 0, E (i) Andall represent lagrangian multipliers;
step 4, solving the formula (7) to obtain a third-order tensorAnd finishing the reconstruction of the data of the Internet of things.
Further, the alternative solving process of equation (7) is:
wherein, k represents the number of iterations,representing lagrange multipliersThe fold represents to restore the mode i expansion matrix to tensor;
sub-problem equations (8) and (9) can be expanded as:
solving equations (14) and (15) by a conjugate gradient algorithm:
in the formula, H represents Hermite transpose, and I represents a unit matrix;
the subproblem equation (10) can be expanded as:
further, the equation (18) is solved through singular value interception operation, so as to obtain:
in the formula (20), shrnk (a, τ) represents a nonlinear function, and the specific operation is to use a soft threshold operatorApplied to the singular values of matrix a; performing singular value decomposition on the matrix A, performing soft threshold operator operation on diagonal lines of the diagonal matrix of the nonnegative real numbers obtained by decomposition, putting vectors subjected to soft threshold back into the diagonal matrix of the nonnegative real numbers, and multiplying the decomposed components to obtain a matrix subjected to singular value interception operation, namely an output result of a nonlinear function shrnk (A, tau);
in the formula, q j Representing the jth singular value of the matrix a.
Compared with the prior art, the invention has the beneficial effects that:
on one hand, the method arranges the data acquired at continuous time in a third-order tensor form, makes full use of the spatial correlation of the data acquired by adjacent sensor nodes, and is beneficial to improving the reconstruction precision. On the other hand, in order to further improve the utilization of the space-time correlation of the data, block Hankel matrix change is carried out on each mode i expansion matrix of the third-order tensor, data reconstruction is carried out through structuralization processing and low-rank tensor completion, and the negative influence of base mismatch on reconstruction performance based on the sparse constraint method is relieved. The experimental comparison result shows that compared with a reconstruction method based on structured matrix completion and basic low-rank tensor completion, the method provided by the invention has the advantages that the reconstruction error is smaller and the reconstruction precision is higher under the same adoption proportion and different sampling proportions.
Drawings
FIG. 1 is a diagram of third order tensorsThe schematic diagram of the mode i expansion matrix of (1);
FIG. 2 is an error contrast plot of NDBC ocean surface temperature data reconstructed using different methods;
fig. 3 is a graph of error versus temperature data for a berkeley laboratory reconstructed using different methods.
Detailed Description
The technical solutions of the present invention are described in detail below with reference to the accompanying drawings and specific embodiments, but the scope of the present invention is not limited thereto.
The invention relates to an Internet of things data reconstruction method (a method for short, see figures 1-3) based on structured low-rank tensor completion, which comprises the following steps:
due to data loss, only D environment information measurements are transmitted to the base station in time T, D < M × N × L, then mathematically, the data received by the base station can be viewed as a tensor of third orderIs obtained by random sampling in a certain proportion, and the data received by the base station is passed throughIt is shown that,a random sampling operator is represented that is,representing tensor from third orderThe data obtained by the middle random sampling comprises D environment information measurement values, and the positions of the non-sampling points are filled with zeros; Ω represents an observation set; the sampling ratio rho = D/(M × N × L), 0 < rho < 1; wherein from the third order tensorData points obtained by medium random samplingExpressed as:
in the formula, x m,n,l Tensor representing the third orderThe data points in (a), M =1,2,. Cndot, M, N =1,2,. Cndot, N, L =1,2,. Cndot, L;
wherein alpha is i Tensor representing the third orderI =1,2,3, satisfying α i > 0 andX (i) tensor representing the third orderThe mode i expands the matrix, | · | | non-woven phosphor * And | · | non-counting F Respectively representing a matrix nuclear norm and an F-norm, wherein lambda is a regularization parameter;
wherein, the first and the second end of the pipe are connected with each other,an operator for converting the matrix into a block hankel matrix;
wherein k is 1 And k 2 Both represent the penil parameter, and P and Q represent the row number and column number of the matrix Y, respectively;
block Hankel matrixAny one of the sub-matrices Y of p Are all hankel matrices satisfying the following equation:
converting the low-rank tensor completion model of the formula (3) into an equivalent constraint optimization problem of the formula (6) by introducing variable splitting;
solving the constraint equation (6) by using an alternating direction multiplier method, and firstly obtaining an augmented Lagrange function of an original objective function, so the augmented Lagrange function of the equation (6) is expressed as:
in the formula, beta > 0 represents penalty coefficient, E (i) Andall represent lagrangian multipliers;
step 4, solving the formula (7) by adopting an alternating direction multiplier method to obtain a third-order tensorCompleting data reconstruction of the Internet of things; the update solving process of equation (7) is:
wherein k represents the number of iterations,to representThe mode i expansion matrix of (1), fold represents to restore the mode expansion matrix to tensor;
sub-problem equations (8) and (9) can be expressed as:
equations (14) and (15) are standard linear least squares problems solved by a conjugate gradient algorithm:
in the formula, H represents Hermite transpose, and I represents a unit matrix;
the subproblem equation (10) is expressed as:
solving equation (18) by singular value intercept operations to obtain:
in equation (20), shrink (A, τ) is a non-linear function that is specifically operated as a soft threshold operatorApplied to the singular values of matrix a; performing singular value decomposition on the matrix A, performing soft threshold operator operation on a diagonal line (elements on the diagonal line are singular values of the matrix A) of the diagonal matrix of the nonnegative real number obtained by decomposition, distinguishing by using a value tau, putting a vector after soft threshold back into the diagonal matrix of the nonnegative real number, and multiplying three decomposed components to obtain a matrix after singular value interception operation, namely an output result of a shrnk (A, tau) nonlinear function;
in the formula, q j Representing the jth singular value of the matrix a.
In order to verify the effectiveness of the method, the method and two methods in the prior art including the structured matrix completion and the basic low-rank tensor completion are respectively utilized to reconstruct the data of the Internet of things; selecting ocean surface temperature collected by a National Data Buoy Center (NDBC for short) and temperature Data collected by Berkeley research laboratory as test Data; because data loss in the Internet of things is inevitable, a small part of complete data subset is selected as real test data; specifically, NDBC data subsetsComprises ocean surface temperature data sensed by 40 sensor nodes in 50 time slots, a Berkeley research laboratory data subsetTemperature data sensed by 54 sensor nodes in 50 time slots are contained; in the experimental simulation, the data tensor containing the missing informationBy pairsAndobtaining by random sampling, i.e. using random sampling operatorsRandomly collecting D data on the real test data, and discarding other data; the reconstruction Error is characterized by a Normalized Mean Absolute Error (NMAE), which reflects the relative difference between the real test data and the reconstructed data, with a lower NAME generally indicating a better reconstruction accuracy, defined as:
in the formula (22), the reaction mixture is,respectively representing real test data and reconstructed data, wherein pi represents a sampling index subset of a complete entry set, namely the reconstruction error of only missing data is considered when NMAE is calculated;
for each method, the random sampling and reconstruction process was repeated 10 times and the average NMAE was calculated; the optimal parameters for each method were chosen individually to ensure convincing experimental results.
In the experiment, the third order tensor of the method of the inventionWeight of three modes [ alpha ] 1 ,α 2 ,α 3 ]Set to [0.33, 0.34](ii) a FIGS. 2 and 3 are respectively the reconstruction errors for NDBC data and Berkeley laboratory data at different sampling ratios; as can be seen from the figure, under the same sampling proportion and different sampling proportions, the reconstruction error of the data by utilizing the method of the invention is smaller than that of the other two methods; the data collected by the sensor nodes are arranged in the two-dimensional matrix, only the spatial correlation can be reflected, and the data collected by the sensor nodes distributed in the monitoring area are arranged in the third-order tensor, so that the damage to the time-space correlation of the data is avoided, and the reconstruction precision is improved; compared with the basic low n-rank tensor completion method, the method of the invention completes the third-order tensorThe block Hankel matrix change is carried out on each mode expansion matrix to enhance the data structure, the utilization of the data space-time correlation is further enhanced, and even under the condition that only a few data can be obtained at an extremely low sampling ratio, the method can reconstruct the complete data at high precision.
Nothing in this specification is said to apply to the prior art.
Claims (3)
1. An Internet of things data reconstruction method based on structured low-rank tensor completion is characterized by comprising the following steps:
step 1, dispersing an internet of things monitoring area into M multiplied by N grid points, wherein a sensor node is deployed in each grid point; suppose that the sensor node senses data once every time slot tau and transmits the data to the base station, so that the data received by the base station in the time T = L x tau forms a third-order tensor Representing a real number field, wherein M and N are positive integers;
only D environmental information measured values are transmitted to the base station in time T due to data loss, D < M multiplied by N multiplied by L, and then the data received by the base station passesIt is shown that the process of the present invention,a random sampling operator is represented that is,representing tensor from third orderThe data obtained by the middle random sampling comprises D environment information measurement values, and the positions of the non-sampling points are filled with zeros; Ω represents an observation set;
step 2, third order tensorHave spatial correlation between horizontal slice data and between side slice data, have temporal correlation between front slice data, and thus tensorThe method has low rank, so that data reconstruction can be converted into a low rank tensor completion problem, and the expression of a basic low rank tensor completion model is as follows:
wherein alpha is i Tensor representing the third orderI =1,2,3, satisfying α, the weight of the kernel norm of the expanded matrix of (a) i > 0 andX (i) tensor representing the third orderThe mode i of (1) expands the matrix, | | caldol |, L * And | · | non-conducting phosphor F Respectively representing a matrix kernel norm and an F-norm, wherein lambda is a regularization parameter;
step 3, for the third order tensorThe expansion matrix of each mode i carries out block Hankel matrix change, and punishment is given by third-order tensorMode i of (2) unfolding matrix X (i) And (3) forming a kernel norm of a block Hankel matrix, and improving a basic low-rank tensor completion model of the formula (2) into the following steps:
wherein the content of the first and second substances,an operator for converting the matrix into a block hankel matrix;
converting the low rank tensor completion model of equation (3) into an equivalence constraint optimization problem of equation (6) by introducing variable splitting:
solving the constraint equation (6) by using an alternating direction multiplier method, and firstly obtaining an augmented Lagrange function of an original objective function, so the augmented Lagrange function of the equation (6) is expressed as:
in the formula, beta represents a penalty coefficient, beta > 0, E (i) Andall represent lagrangian multipliers;
2. The method for reconstructing data of the internet of things based on the structured low-rank tensor completion as claimed in claim 1, wherein the alternative solution process of equation (7) is as follows:
wherein k represents the number of iterations,representing lagrange multipliersThe fold represents to restore the mode i expansion matrix to tensor;
sub-problem equations (8) and (9) can be expanded as:
solving equations (14) and (15) by a conjugate gradient algorithm:
in the formula, H represents Hermite transpose, and I represents a unit matrix;
the subproblem equation (10) can be expanded as:
3. the method for reconstructing data of the internet of things based on structured low-rank tensor completion as claimed in claim 1, wherein equation (18) is solved by singular value intercept operation to obtain:
in the formula (20), shrnk (a, τ) represents a nonlinear function, and the specific operation is to use a soft threshold operatorApplied to the singular values of the matrix a; performing singular value decomposition on the matrix A, performing soft threshold operator operation on diagonal lines of a non-negative real number diagonal matrix obtained by decomposition, putting vectors subjected to soft threshold back into the non-negative real number diagonal matrix, and multiplying decomposed components to obtain a matrix subjected to singular value interception operation, namely an output result of a nonlinear function shrink (A, tau);
in the formula, q j Representing the jth singular value of the matrix a.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210943271.5A CN115309814A (en) | 2022-08-08 | 2022-08-08 | Internet of things data reconstruction method based on structured low-rank tensor completion |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210943271.5A CN115309814A (en) | 2022-08-08 | 2022-08-08 | Internet of things data reconstruction method based on structured low-rank tensor completion |
Publications (1)
Publication Number | Publication Date |
---|---|
CN115309814A true CN115309814A (en) | 2022-11-08 |
Family
ID=83860354
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210943271.5A Pending CN115309814A (en) | 2022-08-08 | 2022-08-08 | Internet of things data reconstruction method based on structured low-rank tensor completion |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN115309814A (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116450636A (en) * | 2023-06-20 | 2023-07-18 | 石家庄学院 | Internet of things data completion method, equipment and medium based on low-rank tensor decomposition |
CN116955334A (en) * | 2023-06-27 | 2023-10-27 | 香港理工大学深圳研究院 | Structural health monitoring sensing data loss recovery method based on low-rank Hank matrix |
-
2022
- 2022-08-08 CN CN202210943271.5A patent/CN115309814A/en active Pending
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116450636A (en) * | 2023-06-20 | 2023-07-18 | 石家庄学院 | Internet of things data completion method, equipment and medium based on low-rank tensor decomposition |
CN116450636B (en) * | 2023-06-20 | 2023-08-18 | 石家庄学院 | Internet of things data completion method, equipment and medium based on low-rank tensor decomposition |
CN116955334A (en) * | 2023-06-27 | 2023-10-27 | 香港理工大学深圳研究院 | Structural health monitoring sensing data loss recovery method based on low-rank Hank matrix |
CN116955334B (en) * | 2023-06-27 | 2024-06-04 | 香港理工大学深圳研究院 | Structural health monitoring sensing data loss recovery method based on low-rank Hank matrix |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN115309814A (en) | Internet of things data reconstruction method based on structured low-rank tensor completion | |
Yuan et al. | High-order tensor completion via gradient-based optimization under tensor train format | |
CN109001802B (en) | Seismic signal reconstructing method based on Hankel tensor resolution | |
Osipov et al. | PMU missing data recovery using tensor decomposition | |
Zhang et al. | Spectrum cartography via coupled block-term tensor decomposition | |
CN107817465A (en) | DOA estimation method based on non-grid compressed sensing under super-Gaussian noise background | |
CN106646303A (en) | Quick reconstruction method for under-sampling magnetic resonance spectra | |
CN107192878A (en) | A kind of trend of harmonic detection method of power and device based on compressed sensing | |
CN106441575B (en) | A kind of sparse imaging method of terahertz time-domain spectroscopy | |
CN110244259A (en) | The two-dimentional angle estimation method of tensor filling is minimized in the case of shortage of data based on low n- order | |
He et al. | Improved FOCUSS method with conjugate gradient iterations | |
CN109143151B (en) | Uniform area array tensor reconstruction method and information source positioning method for partial array element damage | |
Pye et al. | Locality and entanglement in bandlimited quantum field theory | |
CN111324861A (en) | Deep learning magnetic resonance spectrum reconstruction method based on matrix decomposition | |
CN112733327A (en) | non-Gaussian signal-oriented continuous sum-matrix sparse array and design method thereof | |
CN107423543A (en) | A kind of fast reconstructing method of supercomplex Magnetic Resonance Spectrum | |
CN108828482B (en) | In conjunction with the method for reconstructing of sparse and low-rank characteristic lack sampling magnetic resonance diffusion spectrum | |
Zhang et al. | 3-D seismic data recovery via neural network-based matrix completion | |
Pournaghshband et al. | A novel block compressive sensing algorithm for SAR image formation | |
CN113644916A (en) | Power system steady-state data compression method based on edge calculation | |
CN105242237A (en) | Electromagnetic vector array parameter estimation method based on compressed sensing | |
CN108288295A (en) | The method for fast reconstruction and system of infrared small target image based on structural information | |
CN114624646B (en) | DOA estimation method based on model driven complex neural network | |
Yang et al. | Two‐Dimensional Multiple‐Snapshot Grid‐Free Compressive Beamforming Using Alternating Direction Method of Multipliers | |
Liu et al. | HTR-CTO algorithm for wireless data recovery |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |