CN102521143B - Heap data processing method and device - Google Patents
Heap data processing method and device Download PDFInfo
- Publication number
- CN102521143B CN102521143B CN201110418751.1A CN201110418751A CN102521143B CN 102521143 B CN102521143 B CN 102521143B CN 201110418751 A CN201110418751 A CN 201110418751A CN 102521143 B CN102521143 B CN 102521143B
- Authority
- CN
- China
- Prior art keywords
- heap
- metadata
- matching
- idle
- allocated
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 23
- 238000000034 method Methods 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims abstract description 7
- 238000004064 recycling Methods 0.000 claims description 2
- 230000004048 modification Effects 0.000 abstract description 6
- 238000012986 modification Methods 0.000 abstract description 6
- 230000008569 process Effects 0.000 description 9
- 230000002457 bidirectional effect Effects 0.000 description 3
- 101100108191 Vibrio parahaemolyticus serotype O3:K6 (strain RIMD 2210633) add gene Proteins 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a heap data processing method and a heap data processing device. The heap data processing method comprises the following steps: selecting an idle heap for distribution from a heap zone when confirming that a users demands to distribute memory write-in data, and updating heap metadata corresponding to the distributed heap in a set zone dynamically; or when confirming that the user requests to release the distributed heap, releasing the distributed heap which is requested by the user, and dynamically updating the heap metadata corresponding to the released heap in the set zone, wherein the heap metadata of all heaps are centrally stored in a set zone in the memory except zones occupied by all heaps. With the method disclosed by the invention, the heap metadata of all heaps are stored in a safe memory zone in concentration mode, the heap metadata are separated from heap data in terms of storage mode; and only the heap data can be the modification object of a user but the heap metadata cannot be modified, such that the problem that an attacker modifies the heap metadata through heap overflow can be prevented.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a heap data processing method and device.
Background
Heap is a computer memory area with continuous addresses, a heap area comprises a plurality of continuous heaps, and heap overflow refers to a phenomenon that when the amount of data filled into a certain heap by a computer program exceeds the capacity of the heap, the length of the data exceeds the storage boundary of the heap. When a very long data is input into a certain heap, the excess part is written into other adjacent heaps in the heap area, and other heaps may already store data, such as a pointer of a next instruction executed by a program, or output contents of other programs, so that all or part of the contents may be covered or damaged, if the covered or damaged contents are important, the subsequent normal operation may be affected, and it is seen that overflow of a small part of data or a set of instructions may cause a program or an operating system to crash.
Taking Linux system as an example, due to continuous dynamic allocation and recovery, the allocated heap and the free heap are usually distributed in a staggered manner, and the storage structure of each heap is as follows: the method comprises the steps of continuously storing heap metadata and heap data, wherein the heap data in the allocated heap is various application data used by a user program, and the heap data in the idle heap is initialization data, such as full FF; the heap metadata records attribute information of free heaps or allocated heaps (such as the heap length of the current heap, the heap length of the previous free heap, a pointer linking the previous heap, a pointer linking the next heap, and the like), and is an important parameter for allocating and recycling heaps.
For example, the current mechanism for allocating the free heap in the Linux system is as follows: the heap metadata of the idle heap is linked to a bi-directional linked list called Bin, 128 Bin bi-directional linked lists are arranged in a memory, different Bin bi-directional linked lists are linked with the heap metadata of the idle heap with different lengths, and one Bin bi-directional linked list is possible to link the heap metadata of a plurality of idle heaps with the same length; when the system allocates the idle heap, according to the length of the memory to be occupied and the length of the idle heap in which the heap metadata linked by the Bin doubly linked list is located, searching whether an idle heap with a heap length matched with the length of the memory to be occupied is provided for a user, if so, allocating the idle heap to the user, if not, re-dividing a proper idle heap for the user by taking a high address of a heap area (the use of the memory space of the heap area is increased from a low address to a high address) as an initial position, and in the process, the system adopts an optimal matching algorithm to search the idle heap with the heap length matched with the length of the memory to be occupied, namely allocating the idle heap with the first queried heap length matched with the length of the memory to the user.
However, since the storage structure of the heap is currently composed of heap metadata and data in the heap, the allocated heap and the free heap are distributed in an interlaced manner due to the dynamic allocation and reclamation of the heap, and since there is no boundary check in the process of storing the data, an attacker can easily cause a heap overflow phenomenon by inputting a large amount of data. Heap metadata is generally located at the initial position of the heap, so that the probability of the heap metadata being covered is the largest, and the heap metadata is relatively important information, so that if an attacker maliciously causes heap overflow, the operating system or the computer network system is damaged greatly. Moreover, the optimal matching algorithm in the heap allocation process has obvious regularity, and an attacker can completely utilize the algorithm to execute the heap allocation. Therefore, heap metadata is an important target for attackers, and it is very important for security of heap regions to protect the heap metadata from the attackers.
Therefore, a heap overflow prevention mechanism for safely and effectively protecting heap metadata is provided, which is one of important means for preventing malicious attacks on heap buffer overflow.
Disclosure of Invention
The invention provides a heap data processing method and device, which are used for overcoming the problem that an attacker modifies heap metadata by utilizing a heap overflow phenomenon in the prior art so as to damage a program or an operating system.
The method comprises the following steps:
a heap data processing method, comprising:
when determining that a user needs to allocate memory write-in data, selecting an idle heap from the heap area for allocation, and dynamically updating heap metadata corresponding to the allocated heap in the set area; or,
when determining that a user requests to release the allocated heap, releasing the allocated heap requested by the user, and dynamically updating heap metadata corresponding to the released heap in the setting area;
wherein the heap metadata of all the heaps is centrally stored in a set area in the memory outside the occupied area of all the heaps.
A heap data processing apparatus comprising:
the heap allocation module is used for selecting an idle heap from the heap area to allocate when determining that a user needs to allocate memory write-in data;
the first heap metadata updating module is used for dynamically updating heap metadata corresponding to the allocated heap in the set area;
the heap release module is used for releasing the allocated heap requested by the user when the user requests to release the allocated heap;
the second heap metadata updating module is used for dynamically updating heap metadata corresponding to the released heap in the set region;
wherein the heap metadata of all the heaps is centrally stored in a set area in the memory outside the occupied area of all the heaps.
The heap metadata of all the heaps are stored in a safe memory area in a centralized mode, and the heap metadata is separated from the data in the heaps in the storage form, namely, the heap metadata is separated from the interior of the heaps and is stored in a set area outside the area occupied by the heaps in a centralized mode.
Drawings
FIG. 1 is a flow chart of a heap data processing method according to the present invention;
fig. 2 is a schematic structural diagram of a heap data processing apparatus according to the present invention.
Detailed Description
The following describes in detail embodiments of the heap data processing method and apparatus according to the present invention with reference to the accompanying drawings and specific embodiments.
The invention provides a heap data processing method, as shown in fig. 1, comprising:
when determining that a user needs to allocate memory write-in data, selecting an idle heap from the heap area for allocation, and dynamically updating heap metadata corresponding to the allocated heap in the set area; or,
when determining that a user requests to release the allocated heap, releasing the allocated heap requested by the user, and dynamically updating heap metadata corresponding to the released heap in the setting area;
wherein the heap metadata of all the heaps is centrally stored in a set area in the memory outside the occupied area of all the heaps.
The heap is a memory area used by a user program for dynamically storing data during running, and a whole heap area is formed by a free heap and an allocated heap. Each free heap or allocated heap stores therein heap metadata and data in the heap, where the heap metadata records information such as use and address of the free heap or allocated heap, for example, the heap use information indicates whether the corresponding heap is in a free state or an allocated state, and the heap address information indicates a heap address, a heap length, a heap address of a previous heap, a heap address of a next heap, and the like of the corresponding heap.
In the prior art, heap metadata is stored in a corresponding heap, so that an attacker can easily modify the heap metadata by using a heap overflow phenomenon, and a free heap cannot be normally allocated or an allocated heap is recycled, so that a program or an operating system is crashed.
The heap data processing method provided by the invention stores heap metadata of all heaps (idle heaps or allocated heaps) in a safe memory area in a centralized mode, separates the heap metadata from data in the heaps in a storage form, namely separates the heap metadata from the interior of the heaps and stores the heap metadata in a set area outside the area occupied by the heaps in a centralized mode.
In the prior art, an attacker often destroys heap metadata in a heap overflow mode, so that a system cannot normally distribute and recycle the heap, and in order to better protect the heap metadata from being attacked, the invention can further set that the heap metadata can be modified only with certain authority, otherwise, the heap metadata cannot be modified, thereby fundamentally separating the management heap from the use heap. The execution main body of the heap data processing method is an operating system or other background programs, so that the heap metadata can be modified only if the operating system or other background programs have the authority for managing the heap area, and the user program cannot operate the heap metadata because the user program does not have the authority.
In order to protect the heap metadata from being modified by an attacker, the heap metadata corresponding to all the heaps needs to be stored in a continuous, protected and safe memory area, and considering that the data storage mode in the heap area is increased from a low address to a high address and the alternate connection mode of a free heap and an allocated heap in the heap area, the method can open up a space with a certain length as a set area from the starting address of the heap area.
In the heap data processing method provided by the invention, a user writes data into the allocated heap firstly, and when the allocated heap is full of data and the data is not written completely, the unwritten data is written into the heap other than the allocated heap. Therefore, users (including attackers) can only execute write operation in the heap, the modification objects can only be data in the heap, and heap metadata cannot be modified, so that difficulties in heap allocation and heap reclamation are avoided, and the problem that the attackers modify the heap metadata through heap overflow can be effectively prevented.
Preferably, in the heap data processing method provided by the present invention, the heap metadata includes heap address information and heap usage information of the corresponding heap, where the heap address information is address information of an area occupied by the corresponding heap, and by this information, after the heap metadata is separated from the inside of the heap, the area occupied by the corresponding heap can be accurately located by the heap metadata, and the heap length of the corresponding heap is determined; and the heap use information is idle or allocated state information of the corresponding heap.
Based on the idea of separating the heap metadata from the heap itself, the heap data processing method of the present invention selects an idle heap from the heap area for allocation when determining that a user needs to allocate memory for writing data, and dynamically updates the heap metadata corresponding to the allocated heap in the setting area, and specifically includes:
determining the length L of a memory requested to be allocated; searching first matching heap metadata in the set area, wherein the first matching heap metadata is heap metadata with heap length and length L which are determined by idle heap use information and heap address information and matched with the heap length; and allocating the heap corresponding to the first matching heap metadata to a user, and updating the heap usage information in the first matching heap metadata to be allocated.
Preferably, when it is determined that there is no first matching heap metadata, the heap data processing method of the present invention further includes: dividing a new idle heap for distribution in the memory space outside the occupied area and the set area of all the current heaps, wherein the heap length of the new idle heap is matched with the length L; and simultaneously, creating heap metadata corresponding to the new free heap in the set area. Specifically, the stack length of the new free stack may be the same as or different from the stack length L requested by the user, a threshold may be set according to actual needs, and the stack lengths within the threshold range all belong to the matched stack length.
In the existing Linux system, the heap allocation process adopts an optimal matching algorithm, namely, the first idle heap which meets the demand of a heap user is allocated to the heap user, because the algorithm has obvious regularity, an attacker can easily use the vulnerability to attack, and in order to avoid the situation, the heap allocation process of the invention adopts a random matching algorithm: when at least two first matching heap metadata are determined, one first matching heap metadata is randomly selected from the at least two first matching heap metadata, and the heap corresponding to the selected first matching heap metadata is distributed. Because the algorithm of random matching has great randomness, an attacker is difficult to find a vulnerability to attack, and therefore, the algorithm has higher security.
In order to quickly and accurately find a proper idle heap for allocation after determining the length of a memory requested to be allocated by a user, preferably, different linked lists are used for linking heap metadata of idle heaps with different heap lengths; searching for the first matching heap metadata specifically includes: searching a first matching linked list, wherein the first matching linked list is linked with stack metadata of an idle stack with the stack length matched with the length L; and determining heap metadata linked by the first matching linked list as first matching heap metadata. Therefore, because different linked lists are used for linking the heap metadata of the idle heaps with different heap lengths, the corresponding linked lists can be positioned according to the memory length allocated by the user request, and the idle heaps meeting the user request can be easily found in the mode.
Based on the idea of separating the heap metadata from the heap itself, the heap data processing method of the present invention, when determining that a user requests to release an allocated heap, releases the allocated heap requested by the user, and dynamically updates the heap metadata corresponding to the released heap in a setting region, specifically includes:
searching second matching heap metadata in the set area according to the heap address information of the allocated heap requested to be released, wherein the second matching heap metadata is heap metadata of which the heap use information is allocated and the heap address information is consistent with the heap address information of the allocated heap requested to be released; and releasing the allocated heap corresponding to the second matching heap metadata, and updating the heap use information in the second matching heap metadata to be idle.
The heap data processing method needs to dynamically update the heap metadata in the set area, so that the second matching heap metadata in the set area needs to be correspondingly modified while the heap is recycled, and the second matching heap metadata can correctly correspond to the released heap. Preferably, different linked lists are used to correspondingly link heap metadata of idle heaps with different heap lengths, and the heap data processing method of the present invention further includes, after updating the heap usage information in the second matching heap metadata to be idle: and linking the second matching heap metadata to a corresponding linked list according to the heap length determined by the heap address information in the second matching heap metadata.
Since the free heap and the allocated heap are generally alternately connected in the heap area, when the allocated heap requested to be released is released as a free heap, if the previous heap or the next heap of the heap is a free heap, the free heap and the allocated heap should be merged into a larger free heap at the same time of releasing. Preferably, when determining that the adjacent heap of the allocated heap requested to be released in the heap area is a free heap, combining the allocated heap requested to be released and the adjacent heap into a free heap; meanwhile, the following steps are executed:
deleting the heap metadata of the adjacent heap in the set area, and modifying the heap address information in the second matching heap metadata into the heap address information of the merged idle heap; or deleting the second matching heap metadata in the set region, and modifying the heap address information in the heap metadata of the adjacent heap into the heap address information of the merged idle heap; or deleting the second matching heap metadata in the set area and the heap metadata of the adjacent heap, and creating the heap metadata of the merged free heap in the set area.
Based on the above idea of merging idle heaps, the heap data processing method of the present invention further includes:
after the heap address information in the second matching heap metadata is modified into the heap address information of the merged idle heap, linking the second matching heap metadata to a corresponding linked list according to the heap length determined by the heap address information of the merged idle heap;
after the heap address information in the heap metadata of the adjacent heap is modified into the heap address information of the merged idle heap, according to the heap length determined by the heap address information of the merged idle heap, linking the heap metadata of the adjacent heap to a corresponding linked list;
after the pile metadata of the merged idle pile is created in the setting area, the pile metadata of the merged idle pile is linked to a corresponding linked list according to the pile length of the merged idle pile determined by the pile address information of the merged idle pile.
According to the above description, the linked list technology is used in many places in the method of the present invention, because the linked list is a non-continuous and non-sequential storage structure on the physical storage unit, and compared with the linear list sequential structure, the insertion and deletion operations of the linked list are more convenient, therefore, the linked list is used to link the pile metadata of the idle pile, and the corresponding linked list can be conveniently inserted and deleted according to the change of the idle pile in the pile space. The linked list is divided into a one-way linked list and a two-way linked list, wherein the link direction of the one-way linked list is one-way, and the access to the one-way linked list is started from a head part through sequential reading; each data node of the double linked list has two pointers which point to the direct successor and the direct predecessor, so that starting from any node in the double linked list (each element in the linked list is called node), the predecessor node and the successor node can be accessed conveniently. In order to make the operation of searching the linked list in the method of the present invention more convenient and faster, preferably, the present invention uses the doubly linked list to link the heap metadata of the idle heap.
The embodiment of the invention provides a heap data processing method, which is used for overflow protection of a heap area in a Linux system and comprises the following steps:
opening a certain length of space in the low address area of the heap area as a setting area for storing heap metadata, and setting the structure of the heap metadata as follows:
the significance of each field in the heap metadata structure is as follows: if the previous pile of the current pile is a free pile, the prev _ size field stores the pile length of the previous pile; the size field contains the heap length of the current heap and other heap management information, wherein the lowest bit of the size field indicates whether the previous heap is a free heap, for example, 1 indicates an allocated heap, and 0 indicates a free heap, so that it is only necessary to determine whether the lowest bit of the size field is 0 to determine whether the previous heap is a free heap; the status field indicates whether the current heap is a free heap or an allocated heap, e.g., 1 indicates an allocated heap and 0 indicates a free heap; pointer md points to the address currently heap in memory (i.e., heap address); the pointer bk points to the heap address of the next heap. Wherein, the heap address information includes prev _ size field, pointer md and pointer bk, and the heap usage information is indicated by status field.
The Linux system of this embodiment includes 128 Bin doubly linked lists, and the use of these Bin doubly linked lists to link heap metadata of an idle heap of a specific length in the heap area (implemented by the Bin doubly linked list storing an address pointer pointing to the heap metadata) specifically includes: the system divides all idle heaps into two parts of more than 512 bytes and less than 512 bytes according to the size of heap length, the first 62 Bin double linked lists are used to link idle heap metadata of less than 512 bytes, the first Bin double linked list is linked with idle heap metadata of 16 bytes, the second Bin double linked list is linked with idle heap metadata of 24 bytes, and so on; linking idle heap metadata larger than 512 bytes by using the last 66 Bin doubly linked lists; thus, different Bin doubly-linked lists are in one-to-one correspondence with heap metadata of different heap lengths, and here, the present embodiment uses a Bin doubly-linked list at a predetermined position to link the heap metadata including a predetermined heap length.
When a user program requests to allocate an idle heap corresponding to the length of S bytes in the running period, the operating system searches all Bin bidirectional linked lists according to the length S of the applied idle heap, so that a first matching Bin bidirectional linked list is found, the matching Bin bidirectional linked list is further linked to first matching heap metadata in a set area, and finally, an md pointer in the first matching heap metadata determines a corresponding idle heap and allocates the idle heap to the user program. Accordingly, the operating system is to modify the status field of the first matching heap metadata in the set area to the allocated heap identification 1.
After the user is allocated with the proper heap, the user firstly writes data in the allocated heap, and when the allocated heap is full of data and the data is not written completely, the data which is not written completely is written into the heap outside the allocated heap. Therefore, a user can only execute write operation in the heap, even if an attacker, the modification object of the user can only be the data in the heap, and the heap metadata cannot be influenced, so that the problem that the attacker modifies the heap metadata through heap overflow can be effectively prevented.
Since heap metadata with the same heap length (the free heap lengths corresponding to such heap metadata are all the same) are linked to the same Bin doubly linked list, in the above-mentioned heap allocation process, if the operating system searches all Bin doubly linked lists, and the found first matching Bin doubly linked list links N (N > 1) matched heap metadata, the operating system determines a random matching heap metadata to use among the N matched heap metadata through the random number generator f (x).
In addition, in the above allocation process, if the operating system searches all the Bin double linked lists, and does not find the first matching Bin double linked list, that is, does not determine any matching heap metadata, then in the free heap and the high address storage space after the allocated heap that are alternately linked in the heap area, a suitable heap space is re-divided to be allocated to the user program for use.
When a user program requests to release an allocated heap with the heap address add1 in the running period, an operating system searches second matching heap metadata containing the heap address add1 in a set area, modifies a status field in the second matching heap metadata into a free heap identifier 0, finds a corresponding allocated heap according to an md pointer in the second matching heap metadata, and releases the allocated heap into a free heap.
In the above reclamation process, if the request releases the adjacent heap of the allocated heap as a free heap, the operating system merges the two free heaps into one larger free heap. Correspondingly, the operating system can delete the heap metadata of the adjacent idle heap in a set area, and respectively modify the heap length and the heap address contained in the second matching heap metadata into the heap length and the heap address of the merged idle heap; meanwhile, the operating system needs to link the second matching heap metadata to the Bin doubly linked list at the corresponding position according to the heap length of the merged idle heap.
In addition, in this embodiment, when allocating a heap for a user program, a Bin doubly linked list index pointer may be used to locate a corresponding Bin doubly linked list according to the heap length requested to be allocated by the user program. Specifically, the index pointers of all the Bin double linked lists are stored in an array, when the user program requests to allocate the heap, the corresponding Bin double linked list is found in the array according to the heap length allocated by the user program request, and further the corresponding Bin double linked list can be linked to the matched heap metadata, and a proper idle heap is determined to be allocated to the user program for use.
An embodiment of the present invention further provides a heap data processing apparatus, as shown in fig. 2, including the following parts:
the heap allocation module 201 is configured to select an idle heap from the heap area for allocation when determining that a user needs to allocate memory write data;
a first heap metadata updating module 202, configured to dynamically update heap metadata corresponding to the allocated heap in the setting area;
the heap reclamation module 203 is used for releasing the allocated heap requested by the user when the user requests to release the allocated heap;
a second heap metadata updating module 204, configured to dynamically update heap metadata corresponding to the released heap in the setting area;
wherein the heap metadata of all the heaps is centrally stored in a set area in the memory outside the occupied area of all the heaps.
Preferably, the heap metadata includes heap address information and heap usage information of the corresponding heap;
when determining that the user needs to allocate the memory write data, the heap allocation module 201 selects an idle heap from the heap area for allocation, which specifically includes:
determining the length L of a memory requested to be allocated; searching first matching heap metadata in the set area, wherein the first matching heap metadata is heap metadata with heap length and length L which are determined by idle heap use information and heap address information and matched with the heap length; allocating the heap corresponding to the first matching heap metadata to a user;
moreover, the dynamically updating the heap metadata corresponding to the allocated heap in the setting area by the first heap metadata updating module 202 specifically includes: updating heap usage information in the first matching heap metadata as allocated.
Preferably, when the heap allocation module 201 determines that there is no first matching heap metadata, it is further configured to: dividing a new idle heap for distribution in the memory space outside the occupied area and the set area of all the current heaps, wherein the heap length of the new idle heap is matched with the length L;
meanwhile, the first heap metadata update module 202 creates heap metadata corresponding to the new free heap in the set area.
Preferably, when the heap allocation module 201 determines that there are at least two first matching heap metadata, one first matching heap metadata is randomly selected from the at least two first matching heap metadata, and a heap corresponding to the selected first matching heap metadata is allocated to the user.
Preferably, different linked lists are used for linking heap metadata of idle heaps with different heap lengths; the heap allocation module 203 searches for the first matching heap metadata, which specifically includes:
searching a first matching linked list, wherein the first matching linked list is linked with stack metadata of an idle stack with the stack length matched with the length L; and determining heap metadata linked by the first matching linked list as first matching heap metadata.
Preferably, the heap metadata includes heap address information and heap usage information of the corresponding heap;
the heap reclamation module 203 releases the allocated heap requested by the user when determining that the user requests to release the allocated heap, specifically including:
searching second matching heap metadata in the set area according to the heap address information of the allocated heap requested to be released, wherein the second matching heap metadata is heap metadata of which the heap use information is allocated and the heap address information is consistent with the heap address information of the allocated heap requested to be released; releasing the allocated heap corresponding to the second matching heap metadata;
moreover, the dynamically updating the heap metadata corresponding to the released heap in the setting area by the second heap metadata updating module 204 specifically includes:
updating heap usage information in the second matching heap metadata to be free.
Preferably, when different linked lists are used to correspondingly link heap metadata of idle heaps with different heap lengths, the second heap metadata update module 204 is further configured to, after updating the usage information of the heap in the second matching heap metadata to be idle: and linking the second matching heap metadata to a corresponding linked list according to the heap length determined by the heap address information in the second matching heap metadata.
Preferably, the heap data processing apparatus further comprises:
a merging module 205, configured to merge the allocated heap requested to be released and the adjacent heap into a free heap when determining that the adjacent heap of the allocated heap requested to be released in the heap area is a free heap;
meanwhile, the second heap metadata update module 204 performs the following steps:
deleting the heap metadata of the adjacent heap in the set area, and modifying the heap address information in the second matching heap metadata into the heap address information of the merged idle heap; or,
deleting the second matching heap metadata in the set area, and modifying the heap address information in the heap metadata of the adjacent heap into the heap address information of the merged idle heap; or,
deleting the second matching heap metadata in the set area and the heap metadata of the adjacent heap, and creating the heap metadata of the merged free heap in the set area.
Preferably, the second heap metadata update module 204 is further configured to:
after the heap address information in the second matching heap metadata is modified into the heap address information of the merged idle heap, linking the second matching heap metadata to a corresponding linked list according to the heap length determined by the heap address information of the merged idle heap;
after the heap address information in the heap metadata of the adjacent heap is modified into the heap address information of the merged idle heap, according to the heap length determined by the heap address information of the merged idle heap, linking the heap metadata of the adjacent heap to a corresponding linked list;
after the pile metadata of the merged idle pile is created in the setting area, the pile metadata of the merged idle pile is linked to a corresponding linked list according to the pile length of the merged idle pile determined by the pile address information of the merged idle pile.
For specific implementation functions of each module of the heap data processing apparatus, reference is made to the specific implementation process of the above heap data processing method, which is not described herein again.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (19)
1. A heap data processing method, comprising:
when determining that a user needs to allocate memory write-in data, selecting an idle heap from the heap area for allocation, and dynamically updating heap metadata corresponding to the allocated heap in the set area; or,
when determining that a user requests to release the allocated heap, releasing the allocated heap requested by the user, and dynamically updating heap metadata corresponding to the released heap in the setting area;
the structure of the heap metadata comprises a prev _ size field, a status field, a pointer md and a pointer bk, wherein the prev _ size field represents the heap length of a previous heap, the size field contains the heap length of a current heap and heap management information, the heap management information includes whether the previous heap is a free heap or an allocated heap, the pointer md points to the address of the current heap in the memory, and the pointer bk points to the heap address of a next heap.
2. The method of claim 1, wherein the heap metadata includes heap address information and heap usage information for the corresponding heap;
when it is determined that the user needs to allocate the memory write data, selecting a free heap from the heap area for allocation, and dynamically updating heap metadata corresponding to the allocated heap in the setting area, specifically including:
determining the length L of a memory requested to be allocated;
searching first matching heap metadata in the set area, wherein the first matching heap metadata is heap metadata with heap length and length L which are determined by idle heap use information and heap address information and matched with the heap length;
and allocating the heap corresponding to the first matching heap metadata to a user, and updating the heap usage information in the first matching heap metadata to be allocated.
3. The method of claim 2, wherein determining that there is no first matching heap metadata, further comprising:
dividing a new idle heap for distribution in the memory space outside the occupied area and the set area of all the current heaps, wherein the heap length of the new idle heap is matched with the length L; at the same time, the user can select the desired position,
heap metadata corresponding to the new free heap is created in the set area.
4. The method of claim 2, wherein when at least two first matching heap metadata are determined, randomly selecting one first matching heap metadata from the at least two first matching heap metadata, and allocating a heap corresponding to the selected first matching heap metadata.
5. The method of claim 2, wherein heap metadata for free heaps of different heap lengths are linked using different linked lists;
searching for the first matching heap metadata specifically includes:
searching a first matching linked list, wherein the first matching linked list is linked with stack metadata of an idle stack with the stack length matched with the length L;
and determining heap metadata linked by the first matching linked list as first matching heap metadata.
6. The method of claim 1, wherein the heap metadata includes heap address information and heap usage information for the corresponding heap;
when it is determined that the user requests to release the allocated heap, releasing the allocated heap requested by the user, and dynamically updating heap metadata corresponding to the released heap in the setting area, specifically including:
searching second matching heap metadata in the set area according to the heap address information of the allocated heap requested to be released, wherein the second matching heap metadata is heap metadata of which the heap use information is allocated and the heap address information is consistent with the heap address information of the allocated heap requested to be released;
and releasing the allocated heap corresponding to the second matching heap metadata, and updating the heap use information in the second matching heap metadata to be idle.
7. The method of claim 6, wherein if different linked lists are used to correspondingly link heap metadata of free heaps with different heap lengths, then after updating heap usage information in the second matching heap metadata to be free, further comprising:
and linking the second matching heap metadata to a corresponding linked list according to the heap length determined by the heap address information in the second matching heap metadata.
8. The method of claim 6, further comprising:
when determining that the adjacent heap of the allocated heap requested to be released in the heap area is a free heap, combining the allocated heap requested to be released and the adjacent heap into a free heap; meanwhile, the following steps are executed:
deleting the heap metadata of the adjacent heap in the set area, and modifying the heap address information in the second matching heap metadata into the heap address information of the merged idle heap; or,
deleting the second matching heap metadata in the set area, and modifying the heap address information in the heap metadata of the adjacent heap into the heap address information of the merged idle heap; or,
deleting the second matching heap metadata in the set area and the heap metadata of the adjacent heap, and creating the heap metadata of the merged free heap in the set area.
9. The method of claim 8, further comprising:
after the heap address information in the second matching heap metadata is modified into the heap address information of the merged idle heap, linking the second matching heap metadata to a corresponding linked list according to the heap length determined by the heap address information of the merged idle heap;
after the heap address information in the heap metadata of the adjacent heap is modified into the heap address information of the merged idle heap, according to the heap length determined by the heap address information of the merged idle heap, linking the heap metadata of the adjacent heap to a corresponding linked list;
after the pile metadata of the merged idle pile is created in the setting area, the pile metadata of the merged idle pile is linked to a corresponding linked list according to the pile length of the merged idle pile determined by the pile address information of the merged idle pile.
10. A method as claimed in any one of claims 5, 7 or 9, wherein the linked list is a doubly linked list.
11. A heap data processing apparatus, comprising:
the heap allocation module is used for selecting an idle heap from the heap area to allocate when determining that a user needs to allocate memory write-in data;
the first heap metadata updating module is used for dynamically updating heap metadata corresponding to the allocated heap in the set area;
the heap recycling module is used for releasing the allocated heap requested by the user when the user requests to release the allocated heap;
the second heap metadata updating module is used for dynamically updating heap metadata corresponding to the released heap in the set region;
the structure of the heap metadata comprises a prev _ size field, a status field, a pointer md and a pointer bk, wherein the prev _ size field represents the heap length of a previous heap, the size field contains the heap length of a current heap and other heap management information, the heap management information includes whether the previous heap is a free heap or an allocated heap, the pointer md points to the address of the current heap in the memory, and the pointer bk points to the heap address of a next heap.
12. The apparatus of claim 11, wherein the heap metadata includes heap address information and heap usage information for the corresponding heap;
when the heap allocation module determines that the user needs to allocate the memory write-in data, the free heap is selected from the heap area for allocation, which specifically includes:
determining the length L of a memory requested to be allocated;
searching first matching heap metadata in the set area, wherein the first matching heap metadata is heap metadata with heap length and length L which are determined by idle heap use information and heap address information and matched with the heap length;
allocating the heap corresponding to the first matching heap metadata to a user;
the dynamic update of the heap metadata corresponding to the allocated heap in the setting area by the first heap metadata update module specifically includes:
updating heap usage information in the first matching heap metadata as allocated.
13. The apparatus of claim 12,
the heap allocation module, when determining that there is no first matching heap metadata, is further to: dividing a new idle heap for distribution in the memory space outside the occupied area and the set area of all the current heaps, wherein the heap length of the new idle heap is matched with the length L;
meanwhile, the first heap metadata updating module creates heap metadata corresponding to the new free heap in the setting area.
14. The apparatus of claim 12,
and when the heap allocation module determines that at least two first matching heap metadata exist, randomly selecting one first matching heap metadata from the at least two first matching heap metadata, and allocating the heap corresponding to the selected first matching heap metadata to the user.
15. The apparatus of claim 12, wherein heap metadata for free heaps of different heap lengths are linked using different linked lists;
the searching, by the heap allocation module, for the first matching heap metadata specifically includes:
searching a first matching linked list, wherein the first matching linked list is linked with stack metadata of an idle stack with the stack length matched with the length L;
and determining heap metadata linked by the first matching linked list as first matching heap metadata.
16. The apparatus of claim 11, wherein the heap metadata includes heap address information and heap usage information for the corresponding heap;
the releasing the allocated heap requested by the user when the heap reclamation module determines that the user requests to release the allocated heap specifically includes:
searching second matching heap metadata in the set area according to the heap address information of the allocated heap requested to be released, wherein the second matching heap metadata is heap metadata of which the heap use information is allocated and the heap address information is consistent with the heap address information of the allocated heap requested to be released;
releasing the allocated heap corresponding to the second matching heap metadata;
the second heap metadata updating module dynamically updates heap metadata corresponding to the released heap in the setting area, and specifically includes:
updating heap usage information in the second matching heap metadata to be free.
17. The apparatus of claim 16, wherein the heap metadata of free heaps with different heap lengths are correspondingly linked using different linked lists, and after the second heap metadata update module updates the usage information of heaps in the second matching heap metadata to be free, the second heap metadata update module is further configured to:
and linking the second matching heap metadata to a corresponding linked list according to the heap length determined by the heap address information in the second matching heap metadata.
18. The apparatus of claim 16, further comprising:
a merging module, configured to merge the allocated heap requested to be released and an adjacent heap into a free heap when determining that the adjacent heap of the allocated heap requested to be released in the heap area is a free heap;
meanwhile, the second heap metadata updating module executes the following steps:
deleting the heap metadata of the adjacent heap in the set area, and modifying the heap address information in the second matching heap metadata into the heap address information of the merged idle heap; or,
deleting the second matching heap metadata in the set area, and modifying the heap address information in the heap metadata of the adjacent heap into the heap address information of the merged idle heap; or,
deleting the second matching heap metadata in the set area and the heap metadata of the adjacent heap, and creating the heap metadata of the merged free heap in the set area.
19. The apparatus of claim 18, wherein the second heap metadata update module is further to:
after the heap address information in the second matching heap metadata is modified into the heap address information of the merged idle heap, linking the second matching heap metadata to a corresponding linked list according to the heap length determined by the heap address information of the merged idle heap;
after the heap address information in the heap metadata of the adjacent heap is modified into the heap address information of the merged idle heap, according to the heap length determined by the heap address information of the merged idle heap, linking the heap metadata of the adjacent heap to a corresponding linked list;
after the pile metadata of the merged idle pile is created in the setting area, the pile metadata of the merged idle pile is linked to a corresponding linked list according to the pile length of the merged idle pile determined by the pile address information of the merged idle pile.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110418751.1A CN102521143B (en) | 2011-12-14 | 2011-12-14 | Heap data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110418751.1A CN102521143B (en) | 2011-12-14 | 2011-12-14 | Heap data processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102521143A CN102521143A (en) | 2012-06-27 |
| CN102521143B true CN102521143B (en) | 2015-04-15 |
Family
ID=46292073
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110418751.1A Active CN102521143B (en) | 2011-12-14 | 2011-12-14 | Heap data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102521143B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102915276B (en) * | 2012-09-25 | 2015-06-03 | 武汉邮电科学研究院 | Memory control method for embedded systems |
| WO2014142836A1 (en) * | 2013-03-13 | 2014-09-18 | Empire Technology Development, Llc | Memory allocation accelerator |
| CN103761053B (en) * | 2013-12-30 | 2017-08-25 | 华为技术有限公司 | A kind of data processing method and device |
| CN104991747A (en) * | 2015-07-30 | 2015-10-21 | 湖南亿谷科技发展股份有限公司 | Method and system for data management |
| CN106856470A (en) * | 2015-12-09 | 2017-06-16 | 中国电信股份有限公司 | For the method and device of guarding network attack |
| CN107391388B (en) * | 2016-05-17 | 2021-06-11 | 阿里巴巴集团控股有限公司 | Method and equipment for storing data based on instant messaging |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1303534C (en) * | 2003-03-03 | 2007-03-07 | 华为技术有限公司 | Memory pool managing method |
| CN1963788A (en) * | 2005-11-08 | 2007-05-16 | 中兴通讯股份有限公司 | A managing method for EMS memory |
| CN102063385A (en) * | 2010-12-23 | 2011-05-18 | 深圳市金宏威实业发展有限公司 | Memory management method and system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7908455B2 (en) * | 2008-02-27 | 2011-03-15 | Microchip Technology Incorporated | Low overhead memory management system and method |
-
2011
- 2011-12-14 CN CN201110418751.1A patent/CN102521143B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1303534C (en) * | 2003-03-03 | 2007-03-07 | 华为技术有限公司 | Memory pool managing method |
| CN1963788A (en) * | 2005-11-08 | 2007-05-16 | 中兴通讯股份有限公司 | A managing method for EMS memory |
| CN102063385A (en) * | 2010-12-23 | 2011-05-18 | 深圳市金宏威实业发展有限公司 | Memory management method and system |
Non-Patent Citations (1)
| Title |
|---|
| 一种防止堆溢出的有效方法;杨海军;《计算机科学》;20090430;第36卷(第4B期);第118 -119页，156页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102521143A (en) | 2012-06-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102521143B (en) | Heap data processing method and device | |
| US7181585B2 (en) | Defensive heap memory management | |
| CN111344684B (en) | Multi-layer cache placement mechanism | |
| CN108628753B (en) | Memory space management method and device | |
| US7827375B2 (en) | Defensive heap memory management | |
| US7831783B2 (en) | Effective wear-leveling and concurrent reclamation method for embedded linear flash file systems | |
| US7716448B2 (en) | Page oriented memory management | |
| US9710397B2 (en) | Data migration for composite non-volatile storage device | |
| US7802070B2 (en) | Approach for de-fragmenting physical memory by grouping kernel pages together based on large pages | |
| US20070094440A1 (en) | Enhanced data access in a storage device | |
| US7962684B2 (en) | Overlay management in a flash memory storage device | |
| CN107066498B (en) | Key value KV storage method and device | |
| JP2007523412A (en) | Memory allocation | |
| CN108959113B (en) | Method and system for flash aware heap memory management | |
| US20080120456A1 (en) | Method for flash memory data management | |
| JPWO2006107095A1 (en) | Computer system, memory management method, and program thereof | |
| US10838624B2 (en) | Extent pool allocations based on file system instance identifiers | |
| US20140223072A1 (en) | Tiered Caching Using Single Level Cell and Multi-Level Cell Flash Technology | |
| CA2758235A1 (en) | Device and method for storage, retrieval, relocation, insertion or removal of data in storage units | |
| WO2024099448A1 (en) | Memory release method and apparatus, memory recovery method and apparatus, and computer device and storage medium | |
| CN114327917A (en) | Memory management method, computing device and readable storage medium | |
| US7711921B2 (en) | Page oriented memory management | |
| CN109213423A (en) | Concurrent I/O command is handled without lock based on address barrier | |
| US10073851B2 (en) | Fast new file creation cache | |
| CN116661690A (en) | Method, device, computer equipment and storage medium for recording memory state |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |