CN111371704A - Data caching method and device, terminal equipment and storage medium - Google Patents
Data caching method and device, terminal equipment and storage medium Download PDFInfo
- Publication number
- CN111371704A CN111371704A CN202010081796.3A CN202010081796A CN111371704A CN 111371704 A CN111371704 A CN 111371704A CN 202010081796 A CN202010081796 A CN 202010081796A CN 111371704 A CN111371704 A CN 111371704A
- Authority
- CN
- China
- Prior art keywords
- cache region
- data packet
- size
- data
- extended
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 71
- 238000004590 computer program Methods 0.000 claims description 18
- 230000005540 biological transmission Effects 0.000 claims description 10
- 238000012544 monitoring process Methods 0.000 claims description 10
- 230000006854 communication Effects 0.000 abstract description 17
- 238000004891 communication Methods 0.000 abstract description 17
- 230000008569 process Effects 0.000 abstract description 14
- 238000010586 diagram Methods 0.000 description 12
- 238000012545 processing Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000007175 bidirectional communication Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/90—Buffering arrangements
- H04L49/9005—Buffering arrangements using dynamic buffer space allocation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The embodiment of the invention provides a data caching method, a data caching device, terminal equipment and a storage medium, wherein the data caching method comprises the following steps: acquiring a data packet sent by a target terminal through a pre-established IOCP model; determining the size of a data packet and the size of an available cache area in a virtual container corresponding to a target terminal; if the size of the data packet is larger than the size of the available cache region, calling the dynamic cache region to expand the available cache region to obtain an expanded cache region; the data packet is stored in the extended cache region, and by adopting the embodiment of the invention, under the condition that an IOCP model is adopted to process large concurrent data, the cache region in the virtual container corresponding to each terminal can be dynamically extended in real time, so that the IOCP network communication capacity is improved to the maximum extent.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a data caching method and apparatus, a terminal device, and a storage medium.
Background
With the rapid development of network technologies, bidirectional communications such as video conferences and video teaching are widely popularized in the aspects of life, work, learning and the like of users.
The IOCP framework mode supports large concurrent data, each network communication connection needs to consume a certain memory space to bear the network data acquired from the I/O layer, single-channel resources are relatively small, and after the concurrency amount reaches a certain value, the required memory space is huge until a server cannot bear the network data, so that the large concurrency amount and the service equipment performance of the whole IOCP framework are limited.
Generally, when performing service processing, each established network communication connection does not perform service processing in real time, and only occupies a small portion of memory resources, and most of the memory resources applied by the network communication connection are in a dormant state and a dead state, and are not actually utilized.
Disclosure of Invention
In view of the above problems, embodiments of the present invention are proposed to provide a data caching method, apparatus, terminal device and storage medium that overcome or at least partially solve the above problems.
In a first aspect, an embodiment of the present invention provides a data caching method, which is applied to a server, and the method includes:
acquiring a data packet sent by a target terminal through a pre-established IOCP model;
determining the size of the data packet and the size of an available cache area in a virtual container corresponding to the target terminal;
if the size of the data packet is larger than the size of the available cache region, calling a dynamic cache region to expand the available cache region to obtain an expanded cache region;
and storing the data packet into the extended buffer area.
Optionally, before the obtaining, through the pre-established IOCP model, the data packet sent by the target terminal, the method further includes:
creating a plurality of virtual containers in advance;
and when a monitoring thread in the IOCP model detects that a target terminal sends a data packet, associating the virtual container with the target terminal to form a network channel between the target terminal and the virtual container so as to enable the target terminal and a server to carry out data transmission.
Optionally, if the size of the data packet is larger than the size of the available cache region, calling a dynamic cache region to expand the available cache region, and obtaining an expanded cache region, including:
determining the size of a cache region needing capacity expansion according to the difference value between the size of the data packet and the size of the available cache region;
and calling the dynamic cache region to expand the available cache region according to the size of the cache region needing to be expanded, and obtaining the expanded cache region, wherein the dynamic cache region comprises a cache region except a preset cache region which is applied in advance for the virtual container corresponding to each terminal and/or a preset cache region of a virtual container corresponding to other terminals.
Optionally, the available buffer is determined according to a buffer occupied by the data packet written into the extended buffer and a released buffer after the data packet has been read from the extended buffer.
Optionally, the storing the data packet into the extended buffer includes:
and after analyzing the data packet sent by the terminal, writing the analyzed data into the extended cache region, wherein the analyzed data at least comprises the byte length of the data packet, and the byte length of the data packet occupies the extended cache region with the preset byte length.
Optionally, after storing the data packet in the extended buffer, the method further includes:
reading a data packet from the extended cache region;
releasing the extended cache area occupied by the read data packet;
specifically, the reading the data packet from the extended buffer includes:
and if the length of the residual bytes in the extended cache region is greater than the preset byte length, reading the data packet from the extended cache region.
In a second aspect, an embodiment of the present invention provides a data caching apparatus, which is applied to a server, and the apparatus includes:
the acquisition module is used for acquiring a data packet sent by a target terminal through a pre-established IOCP model;
a determining module, configured to determine a size of the data packet and a size of an available buffer in a virtual container corresponding to the target terminal;
the capacity expansion module is used for calling the dynamic cache area to expand the available cache area to obtain an expanded cache area if the size of the data packet is larger than the size of the available cache area;
and the writing module is used for storing the data packet into the extended cache region.
Optionally, before the obtaining module, the apparatus further includes an association module, configured to:
creating a plurality of virtual containers in advance;
and when a monitoring thread in the IOCP model detects that a target terminal sends a data packet, associating the virtual container with the target terminal to form a network channel between the target terminal and the virtual container so as to enable the target terminal and a server to carry out data transmission.
Optionally, the capacity expansion module is specifically configured to:
determining the size of a cache region needing capacity expansion according to the difference value between the size of the data packet and the size of the available cache region;
and calling the dynamic cache region to expand the available cache region according to the size of the cache region needing to be expanded, and obtaining the expanded cache region, wherein the dynamic cache region comprises a cache region except a preset cache region which is applied in advance for the virtual container corresponding to each terminal and/or a preset cache region of a virtual container corresponding to other terminals.
Optionally, the available buffer is determined according to a buffer occupied by the data packet written into the extended buffer and a released buffer after the data packet has been read from the extended buffer.
Optionally, the writing module is specifically configured to:
and after analyzing the data packet sent by the terminal, writing the analyzed data into the extended cache region, wherein the analyzed data at least comprises the byte length of the data packet, and the byte length of the data packet occupies the extended cache region with the preset byte length.
Optionally, after the writing module, the apparatus further includes a reading module, configured to:
reading a data packet from the extended cache region;
releasing the extended cache area occupied by the read data packet;
the reading module is specifically configured to:
and if the length of the residual bytes in the extended cache region is greater than the preset byte length, reading the data packet from the extended cache region.
In a third aspect, an embodiment of the present invention provides a terminal device, including: at least one processor and memory;
the memory stores a computer program; the at least one processor executes the computer program stored by the memory to implement the data caching method provided by the first aspect.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored, and when the computer program is executed, the data caching method provided in the first aspect is implemented.
The technical scheme provided by the embodiment of the invention is applied to a server and comprises the following steps: acquiring a data packet sent by a target terminal through a pre-established IOCP model; determining the size of the data packet and the size of an available cache area in a virtual container corresponding to a target terminal; if the size of the data packet is larger than the size of the available cache region, calling the dynamic cache region to expand the available cache region to obtain an expanded cache region; the data packet is stored in the extended cache region, and by adopting the embodiment of the invention, under the condition that an IOCP model is adopted to process large concurrent data, the cache region in the virtual container corresponding to each terminal can be dynamically extended in real time, so that the IOCP network communication capacity is improved to the maximum extent.
Drawings
FIG. 1 is a flow chart of the steps of an embodiment of a data caching method of the present invention;
FIG. 2 is a flow chart of steps in another embodiment of a data caching method of the present invention;
FIG. 3 is a flow chart of steps in yet another data caching method embodiment of the present invention;
FIG. 4 is a flow chart of steps in yet another data caching method embodiment of the present invention;
FIG. 5 is a flow chart of steps in yet another data caching method embodiment of the present invention;
FIG. 6 is a flow chart of steps in another data caching method embodiment of the present invention;
FIG. 7 is a block diagram of an embodiment of a data caching apparatus according to the present invention;
fig. 8 is a schematic structural diagram of a terminal device of the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
The nouns are explained as follows:
IOCP: I/O Completion Port, often referred to as an I/O Completion Port. The IOCP model belongs to a communication model, and is a technique suitable for a highly loaded server capable of controlling concurrent execution. Colloquially, it is a model for efficiently handling data exchange for many clients. Or, as it were, a model for asynchronous I/O operations; "complete" in an IOCP (i.e., completion port) means that the system is notified that data from the network has been received after the network I/O operation has "completed". This may allow the application to do other things during the I/O transfer. The IOCP network communication model is currently the most widely used network communication infrastructure supporting large concurrency and high load.
Referring to fig. 1, a flowchart illustrating steps of an embodiment of a data caching method according to the present invention is shown, where the method may be applied to a server, and specifically may include the following steps:
s101, acquiring a data packet sent by a target terminal through a pre-established IOCP model;
specifically, on the server side, the IOCP model is pre-established, that is, the server may receive the data packets sent by the multiple internet terminals through the multiple asynchronous I/os at the same time, that is, the server receives the I/O layer network data from each channel through the IOCP model.
When the IOCP model is established, a plurality of network channels are established, specifically, a plurality of virtual containers are established in advance on a server side, a pre-applied cache space corresponding to each virtual container is a preset cache region, when a monitoring thread of the established IOCP model monitors a data packet sent by a terminal, the terminal is associated with the pre-established virtual container to form the network channel for data transmission between the terminal and the server, wherein the virtual container is used for storing the data packet sent by the terminal, and the data packet comprises terminal node information of the terminal, content of the data packet and the like.
S102, determining the size of the data packet and the size of an available buffer area in a virtual container corresponding to the target terminal;
specifically, after receiving a data packet sent by a target terminal, the server calculates the size of the data packet, and calculates the size of the actually available buffer area in the virtual container corresponding to the target terminal at that time.
S103, if the size of the data packet is larger than the size of the available cache region, calling a dynamic cache region to expand the available cache region to obtain an expanded cache region;
the available buffer area is determined according to the buffer area occupied by the stored data packet and the released buffer area after the data packet is read from the preset buffer area.
Specifically, if the data packet is written into the virtual container for the first time, the available cache area in the virtual container is the preset cache area;
when the data packet is written into the virtual container subsequently, the available buffer area is determined according to the buffer area occupied by the data packet written into the extended buffer area and the released buffer area after the data packet is read from the extended buffer area.
The server judges the size of the data packet to be cached and the size of an actually available cache region, and if the size of the data packet to be cached is smaller than the size of the actually available cache region, the data packet is directly stored into the actually available cache region in the virtual container corresponding to the target terminal;
if the size of the data packet to be cached is larger than the size of the actually available cache region, calling the dynamic cache region to expand the actually available cache region to obtain an expanded cache region, wherein the dynamic cache region may be a memory region other than a cache region applied for each virtual container in advance, or may be a preset cache region corresponding to other unoccupied virtual containers.
And S104, storing the data packet into the extended buffer area.
Illustratively, the server establishes 3 network channels in advance, namely 3 virtual containers in advance, receives data packets sent by a terminal A, a terminal B and a terminal C through the established IOCP model, and associates the terminal A with the virtual container A, the terminal B with the virtual container B and the terminal C with the virtual container C;
the method comprises the steps that a server receives a data packet M and a data packet N sent by a terminal A, the data packet M is stored in a virtual container A, the size of the data packet M and the size of an available cache region in the virtual container at the moment are calculated firstly (because data are not stored in a preset cache region, the available cache region is the preset cache region at the moment), and if the size of the data packet M is smaller than the size of the available cache region, the data packet M is directly stored in an available storage region corresponding to the virtual container A;
continuously storing the data packet N into the virtual container A, and calculating the size of the data packet N and the size of an actually available cache region, wherein at the moment, the actually available cache region is the preset cache region minus the space occupied by the data packet M, and if the size of the data packet N is larger than the size of the available cache region, a dynamic cache region is called to expand the actually available cache region, wherein the dynamic cache region can be a pre-applied cache region corresponding to a virtual container corresponding to a terminal or a cache space except the pre-applied cache region corresponding to the virtual container corresponding to the terminal;
specifically, in the process of data communication between the terminal a and the server, only a preset cache region corresponding to the virtual container a is occupied, at this time, the preset cache region applied for the terminal B and the terminal C is in a dormant or stagnant state in advance, when the data packet N of the terminal a is stored, the size of the data packet N and the size of an available cache region are calculated, and if the size of the data packet N is larger than the size of the available cache region, the dynamic cache region is invoked to expand the available cache region, specifically, the preset cache region applied for other terminals is invoked to expand the available cache region, or a memory space except the preset cache region applied for the terminal is invoked to expand the available cache region, so as to obtain an expanded cache region; and storing the data packet N into an extended buffer area in the virtual container A.
According to the data caching method provided by the embodiment of the invention, a data packet sent by a target terminal is obtained through a pre-established IOCP model; determining the size of the data packet and the size of an available buffer area in a virtual container corresponding to a target terminal; if the size of the data packet is larger than the size of the available cache region, calling the dynamic cache region to expand the available cache region to obtain an expanded cache region; the data packet is stored in an expansion cache region which is pre-established in a virtual container corresponding to a target terminal, and through the embodiment of the invention, under the condition that an IOCP model is adopted to process large concurrent data, the preset memory in the virtual container corresponding to each terminal can be dynamically expanded in real time, so that the IOCP network communication capacity is improved to the maximum extent.
The method provided by the above embodiment is further described in an additional embodiment of the present invention.
As shown in fig. 2, a flowchart illustrating steps of another embodiment of a data caching method of the present invention is shown, where the method further includes:
s201, creating a plurality of virtual containers in advance;
specifically, the server establishes a plurality of virtual containers based on the IOCP model, and the virtual containers are used for storing information of each terminal and transmitted data packets.
S202, when a monitoring thread in the IOCP model detects that a target terminal sends a data packet, associating the virtual container with the target terminal to form a network channel between the target terminal and the virtual container so as to enable the target terminal and a server to perform data transmission;
specifically, the server and the terminal establish a socket connection, and the terminal sends a data packet to the server through the socket, where the socket may be included in the internet communication protocol, and may be understood as an interface for interaction between an application layer protocol and a transport layer protocol, which may be used to distinguish network communication and connection between different application program processes.
When monitoring a data packet sent by a terminal by a monitoring thread of an IOCP model pre-established on a server, allocating a virtual container for the terminal, establishing a network channel for data transmission, and carrying out data transmission between the terminal and the server through the IOCP model and the virtual container corresponding to the terminal.
As shown in fig. 3, which shows a flowchart of steps of another embodiment of the data caching method of the present invention, step S103 specifically includes:
s1031, determining the size of the cache region needing capacity expansion according to the difference value between the size of the data packet and the size of the available cache region;
s1032, calling the dynamic cache region to expand the available cache region according to the size of the cache region needing to be expanded, and obtaining the expanded cache region, wherein the dynamic cache region comprises cache regions except the preset cache region which is applied for the virtual container corresponding to each terminal in advance and/or preset cache regions of virtual containers corresponding to other terminals.
Specifically, the server calculates the size of the cache region to be expanded according to the difference between the size of the data packet to be cached and the size of the available cache region;
as an implementable manner, according to the size of a cache region to be expanded, calling a cache region other than a preset cache region, which is applied for the virtual container of each terminal in advance, to expand an available cache region corresponding to a target terminal, and obtaining an expanded cache region;
as another implementable manner, according to the size of the cache region to be expanded, calling a preset cache region corresponding to another terminal to expand the available cache region corresponding to the target terminal, and obtaining an expanded cache region;
as another implementable manner, according to the size of the cache region to be expanded, a preset cache region corresponding to another terminal and an available cache region corresponding to a target terminal except for the preset cache region previously applied for the virtual container of the terminal are simultaneously called for expansion, so as to obtain an expanded cache region;
as shown in fig. 4, a flowchart illustrating steps of another embodiment of a data caching method of the present invention is shown, where the data caching method includes:
s401, acquiring a data packet sent by a target terminal through a pre-established IOCP model.
S402, determining the size of the data packet and the size of an available buffer area in a virtual container corresponding to the target terminal;
s403, if the size of the data packet is larger than the size of the available cache region, calling a dynamic cache region to expand the available cache region to obtain an expanded cache region;
since steps S401 to S403 are the same as steps S101 to S103 in the embodiment shown in fig. 1, steps S101 to S103 have already been described in detail in fig. 1, and therefore, steps S401 to S403 are not described again.
S404, after a data packet sent by the terminal is analyzed, writing the analyzed data into the extended cache region, wherein the analyzed data at least comprises the byte length of the data packet, and the byte length of the data packet occupies the extended cache region with a preset byte length;
specifically, the server stores the data packet obtained from the IOCP model into an extended buffer in a virtual container corresponding to the target terminal, and obtains the parsed data after parsing the data packet, where the parsed data is written into the extended buffer in a manner of a byte length of the data packet and a data packet content, specifically, the byte length of the data packet occupies the extended buffer of a preset byte length, for example, the byte length of the data packet occupies an extended buffer of 4 byte lengths, and the data packet content may be placed after the byte length of the data packet according to an actual byte length.
As shown in fig. 5, a flowchart illustrating steps of another embodiment of a data caching method of the present invention is shown, where the data caching method includes:
s501, acquiring a data packet sent by a target terminal through a pre-established IOCP model.
S502, determining the size of the data packet and the size of an available buffer area in a virtual container corresponding to the target terminal;
s503, if the size of the data packet is larger than the size of the available cache region, calling a dynamic cache region to expand the available cache region to obtain an expanded cache region;
s504, after a data packet sent by the terminal is analyzed, the analyzed data is written into the extended cache region, wherein the analyzed data at least comprises the byte length of the data packet, and the byte length of the data packet occupies the extended cache region with the preset byte length;
since steps S501 to S504 are the same as steps S401 to S404 in the embodiment shown in fig. 4, steps S401 to S404 have already been described in detail in fig. 4, and therefore, steps S501 to S504 are not described again here.
S505, reading a data packet from the expansion cache region;
and releasing the read extended buffer area occupied by the data packet.
Specifically, as an executable manner, if the length of the remaining bytes in the extended cache area is greater than the preset byte length, reading the data packet from the extended cache area;
as another executable manner, if the remaining byte length in the extended buffer is smaller than the preset byte length, a new data packet is read from the pre-established IOCP model.
As shown in fig. 6, a flowchart illustrating steps of another embodiment of a data caching method of the present invention is shown, where the data caching method includes:
s601, generating a certain required number of network channels, namely, pre-creating a plurality of virtual containers on a server, wherein the network channels are used for data transmission between a terminal and the virtual containers;
when a monitoring thread in the IOCP model detects that a terminal sends a data packet to a server, the pre-established virtual containers are associated with the terminal, and the pre-established network channel is directly adopted, so that the time spent on reproducing the network channel when receiving the data packet sent by the terminal can be avoided, and meanwhile, a relatively small memory space buffer is applied in each virtual container in advance;
s602, selecting an idle network channel A, and receiving a data packet sent by a terminal through an IOCP model by a server;
s603, comparing the size of the data packet to be cached with the size of an effective cache region, namely tbuffer, wherein the effective cache region size is determined according to the occupied cache region and the cache region read from the cache region;
and if the size Buffer of the received data packet is smaller than the effective Buffer area size tbuffer, indicating that the Buffer area is sufficient, and storing the received data packet into the Buffer area Buffer.
S604, if the size of the received data packet Buffer is larger than the effective Buffer area size tbuffer, calculating the memory space Buffer-tbuffer which needs to be expanded actually, applying for a new memory space on the basis of the existing available Buffer area to obtain an expanded Buffer area, and writing the received data packet into the expanded Buffer area;
s605, assigning and updating the effective data length in the buffer of the extended cache region;
s606, reading the data packet written in the buffer of the extended buffer area;
s607, specifically, after analyzing the received data packet, writing the data packet into the extended cache region in a manner of data packet length and data packet content, wherein the data packet length occupies 4 bytes, and the data packet content is written according to the actual length;
s608, if the length of the remaining effective bytes is greater than 4 bytes when the data is read from the available buffer area tbuffer in the extended buffer area, the data packet can be continuously read;
and S609, if the length of the remaining effective byte is less than 4 bytes, acquiring a new data packet from the IOCP model, and executing the step S603 again.
According to the data caching method provided by the embodiment of the invention, a data packet sent by a target terminal is obtained through a pre-established IOCP model; determining the size of the data packet and the size of an available buffer area in a preset buffer area; if the size of the data packet is larger than the size of the available cache region, calling the dynamic cache region to expand the available cache region to obtain an expanded cache region; the data packet is stored in an expansion cache region which is pre-established in a virtual container corresponding to the terminal, and through the embodiment of the invention, under the condition that an IOCP model is adopted to process large concurrent data, the preset memory in the virtual container corresponding to each terminal can be dynamically expanded in real time, so that the IOCP network communication capacity is improved to the maximum extent.
Another embodiment of the present invention provides a data caching apparatus, configured to execute the data caching method provided in the foregoing embodiment.
Referring to fig. 7, a block diagram of a data caching apparatus according to an embodiment of the present invention is shown, and the data caching apparatus is applied to a server, and the apparatus includes: an obtaining module 701, a determining module 702, a capacity expanding module 703 and a writing module 704, wherein:
the obtaining module 701 is configured to obtain a data packet sent by a target terminal through a pre-established IOCP model;
a determining module 702 is configured to determine the size of the data packet and the size of an available buffer in a virtual container corresponding to the target terminal;
the capacity expansion module 703 is configured to invoke a dynamic cache area to expand the available cache area if the size of the data packet is larger than the size of the available cache area, so as to obtain an expanded cache area;
the writing module 704 is configured to store the data packet into the extended buffer.
The data caching device provided by the embodiment of the invention obtains the data packet sent by the target terminal through the pre-established IOCP model; determining the size of the data packet and the size of an available buffer area in a preset buffer area; if the size of the data packet is larger than the size of the available cache region, calling the dynamic cache region to expand the available cache region to obtain an expanded cache region; the data packet is stored in an expansion cache region which is pre-established in a virtual container corresponding to the terminal, and through the embodiment of the invention, under the condition that an IOCP model is adopted to process large concurrent data, the preset memory in the virtual container corresponding to each terminal can be dynamically expanded in real time, so that the IOCP network communication capacity is improved to the maximum extent.
The device provided by the above embodiment is further described in an additional embodiment of the present invention.
Optionally, before the obtaining module, the apparatus further includes an association module, configured to:
creating a plurality of virtual containers in advance;
and when a monitoring thread in the IOCP model detects that a target terminal sends a data packet, associating the virtual container with the target terminal to form a network channel between the target terminal and the virtual container so as to enable the target terminal and a server to carry out data transmission.
Optionally, the capacity expansion module 703 is specifically configured to:
determining the size of a cache region needing capacity expansion according to the difference value between the size of the data packet and the size of the available cache region;
and calling the dynamic cache region to expand the available cache region according to the size of the cache region needing to be expanded, and obtaining the expanded cache region, wherein the dynamic cache region comprises a cache region except a preset cache region which is applied in advance for the virtual container corresponding to each terminal and/or a preset cache region of a virtual container corresponding to other terminals.
Optionally, the available buffer is determined according to a buffer occupied by the data packet written into the extended buffer and a released buffer after the data packet has been read from the extended buffer.
Optionally, the writing module 704 is specifically configured to:
and after analyzing the data packet sent by the terminal, writing the analyzed data into the extended cache region, wherein the analyzed data at least comprises the byte length of the data packet, and the byte length of the data packet occupies the extended cache region with the preset byte length.
Optionally, after the writing module, the apparatus further includes a reading module, configured to:
reading a data packet from the extended cache region;
releasing the extended cache area occupied by the read data packet;
the reading module is specifically configured to:
and if the length of the residual bytes in the extended cache region is greater than the preset byte length, reading the data packet from the extended cache region.
It should be noted that the respective implementable modes in the present embodiment may be implemented individually, or may be implemented in combination in any combination without conflict, and the present application is not limited thereto.
For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.
The data caching device provided by the embodiment of the invention obtains the data packet sent by the target terminal through the pre-established IOCP model; determining the size of the data packet and the size of an available buffer area in a preset buffer area; if the size of the data packet is larger than the size of the available cache region, calling the dynamic cache region to expand the available cache region to obtain an expanded cache region; the data packet is stored in an expansion cache region which is pre-established in a virtual container corresponding to the terminal, and through the embodiment of the invention, under the condition that an IOCP model is adopted to process large concurrent data, the preset memory in the virtual container corresponding to each terminal can be dynamically expanded in real time, so that the IOCP network communication capacity is improved to the maximum extent.
Another embodiment of the present invention provides a terminal device, configured to execute the data caching method provided in the foregoing embodiment.
Fig. 8 is a schematic structural diagram of a terminal device according to the present invention, and as shown in fig. 8, the electronic device includes: at least one processor 801 and memory 802;
the memory stores a computer program; the at least one processor executes the computer program stored in the memory to implement the data caching method provided by the above embodiments.
The terminal device provided in this embodiment obtains a data packet sent by a target terminal through a pre-established IOCP model; determining the size of the data packet and the size of an available buffer area in a virtual container corresponding to a target terminal; if the size of the data packet is larger than the size of the available cache region, calling the dynamic cache region to expand the available cache region to obtain an expanded cache region; the data packet is stored in an expansion cache region which is pre-established in a virtual container corresponding to the terminal, and through the embodiment of the invention, under the condition that an IOCP model is adopted to process large concurrent data, the preset memory in the virtual container corresponding to each terminal can be dynamically expanded in real time, so that the IOCP network communication capacity is improved to the maximum extent.
Yet another embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed, the data caching method provided in any one of the above embodiments is implemented.
According to the computer-readable storage medium of the embodiment, a data packet sent by a target terminal is acquired through a pre-established IOCP model; determining the size of the data packet and the size of an available buffer area in a virtual container corresponding to the target terminal; if the size of the data packet is larger than the size of the available cache region, calling the dynamic cache region to expand the available cache region to obtain an expanded cache region; the data packet is stored in an expansion cache region which is pre-established in a virtual container corresponding to the terminal, and through the embodiment of the invention, under the condition that an IOCP model is adopted to process large concurrent data, the preset memory in the virtual container corresponding to each terminal can be dynamically expanded in real time, so that the IOCP network communication capacity is improved to the maximum extent.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.
The data caching method and the data caching device provided by the invention are described in detail, specific examples are applied in the text to explain the principle and the implementation of the invention, and the description of the above embodiments is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (10)
1. A data caching method is applied to a server, and the method comprises the following steps:
acquiring a data packet sent by a target terminal through a pre-established IOCP model;
determining the size of the data packet and the size of an available cache area in a virtual container corresponding to the target terminal;
if the size of the data packet is larger than the size of the available cache region, calling a dynamic cache region to expand the available cache region to obtain an expanded cache region;
and storing the data packet into the extended buffer area.
2. The method of claim 1, wherein before the obtaining the data packet sent by the target terminal through the pre-established IOCP model, the method further comprises:
creating a plurality of virtual containers in advance;
and when a monitoring thread in the IOCP model detects that a target terminal sends a data packet, associating the virtual container with the target terminal to form a network channel between the target terminal and the virtual container so as to enable the target terminal and a server to carry out data transmission.
3. The method of claim 1, wherein if the size of the packet is larger than the size of the available buffer, invoking a dynamic buffer to expand the available buffer to obtain an expanded buffer, comprises:
determining the size of a cache region needing capacity expansion according to the difference value between the size of the data packet and the size of the available cache region;
and calling the dynamic cache region to expand the available cache region according to the size of the cache region needing to be expanded, and obtaining the expanded cache region, wherein the dynamic cache region comprises a cache region except a preset cache region which is applied in advance for the virtual container corresponding to each terminal and/or a preset cache region of a virtual container corresponding to other terminals.
4. The method of claim 1, wherein the available buffer is determined according to a buffer occupied by the packet written into the extended buffer and a released buffer after the packet has been read from the extended buffer.
5. The method of claim 1, wherein storing the packet into the extended buffer comprises:
and after analyzing the data packet sent by the terminal, writing the analyzed data into the extended cache region, wherein the analyzed data at least comprises the byte length of the data packet, and the byte length of the data packet occupies the extended cache region with the preset byte length.
6. The method of claim 5, wherein after storing the packet in the extended buffer, the method further comprises:
reading a data packet from the extended cache region;
releasing the extended cache area occupied by the read data packet;
specifically, the reading the data packet from the extended buffer includes:
and if the length of the residual bytes in the extended cache region is greater than the preset byte length, reading the data packet from the extended cache region.
7. A data caching apparatus, applied to a server, the apparatus comprising:
the acquisition module is used for acquiring a data packet sent by a target terminal through a pre-established IOCP model;
a determining module, configured to determine a size of the data packet and a size of an available buffer in a virtual container corresponding to the target terminal;
the capacity expansion module is used for calling the dynamic cache area to expand the available cache area to obtain an expanded cache area if the size of the data packet is larger than the size of the available cache area;
and the writing module is used for storing the data packet into the extended cache region.
8. The apparatus of claim 7, wherein before the obtaining module, the apparatus further comprises an associating module configured to:
creating a plurality of virtual containers in advance;
and when a monitoring thread in the IOCP model detects that a target terminal sends a data packet, associating the virtual container with the target terminal to form a network channel between the target terminal and the virtual container so as to enable the target terminal and a server to carry out data transmission.
9. A terminal device, comprising: at least one processor and memory;
the memory stores a computer program; the at least one processor executes the memory-stored computer program to implement the data caching method of any one of claims 1 to 6.
10. A computer-readable storage medium, in which a computer program is stored which, when executed, implements the data caching method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010081796.3A CN111371704B (en) | 2020-02-06 | 2020-02-06 | Data caching method and device, terminal equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010081796.3A CN111371704B (en) | 2020-02-06 | 2020-02-06 | Data caching method and device, terminal equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111371704A true CN111371704A (en) | 2020-07-03 |
| CN111371704B CN111371704B (en) | 2024-03-15 |
Family
ID=71209998
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010081796.3A Active CN111371704B (en) | 2020-02-06 | 2020-02-06 | Data caching method and device, terminal equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111371704B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112737977A (en) * | 2020-12-28 | 2021-04-30 | 苏州浪潮智能科技有限公司 | Data packet processing method and device |
| CN112887319A (en) * | 2021-02-01 | 2021-06-01 | 上海帆一尚行科技有限公司 | Network state monitoring method and device based on downlink traffic and electronic equipment |
| CN112905697A (en) * | 2021-02-10 | 2021-06-04 | 北京锐服信科技有限公司 | Method and system for capturing high-speed network data packet |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5920732A (en) * | 1996-07-01 | 1999-07-06 | Apple Computer, Inc. | System for preallocating additional larger buffer sizes in accordance with packet sizes of discarded packets that can't be stored in existing preallocated buffer sizes |
| KR20040058032A (en) * | 2002-12-26 | 2004-07-03 | 한국과학기술정보연구원 | File transfer management system and method, and storage media having program thereof |
| CN1677946A (en) * | 2004-04-02 | 2005-10-05 | 华为技术有限公司 | Buffer distribution method and apparatus |
| US20150334058A1 (en) * | 2014-05-13 | 2015-11-19 | Red Hat Israel, Ltd. | Adjusting buffer size for network interface controller |
| CN107689923A (en) * | 2016-08-04 | 2018-02-13 | 华为技术有限公司 | Message processing method and router |
| CN108834086A (en) * | 2018-06-25 | 2018-11-16 | 平安科技（深圳）有限公司 | Method, apparatus, computer equipment and the storage medium that short message is sent |
-
2020
- 2020-02-06 CN CN202010081796.3A patent/CN111371704B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5920732A (en) * | 1996-07-01 | 1999-07-06 | Apple Computer, Inc. | System for preallocating additional larger buffer sizes in accordance with packet sizes of discarded packets that can't be stored in existing preallocated buffer sizes |
| KR20040058032A (en) * | 2002-12-26 | 2004-07-03 | 한국과학기술정보연구원 | File transfer management system and method, and storage media having program thereof |
| CN1677946A (en) * | 2004-04-02 | 2005-10-05 | 华为技术有限公司 | Buffer distribution method and apparatus |
| US20150334058A1 (en) * | 2014-05-13 | 2015-11-19 | Red Hat Israel, Ltd. | Adjusting buffer size for network interface controller |
| CN107689923A (en) * | 2016-08-04 | 2018-02-13 | 华为技术有限公司 | Message processing method and router |
| CN108834086A (en) * | 2018-06-25 | 2018-11-16 | 平安科技（深圳）有限公司 | Method, apparatus, computer equipment and the storage medium that short message is sent |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112737977A (en) * | 2020-12-28 | 2021-04-30 | 苏州浪潮智能科技有限公司 | Data packet processing method and device |
| CN112887319A (en) * | 2021-02-01 | 2021-06-01 | 上海帆一尚行科技有限公司 | Network state monitoring method and device based on downlink traffic and electronic equipment |
| CN112887319B (en) * | 2021-02-01 | 2022-07-01 | 上海帆一尚行科技有限公司 | Network state monitoring method and device based on downlink traffic and electronic equipment |
| CN112905697A (en) * | 2021-02-10 | 2021-06-04 | 北京锐服信科技有限公司 | Method and system for capturing high-speed network data packet |
| CN112905697B (en) * | 2021-02-10 | 2024-04-16 | 北京锐服信科技有限公司 | High-speed network data packet capturing method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111371704B (en) | 2024-03-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111371704B (en) | Data caching method and device, terminal equipment and storage medium | |
| CN101316226B (en) | Method, device and system for acquiring resources | |
| KR102376713B1 (en) | Composite partition functions | |
| CN107241281B (en) | Data processing method and device | |
| CN110268751B (en) | Method and system for selecting access and mobility management functions in an access network environment | |
| CN109408590B (en) | Method, device and equipment for expanding distributed database and storage medium | |
| CN109547524B (en) | User behavior storage method, device, equipment and storage medium based on Internet of things | |
| CN103237060B (en) | A kind of data object acquisition, Apparatus and system | |
| US8935377B2 (en) | Dynamic registration of listener resources for cloud services | |
| CN107402956B (en) | Data processing method and device for large task and computer readable storage medium | |
| US9483493B2 (en) | Method and system for accessing a distributed file system | |
| CN111124589B (en) | Service discovery system, method, device and equipment | |
| US8832215B2 (en) | Load-balancing in replication engine of directory server | |
| US7257615B2 (en) | Server side execution of application modules in a client and server system | |
| CN109564502A (en) | Treating method and apparatus applied to the access request in storage equipment | |
| CN107579929B (en) | Method, system and related device for setting reliable connection communication queue pair | |
| CN115686875A (en) | Method, apparatus and program product for transferring data between multiple processes | |
| US20160112540A1 (en) | System and Method for Virtual Desktop Infrastructure User Level Virtual Channel | |
| CN106571935B (en) | Resource scheduling method and equipment | |
| CN110958138B (en) | Container expansion method and device | |
| CN112346980A (en) | Software performance testing method, system and readable storage medium | |
| CN110581826B (en) | Core network communication method, device and system | |
| CN109005071B (en) | Decision deployment method and scheduling equipment | |
| CN104079368B (en) | A kind of the test data transmission method and server of application software | |
| CN105554134A (en) | Information synchronization and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |