WO2021164262A1 - Procédé et appareil de collecte de trafic pour réseau virtuel, et dispositif informatique et support de stockage - Google Patents

Procédé et appareil de collecte de trafic pour réseau virtuel, et dispositif informatique et support de stockage Download PDF

Info

Publication number
WO2021164262A1
WO2021164262A1 PCT/CN2020/119067 CN2020119067W WO2021164262A1 WO 2021164262 A1 WO2021164262 A1 WO 2021164262A1 CN 2020119067 W CN2020119067 W CN 2020119067W WO 2021164262 A1 WO2021164262 A1 WO 2021164262A1
Authority
WO
WIPO (PCT)
Prior art keywords
address
kernel
ring buffer
collection module
page memory
Prior art date
Application number
PCT/CN2020/119067
Other languages
English (en)
Chinese (zh)
Inventor
张齐国
王继海
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Publication of WO2021164262A1 publication Critical patent/WO2021164262A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0876—Network utilisation, e.g. volume of load or congestion level
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/50—Testing arrangements
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate

Definitions

  • This application relates to the technical field of virtual machines, and in particular to a method, device, computer equipment and storage medium for traffic collection of a virtual network.
  • the preferred user mode collection unit uses the system to call the AF_PACKET socket interface to monitor the target virtual network card in the shared memory, and then the kernel collection unit puts a newly added network packet on the received network packet through mmap. The network packet is mapped to the shared memory, and finally the user mode unit detects that there is a new network packet in the shared memory, and then collects the new network packet to complete the flow collection.
  • the user mode unit calls the AF_PACKET interface for each net packet, the system call overhead is high, the calling frequency is high, and the resource consumption is high.
  • the memory mmap of each net packet to the shared memory is expensive and the memory is fragmented. Difficult to manage. Due to the excessive network traffic between virtual machines, the overhead of collecting traffic is too large.
  • a method for collecting traffic in a virtual network including:
  • the communication channel being a data transmission channel between the user mode acquisition module and the kernel acquisition module;
  • the user mode collection module applies for a large page memory, creates a ring buffer area on the large page memory, and transmits the physical address of the large page memory to the kernel collection module through the communication channel;
  • the kernel collection module receives the physical address of the large page memory, maps the physical address of the large page memory to a linear address in the kernel address space, and obtains the buffer of the ring buffer in the kernel collection module address;
  • the kernel collection module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer position of the ring buffer;
  • the user state acquisition module detects that the ring buffer is updated, it accesses the ring buffer and obtains the data packet, counts the data traffic flowing through the virtual switch, and updates the reading of the ring buffer at the same time. Pointer position.
  • a virtual network traffic collection device including:
  • the initialization unit is used to initialize the flow collection software operating environment of the user mode collection module and the core collection module;
  • a channel establishing unit configured to establish a communication channel, the communication channel being a data transmission channel between the user mode acquisition module and the kernel acquisition module;
  • the cache creation unit is used for the user mode collection module to apply for a large page memory, create a ring buffer area on the large page memory, and transmit the physical address of the large page memory to the kernel collection via the communication channel Module
  • the address acquisition unit is configured to receive the physical address of the large page memory by the kernel acquisition module, map the physical address of the large page memory to a linear address in the kernel address space, and obtain the ring buffer in the The buffer address of the kernel acquisition module;
  • the data storage unit is used for the kernel collection module to collect the data packet received by the virtual switch, and save the data packet to the ring buffer according to the buffer address, and at the same time update the write pointer of the ring buffer Location;
  • the traffic statistics unit is configured to, if the user state acquisition module detects that the ring buffer is updated, it will access the ring buffer and obtain the data packets, count the data traffic flowing through the virtual switch, and update all the data at the same time. The position of the read pointer of the ring buffer.
  • a storage medium is provided, and at least one executable instruction is stored in the storage medium, and the executable instruction causes a processor to perform the following steps:
  • the communication channel being a data transmission channel between the user mode acquisition module and the kernel acquisition module;
  • the user mode collection module applies for a large page memory, creates a ring buffer area on the large page memory, and transmits the physical address of the large page memory to the kernel collection module through the communication channel;
  • the kernel collection module receives the physical address of the large page memory, maps the physical address of the large page memory to a linear address in the kernel address space, and obtains the buffer of the ring buffer in the kernel collection module address;
  • the kernel collection module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer position of the ring buffer;
  • the user state acquisition module detects that the ring buffer is updated, it accesses the ring buffer and obtains the data packet, counts the data traffic flowing through the virtual switch, and updates the reading of the ring buffer at the same time. Pointer position.
  • a computer device including:
  • a processor a memory, a communication interface, and a communication bus.
  • the processor, the memory, and the communication interface communicate with each other through the communication bus;
  • the memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform the following steps:
  • the communication channel being a data transmission channel between the user mode acquisition module and the kernel acquisition module;
  • the user mode collection module applies for a large page memory, creates a ring buffer area on the large page memory, and transmits the physical address of the large page memory to the kernel collection module through the communication channel;
  • the kernel collection module receives the physical address of the large page memory, maps the physical address of the large page memory to a linear address in the kernel address space, and obtains the buffer of the ring buffer in the kernel collection module address;
  • the kernel collection module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer position of the ring buffer;
  • the user state acquisition module detects that the ring buffer is updated, it accesses the ring buffer and obtains the data packet, counts the data traffic flowing through the virtual switch, and updates the reading of the ring buffer at the same time. Pointer position.
  • FIG. 1 shows a flow chart of a method for collecting traffic on a virtual network according to an embodiment of the present application
  • FIG. 2 shows a flowchart of another method for collecting traffic on a virtual network according to an embodiment of the present application
  • FIG. 3 shows a block diagram of the composition of a virtual network traffic collection device provided by an embodiment of the present application
  • Figure 4 shows a block diagram of another virtual network traffic collection device provided by an embodiment of the present application.
  • Fig. 5 is a schematic structural diagram of a computer device provided by an embodiment of the present application.
  • This application is the traffic collection environment of the embodiment, which can be between virtual machines on the same host machine, or between virtual machines that do not use host machines.
  • the embodiment of the application is a software method running on a host computer, which mainly includes a user mode acquisition module, a ring buffer, and a kernel acquisition module.
  • the kernel collection module is mainly used to collect the data packets received by the virtual switch and save the data packets to the ring buffer.
  • the user mode collection module is mainly used to obtain the data packets in the ring buffer to count the traffic flowing through the virtual switch.
  • the embodiment of the present application provides a method for collecting traffic on a virtual network. As shown in FIG. 1, the method includes:
  • Initializing the software operating environment refers to the software environment, function library, and loading operating environment used to load the user mode acquisition module and the kernel acquisition module.
  • the communication channel is a path for data transmission.
  • the communication channel is a kind of logical channel, which specifically refers to a logical path implemented by an intermediate node between the parties sending and receiving signals on the basis of a physical channel.
  • the communication channel is a data transmission channel between the user state acquisition module and the core acquisition module. Establish a communication channel, that is, establish a logical path between the user mode acquisition module and the core acquisition module.
  • the user mode collection module applies for a large page memory, creates a ring buffer area on the large page memory, and transmits the physical address of the large page memory to the kernel collection module through the communication channel.
  • the use of large page memory reduces the address translation information, thereby reducing the use of the CPU cache, reducing the pressure on the CPU cache, and allowing the CPU cache to use more Data caching for applications can also improve the performance of the system as a whole.
  • the use of large page memory in the host machine of a memory-intensive virtual machine can significantly improve the performance of the host machine.
  • Large page memory helps Linux system to manage virtual memory, and can define a page size of 1GB, which can reserve part of the memory for applications.
  • the reserved large page memory will always be reserved and will not be swapped out of the memory unless the memory configuration is modified.
  • the large page memory requested by the user state acquisition module that is, the large page memory belongs to the user state acquisition module.
  • the ring buffer is a data structure used to represent a fixed-size, head-to-tail vector buffer, suitable for buffering data streams, and used to store data sent and received in communications.
  • the ring buffer is a first-in, first-out circular buffer area that can provide mutually exclusive access to the buffer for communication programs.
  • the physical address of the large page memory requested by the user mode acquisition module is transmitted to the kernel acquisition module through the communication channel, which is equivalent to that the kernel acquisition module can receive the physical address of the large page memory and can calculate the physical address of the ring buffer at the same time.
  • the kernel collection module receives the physical address of the large page memory, maps the physical address of the large page memory to a linear address in the kernel address space, and obtains the buffer of the ring buffer in the kernel collection module address.
  • the kernel acquisition module receives the physical address of the large page memory, and maps the large page memory to a linear address and the ring buffer to a buffer address according to the physical address of the large page memory, the physical address of the ring buffer, and the kernel address space. .
  • the kernel collection module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer position of the ring buffer.
  • the sampling ratio can be 1:1 or 1:n.
  • the sampling ratio is determined according to the subsequent traffic collection function.
  • the sampling ratio is not limited in the embodiment of this application.
  • the kernel collection module collects the data packets received by the virtual switch and saves them to the ring buffer. Each data packet corresponds to a collection record and occupies a fixed-size space in the ring buffer. In the process of collecting data packets, the header of the data packet can be collected, or the entire data packet can be collected.
  • the user mode collection module If the user mode collection module detects that the ring buffer is updated, it accesses the ring buffer and obtains the data packet, counts the data traffic flowing through the virtual switch, and updates the reading of the ring buffer at the same time. Pointer position.
  • the user mode acquisition module After the user mode acquisition module obtains the data packets in the ring buffer, it can count the data traffic, count the number of data packets, and can also perform detailed analysis on the data traffic. Monitor virtual networks based on data traffic to improve network performance, improve network transmission capacity, analyze network channel utilization, and more.
  • This application provides a virtual network traffic collection method.
  • initialize the software operating environment for traffic collection then establish a communication channel between the user mode collection module and the kernel collection module, and then the user mode collection module applies for a large page of memory.
  • Create a ring buffer area on the large page memory and transmit the physical address of the large page memory to the kernel acquisition module through the communication channel, and then the kernel acquisition module maps the physical address of the large page memory to a linear address in the kernel address space , And obtain the buffer address of the ring buffer in the kernel acquisition module, and then the kernel acquisition module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer of the ring buffer Finally, if the user mode acquisition module detects the ring buffer update, it accesses the ring buffer and obtains data packets, counts the data traffic flowing through the virtual switch, and updates the read pointer position of the ring buffer at the same time.
  • the embodiment of the present application directly reads the collected data packet through the ring buffer through the user mode collection module, there is no copying overhead, no system call overhead, and very low resource consumption.
  • the core acquisition module transfers the acquisition data packet to the user mode acquisition module through the ring buffer, and synchronizes with the user mode acquisition module through the write pointer position and the read pointer position in the ring buffer to reduce system overhead.
  • the smallest granularity of the monitoring object is the virtual switch, which monitors the virtual machine and network performance without intruding the virtual machine to ensure the security and performance of the virtual network.
  • the embodiment of the present application provides another method for collecting traffic on a virtual network. As shown in FIG. 2, the method includes:
  • Initializing the software operating environment refers to the software environment, function library, and loading operating environment used to load the user mode acquisition module and the kernel acquisition module.
  • the communication channel is a data transmission channel between the user state acquisition module and the core acquisition module. Establish a communication channel, that is, establish a logical path between the user mode acquisition module and the core acquisition module.
  • the communication channel can be a Netlink channel, a pipe channel, or a Socket channel. Among them, the Netlink channel is a special Socket channel, and the establishment of the two channels is similar.
  • establishing the communication channel includes: creating a socket according to the socket creation function; setting the local address and destination address of the socket, and the local address is the user mode acquisition module The destination address is the physical address of the core collection module; if the first monitoring function detects that the socket receives a connection request, the communication channel is established.
  • Netlink socket is a special inter-process communication (IPC) used to realize the communication between the user process and the kernel process, and it is also the most commonly used interface for communication between network applications and the kernel. It is easily applied to netlink channels through socket(), bind(), connect(), listen(), sendmsg(), recvmsg() and close(). netlink is included in the header file linux/netlink.h.
  • the socket() is used to create a socket.
  • bind() is used to specify the local address. After a socket is created with socket(), it is not actually associated with any specific local or destination address. Once a socket is created, the bind() system call must be used to establish a local address for the socket.
  • connect() is used to connect the socket to the destination address.
  • listen() is used to set the status of waiting for connection. For a server program, after applying for a socket and calling bind() to bind to the local address, it should wait for a client program to request a connection. listen() is a function that sets a socket to this state. sendmsg() and recvmsg() are used to send and receive data. In the data stream mode, after a connection is established, or in the datagram mode, after calling connect() to bind the socket and the destination address, you can call sendmsg() and recvmsg() functions for data transmission . closesocket() is used to close the socket.
  • establishing the communication channel includes: in the user mode acquisition module, establishing the pipeline channel according to the creation function; if the second listening function listens to the core acquisition module to send a connection request, then through all The pipeline channel connects the user state acquisition module and the core acquisition module.
  • the pipeline channel When the pipeline channel is created, a page size is allocated for the buffer.
  • the pipeline transmits an unformatted byte stream. This requires that the read and write parties of the pipeline must agree on the format of the data in advance.
  • the creation process uses the specific functions of the pipeline channel to create, such as the creation function CreateNamedPipe(), the monitoring function ConnectNamedPipe(), the data transfer functions ReadFile() and WriteFile(), and the closing function CloseHandle().
  • the user mode collection module applies for a large page memory, creates a ring buffer area on the large page memory, and transmits the physical address of the large page memory to the kernel collection module through the communication channel.
  • large page memory can be allocated in local memory for each NUMA, and a separate ring buffer can be created for each CPU core.
  • the kernel maintains a table that maps virtual memory addresses to physical addresses. For each page operation, the kernel needs to load related mappings. If the memory page is small, there will be a lot of pages that need to be loaded, causing the kernel to load more mapping tables, which will reduce performance. With “large memory pages”, fewer pages are needed. Thereby greatly reducing the number of mapping tables loaded by the kernel, improving the performance of the kernel level and ultimately benefiting the performance of the application. In short, by enabling "large memory pages", the system only needs to process fewer page mapping tables, thereby reducing the overhead of accessing/maintaining them.
  • sendmsg() in Netlink sockets are used to send physical addresses.
  • writeFile in pipe communication
  • send() and sendto() in sockets are used to send physical addresses.
  • the kernel collection module receives the physical address of the large page memory, and obtains the memory allocation relationship between the host machine and the virtual machine.
  • the response mechanism can be set to ensure the effective transmission of the physical address. For example, after the kernel collection module receives the physical address of the large page memory, it needs to feed back the received message to the user mode collection module.
  • the received message can carry the data header information of the physical address, and the user mode module receives the feedback message from the kernel collection module. Then you can correct whether the received information is correct.
  • the kernel acquisition module maps the physical address of the large page memory to the linear address of the kernel address space according to the memory allocation relationship.
  • the kernel collection module calculates the buffer address of the ring buffer in the kernel collection module according to the address allocation relationship of the linear address, the large page memory, and the ring buffer.
  • the kernel acquisition module maps the large page memory into a linear address and the ring buffer into a buffer address according to the physical address of the large page memory, the physical address of the ring buffer, and the kernel address space.
  • the kernel collection module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer position of the ring buffer.
  • the sampling ratio can be 1:1 or 1:n.
  • the sampling ratio is determined according to the subsequent traffic collection function.
  • the sampling ratio is not limited in the embodiment of this application. .
  • the kernel collection module collects the data packets received by the virtual switch and saves them to the ring buffer. Each data packet corresponds to a collection record and occupies a fixed-size space in the ring buffer. In the process of collecting data packets, the header of the data packet can be collected, or the entire data packet can be collected.
  • the kernel collection module collects data packets received by the virtual switch, which specifically includes: judging the type of the virtual switch, the types include Linux Bridge and OpenvSwitch; if the virtual switch is Type is Linux Bridge, the kernel collection module registers the hook function, and switches into the kernel collection module according to the hook function to collect data packets received by the virtual switch; if the type of the virtual switch is OpenvSwitch, the kernel The collection module uses the flow table structure ovsdatapath to execute the collection action action in the management operation flow, and collect the data packets received by the virtual switch.
  • the write pointer refers to the location of the data to be written in the ring buffer.
  • the user mode collection module If the user mode collection module detects that the ring buffer is updated, it accesses the ring buffer and obtains the data packet, counts the data traffic flowing through the virtual switch, and updates the reading of the ring buffer at the same time. Pointer position.
  • Obtaining the data packet specifically includes: obtaining the current write pointer position, and storing the current write pointer position; if the current write pointer position is different from the previously stored write pointer position, accessing and Acquire a data packet in the ring buffer, and the data packet is stored between the read pointer position of the ring buffer and the current write pointer position.
  • the position of the write pointer in the ring buffer By detecting the position of the write pointer in the ring buffer, it is monitored whether there is an update packet in the ring buffer.
  • the position of the read pointer indicates the position of the data packet in the ring buffer that has not yet been counted.
  • This application provides a virtual network traffic collection method.
  • initialize the software operating environment for traffic collection then establish a communication channel between the user mode collection module and the kernel collection module, and then the user mode collection module applies for a large page of memory.
  • Create a ring buffer area on the large page memory and transmit the physical address of the large page memory to the kernel acquisition module through the communication channel, and then the kernel acquisition module maps the physical address of the large page memory to a linear address in the kernel address space , And obtain the buffer address of the ring buffer in the kernel acquisition module, and then the kernel acquisition module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer of the ring buffer Finally, if the user mode acquisition module detects the ring buffer update, it accesses the ring buffer and obtains data packets, counts the data traffic flowing through the virtual switch, and updates the read pointer position of the ring buffer at the same time.
  • the embodiment of the present application directly reads the collected data packet through the ring buffer through the user mode collection module, there is no copying overhead, no system call overhead, and very low resource consumption.
  • the core acquisition module transfers the acquisition data packet to the user mode acquisition module through the ring buffer, and synchronizes with the user mode acquisition module through the write pointer position and the read pointer position in the ring buffer to reduce system overhead.
  • the smallest granularity of the monitoring object is the virtual switch, which monitors the virtual machine and network performance without intruding the virtual machine to ensure the security and performance of the virtual network.
  • an embodiment of the present application provides a virtual network traffic collection device.
  • the device includes:
  • the initialization unit 31 is used to initialize the flow collection software operating environment of the user mode collection module and the kernel collection module;
  • the channel establishment unit 32 is configured to establish a communication channel, the communication channel being a data transmission channel between the user state acquisition module and the core acquisition module;
  • the cache creation unit 33 is used for the user mode acquisition module to apply for a large page memory, create a ring buffer area on the large page memory, and transmit the physical address of the large page memory to the kernel through the communication channel Acquisition module;
  • the address acquiring unit 34 is configured to receive the physical address of the large page memory by the kernel acquisition module, map the physical address of the large page memory to a linear address in the kernel address space, and acquire the location of the ring buffer The buffer address of the kernel acquisition module;
  • the data storage unit 35 is configured to collect the data packet received by the virtual switch by the kernel collection module, save the data packet to the ring buffer according to the buffer address, and update the write of the ring buffer at the same time. Pointer position
  • the traffic statistics unit 36 is configured to access the ring buffer and obtain the data packet if the user mode collection module detects that the ring buffer is updated, and then count the data traffic flowing through the virtual switch, and update at the same time The position of the read pointer of the ring buffer.
  • This application provides a virtual network traffic collection device.
  • First initialize the software operating environment for traffic collection, then establish a communication channel between the user mode collection module and the kernel collection module, and then the user mode collection module applies for a large page of memory. Create a ring buffer area on the large page memory, and transmit the physical address of the large page memory to the kernel acquisition module through the communication channel, and then the kernel acquisition module maps the physical address of the large page memory to a linear address in the kernel address space , And obtain the buffer address of the ring buffer in the kernel acquisition module, and then the kernel acquisition module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer of the ring buffer Finally, if the user mode acquisition module detects the ring buffer update, it accesses the ring buffer and obtains data packets, counts the data traffic flowing through the virtual switch, and updates the read pointer position of the ring buffer at the same time.
  • the embodiment of the present application directly reads the collected data packet through the ring buffer through the user mode collection module, there is no copying overhead, no system call overhead, and very low resource consumption.
  • the core acquisition module transfers the acquisition data packet to the user mode acquisition module through the ring buffer, and synchronizes with the user mode acquisition module through the write pointer position and the read pointer position in the ring buffer to reduce system overhead.
  • the smallest granularity of the monitoring object is the virtual switch, which monitors the virtual machine and network performance without intruding the virtual machine to ensure the security and performance of the virtual network.
  • an embodiment of the present application provides another virtual network traffic collection device.
  • the device includes:
  • the initialization unit 41 is used to initialize the flow collection software operating environment of the user mode collection module and the kernel collection module;
  • the channel establishment unit 42 is configured to establish a communication channel, and the communication channel is a data transmission channel between the user state acquisition module and the kernel acquisition module;
  • the cache creation unit 43 is used for the user mode acquisition module to apply for a large page memory, create a ring buffer area on the large page memory, and transmit the physical address of the large page memory to the kernel through the communication channel Acquisition module;
  • the address obtaining unit 44 is configured to receive the physical address of the large page memory by the kernel acquisition module, map the physical address of the large page memory to a linear address in the kernel address space, and obtain the location of the ring buffer The buffer address of the kernel acquisition module;
  • the data storage unit 45 is configured to collect the data packet received by the virtual switch by the kernel collection module, save the data packet to the ring buffer according to the buffer address, and update the writing of the ring buffer at the same time. Pointer position
  • the traffic statistics unit 46 is configured to, if the user mode collection module detects that the ring buffer is updated, it will access the ring buffer and obtain the data packet, count the data traffic flowing through the virtual switch, and update at the same time The position of the read pointer of the ring buffer.
  • the communication channel is a Netlink channel
  • the channel establishment unit 42 includes:
  • a setting subunit 422 configured to set a local address and a destination address of the socket, where the local address is the physical address of the user mode collection module, and the destination address is the physical address of the kernel collection module;
  • connection subunit 423 is configured to establish the communication channel if the first monitoring function monitors that the socket receives a connection request.
  • the communication channel is a pipeline channel
  • the channel establishment unit 42 includes:
  • the creation subunit 421 is further configured to establish the pipeline channel according to the creation function in the user state acquisition module;
  • connection subunit 423 is configured to connect the user state collection module and the core collection module through the pipeline channel if the second monitoring function monitors that the core collection module sends a connection request.
  • the cache creating unit 43 includes:
  • the searching subunit 431 is configured to search for a transmission function corresponding to the communication channel
  • the transmission subunit 432 is configured to transmit the physical address of the large page memory to the kernel collection module according to the transmission function.
  • the address obtaining unit 44 includes:
  • the obtaining subunit 441 is used to obtain the memory allocation relationship between the host machine and the virtual machine;
  • the mapping subunit 442 is configured to map the physical address of the large page memory to the linear address of the kernel address space according to the memory allocation relationship;
  • the calculation subunit 443 is configured to calculate the buffer address of the ring buffer in the kernel acquisition module according to the address allocation relationship of the linear address, the large page memory, and the ring buffer.
  • the data storage unit 45 includes:
  • the judging subunit 451 is configured to judge the type of the virtual switch, and the type includes Linux Bridge and OpenvSwitch;
  • the collection subunit 452 is configured to, if the type of the virtual switch is Linux Bridge, the kernel collection module registers a hook function, and switches to the kernel collection module according to the hook function to collect data received by the virtual switch Bag;
  • the collection subunit 452 is configured to, if the type of the virtual switch is OpenvSwitch, the kernel collection module uses the flow table structure ovsdatapath to execute the collection action action in the management operation flow, and collect the data packets received by the virtual switch .
  • the data storage unit 45 includes:
  • the storage subunit 454 is configured to start from the write pointer position in the buffer address, store the data packet in the ring buffer, and update the write pointer position.
  • traffic statistics unit 46 is configured to:
  • This application provides a virtual network traffic collection device.
  • First initialize the software operating environment for traffic collection, then establish a communication channel between the user mode collection module and the kernel collection module, and then the user mode collection module applies for a large page of memory. Create a ring buffer area on the large page memory, and transmit the physical address of the large page memory to the kernel acquisition module through the communication channel, and then the kernel acquisition module maps the physical address of the large page memory to a linear address in the kernel address space , And obtain the buffer address of the ring buffer in the kernel acquisition module, and then the kernel acquisition module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer of the ring buffer Finally, if the user mode acquisition module detects the ring buffer update, it accesses the ring buffer and obtains data packets, counts the data traffic flowing through the virtual switch, and updates the read pointer position of the ring buffer at the same time.
  • the embodiment of the present application directly reads the collected data packet through the ring buffer through the user mode collection module, there is no copying overhead, no system call overhead, and very low resource consumption.
  • the core acquisition module transfers the acquisition data packet to the user mode acquisition module through the ring buffer, and synchronizes with the user mode acquisition module through the write pointer position and the read pointer position in the ring buffer to reduce system overhead.
  • the smallest granularity of the monitoring object is the virtual switch, which monitors the virtual machine and network performance without intruding the virtual machine to ensure the security and performance of the virtual network.
  • FIG. 5 shows a schematic structural diagram of a traffic collection device for a virtual network according to an embodiment of the present application.
  • the specific embodiment of the present application does not limit the specific implementation of the traffic collection device for the virtual network.
  • the traffic collection device of the virtual network may include a processor 502, a communications interface 504, a memory 506, and a communications bus 508.
  • the processor 502, the communication interface 504, and the memory 506 communicate with each other through the communication bus 508.
  • the communication interface 504 is used to communicate with other devices, such as network elements such as clients or other servers.
  • the processor 502 is configured to execute the program 510, and specifically can execute the relevant steps in the embodiment of the above-mentioned virtual network traffic collection method.
  • the program 510 may include program code, and the program code includes a computer operation instruction.
  • the processor 502 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.
  • the one or more processors included in the computer device may be the same type of processor, such as one or more CPUs, or different types of processors, such as one or more CPUs and one or more ASICs.
  • the memory 506 is used to store the program 510.
  • the memory 506 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), for example, at least one magnetic disk memory.
  • the program 510 may be specifically used to cause the processor 502 to perform the following operations:
  • the communication channel being a data transmission channel between the user mode acquisition module and the kernel acquisition module;
  • the user mode collection module applies for a large page memory, creates a ring buffer area on the large page memory, and transmits the physical address of the large page memory to the kernel collection module through the communication channel;
  • the kernel collection module receives the physical address of the large page memory, maps the physical address of the large page memory to a linear address in the kernel address space, and obtains the buffer of the ring buffer in the kernel collection module address;
  • the kernel collection module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer position of the ring buffer;
  • the user state acquisition module detects that the ring buffer is updated, it accesses the ring buffer and obtains the data packet, counts the data traffic flowing through the virtual switch, and updates the reading of the ring buffer at the same time. Pointer position.
  • This application provides a virtual network traffic collection device.
  • First initialize the software operating environment for traffic collection, then establish a communication channel between the user mode collection module and the kernel collection module, and then the user mode collection module applies for a large page of memory. Create a ring buffer area on the large page memory, and transmit the physical address of the large page memory to the kernel acquisition module through the communication channel, and then the kernel acquisition module maps the physical address of the large page memory to a linear address in the kernel address space , And obtain the buffer address of the ring buffer in the kernel acquisition module, and then the kernel acquisition module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer of the ring buffer Finally, if the user mode acquisition module detects the ring buffer update, it accesses the ring buffer and obtains data packets, counts the data traffic flowing through the virtual switch, and updates the read pointer position of the ring buffer at the same time.
  • the embodiment of the present application directly reads the collected data packet through the ring buffer through the user mode collection module, there is no copying overhead, no system call overhead, and very low resource consumption.
  • the core acquisition module transfers the acquisition data packet to the user mode acquisition module through the ring buffer, and synchronizes with the user mode acquisition module through the write pointer position and the read pointer position in the ring buffer to reduce system overhead.
  • the smallest granularity of the monitoring object is the virtual switch, which monitors the virtual machine and network performance without intruding the virtual machine to ensure the security and performance of the virtual network.
  • a storage medium may be volatile or non-volatile.
  • the storage medium stores at least one executable instruction.
  • the computer executable instruction can execute the foregoing Any method embodiment of the virtual network traffic collection method.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Environmental & Geological Engineering (AREA)
  • Information Transfer Systems (AREA)

Abstract

L'invention concerne un procédé et appareil de collecte de trafic pour un réseau virtuel, ainsi qu'un dispositif informatique et un support de stockage, qui se rapportent au domaine technique de machines virtuelles, et résolvent le problème, rencontré dans l'état antérieur de la technique, des surdébits excessifs de collecte de trafic. Le procédé comporte principalement les étapes consistant à: initialiser un environnement d'exécution de logiciel en vue d'une collecte de trafic; établir un canal de communication; un module de collecte d'état d'utilisateur sollicite une grande mémoire par pages, et crée une zone tampon annulaire dans la grande mémoire par pages; un module de collecte de noyau reçoit une adresse physique de la grande mémoire par pages, transpose l'adresse physique de la grande mémoire par pages en une adresse linéaire d'un espace d'adresses de noyau, et acquiert une adresse de zone tampon, dans le module de collecte de noyau, de la zone tampon annulaire; le module de collecte de noyau collecte un paquet de données reçu par un commutateur virtuel, et sauvegarde le paquet de données dans la zone tampon annulaire selon l'adresse de zone tampon; et si une mise à jour dans la zone tampon annulaire est détectée, le module de collecte d'état d'utilisateur accède à la zone tampon annulaire et acquiert le paquet de données, et compile des statistiques sur le trafic de données circulant à travers le commutateur virtuel. La présente invention est principalement appliquée à un processus de collecte de trafic.
PCT/CN2020/119067 2020-02-18 2020-09-29 Procédé et appareil de collecte de trafic pour réseau virtuel, et dispositif informatique et support de stockage WO2021164262A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010098775.2A CN111431757B (zh) 2020-02-18 2020-02-18 虚拟网络的流量采集方法及装置
CN202010098775.2 2020-02-18

Publications (1)

Publication Number Publication Date
WO2021164262A1 true WO2021164262A1 (fr) 2021-08-26

Family

ID=71551573

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/119067 WO2021164262A1 (fr) 2020-02-18 2020-09-29 Procédé et appareil de collecte de trafic pour réseau virtuel, et dispositif informatique et support de stockage

Country Status (2)

Country Link
CN (1) CN111431757B (fr)
WO (1) WO2021164262A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113794605A (zh) * 2021-09-10 2021-12-14 杭州谐云科技有限公司 一种基于eBPF的内核丢包检测方法、***和装置
CN115033407A (zh) * 2022-08-09 2022-09-09 微栈科技(浙江)有限公司 一种适用于云计算的采集识别流量的***和方法
CN117579386A (zh) * 2024-01-16 2024-02-20 麒麟软件有限公司 网络流量安全管控方法、装置及存储介质

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111431757B (zh) * 2020-02-18 2023-03-24 平安科技（深圳）有限公司 虚拟网络的流量采集方法及装置
CN113132349A (zh) * 2021-03-12 2021-07-16 中国科学院信息工程研究所 一种免代理云平台虚拟流量入侵检测方法及装置
CN113300917B (zh) * 2021-07-27 2021-10-15 苏州浪潮智能科技有限公司 Open Stack租户网络的流量监控方法、装置
CN113986811B (zh) * 2021-09-23 2022-05-10 北京东方通网信科技有限公司 一种高性能内核态网络数据包加速方法
CN116915667B (zh) * 2023-07-19 2024-03-08 上海螣龙科技有限公司 网络流量采集方法、装置、电子设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6351791B1 (en) * 1998-06-25 2002-02-26 International Business Machines Corporation Circuit arrangement and method of maintaining cache coherence utilizing snoop response collection logic that disregards extraneous retry responses
CN107294869A (zh) * 2017-06-22 2017-10-24 郑州云海信息技术有限公司 一种虚拟网卡报文抓取的方法及***
CN108964959A (zh) * 2017-05-27 2018-12-07 阿里巴巴集团控股有限公司 一种用于虚拟化平台的网卡直通***及数据包监管方法
CN108989151A (zh) * 2018-07-20 2018-12-11 北京云杉世纪网络科技有限公司 用于网络或应用性能管理的流量采集方法
CN110380992A (zh) * 2019-07-24 2019-10-25 南京中孚信息技术有限公司 报文处理方法、装置及网络流量采集设备
CN111431757A (zh) * 2020-02-18 2020-07-17 平安科技（深圳）有限公司 虚拟网络的流量采集方法及装置

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6961941B1 (en) * 2001-06-08 2005-11-01 Vmware, Inc. Computer configuration for resource management in systems including a virtual machine
CN100477643C (zh) * 2006-09-22 2009-04-08 中国科学院计算技术研究所 基于共享内存实现的数据包捕获方法
CN102467473B (zh) * 2010-11-03 2015-02-11 Tcl集团股份有限公司 一种在用户空间和内核之间传输数据的方法和装置
CN103034684A (zh) * 2012-11-27 2013-04-10 北京航空航天大学 一种基于内容寻址存储的虚拟机镜像存储优化方法
CN104063267B (zh) * 2014-07-11 2017-11-14 孙强强 一种虚拟机流量监控方法及***
CN108062253A (zh) * 2017-12-11 2018-05-22 北京奇虎科技有限公司 一种内核态与用户态的通信方法、装置及终端
CN109889403A (zh) * 2019-01-30 2019-06-14 北京云端智度科技有限公司 一种基于轮询机制的cdn网络流量检测方法
CN109981403A (zh) * 2019-03-05 2019-07-05 北京勤慕数据科技有限公司 虚拟机网络数据流量监控方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6351791B1 (en) * 1998-06-25 2002-02-26 International Business Machines Corporation Circuit arrangement and method of maintaining cache coherence utilizing snoop response collection logic that disregards extraneous retry responses
CN108964959A (zh) * 2017-05-27 2018-12-07 阿里巴巴集团控股有限公司 一种用于虚拟化平台的网卡直通***及数据包监管方法
CN107294869A (zh) * 2017-06-22 2017-10-24 郑州云海信息技术有限公司 一种虚拟网卡报文抓取的方法及***
CN108989151A (zh) * 2018-07-20 2018-12-11 北京云杉世纪网络科技有限公司 用于网络或应用性能管理的流量采集方法
CN110380992A (zh) * 2019-07-24 2019-10-25 南京中孚信息技术有限公司 报文处理方法、装置及网络流量采集设备
CN111431757A (zh) * 2020-02-18 2020-07-17 平安科技（深圳）有限公司 虚拟网络的流量采集方法及装置

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113794605A (zh) * 2021-09-10 2021-12-14 杭州谐云科技有限公司 一种基于eBPF的内核丢包检测方法、***和装置
CN115033407A (zh) * 2022-08-09 2022-09-09 微栈科技(浙江)有限公司 一种适用于云计算的采集识别流量的***和方法
CN115033407B (zh) * 2022-08-09 2022-11-04 微栈科技(浙江)有限公司 一种适用于云计算的采集识别流量的***和方法
CN117579386A (zh) * 2024-01-16 2024-02-20 麒麟软件有限公司 网络流量安全管控方法、装置及存储介质
CN117579386B (zh) * 2024-01-16 2024-04-12 麒麟软件有限公司 网络流量安全管控方法、装置及存储介质

Also Published As

Publication number Publication date
CN111431757B (zh) 2023-03-24
CN111431757A (zh) 2020-07-17

Similar Documents

Publication Publication Date Title
WO2021164262A1 (fr) Procédé et appareil de collecte de trafic pour réseau virtuel, et dispositif informatique et support de stockage
US11500689B2 (en) Communication method and apparatus
Cheriton et al. The distributed V kernel and its performance for diskless workstations
Welch et al. Pseudo devices: User-level extensions to the Sprite file system
US11249938B2 (en) Programmed input/output mode
US8234407B2 (en) Network use of virtual addresses without pinning or registration
Takahashi et al. PM2: High performance communication middleware for heterogeneous network environments
US7089289B1 (en) Mechanisms for efficient message passing with copy avoidance in a distributed system using advanced network devices
US7996569B2 (en) Method and system for zero copy in a virtualized network environment
US6799200B1 (en) Mechanisms for efficient message passing with copy avoidance in a distributed system
EP0889622B1 (fr) Appareil et procédé pour l'allocation et la gestion à distance de mémoire tampon pour la communication de messages entre des noeuds du réseau
US20030145230A1 (en) System for exchanging data utilizing remote direct memory access
Anderson et al. Cheating the I/O Bottleneck: Network Storage with Trapeze/Myrinet.
JPH1185710A (ja) サーバ装置およびファイル管理方法
CN114756388B (zh) 一种基于rdma的集群***节点间按需共享内存的方法
CA2241994A1 (fr) Systeme et methode d'execution efficace d'operations d'entree-sortie sur un disque eloigne
JPH09231157A (ja) コンピュータに接続されている入力／出力（ｉ／ｏ）デバイスを制御する方法
JP2004520646A (ja) 周辺デバイスからホスト・コンピュータ・システムに割込みを転送する方法および装置
CN109857545B (zh) 一种数据传输方法及装置
CN102375789A (zh) 一种通用网卡非缓存的零拷贝方法及零拷贝***
Salehi et al. The effectiveness of affinity-based scheduling in multiprocessor networking
Li et al. A high-efficient inter-domain data transferring system for virtual machines
US20240211392A1 (en) Buffer allocation
CN118093225A (zh) 订阅消息的传输方法及装置、存储介质及电子设备
Welch et al. to the Sprite File System

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20919864

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20919864

Country of ref document: EP

Kind code of ref document: A1