WO2021164262A1 - 虚拟网络的流量采集方法、装置、计算机设备及存储介质 - Google Patents

虚拟网络的流量采集方法、装置、计算机设备及存储介质 Download PDF

Info

Publication number
WO2021164262A1
WO2021164262A1 PCT/CN2020/119067 CN2020119067W WO2021164262A1 WO 2021164262 A1 WO2021164262 A1 WO 2021164262A1 CN 2020119067 W CN2020119067 W CN 2020119067W WO 2021164262 A1 WO2021164262 A1 WO 2021164262A1
Authority
WO
WIPO (PCT)
Prior art keywords
address
kernel
ring buffer
collection module
page memory
Prior art date
Application number
PCT/CN2020/119067
Other languages
English (en)
French (fr)
Inventor
张齐国
王继海
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Publication of WO2021164262A1 publication Critical patent/WO2021164262A1/zh

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0876—Network utilisation, e.g. volume of load or congestion level
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/50—Testing arrangements
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate

Definitions

  • This application relates to the technical field of virtual machines, and in particular to a method, device, computer equipment and storage medium for traffic collection of a virtual network.
  • the preferred user mode collection unit uses the system to call the AF_PACKET socket interface to monitor the target virtual network card in the shared memory, and then the kernel collection unit puts a newly added network packet on the received network packet through mmap. The network packet is mapped to the shared memory, and finally the user mode unit detects that there is a new network packet in the shared memory, and then collects the new network packet to complete the flow collection.
  • the user mode unit calls the AF_PACKET interface for each net packet, the system call overhead is high, the calling frequency is high, and the resource consumption is high.
  • the memory mmap of each net packet to the shared memory is expensive and the memory is fragmented. Difficult to manage. Due to the excessive network traffic between virtual machines, the overhead of collecting traffic is too large.
  • a method for collecting traffic in a virtual network including:
  • the communication channel being a data transmission channel between the user mode acquisition module and the kernel acquisition module;
  • the user mode collection module applies for a large page memory, creates a ring buffer area on the large page memory, and transmits the physical address of the large page memory to the kernel collection module through the communication channel;
  • the kernel collection module receives the physical address of the large page memory, maps the physical address of the large page memory to a linear address in the kernel address space, and obtains the buffer of the ring buffer in the kernel collection module address;
  • the kernel collection module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer position of the ring buffer;
  • the user state acquisition module detects that the ring buffer is updated, it accesses the ring buffer and obtains the data packet, counts the data traffic flowing through the virtual switch, and updates the reading of the ring buffer at the same time. Pointer position.
  • a virtual network traffic collection device including:
  • the initialization unit is used to initialize the flow collection software operating environment of the user mode collection module and the core collection module;
  • a channel establishing unit configured to establish a communication channel, the communication channel being a data transmission channel between the user mode acquisition module and the kernel acquisition module;
  • the cache creation unit is used for the user mode collection module to apply for a large page memory, create a ring buffer area on the large page memory, and transmit the physical address of the large page memory to the kernel collection via the communication channel Module
  • the address acquisition unit is configured to receive the physical address of the large page memory by the kernel acquisition module, map the physical address of the large page memory to a linear address in the kernel address space, and obtain the ring buffer in the The buffer address of the kernel acquisition module;
  • the data storage unit is used for the kernel collection module to collect the data packet received by the virtual switch, and save the data packet to the ring buffer according to the buffer address, and at the same time update the write pointer of the ring buffer Location;
  • the traffic statistics unit is configured to, if the user state acquisition module detects that the ring buffer is updated, it will access the ring buffer and obtain the data packets, count the data traffic flowing through the virtual switch, and update all the data at the same time. The position of the read pointer of the ring buffer.
  • a storage medium is provided, and at least one executable instruction is stored in the storage medium, and the executable instruction causes a processor to perform the following steps:
  • the communication channel being a data transmission channel between the user mode acquisition module and the kernel acquisition module;
  • the user mode collection module applies for a large page memory, creates a ring buffer area on the large page memory, and transmits the physical address of the large page memory to the kernel collection module through the communication channel;
  • the kernel collection module receives the physical address of the large page memory, maps the physical address of the large page memory to a linear address in the kernel address space, and obtains the buffer of the ring buffer in the kernel collection module address;
  • the kernel collection module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer position of the ring buffer;
  • the user state acquisition module detects that the ring buffer is updated, it accesses the ring buffer and obtains the data packet, counts the data traffic flowing through the virtual switch, and updates the reading of the ring buffer at the same time. Pointer position.
  • a computer device including:
  • a processor a memory, a communication interface, and a communication bus.
  • the processor, the memory, and the communication interface communicate with each other through the communication bus;
  • the memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform the following steps:
  • the communication channel being a data transmission channel between the user mode acquisition module and the kernel acquisition module;
  • the user mode collection module applies for a large page memory, creates a ring buffer area on the large page memory, and transmits the physical address of the large page memory to the kernel collection module through the communication channel;
  • the kernel collection module receives the physical address of the large page memory, maps the physical address of the large page memory to a linear address in the kernel address space, and obtains the buffer of the ring buffer in the kernel collection module address;
  • the kernel collection module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer position of the ring buffer;
  • the user state acquisition module detects that the ring buffer is updated, it accesses the ring buffer and obtains the data packet, counts the data traffic flowing through the virtual switch, and updates the reading of the ring buffer at the same time. Pointer position.
  • FIG. 1 shows a flow chart of a method for collecting traffic on a virtual network according to an embodiment of the present application
  • FIG. 2 shows a flowchart of another method for collecting traffic on a virtual network according to an embodiment of the present application
  • FIG. 3 shows a block diagram of the composition of a virtual network traffic collection device provided by an embodiment of the present application
  • Figure 4 shows a block diagram of another virtual network traffic collection device provided by an embodiment of the present application.
  • Fig. 5 is a schematic structural diagram of a computer device provided by an embodiment of the present application.
  • This application is the traffic collection environment of the embodiment, which can be between virtual machines on the same host machine, or between virtual machines that do not use host machines.
  • the embodiment of the application is a software method running on a host computer, which mainly includes a user mode acquisition module, a ring buffer, and a kernel acquisition module.
  • the kernel collection module is mainly used to collect the data packets received by the virtual switch and save the data packets to the ring buffer.
  • the user mode collection module is mainly used to obtain the data packets in the ring buffer to count the traffic flowing through the virtual switch.
  • the embodiment of the present application provides a method for collecting traffic on a virtual network. As shown in FIG. 1, the method includes:
  • Initializing the software operating environment refers to the software environment, function library, and loading operating environment used to load the user mode acquisition module and the kernel acquisition module.
  • the communication channel is a path for data transmission.
  • the communication channel is a kind of logical channel, which specifically refers to a logical path implemented by an intermediate node between the parties sending and receiving signals on the basis of a physical channel.
  • the communication channel is a data transmission channel between the user state acquisition module and the core acquisition module. Establish a communication channel, that is, establish a logical path between the user mode acquisition module and the core acquisition module.
  • the user mode collection module applies for a large page memory, creates a ring buffer area on the large page memory, and transmits the physical address of the large page memory to the kernel collection module through the communication channel.
  • the use of large page memory reduces the address translation information, thereby reducing the use of the CPU cache, reducing the pressure on the CPU cache, and allowing the CPU cache to use more Data caching for applications can also improve the performance of the system as a whole.
  • the use of large page memory in the host machine of a memory-intensive virtual machine can significantly improve the performance of the host machine.
  • Large page memory helps Linux system to manage virtual memory, and can define a page size of 1GB, which can reserve part of the memory for applications.
  • the reserved large page memory will always be reserved and will not be swapped out of the memory unless the memory configuration is modified.
  • the large page memory requested by the user state acquisition module that is, the large page memory belongs to the user state acquisition module.
  • the ring buffer is a data structure used to represent a fixed-size, head-to-tail vector buffer, suitable for buffering data streams, and used to store data sent and received in communications.
  • the ring buffer is a first-in, first-out circular buffer area that can provide mutually exclusive access to the buffer for communication programs.
  • the physical address of the large page memory requested by the user mode acquisition module is transmitted to the kernel acquisition module through the communication channel, which is equivalent to that the kernel acquisition module can receive the physical address of the large page memory and can calculate the physical address of the ring buffer at the same time.
  • the kernel collection module receives the physical address of the large page memory, maps the physical address of the large page memory to a linear address in the kernel address space, and obtains the buffer of the ring buffer in the kernel collection module address.
  • the kernel acquisition module receives the physical address of the large page memory, and maps the large page memory to a linear address and the ring buffer to a buffer address according to the physical address of the large page memory, the physical address of the ring buffer, and the kernel address space. .
  • the kernel collection module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer position of the ring buffer.
  • the sampling ratio can be 1:1 or 1:n.
  • the sampling ratio is determined according to the subsequent traffic collection function.
  • the sampling ratio is not limited in the embodiment of this application.
  • the kernel collection module collects the data packets received by the virtual switch and saves them to the ring buffer. Each data packet corresponds to a collection record and occupies a fixed-size space in the ring buffer. In the process of collecting data packets, the header of the data packet can be collected, or the entire data packet can be collected.
  • the user mode collection module If the user mode collection module detects that the ring buffer is updated, it accesses the ring buffer and obtains the data packet, counts the data traffic flowing through the virtual switch, and updates the reading of the ring buffer at the same time. Pointer position.
  • the user mode acquisition module After the user mode acquisition module obtains the data packets in the ring buffer, it can count the data traffic, count the number of data packets, and can also perform detailed analysis on the data traffic. Monitor virtual networks based on data traffic to improve network performance, improve network transmission capacity, analyze network channel utilization, and more.
  • This application provides a virtual network traffic collection method.
  • initialize the software operating environment for traffic collection then establish a communication channel between the user mode collection module and the kernel collection module, and then the user mode collection module applies for a large page of memory.
  • Create a ring buffer area on the large page memory and transmit the physical address of the large page memory to the kernel acquisition module through the communication channel, and then the kernel acquisition module maps the physical address of the large page memory to a linear address in the kernel address space , And obtain the buffer address of the ring buffer in the kernel acquisition module, and then the kernel acquisition module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer of the ring buffer Finally, if the user mode acquisition module detects the ring buffer update, it accesses the ring buffer and obtains data packets, counts the data traffic flowing through the virtual switch, and updates the read pointer position of the ring buffer at the same time.
  • the embodiment of the present application directly reads the collected data packet through the ring buffer through the user mode collection module, there is no copying overhead, no system call overhead, and very low resource consumption.
  • the core acquisition module transfers the acquisition data packet to the user mode acquisition module through the ring buffer, and synchronizes with the user mode acquisition module through the write pointer position and the read pointer position in the ring buffer to reduce system overhead.
  • the smallest granularity of the monitoring object is the virtual switch, which monitors the virtual machine and network performance without intruding the virtual machine to ensure the security and performance of the virtual network.
  • the embodiment of the present application provides another method for collecting traffic on a virtual network. As shown in FIG. 2, the method includes:
  • Initializing the software operating environment refers to the software environment, function library, and loading operating environment used to load the user mode acquisition module and the kernel acquisition module.
  • the communication channel is a data transmission channel between the user state acquisition module and the core acquisition module. Establish a communication channel, that is, establish a logical path between the user mode acquisition module and the core acquisition module.
  • the communication channel can be a Netlink channel, a pipe channel, or a Socket channel. Among them, the Netlink channel is a special Socket channel, and the establishment of the two channels is similar.
  • establishing the communication channel includes: creating a socket according to the socket creation function; setting the local address and destination address of the socket, and the local address is the user mode acquisition module The destination address is the physical address of the core collection module; if the first monitoring function detects that the socket receives a connection request, the communication channel is established.
  • Netlink socket is a special inter-process communication (IPC) used to realize the communication between the user process and the kernel process, and it is also the most commonly used interface for communication between network applications and the kernel. It is easily applied to netlink channels through socket(), bind(), connect(), listen(), sendmsg(), recvmsg() and close(). netlink is included in the header file linux/netlink.h.
  • the socket() is used to create a socket.
  • bind() is used to specify the local address. After a socket is created with socket(), it is not actually associated with any specific local or destination address. Once a socket is created, the bind() system call must be used to establish a local address for the socket.
  • connect() is used to connect the socket to the destination address.
  • listen() is used to set the status of waiting for connection. For a server program, after applying for a socket and calling bind() to bind to the local address, it should wait for a client program to request a connection. listen() is a function that sets a socket to this state. sendmsg() and recvmsg() are used to send and receive data. In the data stream mode, after a connection is established, or in the datagram mode, after calling connect() to bind the socket and the destination address, you can call sendmsg() and recvmsg() functions for data transmission . closesocket() is used to close the socket.
  • establishing the communication channel includes: in the user mode acquisition module, establishing the pipeline channel according to the creation function; if the second listening function listens to the core acquisition module to send a connection request, then through all The pipeline channel connects the user state acquisition module and the core acquisition module.
  • the pipeline channel When the pipeline channel is created, a page size is allocated for the buffer.
  • the pipeline transmits an unformatted byte stream. This requires that the read and write parties of the pipeline must agree on the format of the data in advance.
  • the creation process uses the specific functions of the pipeline channel to create, such as the creation function CreateNamedPipe(), the monitoring function ConnectNamedPipe(), the data transfer functions ReadFile() and WriteFile(), and the closing function CloseHandle().
  • the user mode collection module applies for a large page memory, creates a ring buffer area on the large page memory, and transmits the physical address of the large page memory to the kernel collection module through the communication channel.
  • large page memory can be allocated in local memory for each NUMA, and a separate ring buffer can be created for each CPU core.
  • the kernel maintains a table that maps virtual memory addresses to physical addresses. For each page operation, the kernel needs to load related mappings. If the memory page is small, there will be a lot of pages that need to be loaded, causing the kernel to load more mapping tables, which will reduce performance. With “large memory pages”, fewer pages are needed. Thereby greatly reducing the number of mapping tables loaded by the kernel, improving the performance of the kernel level and ultimately benefiting the performance of the application. In short, by enabling "large memory pages", the system only needs to process fewer page mapping tables, thereby reducing the overhead of accessing/maintaining them.
  • sendmsg() in Netlink sockets are used to send physical addresses.
  • writeFile in pipe communication
  • send() and sendto() in sockets are used to send physical addresses.
  • the kernel collection module receives the physical address of the large page memory, and obtains the memory allocation relationship between the host machine and the virtual machine.
  • the response mechanism can be set to ensure the effective transmission of the physical address. For example, after the kernel collection module receives the physical address of the large page memory, it needs to feed back the received message to the user mode collection module.
  • the received message can carry the data header information of the physical address, and the user mode module receives the feedback message from the kernel collection module. Then you can correct whether the received information is correct.
  • the kernel acquisition module maps the physical address of the large page memory to the linear address of the kernel address space according to the memory allocation relationship.
  • the kernel collection module calculates the buffer address of the ring buffer in the kernel collection module according to the address allocation relationship of the linear address, the large page memory, and the ring buffer.
  • the kernel acquisition module maps the large page memory into a linear address and the ring buffer into a buffer address according to the physical address of the large page memory, the physical address of the ring buffer, and the kernel address space.
  • the kernel collection module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer position of the ring buffer.
  • the sampling ratio can be 1:1 or 1:n.
  • the sampling ratio is determined according to the subsequent traffic collection function.
  • the sampling ratio is not limited in the embodiment of this application. .
  • the kernel collection module collects the data packets received by the virtual switch and saves them to the ring buffer. Each data packet corresponds to a collection record and occupies a fixed-size space in the ring buffer. In the process of collecting data packets, the header of the data packet can be collected, or the entire data packet can be collected.
  • the kernel collection module collects data packets received by the virtual switch, which specifically includes: judging the type of the virtual switch, the types include Linux Bridge and OpenvSwitch; if the virtual switch is Type is Linux Bridge, the kernel collection module registers the hook function, and switches into the kernel collection module according to the hook function to collect data packets received by the virtual switch; if the type of the virtual switch is OpenvSwitch, the kernel The collection module uses the flow table structure ovsdatapath to execute the collection action action in the management operation flow, and collect the data packets received by the virtual switch.
  • the write pointer refers to the location of the data to be written in the ring buffer.
  • the user mode collection module If the user mode collection module detects that the ring buffer is updated, it accesses the ring buffer and obtains the data packet, counts the data traffic flowing through the virtual switch, and updates the reading of the ring buffer at the same time. Pointer position.
  • Obtaining the data packet specifically includes: obtaining the current write pointer position, and storing the current write pointer position; if the current write pointer position is different from the previously stored write pointer position, accessing and Acquire a data packet in the ring buffer, and the data packet is stored between the read pointer position of the ring buffer and the current write pointer position.
  • the position of the write pointer in the ring buffer By detecting the position of the write pointer in the ring buffer, it is monitored whether there is an update packet in the ring buffer.
  • the position of the read pointer indicates the position of the data packet in the ring buffer that has not yet been counted.
  • This application provides a virtual network traffic collection method.
  • initialize the software operating environment for traffic collection then establish a communication channel between the user mode collection module and the kernel collection module, and then the user mode collection module applies for a large page of memory.
  • Create a ring buffer area on the large page memory and transmit the physical address of the large page memory to the kernel acquisition module through the communication channel, and then the kernel acquisition module maps the physical address of the large page memory to a linear address in the kernel address space , And obtain the buffer address of the ring buffer in the kernel acquisition module, and then the kernel acquisition module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer of the ring buffer Finally, if the user mode acquisition module detects the ring buffer update, it accesses the ring buffer and obtains data packets, counts the data traffic flowing through the virtual switch, and updates the read pointer position of the ring buffer at the same time.
  • the embodiment of the present application directly reads the collected data packet through the ring buffer through the user mode collection module, there is no copying overhead, no system call overhead, and very low resource consumption.
  • the core acquisition module transfers the acquisition data packet to the user mode acquisition module through the ring buffer, and synchronizes with the user mode acquisition module through the write pointer position and the read pointer position in the ring buffer to reduce system overhead.
  • the smallest granularity of the monitoring object is the virtual switch, which monitors the virtual machine and network performance without intruding the virtual machine to ensure the security and performance of the virtual network.
  • an embodiment of the present application provides a virtual network traffic collection device.
  • the device includes:
  • the initialization unit 31 is used to initialize the flow collection software operating environment of the user mode collection module and the kernel collection module;
  • the channel establishment unit 32 is configured to establish a communication channel, the communication channel being a data transmission channel between the user state acquisition module and the core acquisition module;
  • the cache creation unit 33 is used for the user mode acquisition module to apply for a large page memory, create a ring buffer area on the large page memory, and transmit the physical address of the large page memory to the kernel through the communication channel Acquisition module;
  • the address acquiring unit 34 is configured to receive the physical address of the large page memory by the kernel acquisition module, map the physical address of the large page memory to a linear address in the kernel address space, and acquire the location of the ring buffer The buffer address of the kernel acquisition module;
  • the data storage unit 35 is configured to collect the data packet received by the virtual switch by the kernel collection module, save the data packet to the ring buffer according to the buffer address, and update the write of the ring buffer at the same time. Pointer position
  • the traffic statistics unit 36 is configured to access the ring buffer and obtain the data packet if the user mode collection module detects that the ring buffer is updated, and then count the data traffic flowing through the virtual switch, and update at the same time The position of the read pointer of the ring buffer.
  • This application provides a virtual network traffic collection device.
  • First initialize the software operating environment for traffic collection, then establish a communication channel between the user mode collection module and the kernel collection module, and then the user mode collection module applies for a large page of memory. Create a ring buffer area on the large page memory, and transmit the physical address of the large page memory to the kernel acquisition module through the communication channel, and then the kernel acquisition module maps the physical address of the large page memory to a linear address in the kernel address space , And obtain the buffer address of the ring buffer in the kernel acquisition module, and then the kernel acquisition module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer of the ring buffer Finally, if the user mode acquisition module detects the ring buffer update, it accesses the ring buffer and obtains data packets, counts the data traffic flowing through the virtual switch, and updates the read pointer position of the ring buffer at the same time.
  • the embodiment of the present application directly reads the collected data packet through the ring buffer through the user mode collection module, there is no copying overhead, no system call overhead, and very low resource consumption.
  • the core acquisition module transfers the acquisition data packet to the user mode acquisition module through the ring buffer, and synchronizes with the user mode acquisition module through the write pointer position and the read pointer position in the ring buffer to reduce system overhead.
  • the smallest granularity of the monitoring object is the virtual switch, which monitors the virtual machine and network performance without intruding the virtual machine to ensure the security and performance of the virtual network.
  • an embodiment of the present application provides another virtual network traffic collection device.
  • the device includes:
  • the initialization unit 41 is used to initialize the flow collection software operating environment of the user mode collection module and the kernel collection module;
  • the channel establishment unit 42 is configured to establish a communication channel, and the communication channel is a data transmission channel between the user state acquisition module and the kernel acquisition module;
  • the cache creation unit 43 is used for the user mode acquisition module to apply for a large page memory, create a ring buffer area on the large page memory, and transmit the physical address of the large page memory to the kernel through the communication channel Acquisition module;
  • the address obtaining unit 44 is configured to receive the physical address of the large page memory by the kernel acquisition module, map the physical address of the large page memory to a linear address in the kernel address space, and obtain the location of the ring buffer The buffer address of the kernel acquisition module;
  • the data storage unit 45 is configured to collect the data packet received by the virtual switch by the kernel collection module, save the data packet to the ring buffer according to the buffer address, and update the writing of the ring buffer at the same time. Pointer position
  • the traffic statistics unit 46 is configured to, if the user mode collection module detects that the ring buffer is updated, it will access the ring buffer and obtain the data packet, count the data traffic flowing through the virtual switch, and update at the same time The position of the read pointer of the ring buffer.
  • the communication channel is a Netlink channel
  • the channel establishment unit 42 includes:
  • a setting subunit 422 configured to set a local address and a destination address of the socket, where the local address is the physical address of the user mode collection module, and the destination address is the physical address of the kernel collection module;
  • connection subunit 423 is configured to establish the communication channel if the first monitoring function monitors that the socket receives a connection request.
  • the communication channel is a pipeline channel
  • the channel establishment unit 42 includes:
  • the creation subunit 421 is further configured to establish the pipeline channel according to the creation function in the user state acquisition module;
  • connection subunit 423 is configured to connect the user state collection module and the core collection module through the pipeline channel if the second monitoring function monitors that the core collection module sends a connection request.
  • the cache creating unit 43 includes:
  • the searching subunit 431 is configured to search for a transmission function corresponding to the communication channel
  • the transmission subunit 432 is configured to transmit the physical address of the large page memory to the kernel collection module according to the transmission function.
  • the address obtaining unit 44 includes:
  • the obtaining subunit 441 is used to obtain the memory allocation relationship between the host machine and the virtual machine;
  • the mapping subunit 442 is configured to map the physical address of the large page memory to the linear address of the kernel address space according to the memory allocation relationship;
  • the calculation subunit 443 is configured to calculate the buffer address of the ring buffer in the kernel acquisition module according to the address allocation relationship of the linear address, the large page memory, and the ring buffer.
  • the data storage unit 45 includes:
  • the judging subunit 451 is configured to judge the type of the virtual switch, and the type includes Linux Bridge and OpenvSwitch;
  • the collection subunit 452 is configured to, if the type of the virtual switch is Linux Bridge, the kernel collection module registers a hook function, and switches to the kernel collection module according to the hook function to collect data received by the virtual switch Bag;
  • the collection subunit 452 is configured to, if the type of the virtual switch is OpenvSwitch, the kernel collection module uses the flow table structure ovsdatapath to execute the collection action action in the management operation flow, and collect the data packets received by the virtual switch .
  • the data storage unit 45 includes:
  • the storage subunit 454 is configured to start from the write pointer position in the buffer address, store the data packet in the ring buffer, and update the write pointer position.
  • traffic statistics unit 46 is configured to:
  • This application provides a virtual network traffic collection device.
  • First initialize the software operating environment for traffic collection, then establish a communication channel between the user mode collection module and the kernel collection module, and then the user mode collection module applies for a large page of memory. Create a ring buffer area on the large page memory, and transmit the physical address of the large page memory to the kernel acquisition module through the communication channel, and then the kernel acquisition module maps the physical address of the large page memory to a linear address in the kernel address space , And obtain the buffer address of the ring buffer in the kernel acquisition module, and then the kernel acquisition module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer of the ring buffer Finally, if the user mode acquisition module detects the ring buffer update, it accesses the ring buffer and obtains data packets, counts the data traffic flowing through the virtual switch, and updates the read pointer position of the ring buffer at the same time.
  • the embodiment of the present application directly reads the collected data packet through the ring buffer through the user mode collection module, there is no copying overhead, no system call overhead, and very low resource consumption.
  • the core acquisition module transfers the acquisition data packet to the user mode acquisition module through the ring buffer, and synchronizes with the user mode acquisition module through the write pointer position and the read pointer position in the ring buffer to reduce system overhead.
  • the smallest granularity of the monitoring object is the virtual switch, which monitors the virtual machine and network performance without intruding the virtual machine to ensure the security and performance of the virtual network.
  • FIG. 5 shows a schematic structural diagram of a traffic collection device for a virtual network according to an embodiment of the present application.
  • the specific embodiment of the present application does not limit the specific implementation of the traffic collection device for the virtual network.
  • the traffic collection device of the virtual network may include a processor 502, a communications interface 504, a memory 506, and a communications bus 508.
  • the processor 502, the communication interface 504, and the memory 506 communicate with each other through the communication bus 508.
  • the communication interface 504 is used to communicate with other devices, such as network elements such as clients or other servers.
  • the processor 502 is configured to execute the program 510, and specifically can execute the relevant steps in the embodiment of the above-mentioned virtual network traffic collection method.
  • the program 510 may include program code, and the program code includes a computer operation instruction.
  • the processor 502 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.
  • the one or more processors included in the computer device may be the same type of processor, such as one or more CPUs, or different types of processors, such as one or more CPUs and one or more ASICs.
  • the memory 506 is used to store the program 510.
  • the memory 506 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), for example, at least one magnetic disk memory.
  • the program 510 may be specifically used to cause the processor 502 to perform the following operations:
  • the communication channel being a data transmission channel between the user mode acquisition module and the kernel acquisition module;
  • the user mode collection module applies for a large page memory, creates a ring buffer area on the large page memory, and transmits the physical address of the large page memory to the kernel collection module through the communication channel;
  • the kernel collection module receives the physical address of the large page memory, maps the physical address of the large page memory to a linear address in the kernel address space, and obtains the buffer of the ring buffer in the kernel collection module address;
  • the kernel collection module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer position of the ring buffer;
  • the user state acquisition module detects that the ring buffer is updated, it accesses the ring buffer and obtains the data packet, counts the data traffic flowing through the virtual switch, and updates the reading of the ring buffer at the same time. Pointer position.
  • This application provides a virtual network traffic collection device.
  • First initialize the software operating environment for traffic collection, then establish a communication channel between the user mode collection module and the kernel collection module, and then the user mode collection module applies for a large page of memory. Create a ring buffer area on the large page memory, and transmit the physical address of the large page memory to the kernel acquisition module through the communication channel, and then the kernel acquisition module maps the physical address of the large page memory to a linear address in the kernel address space , And obtain the buffer address of the ring buffer in the kernel acquisition module, and then the kernel acquisition module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer of the ring buffer Finally, if the user mode acquisition module detects the ring buffer update, it accesses the ring buffer and obtains data packets, counts the data traffic flowing through the virtual switch, and updates the read pointer position of the ring buffer at the same time.
  • the embodiment of the present application directly reads the collected data packet through the ring buffer through the user mode collection module, there is no copying overhead, no system call overhead, and very low resource consumption.
  • the core acquisition module transfers the acquisition data packet to the user mode acquisition module through the ring buffer, and synchronizes with the user mode acquisition module through the write pointer position and the read pointer position in the ring buffer to reduce system overhead.
  • the smallest granularity of the monitoring object is the virtual switch, which monitors the virtual machine and network performance without intruding the virtual machine to ensure the security and performance of the virtual network.
  • a storage medium may be volatile or non-volatile.
  • the storage medium stores at least one executable instruction.
  • the computer executable instruction can execute the foregoing Any method embodiment of the virtual network traffic collection method.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Environmental & Geological Engineering (AREA)
  • Information Transfer Systems (AREA)

Abstract

本申请公开了一种虚拟网络的流量采集方法、装置、计算机设备及存储介质，涉及虚拟机技术领域，为解决现有技术中采集流量开销过大的问题。该方法主要包括：初始化流量采集的软件运行环境；建立通信通道；用户态采集模块申请大页内存，在大页内存上创建环形缓存区；内核采集模块接收大页内存的物理地址，并将大页内存的物理地址映射成内核地址空间的线性地址，并获取环形缓冲区在内核采集模块的缓冲区地址；内核采集模块采集虚拟交换机接收到的数据包，并根据缓冲区地址将数据包保存至环形缓冲区；用户态采集模块如果监测到环形缓冲区更新，则访问环形缓冲区并获取数据包，统计流经虚拟交换机的数据流量。本申请主要应用于流量采集的过程中。

Description

虚拟网络的流量采集方法、装置、计算机设备及存储介质
本申请要求于2020年2月18日提交中国专利局、申请号为CN202010098775.2、名称为“虚拟网络的流量采集方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及一种虚拟机技术领域，特别是涉及一种虚拟网络的流量采集方法、装置、计算机设备及存储介质。
 背景技术
对数据中心网络性能管理和网络优化，需要对流量进行采集分析，把网络流量从不可见变为可见。随着虚拟化和云应用越来越广，更多的网络流量发生在同一主机上的虚拟机之间，甚至虚拟机之间的网络流量已占据到数据中心总流量的80%。
现有技术中，首选用户态采集单元通过***调用AF_PACKET套接字接口，在共享内存中对目标虚拟网卡进行网包监控，然后内核采集单元对收到的网包通过mmap放个把新增的网包映射到共享内存中，最后用户态单元检测到共享内存中存在新增的网包，则采集新增的网包，完成流量采集。
 技术问题
发明人意识到上述现有技术，用户态单元对每个网包调用AF_PACKET接口，***调用开销大，调用频率高，消耗资源多，每个网包的内存mmap到共享内存，开销大且内存零散难于管理。由于虚拟机之间的网络流传流量过多，导致采集流量时开销过大。
 技术解决方案
依据本申请一个方面，提供了一种虚拟网络的流量采集方法，包括：
初始化用户态采集模块和内核采集模块的流量采集的软件运行环境；
建立通信通道，所述通信通道是所述用户态采集模块和所述内核采集模块之间的数据传输通道；
所述用户态采集模块申请大页内存，在所述大页内存上创建环形缓存区，并将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块；
所述内核采集模块接收所述大页内存的物理地址，并将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址；
所述内核采集模块采集虚拟交换机接收到的数据包，并根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置；
所述用户态采集模块如果监测到所述环形缓冲区更新，则访问所述环形缓冲区并获取所述数据包，统计流经所述虚拟交换机的数据流量，同时更新所述环形缓冲区的读指针位置。
依据本申请另一个方面，提供了一种虚拟网络的流量采集装置，包括：
初始化单元，用于初始化用户态采集模块和内核采集模块的流量采集的软件运行环境；
通道建立单元，用于建立通信通道，所述通信通道是所述用户态采集模块和所述内核采集模块之间的数据传输通道；
缓存创建单元，用于所述用户态采集模块申请大页内存，在所述大页内存上创建环形缓存区，并将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块；
地址获取单元，用于所述内核采集模块接收所述大页内存的物理地址，并将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址；
数据存储单元，用于所述内核采集模块采集虚拟交换机接收到的数据包，并根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置；
流量统计单元，用于所述用户态采集模块如果监测到所述环形缓冲区更新，则访问所述环形缓冲区并获取所述数据包，统计流经所述虚拟交换机的数据流量，同时更新所述环形缓冲区的读指针位置。
根据本申请的又一方面，提供了一种存储介质，所述存储介质中存储有至少一可执行指令，所述可执行指令使处理器执行如下步骤：
初始化用户态采集模块和内核采集模块的流量采集的软件运行环境；
建立通信通道，所述通信通道是所述用户态采集模块和所述内核采集模块之间的数据传输通道；
所述用户态采集模块申请大页内存，在所述大页内存上创建环形缓存区，并将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块；
所述内核采集模块接收所述大页内存的物理地址，并将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址；
所述内核采集模块采集虚拟交换机接收到的数据包，并根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置；
所述用户态采集模块如果监测到所述环形缓冲区更新，则访问所述环形缓冲区并获取所述数据包，统计流经所述虚拟交换机的数据流量，同时更新所述环形缓冲区的读指针位置。
依据本申请又一个方面，提供了一种计算机设备，包括：
处理器、存储器、通信接口和通信总线，所述处理器、所述存储器和所述通信接口通过所述通信总线完成相互间的通信；
所述存储器用于存放至少一可执行指令，所述可执行指令使所述处理器执行如下步骤：
初始化用户态采集模块和内核采集模块的流量采集的软件运行环境；
建立通信通道，所述通信通道是所述用户态采集模块和所述内核采集模块之间的数据传输通道；
所述用户态采集模块申请大页内存，在所述大页内存上创建环形缓存区，并将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块；
所述内核采集模块接收所述大页内存的物理地址，并将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址；
所述内核采集模块采集虚拟交换机接收到的数据包，并根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置；
所述用户态采集模块如果监测到所述环形缓冲区更新，则访问所述环形缓冲区并获取所述数据包，统计流经所述虚拟交换机的数据流量，同时更新所述环形缓冲区的读指针位置。
 附图说明
通过阅读下文优选实施方式的详细描述，各种其他的优点和益处对于本领域普通技术人员将变得清楚明了。附图仅用于示出优选实施方式的目的，而并不认为是对本申请的限制。而且在整个附图中，用相同的参考符号表示相同的部件。在附图中：
图1示出了本申请实施例提供的一种虚拟网络的流量采集方法流程图；
图2示出了本申请实施例提供的另一种虚拟网络的流量采集方法流程图；
图3示出了本申请实施例提供的一种虚拟网络的流量采集装置组成框图；
图4示出了本申请实施例提供的另一种虚拟网络的流量采集装置组成框图；
图5本申请实施例提供的一种计算机设备的结构示意图。
 本发明的实施方式
本申请是实施例的流量采集环境，可以为同一宿主机上的虚拟机之间，也可以为不用宿主机上的虚拟机之间。本申请实施例是一种运行与宿主机上的软件方法，主要包括用户态采集模块、环形缓冲区和内核采集模块。内核采集模块主要用于采集虚拟交换机接收到的数据包，并将数据包保存至环形缓冲区，用户态采集模块主要用于获取环形缓冲区中的数据包，以统计流经虚拟交换机的流量。本申请实施例提供了一种虚拟网络的流量采集方法，如图1所示，该方法包括：
101、初始化用户态采集模块和内核采集模块的流量采集的软件运行环境。
初始化软件运行环境，是指加载用户态采集模块和内核采集模块所使用的软件环境、函数库和搭载运行环境。
102、建立通信通道。
通信通道是数据传输的通路，在本申请实施例中通信通道是一种逻辑通道，具体指在物理信道的基础上发送与接收信号的双方通过中间节点所实现的逻辑通路。通信通道是所述用户态采集模块和所述内核采集模块之间的数据传输通道。建立通信通道，也就是建立用户态采集模块和内核采集模块之间的逻辑通路。
103、用户态采集模块申请大页内存，在所述大页内存上创建环形缓存区，并将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块。
如果在***中使用大页内存，则内存页的数量会减少，从而需要更少的页表，节约了页表所占用的内存数量，并且所需的地址转换也减少了，TLB缓存失效的次数就减少了，从而提高了内存访问的性能。另外，由于地址转换所需的信息一般保存在CPU的缓存中，大页内存的使用让地址转换信息减少，从而减少了CPU缓存的使用，减轻了CPU缓存的压力，让CPU缓存能更多地用于应用程序的数据缓存，也能够在整体上提升***的性能。尤其是在内存访问密集型的虚拟机的宿主机中使用大页内存可以明显提高宿主机的性能。大页内存有助于Linux***进行虚拟内存管理，可以定义1GB的页面大小，能够为应用程序预留部分内存。预留的大页内存会一直保留，不会被交换出内存，除非修改了内存配置。用户态采集模块申请的大页内存，也就是大页内存属于用户态采集模块。
环形缓冲区是一种用于表示固定尺寸、头尾向量的缓冲区的数据结构，适合缓存数据流，用于存放通信中发送和接收的数据。环形缓冲区是先进先出的循环缓存区，可以向通信程序提供对缓冲区的互斥访问。在大页内存上创建环形缓冲区，环形缓冲区的位置可以跟大页内存与环形缓冲区的相对位置关系确定。如果能够确定大页内存的物理地址，在创建环形缓冲区后，就能确定环形缓冲区的物理地址。通过通信通道将用户态采集模块申请的大页内存的物理地址传输至内核采集模块，相当于内核采集模块能够接收到大页内存的物理地址，同时能够推算出环形缓冲区的物理地址。
104、内核采集模块接收所述大页内存的物理地址，并将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址。
内核采集模块接收大页内存的物理地址，并根据大页内存的物理地址、环形缓冲区的物理地址，以及内核地址空间，将大页内存映射成线性地址，将环形缓冲区映射成缓冲区地址。
105、内核采集模块采集虚拟交换机接收到的数据包，并根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置。
内核采集模块在采集虚拟交换机接收到的数据包时，其采样比例可以为1:1，或者1：n，采样比例根据后续流量采集的作用确定，在本申请实施例中对采样比例不做限定。内核采集模块采集虚拟交换机接收到的数据包并保存至环形缓冲区，每个数据包对应一条采集记录，占用环形缓冲区的一块固定大小的空间。在采集数据包的过程中，可以值采集数据包的包头，也可采集整个数据包。
106、用户态采集模块如果监测到所述环形缓冲区更新，则访问所述环形缓冲区并获取所述数据包，统计流经所述虚拟交换机的数据流量，同时更新所述环形缓冲区的读指针位置。
用户态采集模块获取环形缓冲区中的数据包之后，可以统计数据流量，统计数据包个数，还可以对数据流量做详细分析。根据数据流量监控虚拟网络，以提高网络性能、提高网络传输能力、分析网络信道利用率等等。
本申请提供了一种虚拟网络的流量采集方法，首先初始化流量采集的软件运行环境，然后建立用户态采集模块和内核采集模块之间的通信通道，再然后用户态采集模块申请大页内存，在大页内存上创建环形缓存区，并将大页内存的物理地址通过所述通信通道传输至所述内核采集模块，再然后内核采集模块将大页内存的物理地址映射成内核地址空间的线性地址，并获取环形缓冲区在内核采集模块的缓冲区地址，然后内核采集模块采集虚拟交换机接收到的数据包，并根据缓冲区地址将数据包保存至环形缓冲区，同时更新环形缓冲区的写指针位置，最后用户态采集模块如果监测到环形缓冲区更新，则访问环形缓冲区并获取数据包，统计流经虚拟交换机的数据流量，同时更新环形缓冲区的读指针位置。与现有技术相比，本申请实施例通过用户态采集模块通过环形缓冲区直接读取采集数据包，无复制开销，无***调用开销，消耗资源极低。内核采集模块通过环形缓冲区传递采集数据包到用户态采集模块，与用户态采集模块通过环形缓冲区中的写指针位置和读指针位置的同步，降低***开销。在流量采集过程中，监控对象的最小粒度为虚拟交换机，在不侵入虚拟机保证虚拟网络安全和性能的情况下监控虚拟机和网络性能。
本申请实施例提供了另一种虚拟网络的流量采集方法，如图2所示，该方法包括：
201、初始化用户态采集模块和内核采集模块的流量采集的软件运行环境。
初始化软件运行环境，是指加载用户态采集模块和内核采集模块所使用的软件环境、函数库和搭载运行环境。
202、建立通信通道。
通信通道是所述用户态采集模块和所述内核采集模块之间的数据传输通道。建立通信通道，也就是建立用户态采集模块和内核采集模块之间的逻辑通路。通信通道可以为Netlink通道，管道通道，或Socket通道。其中Netlink通道是一种特殊的Socket通道，其两种通道的建立方式类似。
如果通信通道是Netlink通道，那么建立通信通道，包括：根据套接字创建函数，创建套接字；设置所述套接字的本地地址和目的地址，所述本地地址为所述用户态采集模块的物理地址，所述目的地址为所述内核采集模块的物理地址；如果第一监听函数监听到所述套接字接收到连接请求，则建立所述通信通道。
Netlink套接字是用以实现用户进程与内核进程通信的一种特殊的进程间通信(IPC) ，也是网络应用程序与内核通信的最常用的接口。通过socket()，bind()，connect()，listen()，sendmsg()，recvmsg() 和 close()很容易地应用到netlink通道。netlink包含于头文件linux/netlink.h中。其中socket()用于创建套接字。bind()用于指定本地地址。一个套接字用socket()创建后，它其实还没有与任何特定的本地或目的地址相关联。一旦创建了一个套接字，就必须使用bind()***调用为套接字建立一个本地地址。connect()用于将套接字连接到目的地址。listen()用于设置等待连接状态。对于一个服务器的程序，当申请到套接字，并调用bind()与本地地址绑定后，就应该等待某个客户机的程序来要求连接。listen()就是把一个套接字设置为这种状态的函数。sendmsg()和recvmsg() 用于发送和接收数据。在数据流方式中，一个连接建立以后，或者在数据报方式下，调用了connect()进行了套接字与目的地址的绑定后，就可以调用sendmsg()和recvmsg() 函数进行数据传输。closesocket()用于关闭套接字。
如果通信通道是管道通道，那么建立通信通道，包括：在所述用户态采集模块，根据创建函数建立所述管道通道；如果第二监听函数监听到所述内核采集模块发送连接请求，则通过所述管道通道连接所述用户态采集模块和所述内核采集模块。
管道通道创建时，为缓冲区分配一个页面大小，管道所传送的是无格式字节流，这就要求管道的读出方和写入方必须事先约定好数据的格式。其创建过程使用管道通道所属的特定函数创建，例如创建函数CreateNamedPipe()，监听函数ConnectNamedPipe()，数据传输函数ReadFile()和WriteFile()，关闭函数CloseHandle()。
203、用户态采集模块申请大页内存，在所述大页内存上创建环形缓存区，并将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块。
在多核及NUMA架构服务器中，可以为每个NUMA在本地内存分配大页内存，为每个CPU核创建单独的环形缓冲区。在虚拟内存管理中，内核维护一个将虚拟内存地址映射到物理地址的表，对于每个页面操作，内核都需要加载相关的映射。如果内存页很小，那么需要加载的页就会很多，导致内核会加载更多的映射表，会降低性能。使用“大内存页”，所需要的页变少了。从而大大减少由内核加载的映射表的数量，提高了内核级别的性能最终有利于应用程序的性能。简而言之，通过启用“大内存页”，***具只需要处理较少的页面映射表，从而减少访问/维护它们的开销。
查找与通信通道的类型对应的数据传输函数，根据数据传输函数将大页内存的物理地址传输至内核采集模块。示例性的，Netlink套接字中的sendmsg()，管道通信中的WriteFile，以及socket中的send()和sendto()，用于发送物理地址。
204、内核采集模块接收所述大页内存的物理地址，获取宿主机和虚拟机之间的内存分配关系。
在用户态采集模块和内核采集模块收发数据时，可以设置应答机制，也确保物理地址的有效传输。例如，在内核采集模块接收到大页内存的物理地址后，需要给用户态采集模块反馈已接收消息，已接收消息可以携带物理地址的数据头信息，用户态模块接收到内核采集模块的反馈消息后可以校正接收信息是否正确。
为了在大页内存中存储数据包，在本步骤中还需要获取宿主机和虚拟机之间的内存分配关系，其分配关系在宿主机建立虚拟机时已经形成，只需要在保存其分配关系的路径中获取即可。
205、内核采集模块根据所述内存分配关系，将所述大页内存的物理地址映射成所述内核地址空间的线性地址。
206、内核采集模块根据所述线性地址、所述大页内存和所述环形缓冲区的地址分配关系，计算所述环形缓冲区的在所述内核采集模块的缓冲区地址。
内核采集模块根据大页内存的物理地址、环形缓冲区的物理地址，以及内核地址空间，将大页内存映射成线性地址，将环形缓冲区映射成缓冲区地址。
207、内核采集模块采集虚拟交换机接收到的数据包，并根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置。
内核采集模块在采集虚拟交换机接收到的数据包时，其采样比例可以为1:1，或者1：n，采样比例根据后续流量采集的作用确定，在本申请实施例中对采样比例不做限定。内核采集模块采集虚拟交换机接收到的数据包并保存至环形缓冲区，每个数据包对应一条采集记录，占用环形缓冲区的一块固定大小的空间。在采集数据包的过程中，可以值采集数据包的包头，也可采集整个数据包。
实际上，虚拟交换器具有多种类型，所以内核采集模块采集虚拟交换机接收到的数据包，具体包括：判断所述虚拟交换机的类型，所述类型包括Linux Bridge和OpenvSwitch；如果所述虚拟交换机的类型为Linux Bridge，则所述内核采集模块注册钩子函数，并根据所述钩子函数切入所述内核采集模块，采集所述虚拟交换机接收到的数据包；如果所述虚拟交换机的类型为OpenvSwitch，则所述内核采集模块利用流表结构ovsdatapath执行管理操作flow中的采集动作action，采集所述虚拟交换机接收到的数据包。
在保存数据包时，具体包括：获取所述写指针位置；从所述缓冲区地址中的所述写指针位置开始，在所述环形缓冲区存储所述数据包，并更新所述写指针位置。写指针是指环形缓冲区中待写入数据的位置。
208、用户态采集模块如果监测到所述环形缓冲区更新，则访问所述环形缓冲区并获取所述数据包，统计流经所述虚拟交换机的数据流量，同时更新所述环形缓冲区的读指针位置。
获取数据包具体包括：获取当前的所述写指针位置，并存储当前的所述写指针位置；如果所述当前的所述写指针位置与前一次存储的所述写指针位置不同，则访问并获取所述环形缓冲区中的数据包，所述数据包保存在所述环形缓冲区的所述读指针位置到当前的所述写指针位置之间。
通过检测环形缓冲区中写指针的位置监测环形缓冲区是否存在更新数据包。通过读指针的位置指示尚未统计的环形缓冲区中数据包位置。通过写指针和读指针的方式，在减小***开销的前提下，保证数据包存储过程中不被覆盖，读取过程中不被重复读取，提高统计数量流量的准确性。
本申请提供了一种虚拟网络的流量采集方法，首先初始化流量采集的软件运行环境，然后建立用户态采集模块和内核采集模块之间的通信通道，再然后用户态采集模块申请大页内存，在大页内存上创建环形缓存区，并将大页内存的物理地址通过所述通信通道传输至所述内核采集模块，再然后内核采集模块将大页内存的物理地址映射成内核地址空间的线性地址，并获取环形缓冲区在内核采集模块的缓冲区地址，然后内核采集模块采集虚拟交换机接收到的数据包，并根据缓冲区地址将数据包保存至环形缓冲区，同时更新环形缓冲区的写指针位置，最后用户态采集模块如果监测到环形缓冲区更新，则访问环形缓冲区并获取数据包，统计流经虚拟交换机的数据流量，同时更新环形缓冲区的读指针位置。与现有技术相比，本申请实施例通过用户态采集模块通过环形缓冲区直接读取采集数据包，无复制开销，无***调用开销，消耗资源极低。内核采集模块通过环形缓冲区传递采集数据包到用户态采集模块，与用户态采集模块通过环形缓冲区中的写指针位置和读指针位置的同步，降低***开销。在流量采集过程中，监控对象的最小粒度为虚拟交换机，在不侵入虚拟机保证虚拟网络安全和性能的情况下监控虚拟机和网络性能。
进一步的，作为对上述图1所示方法的实现，本申请实施例提供了一种虚拟网络的流量采集装置，如图3所示，该装置包括：
初始化单元31，用于初始化用户态采集模块和内核采集模块的流量采集的软件运行环境；
通道建立单元32，用于建立通信通道，所述通信通道是所述用户态采集模块和所述内核采集模块之间的数据传输通道；
缓存创建单元33，用于所述用户态采集模块申请大页内存，在所述大页内存上创建环形缓存区，并将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块；
地址获取单元34，用于所述内核采集模块接收所述大页内存的物理地址，并将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址；
数据存储单元35，用于所述内核采集模块采集虚拟交换机接收到的数据包，并根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置；
流量统计单元36，用于所述用户态采集模块如果监测到所述环形缓冲区更新，则访问所述环形缓冲区并获取所述数据包，统计流经所述虚拟交换机的数据流量，同时更新所述环形缓冲区的读指针位置。
本申请提供了一种虚拟网络的流量采集装置，首先初始化流量采集的软件运行环境，然后建立用户态采集模块和内核采集模块之间的通信通道，再然后用户态采集模块申请大页内存，在大页内存上创建环形缓存区，并将大页内存的物理地址通过所述通信通道传输至所述内核采集模块，再然后内核采集模块将大页内存的物理地址映射成内核地址空间的线性地址，并获取环形缓冲区在内核采集模块的缓冲区地址，然后内核采集模块采集虚拟交换机接收到的数据包，并根据缓冲区地址将数据包保存至环形缓冲区，同时更新环形缓冲区的写指针位置，最后用户态采集模块如果监测到环形缓冲区更新，则访问环形缓冲区并获取数据包，统计流经虚拟交换机的数据流量，同时更新环形缓冲区的读指针位置。与现有技术相比，本申请实施例通过用户态采集模块通过环形缓冲区直接读取采集数据包，无复制开销，无***调用开销，消耗资源极低。内核采集模块通过环形缓冲区传递采集数据包到用户态采集模块，与用户态采集模块通过环形缓冲区中的写指针位置和读指针位置的同步，降低***开销。在流量采集过程中，监控对象的最小粒度为虚拟交换机，在不侵入虚拟机保证虚拟网络安全和性能的情况下监控虚拟机和网络性能。
进一步的，作为对上述图2所示方法的实现，本申请实施例提供了另一种虚拟网络的流量采集装置，如图4所示，该装置包括：
初始化单元41，用于初始化用户态采集模块和内核采集模块的流量采集的软件运行环境；
通道建立单元42，用于建立通信通道，所述通信通道是所述用户态采集模块和所述内核采集模块之间的数据传输通道；
缓存创建单元43，用于所述用户态采集模块申请大页内存，在所述大页内存上创建环形缓存区，并将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块；
地址获取单元44，用于所述内核采集模块接收所述大页内存的物理地址，并将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址；
数据存储单元45，用于所述内核采集模块采集虚拟交换机接收到的数据包，并根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置；
流量统计单元46，用于所述用户态采集模块如果监测到所述环形缓冲区更新，则访问所述环形缓冲区并获取所述数据包，统计流经所述虚拟交换机的数据流量，同时更新所述环形缓冲区的读指针位置。
进一步地，所述通信通道是Netlink通道；
所述通道建立单元42，包括：
创建子单元421，用于根据套接字创建函数，创建套接字；
设置子单元422，用于设置所述套接字的本地地址和目的地址，所述本地地址为所述用户态采集模块的物理地址，所述目的地址为所述内核采集模块的物理地址；
连接子单元423，用于如果第一监听函数监听到所述套接字接收到连接请求，则建立所述通信通道。
进一步地，所述通信通道是管道通道；
所述通道建立单元42，包括：
所述创建子单元421，还用于在所述用户态采集模块，根据创建函数建立所述管道通道；
所述连接子单元423，用于如果第二监听函数监听到所述内核采集模块发送连接请求，则通过所述管道通道连接所述用户态采集模块和所述内核采集模块。
进一步地，所述缓存创建单元43，包括：
查找子单元431，用于查找与所述通信通道对应的传输函数；
传输子单元432，用于根据所述传输函数，将所述大页内存的物理地址传输至所述内核采集模块。
进一步地，所述地址获取单元44，包括：
获取子单元441，用于获取宿主机和虚拟机之间的内存分配关系；
映射子单元442，用于根据所述内存分配关系，将所述大页内存的物理地址映射成所述内核地址空间的线性地址；
计算子单元443，用于根据所述线性地址、所述大页内存和所述环形缓冲区的地址分配关系，计算所述环形缓冲区的在所述内核采集模块的缓冲区地址。
进一步地，所述数据存储单元45，包括：
判断子单元451，用于判断所述虚拟交换机的类型，所述类型包括Linux Bridge和OpenvSwitch；
采集子单元452，用于如果所述虚拟交换机的类型为Linux Bridge，则所述内核采集模块注册钩子函数，并根据所述钩子函数切入所述内核采集模块，采集所述虚拟交换机接收到的数据包；
所述采集子单元452，用于如果所述虚拟交换机的类型为OpenvSwitch，则所述内核采集模块利用流表结构ovsdatapath执行管理操作flow中的采集动作action，采集所述虚拟交换机接收到的数据包。
进一步地，所述数据存储单元45，包括：
获取子单元453，用于获取所述写指针位置；
存储子单元454，用于从所述缓冲区地址中的所述写指针位置开始，在所述环形缓冲区存储所述数据包，并更新所述写指针位置。
进一步地，所述流量统计单元46，用于：
获取当前的所述写指针位置，并存储当前的所述写指针位置；
如果所述当前的所述写指针位置与前一次存储的所述写指针位置不同，则访问并获取所述环形缓冲区中的数据包，所述数据包保存在所述环形缓冲区的所述读指针位置到当前的所述写指针位置之间。
本申请提供了一种虚拟网络的流量采集装置，首先初始化流量采集的软件运行环境，然后建立用户态采集模块和内核采集模块之间的通信通道，再然后用户态采集模块申请大页内存，在大页内存上创建环形缓存区，并将大页内存的物理地址通过所述通信通道传输至所述内核采集模块，再然后内核采集模块将大页内存的物理地址映射成内核地址空间的线性地址，并获取环形缓冲区在内核采集模块的缓冲区地址，然后内核采集模块采集虚拟交换机接收到的数据包，并根据缓冲区地址将数据包保存至环形缓冲区，同时更新环形缓冲区的写指针位置，最后用户态采集模块如果监测到环形缓冲区更新，则访问环形缓冲区并获取数据包，统计流经虚拟交换机的数据流量，同时更新环形缓冲区的读指针位置。与现有技术相比，本申请实施例通过用户态采集模块通过环形缓冲区直接读取采集数据包，无复制开销，无***调用开销，消耗资源极低。内核采集模块通过环形缓冲区传递采集数据包到用户态采集模块，与用户态采集模块通过环形缓冲区中的写指针位置和读指针位置的同步，降低***开销。在流量采集过程中，监控对象的最小粒度为虚拟交换机，在不侵入虚拟机保证虚拟网络安全和性能的情况下监控虚拟机和网络性能。
进一步的，图5示出了根据本申请一个实施例提供的一种虚拟网络的流量采集装置的结构示意图，本申请具体实施例并不对虚拟网络的流量采集装置的具体实现做限定。
如图5所示，该虚拟网络的流量采集装置可以包括：处理器(processor)502、通信接口(Communications Interface)504、存储器(memory)506、以及通信总线508。
其中：处理器502、通信接口504、以及存储器506通过通信总线508完成相互间的通信。
通信接口504，用于与其它设备比如客户端或其它服务器等的网元通信。
处理器502，用于执行程序510，具体可以执行上述虚拟网络的流量采集方法实施例中的相关步骤。
具体地，程序510可以包括程序代码，该程序代码包括计算机操作指令。
处理器502可能是中央处理器CPU，或者是特定集成电路ASIC（Application Specific Integrated Circuit），或者是被配置成实施本申请实施例的一个或多个集成电路。计算机设备包括的一个或多个处理器，可以是同一类型的处理器，如一个或多个CPU；也可以是不同类型的处理器，如一个或多个CPU以及一个或多个ASIC。
存储器506，用于存放程序510。存储器506可能包含高速RAM存储器，也可能还包括非易失性存储器（non-volatile memory），例如至少一个磁盘存储器。
程序510具体可以用于使得处理器502执行以下操作：
初始化用户态采集模块和内核采集模块的流量采集的软件运行环境；
建立通信通道，所述通信通道是所述用户态采集模块和所述内核采集模块之间的数据传输通道；
所述用户态采集模块申请大页内存，在所述大页内存上创建环形缓存区，并将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块；
所述内核采集模块接收所述大页内存的物理地址，并将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址；
所述内核采集模块采集虚拟交换机接收到的数据包，并根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置；
所述用户态采集模块如果监测到所述环形缓冲区更新，则访问所述环形缓冲区并获取所述数据包，统计流经所述虚拟交换机的数据流量，同时更新所述环形缓冲区的读指针位置。
本申请提供了一种虚拟网络的流量采集装置，首先初始化流量采集的软件运行环境，然后建立用户态采集模块和内核采集模块之间的通信通道，再然后用户态采集模块申请大页内存，在大页内存上创建环形缓存区，并将大页内存的物理地址通过所述通信通道传输至所述内核采集模块，再然后内核采集模块将大页内存的物理地址映射成内核地址空间的线性地址，并获取环形缓冲区在内核采集模块的缓冲区地址，然后内核采集模块采集虚拟交换机接收到的数据包，并根据缓冲区地址将数据包保存至环形缓冲区，同时更新环形缓冲区的写指针位置，最后用户态采集模块如果监测到环形缓冲区更新，则访问环形缓冲区并获取数据包，统计流经虚拟交换机的数据流量，同时更新环形缓冲区的读指针位置。与现有技术相比，本申请实施例通过用户态采集模块通过环形缓冲区直接读取采集数据包，无复制开销，无***调用开销，消耗资源极低。内核采集模块通过环形缓冲区传递采集数据包到用户态采集模块，与用户态采集模块通过环形缓冲区中的写指针位置和读指针位置的同步，降低***开销。在流量采集过程中，监控对象的最小粒度为虚拟交换机，在不侵入虚拟机保证虚拟网络安全和性能的情况下监控虚拟机和网络性能。
根据本申请一个实施例提供了一种存储介质，所述存储介质可以是易失性，也可以是非易失性，所述存储介质存储有至少一可执行指令，该计算机可执行指令可执行上述任意方法实施例中的虚拟网络的流量采集方法。

Claims (20)

  1. 一种虚拟网络的流量采集方法，其中，包括：
    初始化用户态采集模块和内核采集模块的流量采集的软件运行环境；
    建立通信通道，所述通信通道是所述用户态采集模块和所述内核采集模块之间的数据传输通道；
    所述用户态采集模块申请大页内存，在所述大页内存上创建环形缓存区，并将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块；
    所述内核采集模块接收所述大页内存的物理地址，并将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址；
    所述内核采集模块采集虚拟交换机接收到的数据包，并根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置；
    所述用户态采集模块如果监测到所述环形缓冲区更新，则访问所述环形缓冲区并获取所述数据包，统计流经所述虚拟交换机的数据流量，同时更新所述环形缓冲区的读指针位置。
  2. 如权利要求1所述的方法，其中，所述通信通道是Netlink通道；
    所述建立通信通道，包括：
    根据套接字创建函数，创建套接字；
    设置所述套接字的本地地址和目的地址，所述本地地址为所述用户态采集模块的物理地址，所述目的地址为所述内核采集模块的物理地址；
    如果第一监听函数监听到所述套接字接收到连接请求，则建立所述通信通道。
  3. 如权利要求1所述的方法，其中，所述通信通道是管道通道；
    所述建立通信通道，包括：
    在所述用户态采集模块，根据创建函数建立所述管道通道；
    如果第二监听函数监听到所述内核采集模块发送连接请求，则通过所述管道通道连接所述用户态采集模块和所述内核采集模块。
  4. 如权利要求1所述的方法，其中，所述将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块，包括：
    查找与所述通信通道对应的传输函数；
    根据所述传输函数，将所述大页内存的物理地址传输至所述内核采集模块。
  5. 如权利要求1所述的方法，其中，所述将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址，包括：
    获取宿主机和虚拟机之间的内存分配关系；
    根据所述内存分配关系，将所述大页内存的物理地址映射成所述内核地址空间的线性地址；
    根据所述线性地址、所述大页内存和所述环形缓冲区的地址分配关系，计算所述环形缓冲区的在所述内核采集模块的缓冲区地址。
  6. 如权利要求1所述的方法，其中，所述内核采集模块采集虚拟交换机接收到的数据包，包括：
    判断所述虚拟交换机的类型，所述类型包括Linux Bridge和OpenvSwitch；
    如果所述虚拟交换机的类型为Linux Bridge，则所述内核采集模块注册钩子函数，并根据所述钩子函数切入所述内核采集模块，采集所述虚拟交换机接收到的数据包；
    如果所述虚拟交换机的类型为OpenvSwitch，则所述内核采集模块利用流表结构ovsdatapath执行管理操作flow中的采集动作action，采集所述虚拟交换机接收到的数据包。
  7. 如权利要求1所述的方法，其中，所述根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置，包括：
    获取所述写指针位置；
    从所述缓冲区地址中的所述写指针位置开始，在所述环形缓冲区存储所述数据包，并更新所述写指针位置。
  8. 一种虚拟网络的流量采集装置，其中，包括：
    初始化单元，用于初始化用户态采集模块和内核采集模块的流量采集的软件运行环境；
    通道建立单元，用于建立通信通道，所述通信通道是所述用户态采集模块和所述内核采集模块之间的数据传输通道；
    缓存创建单元，用于所述用户态采集模块申请大页内存，在所述大页内存上创建环形缓存区，并将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块；
    地址获取单元，用于所述内核采集模块接收所述大页内存的物理地址，并将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址；
    数据存储单元，用于所述内核采集模块采集虚拟交换机接收到的数据包，并根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置；
    流量统计单元，用于所述用户态采集模块如果监测到所述环形缓冲区更新，则访问所述环形缓冲区并获取所述数据包，统计流经所述虚拟交换机的数据流量，同时更新所述环形缓冲区的读指针位置。
  9. 一种存储介质，所述存储介质中存储有至少一可执行指令，所述可执行指令使处理器执行如下步骤：
    初始化用户态采集模块和内核采集模块的流量采集的软件运行环境；
    建立通信通道，所述通信通道是所述用户态采集模块和所述内核采集模块之间的数据传输通道；
    所述用户态采集模块申请大页内存，在所述大页内存上创建环形缓存区，并将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块；
    所述内核采集模块接收所述大页内存的物理地址，并将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址；
    所述内核采集模块采集虚拟交换机接收到的数据包，并根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置；
    所述用户态采集模块如果监测到所述环形缓冲区更新，则访问所述环形缓冲区并获取所述数据包，统计流经所述虚拟交换机的数据流量，同时更新所述环形缓冲区的读指针位置。
  10. 如权利要求9所述的存储介质，其中，所述通信通道是Netlink通道；
    所述建立通信通道，包括：
    根据套接字创建函数，创建套接字；
    设置所述套接字的本地地址和目的地址，所述本地地址为所述用户态采集模块的物理地址，所述目的地址为所述内核采集模块的物理地址；
    如果第一监听函数监听到所述套接字接收到连接请求，则建立所述通信通道。
  11. 如权利要求9所述的存储介质，其中，所述通信通道是管道通道；
    所述建立通信通道，包括：
    在所述用户态采集模块，根据创建函数建立所述管道通道；
    如果第二监听函数监听到所述内核采集模块发送连接请求，则通过所述管道通道连接所述用户态采集模块和所述内核采集模块。
  12. 如权利要求9所述的存储介质，其中，所述将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块，包括：
    查找与所述通信通道对应的传输函数；
    根据所述传输函数，将所述大页内存的物理地址传输至所述内核采集模块。
  13. 如权利要求9所述的存储介质，其中，所述将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址，包括：
    获取宿主机和虚拟机之间的内存分配关系；
    根据所述内存分配关系，将所述大页内存的物理地址映射成所述内核地址空间的线性地址；
    根据所述线性地址、所述大页内存和所述环形缓冲区的地址分配关系，计算所述环形缓冲区的在所述内核采集模块的缓冲区地址。
  14. 如权利要求9所述的存储介质，其中，所述内核采集模块采集虚拟交换机接收到的数据包，包括：
    判断所述虚拟交换机的类型，所述类型包括Linux Bridge和OpenvSwitch；
    如果所述虚拟交换机的类型为Linux Bridge，则所述内核采集模块注册钩子函数，并根据所述钩子函数切入所述内核采集模块，采集所述虚拟交换机接收到的数据包；
    如果所述虚拟交换机的类型为OpenvSwitch，则所述内核采集模块利用流表结构ovsdatapath执行管理操作flow中的采集动作action，采集所述虚拟交换机接收到的数据包。
  15. 如权利要求9所述的存储介质，其中，所述根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置，包括：
    获取所述写指针位置；
    从所述缓冲区地址中的所述写指针位置开始，在所述环形缓冲区存储所述数据包，并更新所述写指针位置。
  16. 一种计算机设备，包括：处理器、存储器、通信接口和通信总线，所述处理器、所述存储器和所述通信接口通过所述通信总线完成相互间的通信；
    所述存储器用于存放至少一可执行指令，所述可执行指令使所述处理器执行如下步骤：
    初始化用户态采集模块和内核采集模块的流量采集的软件运行环境；
    建立通信通道，所述通信通道是所述用户态采集模块和所述内核采集模块之间的数据传输通道；
    所述用户态采集模块申请大页内存，在所述大页内存上创建环形缓存区，并将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块；
    所述内核采集模块接收所述大页内存的物理地址，并将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址；
    所述内核采集模块采集虚拟交换机接收到的数据包，并根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置；
    所述用户态采集模块如果监测到所述环形缓冲区更新，则访问所述环形缓冲区并获取所述数据包，统计流经所述虚拟交换机的数据流量，同时更新所述环形缓冲区的读指针位置。
  17. 如权利要求16所述的计算机设备，其中，所述通信通道是Netlink通道；
    所述建立通信通道，包括：
    根据套接字创建函数，创建套接字；
    设置所述套接字的本地地址和目的地址，所述本地地址为所述用户态采集模块的物理地址，所述目的地址为所述内核采集模块的物理地址；
    如果第一监听函数监听到所述套接字接收到连接请求，则建立所述通信通道。
  18. 如权利要求16所述的计算机设备，其中，所述通信通道是管道通道；
    所述建立通信通道，包括：
    在所述用户态采集模块，根据创建函数建立所述管道通道；
    如果第二监听函数监听到所述内核采集模块发送连接请求，则通过所述管道通道连接所述用户态采集模块和所述内核采集模块。
  19. 如权利要求16所述的计算机设备，其中，所述将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块，包括：
    查找与所述通信通道对应的传输函数；
    根据所述传输函数，将所述大页内存的物理地址传输至所述内核采集模块。
  20. 如权利要求16所述的计算机设备，其中，所述将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址，包括：
    获取宿主机和虚拟机之间的内存分配关系；
    根据所述内存分配关系，将所述大页内存的物理地址映射成所述内核地址空间的线性地址；
    根据所述线性地址、所述大页内存和所述环形缓冲区的地址分配关系，计算所述环形缓冲区的在所述内核采集模块的缓冲区地址。
PCT/CN2020/119067 2020-02-18 2020-09-29 虚拟网络的流量采集方法、装置、计算机设备及存储介质 WO2021164262A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010098775.2A CN111431757B (zh) 2020-02-18 2020-02-18 虚拟网络的流量采集方法及装置
CN202010098775.2 2020-02-18

Publications (1)

Publication Number Publication Date
WO2021164262A1 true WO2021164262A1 (zh) 2021-08-26

Family

ID=71551573

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/119067 WO2021164262A1 (zh) 2020-02-18 2020-09-29 虚拟网络的流量采集方法、装置、计算机设备及存储介质

Country Status (2)

Country Link
CN (1) CN111431757B (zh)
WO (1) WO2021164262A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113794605A (zh) * 2021-09-10 2021-12-14 杭州谐云科技有限公司 一种基于eBPF的内核丢包检测方法、***和装置
CN115033407A (zh) * 2022-08-09 2022-09-09 微栈科技(浙江)有限公司 一种适用于云计算的采集识别流量的***和方法
CN117579386A (zh) * 2024-01-16 2024-02-20 麒麟软件有限公司 网络流量安全管控方法、装置及存储介质

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111431757B (zh) * 2020-02-18 2023-03-24 平安科技（深圳）有限公司 虚拟网络的流量采集方法及装置
CN113132349A (zh) * 2021-03-12 2021-07-16 中国科学院信息工程研究所 一种免代理云平台虚拟流量入侵检测方法及装置
CN113300917B (zh) * 2021-07-27 2021-10-15 苏州浪潮智能科技有限公司 Open Stack租户网络的流量监控方法、装置
CN113986811B (zh) * 2021-09-23 2022-05-10 北京东方通网信科技有限公司 一种高性能内核态网络数据包加速方法
CN116915667B (zh) * 2023-07-19 2024-03-08 上海螣龙科技有限公司 网络流量采集方法、装置、电子设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6351791B1 (en) * 1998-06-25 2002-02-26 International Business Machines Corporation Circuit arrangement and method of maintaining cache coherence utilizing snoop response collection logic that disregards extraneous retry responses
CN107294869A (zh) * 2017-06-22 2017-10-24 郑州云海信息技术有限公司 一种虚拟网卡报文抓取的方法及***
CN108964959A (zh) * 2017-05-27 2018-12-07 阿里巴巴集团控股有限公司 一种用于虚拟化平台的网卡直通***及数据包监管方法
CN108989151A (zh) * 2018-07-20 2018-12-11 北京云杉世纪网络科技有限公司 用于网络或应用性能管理的流量采集方法
CN110380992A (zh) * 2019-07-24 2019-10-25 南京中孚信息技术有限公司 报文处理方法、装置及网络流量采集设备
CN111431757A (zh) * 2020-02-18 2020-07-17 平安科技（深圳）有限公司 虚拟网络的流量采集方法及装置

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6961941B1 (en) * 2001-06-08 2005-11-01 Vmware, Inc. Computer configuration for resource management in systems including a virtual machine
CN100477643C (zh) * 2006-09-22 2009-04-08 中国科学院计算技术研究所 基于共享内存实现的数据包捕获方法
CN102467473B (zh) * 2010-11-03 2015-02-11 Tcl集团股份有限公司 一种在用户空间和内核之间传输数据的方法和装置
CN103034684A (zh) * 2012-11-27 2013-04-10 北京航空航天大学 一种基于内容寻址存储的虚拟机镜像存储优化方法
CN104063267B (zh) * 2014-07-11 2017-11-14 孙强强 一种虚拟机流量监控方法及***
CN108062253A (zh) * 2017-12-11 2018-05-22 北京奇虎科技有限公司 一种内核态与用户态的通信方法、装置及终端
CN109889403A (zh) * 2019-01-30 2019-06-14 北京云端智度科技有限公司 一种基于轮询机制的cdn网络流量检测方法
CN109981403A (zh) * 2019-03-05 2019-07-05 北京勤慕数据科技有限公司 虚拟机网络数据流量监控方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6351791B1 (en) * 1998-06-25 2002-02-26 International Business Machines Corporation Circuit arrangement and method of maintaining cache coherence utilizing snoop response collection logic that disregards extraneous retry responses
CN108964959A (zh) * 2017-05-27 2018-12-07 阿里巴巴集团控股有限公司 一种用于虚拟化平台的网卡直通***及数据包监管方法
CN107294869A (zh) * 2017-06-22 2017-10-24 郑州云海信息技术有限公司 一种虚拟网卡报文抓取的方法及***
CN108989151A (zh) * 2018-07-20 2018-12-11 北京云杉世纪网络科技有限公司 用于网络或应用性能管理的流量采集方法
CN110380992A (zh) * 2019-07-24 2019-10-25 南京中孚信息技术有限公司 报文处理方法、装置及网络流量采集设备
CN111431757A (zh) * 2020-02-18 2020-07-17 平安科技（深圳）有限公司 虚拟网络的流量采集方法及装置

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113794605A (zh) * 2021-09-10 2021-12-14 杭州谐云科技有限公司 一种基于eBPF的内核丢包检测方法、***和装置
CN115033407A (zh) * 2022-08-09 2022-09-09 微栈科技(浙江)有限公司 一种适用于云计算的采集识别流量的***和方法
CN115033407B (zh) * 2022-08-09 2022-11-04 微栈科技(浙江)有限公司 一种适用于云计算的采集识别流量的***和方法
CN117579386A (zh) * 2024-01-16 2024-02-20 麒麟软件有限公司 网络流量安全管控方法、装置及存储介质
CN117579386B (zh) * 2024-01-16 2024-04-12 麒麟软件有限公司 网络流量安全管控方法、装置及存储介质

Also Published As

Publication number Publication date
CN111431757B (zh) 2023-03-24
CN111431757A (zh) 2020-07-17

Similar Documents

Publication Publication Date Title
WO2021164262A1 (zh) 虚拟网络的流量采集方法、装置、计算机设备及存储介质
US11500689B2 (en) Communication method and apparatus
Cheriton et al. The distributed V kernel and its performance for diskless workstations
Welch et al. Pseudo devices: User-level extensions to the Sprite file system
US11249938B2 (en) Programmed input/output mode
US8234407B2 (en) Network use of virtual addresses without pinning or registration
Takahashi et al. PM2: High performance communication middleware for heterogeneous network environments
US7089289B1 (en) Mechanisms for efficient message passing with copy avoidance in a distributed system using advanced network devices
US6799200B1 (en) Mechanisms for efficient message passing with copy avoidance in a distributed system
EP0889622B1 (en) Apparatus and method for remote buffer allocation and management for message passing between network nodes
US20030145230A1 (en) System for exchanging data utilizing remote direct memory access
Anderson et al. Cheating the I/O Bottleneck: Network Storage with Trapeze/Myrinet.
JPH1185710A (ja) サーバ装置およびファイル管理方法
CN114756388B (zh) 一种基于rdma的集群***节点间按需共享内存的方法
CA2241994A1 (en) System and method for efficient remote disk i/o
JPH09231157A (ja) コンピュータに接続されている入力／出力（ｉ／ｏ）デバイスを制御する方法
JP2004520646A (ja) 周辺デバイスからホスト・コンピュータ・システムに割込みを転送する方法および装置
CN109857545B (zh) 一种数据传输方法及装置
US20060123139A1 (en) Intelligent memory interface
Salehi et al. The effectiveness of affinity-based scheduling in multiprocessor networking
Li et al. A high-efficient inter-domain data transferring system for virtual machines
CN118093225A (zh) 订阅消息的传输方法及装置、存储介质及电子设备
Welch et al. to the Sprite File System
SINGH et al. QMP: A Fast Communication Protocol for COW’s
Yamada et al. Implementation and performance evaluation of a memory‐coupled scalable node architecture MESCAR for parallel and distributed processing

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20919864

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20919864

Country of ref document: EP

Kind code of ref document: A1