WO2021164262A1

WO2021164262A1 - Traffic collection method and apparatus for virtual network, and computer device and storage medium

Info

Publication number: WO2021164262A1
Application number: PCT/CN2020/119067
Authority: WO
Inventors: 张齐国; 王继海
Original assignee: 平安科技（深圳）有限公司
Priority date: 2020-02-18
Filing date: 2020-09-29
Publication date: 2021-08-26
Also published as: CN111431757B; CN111431757A

Abstract

Disclosed are a traffic collection method and apparatus for a virtual network, and a computer device and a storage medium, which relate to the technical field of virtual machines, and solve the problem in the prior art of excessive traffic collection overheads. The method mainly comprises: initializing a software running environment for traffic collection; establishing a communication channel; a user state collection module applying for a large page memory, and creating an annular buffer area in the large page memory; a kernel collection module receiving a physical address of the large page memory, mapping the physical address of the large page memory into a linear address of a kernel address space, and acquiring a buffer area address, in the kernel collection module, of the annular buffer area; the kernel collection module collecting a data packet received by a virtual switch, and saving the data packet in the annular buffer area according to the buffer area address; and if an update in the annular buffer area is detected, the user state collection module accessing the annular buffer area and acquiring the data packet, and compiling statistics on data traffic flowing through the virtual switch. The present application is mainly applied to a traffic collection process.

Description

虚拟网络的流量采集方法、装置、计算机设备及存储介质Virtual network traffic collection method, device, computer equipment and storage medium

本申请要求于2020年2月18日提交中国专利局、申请号为CN202010098775.2、名称为“虚拟网络的流量采集方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on February 18, 2020, with the application number CN202010098775. 2, titled "Virtual Network Traffic Collection Method and Device", the entire content of which is incorporated herein by reference. Applying.

技术领域Technical field

本申请涉及一种虚拟机技术领域，特别是涉及一种虚拟网络的流量采集方法、装置、计算机设备及存储介质。This application relates to the technical field of virtual machines, and in particular to a method, device, computer equipment and storage medium for traffic collection of a virtual network.

背景技术Background technique

对数据中心网络性能管理和网络优化，需要对流量进行采集分析，把网络流量从不可见变为可见。随着虚拟化和云应用越来越广，更多的网络流量发生在同一主机上的虚拟机之间，甚至虚拟机之间的网络流量已占据到数据中心总流量的80%。For data center network performance management and network optimization, it is necessary to collect and analyze the traffic to change the network traffic from invisible to visible. As virtualization and cloud applications become more and more widespread, more network traffic occurs between virtual machines on the same host, and even network traffic between virtual machines has accounted for 80% of the total data center traffic.

现有技术中，首选用户态采集单元通过***调用AF_PACKET套接字接口，在共享内存中对目标虚拟网卡进行网包监控，然后内核采集单元对收到的网包通过mmap放个把新增的网包映射到共享内存中，最后用户态单元检测到共享内存中存在新增的网包，则采集新增的网包，完成流量采集。In the prior art, the preferred user mode collection unit uses the system to call the AF_PACKET socket interface to monitor the target virtual network card in the shared memory, and then the kernel collection unit puts a newly added network packet on the received network packet through mmap. The network packet is mapped to the shared memory, and finally the user mode unit detects that there is a new network packet in the shared memory, and then collects the new network packet to complete the flow collection.

技术问题technical problem

发明人意识到上述现有技术，用户态单元对每个网包调用AF_PACKET接口，***调用开销大，调用频率高，消耗资源多，每个网包的内存mmap到共享内存，开销大且内存零散难于管理。由于虚拟机之间的网络流传流量过多，导致采集流量时开销过大。The inventor realizes that in the above-mentioned prior art, the user mode unit calls the AF_PACKET interface for each net packet, the system call overhead is high, the calling frequency is high, and the resource consumption is high. The memory mmap of each net packet to the shared memory is expensive and the memory is fragmented. Difficult to manage. Due to the excessive network traffic between virtual machines, the overhead of collecting traffic is too large.

技术解决方案Technical solutions

依据本申请一个方面，提供了一种虚拟网络的流量采集方法，包括：According to one aspect of the present application, a method for collecting traffic in a virtual network is provided, including:

初始化用户态采集模块和内核采集模块的流量采集的软件运行环境；Initialize the flow collection software operating environment of the user mode collection module and the kernel collection module;

建立通信通道，所述通信通道是所述用户态采集模块和所述内核采集模块之间的数据传输通道；Establishing a communication channel, the communication channel being a data transmission channel between the user mode acquisition module and the kernel acquisition module;

所述用户态采集模块申请大页内存，在所述大页内存上创建环形缓存区，并将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块；The user mode collection module applies for a large page memory, creates a ring buffer area on the large page memory, and transmits the physical address of the large page memory to the kernel collection module through the communication channel;

所述内核采集模块接收所述大页内存的物理地址，并将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址；The kernel collection module receives the physical address of the large page memory, maps the physical address of the large page memory to a linear address in the kernel address space, and obtains the buffer of the ring buffer in the kernel collection module address;

所述内核采集模块采集虚拟交换机接收到的数据包，并根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置；The kernel collection module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer position of the ring buffer;

所述用户态采集模块如果监测到所述环形缓冲区更新，则访问所述环形缓冲区并获取所述数据包，统计流经所述虚拟交换机的数据流量，同时更新所述环形缓冲区的读指针位置。If the user state acquisition module detects that the ring buffer is updated, it accesses the ring buffer and obtains the data packet, counts the data traffic flowing through the virtual switch, and updates the reading of the ring buffer at the same time. Pointer position.

依据本申请另一个方面，提供了一种虚拟网络的流量采集装置，包括：According to another aspect of the present application, there is provided a virtual network traffic collection device, including:

初始化单元，用于初始化用户态采集模块和内核采集模块的流量采集的软件运行环境；The initialization unit is used to initialize the flow collection software operating environment of the user mode collection module and the core collection module;

通道建立单元，用于建立通信通道，所述通信通道是所述用户态采集模块和所述内核采集模块之间的数据传输通道；A channel establishing unit, configured to establish a communication channel, the communication channel being a data transmission channel between the user mode acquisition module and the kernel acquisition module;

缓存创建单元，用于所述用户态采集模块申请大页内存，在所述大页内存上创建环形缓存区，并将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块；The cache creation unit is used for the user mode collection module to apply for a large page memory, create a ring buffer area on the large page memory, and transmit the physical address of the large page memory to the kernel collection via the communication channel Module

地址获取单元，用于所述内核采集模块接收所述大页内存的物理地址，并将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址；The address acquisition unit is configured to receive the physical address of the large page memory by the kernel acquisition module, map the physical address of the large page memory to a linear address in the kernel address space, and obtain the ring buffer in the The buffer address of the kernel acquisition module;

数据存储单元，用于所述内核采集模块采集虚拟交换机接收到的数据包，并根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置；The data storage unit is used for the kernel collection module to collect the data packet received by the virtual switch, and save the data packet to the ring buffer according to the buffer address, and at the same time update the write pointer of the ring buffer Location;

流量统计单元，用于所述用户态采集模块如果监测到所述环形缓冲区更新，则访问所述环形缓冲区并获取所述数据包，统计流经所述虚拟交换机的数据流量，同时更新所述环形缓冲区的读指针位置。The traffic statistics unit is configured to, if the user state acquisition module detects that the ring buffer is updated, it will access the ring buffer and obtain the data packets, count the data traffic flowing through the virtual switch, and update all the data at the same time. The position of the read pointer of the ring buffer.

根据本申请的又一方面，提供了一种存储介质，所述存储介质中存储有至少一可执行指令，所述可执行指令使处理器执行如下步骤：According to another aspect of the present application, a storage medium is provided, and at least one executable instruction is stored in the storage medium, and the executable instruction causes a processor to perform the following steps:

依据本申请又一个方面，提供了一种计算机设备，包括：According to another aspect of the present application, a computer device is provided, including:

处理器、存储器、通信接口和通信总线，所述处理器、所述存储器和所述通信接口通过所述通信总线完成相互间的通信；A processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface communicate with each other through the communication bus;

所述存储器用于存放至少一可执行指令，所述可执行指令使所述处理器执行如下步骤：The memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform the following steps:

附图说明Description of the drawings

通过阅读下文优选实施方式的详细描述，各种其他的优点和益处对于本领域普通技术人员将变得清楚明了。附图仅用于示出优选实施方式的目的，而并不认为是对本申请的限制。而且在整个附图中，用相同的参考符号表示相同的部件。在附图中：By reading the detailed description of the preferred embodiments below, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only used for the purpose of illustrating the preferred embodiments, and are not considered as a limitation to the application. Also, throughout the drawings, the same reference symbols are used to denote the same components. In the attached picture:

图1示出了本申请实施例提供的一种虚拟网络的流量采集方法流程图；FIG. 1 shows a flow chart of a method for collecting traffic on a virtual network according to an embodiment of the present application;

图2示出了本申请实施例提供的另一种虚拟网络的流量采集方法流程图；FIG. 2 shows a flowchart of another method for collecting traffic on a virtual network according to an embodiment of the present application;

图3示出了本申请实施例提供的一种虚拟网络的流量采集装置组成框图；FIG. 3 shows a block diagram of the composition of a virtual network traffic collection device provided by an embodiment of the present application;

图4示出了本申请实施例提供的另一种虚拟网络的流量采集装置组成框图；Figure 4 shows a block diagram of another virtual network traffic collection device provided by an embodiment of the present application;

图5本申请实施例提供的一种计算机设备的结构示意图。Fig. 5 is a schematic structural diagram of a computer device provided by an embodiment of the present application.

本发明的实施方式Embodiments of the present invention

本申请是实施例的流量采集环境，可以为同一宿主机上的虚拟机之间，也可以为不用宿主机上的虚拟机之间。本申请实施例是一种运行与宿主机上的软件方法，主要包括用户态采集模块、环形缓冲区和内核采集模块。内核采集模块主要用于采集虚拟交换机接收到的数据包，并将数据包保存至环形缓冲区，用户态采集模块主要用于获取环形缓冲区中的数据包，以统计流经虚拟交换机的流量。本申请实施例提供了一种虚拟网络的流量采集方法，如图1所示，该方法包括：This application is the traffic collection environment of the embodiment, which can be between virtual machines on the same host machine, or between virtual machines that do not use host machines. The embodiment of the application is a software method running on a host computer, which mainly includes a user mode acquisition module, a ring buffer, and a kernel acquisition module. The kernel collection module is mainly used to collect the data packets received by the virtual switch and save the data packets to the ring buffer. The user mode collection module is mainly used to obtain the data packets in the ring buffer to count the traffic flowing through the virtual switch. The embodiment of the present application provides a method for collecting traffic on a virtual network. As shown in FIG. 1, the method includes:

101、初始化用户态采集模块和内核采集模块的流量采集的软件运行环境。101. Initialize the flow collection software operating environment of the user mode collection module and the kernel collection module.

初始化软件运行环境，是指加载用户态采集模块和内核采集模块所使用的软件环境、函数库和搭载运行环境。Initializing the software operating environment refers to the software environment, function library, and loading operating environment used to load the user mode acquisition module and the kernel acquisition module.

102、建立通信通道。102. Establish a communication channel.

通信通道是数据传输的通路，在本申请实施例中通信通道是一种逻辑通道，具体指在物理信道的基础上发送与接收信号的双方通过中间节点所实现的逻辑通路。通信通道是所述用户态采集模块和所述内核采集模块之间的数据传输通道。建立通信通道，也就是建立用户态采集模块和内核采集模块之间的逻辑通路。The communication channel is a path for data transmission. In the embodiment of the present application, the communication channel is a kind of logical channel, which specifically refers to a logical path implemented by an intermediate node between the parties sending and receiving signals on the basis of a physical channel. The communication channel is a data transmission channel between the user state acquisition module and the core acquisition module. Establish a communication channel, that is, establish a logical path between the user mode acquisition module and the core acquisition module.

103、用户态采集模块申请大页内存，在所述大页内存上创建环形缓存区，并将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块。103. The user mode collection module applies for a large page memory, creates a ring buffer area on the large page memory, and transmits the physical address of the large page memory to the kernel collection module through the communication channel.

如果在***中使用大页内存，则内存页的数量会减少，从而需要更少的页表，节约了页表所占用的内存数量，并且所需的地址转换也减少了，TLB缓存失效的次数就减少了，从而提高了内存访问的性能。另外，由于地址转换所需的信息一般保存在CPU的缓存中，大页内存的使用让地址转换信息减少，从而减少了CPU缓存的使用，减轻了CPU缓存的压力，让CPU缓存能更多地用于应用程序的数据缓存，也能够在整体上提升***的性能。尤其是在内存访问密集型的虚拟机的宿主机中使用大页内存可以明显提高宿主机的性能。大页内存有助于Linux***进行虚拟内存管理，可以定义1GB的页面大小，能够为应用程序预留部分内存。预留的大页内存会一直保留，不会被交换出内存，除非修改了内存配置。用户态采集模块申请的大页内存，也就是大页内存属于用户态采集模块。If large page memory is used in the system, the number of memory pages will be reduced, thus requiring fewer page tables, saving the amount of memory occupied by page tables, and the required address translation is also reduced, and the number of TLB cache invalidations It is reduced, thereby improving the performance of memory access. In addition, because the information required for address translation is generally stored in the CPU cache, the use of large page memory reduces the address translation information, thereby reducing the use of the CPU cache, reducing the pressure on the CPU cache, and allowing the CPU cache to use more Data caching for applications can also improve the performance of the system as a whole. In particular, the use of large page memory in the host machine of a memory-intensive virtual machine can significantly improve the performance of the host machine. Large page memory helps Linux system to manage virtual memory, and can define a page size of 1GB, which can reserve part of the memory for applications. The reserved large page memory will always be reserved and will not be swapped out of the memory unless the memory configuration is modified. The large page memory requested by the user state acquisition module, that is, the large page memory belongs to the user state acquisition module.

环形缓冲区是一种用于表示固定尺寸、头尾向量的缓冲区的数据结构，适合缓存数据流，用于存放通信中发送和接收的数据。环形缓冲区是先进先出的循环缓存区，可以向通信程序提供对缓冲区的互斥访问。在大页内存上创建环形缓冲区，环形缓冲区的位置可以跟大页内存与环形缓冲区的相对位置关系确定。如果能够确定大页内存的物理地址，在创建环形缓冲区后，就能确定环形缓冲区的物理地址。通过通信通道将用户态采集模块申请的大页内存的物理地址传输至内核采集模块，相当于内核采集模块能够接收到大页内存的物理地址，同时能够推算出环形缓冲区的物理地址。The ring buffer is a data structure used to represent a fixed-size, head-to-tail vector buffer, suitable for buffering data streams, and used to store data sent and received in communications. The ring buffer is a first-in, first-out circular buffer area that can provide mutually exclusive access to the buffer for communication programs. Create a ring buffer on the large page memory. The location of the ring buffer can be determined with the relative position of the large page memory and the ring buffer. If the physical address of the large page memory can be determined, after the ring buffer is created, the physical address of the ring buffer can be determined. The physical address of the large page memory requested by the user mode acquisition module is transmitted to the kernel acquisition module through the communication channel, which is equivalent to that the kernel acquisition module can receive the physical address of the large page memory and can calculate the physical address of the ring buffer at the same time.

104、内核采集模块接收所述大页内存的物理地址，并将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址。104. The kernel collection module receives the physical address of the large page memory, maps the physical address of the large page memory to a linear address in the kernel address space, and obtains the buffer of the ring buffer in the kernel collection module address.

内核采集模块接收大页内存的物理地址，并根据大页内存的物理地址、环形缓冲区的物理地址，以及内核地址空间，将大页内存映射成线性地址，将环形缓冲区映射成缓冲区地址。The kernel acquisition module receives the physical address of the large page memory, and maps the large page memory to a linear address and the ring buffer to a buffer address according to the physical address of the large page memory, the physical address of the ring buffer, and the kernel address space. .

105、内核采集模块采集虚拟交换机接收到的数据包，并根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置。105. The kernel collection module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer position of the ring buffer.

内核采集模块在采集虚拟交换机接收到的数据包时，其采样比例可以为1:1，或者1：n，采样比例根据后续流量采集的作用确定，在本申请实施例中对采样比例不做限定。内核采集模块采集虚拟交换机接收到的数据包并保存至环形缓冲区，每个数据包对应一条采集记录，占用环形缓冲区的一块固定大小的空间。在采集数据包的过程中，可以值采集数据包的包头，也可采集整个数据包。When the core collection module collects data packets received by the virtual switch, the sampling ratio can be 1:1 or 1:n. The sampling ratio is determined according to the subsequent traffic collection function. The sampling ratio is not limited in the embodiment of this application. . The kernel collection module collects the data packets received by the virtual switch and saves them to the ring buffer. Each data packet corresponds to a collection record and occupies a fixed-size space in the ring buffer. In the process of collecting data packets, the header of the data packet can be collected, or the entire data packet can be collected.

106、用户态采集模块如果监测到所述环形缓冲区更新，则访问所述环形缓冲区并获取所述数据包，统计流经所述虚拟交换机的数据流量，同时更新所述环形缓冲区的读指针位置。106. If the user mode collection module detects that the ring buffer is updated, it accesses the ring buffer and obtains the data packet, counts the data traffic flowing through the virtual switch, and updates the reading of the ring buffer at the same time. Pointer position.

用户态采集模块获取环形缓冲区中的数据包之后，可以统计数据流量，统计数据包个数，还可以对数据流量做详细分析。根据数据流量监控虚拟网络，以提高网络性能、提高网络传输能力、分析网络信道利用率等等。After the user mode acquisition module obtains the data packets in the ring buffer, it can count the data traffic, count the number of data packets, and can also perform detailed analysis on the data traffic. Monitor virtual networks based on data traffic to improve network performance, improve network transmission capacity, analyze network channel utilization, and more.

本申请提供了一种虚拟网络的流量采集方法，首先初始化流量采集的软件运行环境，然后建立用户态采集模块和内核采集模块之间的通信通道，再然后用户态采集模块申请大页内存，在大页内存上创建环形缓存区，并将大页内存的物理地址通过所述通信通道传输至所述内核采集模块，再然后内核采集模块将大页内存的物理地址映射成内核地址空间的线性地址，并获取环形缓冲区在内核采集模块的缓冲区地址，然后内核采集模块采集虚拟交换机接收到的数据包，并根据缓冲区地址将数据包保存至环形缓冲区，同时更新环形缓冲区的写指针位置，最后用户态采集模块如果监测到环形缓冲区更新，则访问环形缓冲区并获取数据包，统计流经虚拟交换机的数据流量，同时更新环形缓冲区的读指针位置。与现有技术相比，本申请实施例通过用户态采集模块通过环形缓冲区直接读取采集数据包，无复制开销，无***调用开销，消耗资源极低。内核采集模块通过环形缓冲区传递采集数据包到用户态采集模块，与用户态采集模块通过环形缓冲区中的写指针位置和读指针位置的同步，降低***开销。在流量采集过程中，监控对象的最小粒度为虚拟交换机，在不侵入虚拟机保证虚拟网络安全和性能的情况下监控虚拟机和网络性能。This application provides a virtual network traffic collection method. First, initialize the software operating environment for traffic collection, then establish a communication channel between the user mode collection module and the kernel collection module, and then the user mode collection module applies for a large page of memory. Create a ring buffer area on the large page memory, and transmit the physical address of the large page memory to the kernel acquisition module through the communication channel, and then the kernel acquisition module maps the physical address of the large page memory to a linear address in the kernel address space , And obtain the buffer address of the ring buffer in the kernel acquisition module, and then the kernel acquisition module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer of the ring buffer Finally, if the user mode acquisition module detects the ring buffer update, it accesses the ring buffer and obtains data packets, counts the data traffic flowing through the virtual switch, and updates the read pointer position of the ring buffer at the same time. Compared with the prior art, the embodiment of the present application directly reads the collected data packet through the ring buffer through the user mode collection module, there is no copying overhead, no system call overhead, and very low resource consumption. The core acquisition module transfers the acquisition data packet to the user mode acquisition module through the ring buffer, and synchronizes with the user mode acquisition module through the write pointer position and the read pointer position in the ring buffer to reduce system overhead. In the process of traffic collection, the smallest granularity of the monitoring object is the virtual switch, which monitors the virtual machine and network performance without intruding the virtual machine to ensure the security and performance of the virtual network.

本申请实施例提供了另一种虚拟网络的流量采集方法，如图2所示，该方法包括：The embodiment of the present application provides another method for collecting traffic on a virtual network. As shown in FIG. 2, the method includes:

201、初始化用户态采集模块和内核采集模块的流量采集的软件运行环境。201. Initialize the flow collection software operating environment of the user mode collection module and the kernel collection module.

202、建立通信通道。202. Establish a communication channel.

通信通道是所述用户态采集模块和所述内核采集模块之间的数据传输通道。建立通信通道，也就是建立用户态采集模块和内核采集模块之间的逻辑通路。通信通道可以为Netlink通道，管道通道，或Socket通道。其中Netlink通道是一种特殊的Socket通道，其两种通道的建立方式类似。The communication channel is a data transmission channel between the user state acquisition module and the core acquisition module. Establish a communication channel, that is, establish a logical path between the user mode acquisition module and the core acquisition module. The communication channel can be a Netlink channel, a pipe channel, or a Socket channel. Among them, the Netlink channel is a special Socket channel, and the establishment of the two channels is similar.

如果通信通道是Netlink通道，那么建立通信通道，包括：根据套接字创建函数，创建套接字；设置所述套接字的本地地址和目的地址，所述本地地址为所述用户态采集模块的物理地址，所述目的地址为所述内核采集模块的物理地址；如果第一监听函数监听到所述套接字接收到连接请求，则建立所述通信通道。If the communication channel is a Netlink channel, then establishing the communication channel includes: creating a socket according to the socket creation function; setting the local address and destination address of the socket, and the local address is the user mode acquisition module The destination address is the physical address of the core collection module; if the first monitoring function detects that the socket receives a connection request, the communication channel is established.

Netlink套接字是用以实现用户进程与内核进程通信的一种特殊的进程间通信(IPC) ，也是网络应用程序与内核通信的最常用的接口。通过socket()，bind()，connect()，listen()，sendmsg()，recvmsg() 和 close()很容易地应用到netlink通道。netlink包含于头文件linux/netlink.h中。其中socket()用于创建套接字。bind()用于指定本地地址。一个套接字用socket()创建后，它其实还没有与任何特定的本地或目的地址相关联。一旦创建了一个套接字，就必须使用bind()***调用为套接字建立一个本地地址。connect()用于将套接字连接到目的地址。listen()用于设置等待连接状态。对于一个服务器的程序，当申请到套接字，并调用bind()与本地地址绑定后，就应该等待某个客户机的程序来要求连接。listen()就是把一个套接字设置为这种状态的函数。sendmsg()和recvmsg() 用于发送和接收数据。在数据流方式中，一个连接建立以后，或者在数据报方式下，调用了connect()进行了套接字与目的地址的绑定后，就可以调用sendmsg()和recvmsg() 函数进行数据传输。closesocket()用于关闭套接字。Netlink socket is a special inter-process communication (IPC) used to realize the communication between the user process and the kernel process, and it is also the most commonly used interface for communication between network applications and the kernel. It is easily applied to netlink channels through socket(), bind(), connect(), listen(), sendmsg(), recvmsg() and close(). netlink is included in the header file linux/netlink.h. The socket() is used to create a socket. bind() is used to specify the local address. After a socket is created with socket(), it is not actually associated with any specific local or destination address. Once a socket is created, the bind() system call must be used to establish a local address for the socket. connect() is used to connect the socket to the destination address. listen() is used to set the status of waiting for connection. For a server program, after applying for a socket and calling bind() to bind to the local address, it should wait for a client program to request a connection. listen() is a function that sets a socket to this state. sendmsg() and recvmsg() are used to send and receive data. In the data stream mode, after a connection is established, or in the datagram mode, after calling connect() to bind the socket and the destination address, you can call sendmsg() and recvmsg() functions for data transmission . closesocket() is used to close the socket.

如果通信通道是管道通道，那么建立通信通道，包括：在所述用户态采集模块，根据创建函数建立所述管道通道；如果第二监听函数监听到所述内核采集模块发送连接请求，则通过所述管道通道连接所述用户态采集模块和所述内核采集模块。If the communication channel is a pipeline channel, establishing the communication channel includes: in the user mode acquisition module, establishing the pipeline channel according to the creation function; if the second listening function listens to the core acquisition module to send a connection request, then through all The pipeline channel connects the user state acquisition module and the core acquisition module.

管道通道创建时，为缓冲区分配一个页面大小，管道所传送的是无格式字节流，这就要求管道的读出方和写入方必须事先约定好数据的格式。其创建过程使用管道通道所属的特定函数创建，例如创建函数CreateNamedPipe()，监听函数ConnectNamedPipe()，数据传输函数ReadFile()和WriteFile()，关闭函数CloseHandle()。When the pipeline channel is created, a page size is allocated for the buffer. The pipeline transmits an unformatted byte stream. This requires that the read and write parties of the pipeline must agree on the format of the data in advance. The creation process uses the specific functions of the pipeline channel to create, such as the creation function CreateNamedPipe(), the monitoring function ConnectNamedPipe(), the data transfer functions ReadFile() and WriteFile(), and the closing function CloseHandle().

203、用户态采集模块申请大页内存，在所述大页内存上创建环形缓存区，并将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块。203. The user mode collection module applies for a large page memory, creates a ring buffer area on the large page memory, and transmits the physical address of the large page memory to the kernel collection module through the communication channel.

在多核及NUMA架构服务器中，可以为每个NUMA在本地内存分配大页内存，为每个CPU核创建单独的环形缓冲区。在虚拟内存管理中，内核维护一个将虚拟内存地址映射到物理地址的表，对于每个页面操作，内核都需要加载相关的映射。如果内存页很小，那么需要加载的页就会很多，导致内核会加载更多的映射表，会降低性能。使用“大内存页”，所需要的页变少了。从而大大减少由内核加载的映射表的数量，提高了内核级别的性能最终有利于应用程序的性能。简而言之，通过启用“大内存页”，***具只需要处理较少的页面映射表，从而减少访问/维护它们的开销。In multi-core and NUMA architecture servers, large page memory can be allocated in local memory for each NUMA, and a separate ring buffer can be created for each CPU core. In virtual memory management, the kernel maintains a table that maps virtual memory addresses to physical addresses. For each page operation, the kernel needs to load related mappings. If the memory page is small, there will be a lot of pages that need to be loaded, causing the kernel to load more mapping tables, which will reduce performance. With "large memory pages", fewer pages are needed. Thereby greatly reducing the number of mapping tables loaded by the kernel, improving the performance of the kernel level and ultimately benefiting the performance of the application. In short, by enabling "large memory pages", the system only needs to process fewer page mapping tables, thereby reducing the overhead of accessing/maintaining them.

查找与通信通道的类型对应的数据传输函数，根据数据传输函数将大页内存的物理地址传输至内核采集模块。示例性的，Netlink套接字中的sendmsg()，管道通信中的WriteFile，以及socket中的send()和sendto()，用于发送物理地址。Find the data transfer function corresponding to the type of communication channel, and transfer the physical address of the large page memory to the kernel acquisition module according to the data transfer function. Exemplarily, sendmsg() in Netlink sockets, WriteFile in pipe communication, and send() and sendto() in sockets are used to send physical addresses.

204、内核采集模块接收所述大页内存的物理地址，获取宿主机和虚拟机之间的内存分配关系。204. The kernel collection module receives the physical address of the large page memory, and obtains the memory allocation relationship between the host machine and the virtual machine.

在用户态采集模块和内核采集模块收发数据时，可以设置应答机制，也确保物理地址的有效传输。例如，在内核采集模块接收到大页内存的物理地址后，需要给用户态采集模块反馈已接收消息，已接收消息可以携带物理地址的数据头信息，用户态模块接收到内核采集模块的反馈消息后可以校正接收信息是否正确。When the user mode acquisition module and the kernel acquisition module send and receive data, the response mechanism can be set to ensure the effective transmission of the physical address. For example, after the kernel collection module receives the physical address of the large page memory, it needs to feed back the received message to the user mode collection module. The received message can carry the data header information of the physical address, and the user mode module receives the feedback message from the kernel collection module. Then you can correct whether the received information is correct.

为了在大页内存中存储数据包，在本步骤中还需要获取宿主机和虚拟机之间的内存分配关系，其分配关系在宿主机建立虚拟机时已经形成，只需要在保存其分配关系的路径中获取即可。In order to store data packets in large-page memory, in this step, you also need to obtain the memory allocation relationship between the host and the virtual machine. The allocation relationship has been formed when the host creates the virtual machine. You only need to save the allocation relationship. Get it from the path.

205、内核采集模块根据所述内存分配关系，将所述大页内存的物理地址映射成所述内核地址空间的线性地址。205. The kernel acquisition module maps the physical address of the large page memory to the linear address of the kernel address space according to the memory allocation relationship.

206、内核采集模块根据所述线性地址、所述大页内存和所述环形缓冲区的地址分配关系，计算所述环形缓冲区的在所述内核采集模块的缓冲区地址。206. The kernel collection module calculates the buffer address of the ring buffer in the kernel collection module according to the address allocation relationship of the linear address, the large page memory, and the ring buffer.

内核采集模块根据大页内存的物理地址、环形缓冲区的物理地址，以及内核地址空间，将大页内存映射成线性地址，将环形缓冲区映射成缓冲区地址。The kernel acquisition module maps the large page memory into a linear address and the ring buffer into a buffer address according to the physical address of the large page memory, the physical address of the ring buffer, and the kernel address space.

207、内核采集模块采集虚拟交换机接收到的数据包，并根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置。207. The kernel collection module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer position of the ring buffer.

内核采集模块在采集虚拟交换机接收到的数据包时，其采样比例可以为1:1，或者1：n，采样比例根据后续流量采集的作用确定，在本申请实施例中对采样比例不做限定。内核采集模块采集虚拟交换机接收到的数据包并保存至环形缓冲区，每个数据包对应一条采集记录，占用环形缓冲区的一块固定大小的空间。在采集数据包的过程中，可以值采集数据包的包头，也可采集整个数据包。When the kernel collection module collects data packets received by the virtual switch, the sampling ratio can be 1:1 or 1:n. The sampling ratio is determined according to the subsequent traffic collection function. The sampling ratio is not limited in the embodiment of this application. . The kernel collection module collects the data packets received by the virtual switch and saves them to the ring buffer. Each data packet corresponds to a collection record and occupies a fixed-size space in the ring buffer. In the process of collecting data packets, the header of the data packet can be collected, or the entire data packet can be collected.

实际上，虚拟交换器具有多种类型，所以内核采集模块采集虚拟交换机接收到的数据包，具体包括：判断所述虚拟交换机的类型，所述类型包括Linux Bridge和OpenvSwitch；如果所述虚拟交换机的类型为Linux Bridge，则所述内核采集模块注册钩子函数，并根据所述钩子函数切入所述内核采集模块，采集所述虚拟交换机接收到的数据包；如果所述虚拟交换机的类型为OpenvSwitch，则所述内核采集模块利用流表结构ovsdatapath执行管理操作flow中的采集动作action，采集所述虚拟交换机接收到的数据包。In fact, there are many types of virtual switches, so the kernel collection module collects data packets received by the virtual switch, which specifically includes: judging the type of the virtual switch, the types include Linux Bridge and OpenvSwitch; if the virtual switch is Type is Linux Bridge, the kernel collection module registers the hook function, and switches into the kernel collection module according to the hook function to collect data packets received by the virtual switch; if the type of the virtual switch is OpenvSwitch, the kernel The collection module uses the flow table structure ovsdatapath to execute the collection action action in the management operation flow, and collect the data packets received by the virtual switch.

在保存数据包时，具体包括：获取所述写指针位置；从所述缓冲区地址中的所述写指针位置开始，在所述环形缓冲区存储所述数据包，并更新所述写指针位置。写指针是指环形缓冲区中待写入数据的位置。When saving a data packet, it specifically includes: acquiring the write pointer position; starting from the write pointer position in the buffer address, storing the data packet in the ring buffer, and updating the write pointer position . The write pointer refers to the location of the data to be written in the ring buffer.

208、用户态采集模块如果监测到所述环形缓冲区更新，则访问所述环形缓冲区并获取所述数据包，统计流经所述虚拟交换机的数据流量，同时更新所述环形缓冲区的读指针位置。208. If the user mode collection module detects that the ring buffer is updated, it accesses the ring buffer and obtains the data packet, counts the data traffic flowing through the virtual switch, and updates the reading of the ring buffer at the same time. Pointer position.

获取数据包具体包括：获取当前的所述写指针位置，并存储当前的所述写指针位置；如果所述当前的所述写指针位置与前一次存储的所述写指针位置不同，则访问并获取所述环形缓冲区中的数据包，所述数据包保存在所述环形缓冲区的所述读指针位置到当前的所述写指针位置之间。Obtaining the data packet specifically includes: obtaining the current write pointer position, and storing the current write pointer position; if the current write pointer position is different from the previously stored write pointer position, accessing and Acquire a data packet in the ring buffer, and the data packet is stored between the read pointer position of the ring buffer and the current write pointer position.

通过检测环形缓冲区中写指针的位置监测环形缓冲区是否存在更新数据包。通过读指针的位置指示尚未统计的环形缓冲区中数据包位置。通过写指针和读指针的方式，在减小***开销的前提下，保证数据包存储过程中不被覆盖，读取过程中不被重复读取，提高统计数量流量的准确性。By detecting the position of the write pointer in the ring buffer, it is monitored whether there is an update packet in the ring buffer. The position of the read pointer indicates the position of the data packet in the ring buffer that has not yet been counted. By means of writing pointers and reading pointers, under the premise of reducing system overhead, it is ensured that data packets will not be overwritten during storage and will not be read repeatedly during reading, which improves the accuracy of statistical traffic.

进一步的，作为对上述图1所示方法的实现，本申请实施例提供了一种虚拟网络的流量采集装置，如图3所示，该装置包括：Further, as an implementation of the method shown in FIG. 1, an embodiment of the present application provides a virtual network traffic collection device. As shown in FIG. 3, the device includes:

初始化单元31，用于初始化用户态采集模块和内核采集模块的流量采集的软件运行环境；The initialization unit 31 is used to initialize the flow collection software operating environment of the user mode collection module and the kernel collection module;

通道建立单元32，用于建立通信通道，所述通信通道是所述用户态采集模块和所述内核采集模块之间的数据传输通道；The channel establishment unit 32 is configured to establish a communication channel, the communication channel being a data transmission channel between the user state acquisition module and the core acquisition module;

缓存创建单元33，用于所述用户态采集模块申请大页内存，在所述大页内存上创建环形缓存区，并将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块；The cache creation unit 33 is used for the user mode acquisition module to apply for a large page memory, create a ring buffer area on the large page memory, and transmit the physical address of the large page memory to the kernel through the communication channel Acquisition module;

地址获取单元34，用于所述内核采集模块接收所述大页内存的物理地址，并将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址；The address acquiring unit 34 is configured to receive the physical address of the large page memory by the kernel acquisition module, map the physical address of the large page memory to a linear address in the kernel address space, and acquire the location of the ring buffer The buffer address of the kernel acquisition module;

数据存储单元35，用于所述内核采集模块采集虚拟交换机接收到的数据包，并根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置；The data storage unit 35 is configured to collect the data packet received by the virtual switch by the kernel collection module, save the data packet to the ring buffer according to the buffer address, and update the write of the ring buffer at the same time. Pointer position

流量统计单元36，用于所述用户态采集模块如果监测到所述环形缓冲区更新，则访问所述环形缓冲区并获取所述数据包，统计流经所述虚拟交换机的数据流量，同时更新所述环形缓冲区的读指针位置。The traffic statistics unit 36 is configured to access the ring buffer and obtain the data packet if the user mode collection module detects that the ring buffer is updated, and then count the data traffic flowing through the virtual switch, and update at the same time The position of the read pointer of the ring buffer.

本申请提供了一种虚拟网络的流量采集装置，首先初始化流量采集的软件运行环境，然后建立用户态采集模块和内核采集模块之间的通信通道，再然后用户态采集模块申请大页内存，在大页内存上创建环形缓存区，并将大页内存的物理地址通过所述通信通道传输至所述内核采集模块，再然后内核采集模块将大页内存的物理地址映射成内核地址空间的线性地址，并获取环形缓冲区在内核采集模块的缓冲区地址，然后内核采集模块采集虚拟交换机接收到的数据包，并根据缓冲区地址将数据包保存至环形缓冲区，同时更新环形缓冲区的写指针位置，最后用户态采集模块如果监测到环形缓冲区更新，则访问环形缓冲区并获取数据包，统计流经虚拟交换机的数据流量，同时更新环形缓冲区的读指针位置。与现有技术相比，本申请实施例通过用户态采集模块通过环形缓冲区直接读取采集数据包，无复制开销，无***调用开销，消耗资源极低。内核采集模块通过环形缓冲区传递采集数据包到用户态采集模块，与用户态采集模块通过环形缓冲区中的写指针位置和读指针位置的同步，降低***开销。在流量采集过程中，监控对象的最小粒度为虚拟交换机，在不侵入虚拟机保证虚拟网络安全和性能的情况下监控虚拟机和网络性能。This application provides a virtual network traffic collection device. First, initialize the software operating environment for traffic collection, then establish a communication channel between the user mode collection module and the kernel collection module, and then the user mode collection module applies for a large page of memory. Create a ring buffer area on the large page memory, and transmit the physical address of the large page memory to the kernel acquisition module through the communication channel, and then the kernel acquisition module maps the physical address of the large page memory to a linear address in the kernel address space , And obtain the buffer address of the ring buffer in the kernel acquisition module, and then the kernel acquisition module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer of the ring buffer Finally, if the user mode acquisition module detects the ring buffer update, it accesses the ring buffer and obtains data packets, counts the data traffic flowing through the virtual switch, and updates the read pointer position of the ring buffer at the same time. Compared with the prior art, the embodiment of the present application directly reads the collected data packet through the ring buffer through the user mode collection module, there is no copying overhead, no system call overhead, and very low resource consumption. The core acquisition module transfers the acquisition data packet to the user mode acquisition module through the ring buffer, and synchronizes with the user mode acquisition module through the write pointer position and the read pointer position in the ring buffer to reduce system overhead. In the process of traffic collection, the smallest granularity of the monitoring object is the virtual switch, which monitors the virtual machine and network performance without intruding the virtual machine to ensure the security and performance of the virtual network.

进一步的，作为对上述图2所示方法的实现，本申请实施例提供了另一种虚拟网络的流量采集装置，如图4所示，该装置包括：Further, as an implementation of the method shown in FIG. 2, an embodiment of the present application provides another virtual network traffic collection device. As shown in FIG. 4, the device includes:

初始化单元41，用于初始化用户态采集模块和内核采集模块的流量采集的软件运行环境；The initialization unit 41 is used to initialize the flow collection software operating environment of the user mode collection module and the kernel collection module;

通道建立单元42，用于建立通信通道，所述通信通道是所述用户态采集模块和所述内核采集模块之间的数据传输通道；The channel establishment unit 42 is configured to establish a communication channel, and the communication channel is a data transmission channel between the user state acquisition module and the kernel acquisition module;

缓存创建单元43，用于所述用户态采集模块申请大页内存，在所述大页内存上创建环形缓存区，并将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块；The cache creation unit 43 is used for the user mode acquisition module to apply for a large page memory, create a ring buffer area on the large page memory, and transmit the physical address of the large page memory to the kernel through the communication channel Acquisition module;

地址获取单元44，用于所述内核采集模块接收所述大页内存的物理地址，并将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址；The address obtaining unit 44 is configured to receive the physical address of the large page memory by the kernel acquisition module, map the physical address of the large page memory to a linear address in the kernel address space, and obtain the location of the ring buffer The buffer address of the kernel acquisition module;

数据存储单元45，用于所述内核采集模块采集虚拟交换机接收到的数据包，并根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置；The data storage unit 45 is configured to collect the data packet received by the virtual switch by the kernel collection module, save the data packet to the ring buffer according to the buffer address, and update the writing of the ring buffer at the same time. Pointer position

流量统计单元46，用于所述用户态采集模块如果监测到所述环形缓冲区更新，则访问所述环形缓冲区并获取所述数据包，统计流经所述虚拟交换机的数据流量，同时更新所述环形缓冲区的读指针位置。The traffic statistics unit 46 is configured to, if the user mode collection module detects that the ring buffer is updated, it will access the ring buffer and obtain the data packet, count the data traffic flowing through the virtual switch, and update at the same time The position of the read pointer of the ring buffer.

进一步地，所述通信通道是Netlink通道；Further, the communication channel is a Netlink channel;

所述通道建立单元42，包括：The channel establishment unit 42 includes:

创建子单元421，用于根据套接字创建函数，创建套接字；Create a subunit 421, which is used to create a socket according to a socket creation function;

设置子单元422，用于设置所述套接字的本地地址和目的地址，所述本地地址为所述用户态采集模块的物理地址，所述目的地址为所述内核采集模块的物理地址；A setting subunit 422, configured to set a local address and a destination address of the socket, where the local address is the physical address of the user mode collection module, and the destination address is the physical address of the kernel collection module;

连接子单元423，用于如果第一监听函数监听到所述套接字接收到连接请求，则建立所述通信通道。The connection subunit 423 is configured to establish the communication channel if the first monitoring function monitors that the socket receives a connection request.

进一步地，所述通信通道是管道通道；Further, the communication channel is a pipeline channel;

所述创建子单元421，还用于在所述用户态采集模块，根据创建函数建立所述管道通道；The creation subunit 421 is further configured to establish the pipeline channel according to the creation function in the user state acquisition module;

所述连接子单元423，用于如果第二监听函数监听到所述内核采集模块发送连接请求，则通过所述管道通道连接所述用户态采集模块和所述内核采集模块。The connection subunit 423 is configured to connect the user state collection module and the core collection module through the pipeline channel if the second monitoring function monitors that the core collection module sends a connection request.

进一步地，所述缓存创建单元43，包括：Further, the cache creating unit 43 includes:

查找子单元431，用于查找与所述通信通道对应的传输函数；The searching subunit 431 is configured to search for a transmission function corresponding to the communication channel;

传输子单元432，用于根据所述传输函数，将所述大页内存的物理地址传输至所述内核采集模块。The transmission subunit 432 is configured to transmit the physical address of the large page memory to the kernel collection module according to the transmission function.

进一步地，所述地址获取单元44，包括：Further, the address obtaining unit 44 includes:

获取子单元441，用于获取宿主机和虚拟机之间的内存分配关系；The obtaining subunit 441 is used to obtain the memory allocation relationship between the host machine and the virtual machine;

映射子单元442，用于根据所述内存分配关系，将所述大页内存的物理地址映射成所述内核地址空间的线性地址；The mapping subunit 442 is configured to map the physical address of the large page memory to the linear address of the kernel address space according to the memory allocation relationship;

计算子单元443，用于根据所述线性地址、所述大页内存和所述环形缓冲区的地址分配关系，计算所述环形缓冲区的在所述内核采集模块的缓冲区地址。The calculation subunit 443 is configured to calculate the buffer address of the ring buffer in the kernel acquisition module according to the address allocation relationship of the linear address, the large page memory, and the ring buffer.

进一步地，所述数据存储单元45，包括：Further, the data storage unit 45 includes:

判断子单元451，用于判断所述虚拟交换机的类型，所述类型包括Linux Bridge和OpenvSwitch；The judging subunit 451 is configured to judge the type of the virtual switch, and the type includes Linux Bridge and OpenvSwitch;

采集子单元452，用于如果所述虚拟交换机的类型为Linux Bridge，则所述内核采集模块注册钩子函数，并根据所述钩子函数切入所述内核采集模块，采集所述虚拟交换机接收到的数据包；The collection subunit 452 is configured to, if the type of the virtual switch is Linux Bridge, the kernel collection module registers a hook function, and switches to the kernel collection module according to the hook function to collect data received by the virtual switch Bag;

所述采集子单元452，用于如果所述虚拟交换机的类型为OpenvSwitch，则所述内核采集模块利用流表结构ovsdatapath执行管理操作flow中的采集动作action，采集所述虚拟交换机接收到的数据包。The collection subunit 452 is configured to, if the type of the virtual switch is OpenvSwitch, the kernel collection module uses the flow table structure ovsdatapath to execute the collection action action in the management operation flow, and collect the data packets received by the virtual switch .

获取子单元453，用于获取所述写指针位置；An obtaining subunit 453, configured to obtain the write pointer position;

存储子单元454，用于从所述缓冲区地址中的所述写指针位置开始，在所述环形缓冲区存储所述数据包，并更新所述写指针位置。The storage subunit 454 is configured to start from the write pointer position in the buffer address, store the data packet in the ring buffer, and update the write pointer position.

进一步地，所述流量统计单元46，用于：Further, the traffic statistics unit 46 is configured to:

获取当前的所述写指针位置，并存储当前的所述写指针位置；Obtain the current write pointer position, and store the current write pointer position;

如果所述当前的所述写指针位置与前一次存储的所述写指针位置不同，则访问并获取所述环形缓冲区中的数据包，所述数据包保存在所述环形缓冲区的所述读指针位置到当前的所述写指针位置之间。If the current write pointer position is different from the previously stored write pointer position, access and obtain the data packet in the ring buffer, and the data packet is stored in the ring buffer. Between the read pointer position and the current write pointer position.

进一步的，图5示出了根据本申请一个实施例提供的一种虚拟网络的流量采集装置的结构示意图，本申请具体实施例并不对虚拟网络的流量采集装置的具体实现做限定。Further, FIG. 5 shows a schematic structural diagram of a traffic collection device for a virtual network according to an embodiment of the present application. The specific embodiment of the present application does not limit the specific implementation of the traffic collection device for the virtual network.

如图5所示，该虚拟网络的流量采集装置可以包括：处理器(processor)502、通信接口(Communications Interface)504、存储器(memory)506、以及通信总线508。As shown in FIG. 5, the traffic collection device of the virtual network may include a processor 502, a communications interface 504, a memory 506, and a communications bus 508.

其中：处理器502、通信接口504、以及存储器506通过通信总线508完成相互间的通信。Among them, the processor 502, the communication interface 504, and the memory 506 communicate with each other through the communication bus 508.

通信接口504，用于与其它设备比如客户端或其它服务器等的网元通信。The communication interface 504 is used to communicate with other devices, such as network elements such as clients or other servers.

处理器502，用于执行程序510，具体可以执行上述虚拟网络的流量采集方法实施例中的相关步骤。The processor 502 is configured to execute the program 510, and specifically can execute the relevant steps in the embodiment of the above-mentioned virtual network traffic collection method.

具体地，程序510可以包括程序代码，该程序代码包括计算机操作指令。Specifically, the program 510 may include program code, and the program code includes a computer operation instruction.

处理器502可能是中央处理器CPU，或者是特定集成电路ASIC（Application Specific Integrated Circuit），或者是被配置成实施本申请实施例的一个或多个集成电路。计算机设备包括的一个或多个处理器，可以是同一类型的处理器，如一个或多个CPU；也可以是不同类型的处理器，如一个或多个CPU以及一个或多个ASIC。The processor 502 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the computer device may be the same type of processor, such as one or more CPUs, or different types of processors, such as one or more CPUs and one or more ASICs.

存储器506，用于存放程序510。存储器506可能包含高速RAM存储器，也可能还包括非易失性存储器（non-volatile memory），例如至少一个磁盘存储器。The memory 506 is used to store the program 510. The memory 506 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), for example, at least one magnetic disk memory.

程序510具体可以用于使得处理器502执行以下操作：The program 510 may be specifically used to cause the processor 502 to perform the following operations:

根据本申请一个实施例提供了一种存储介质，所述存储介质可以是易失性，也可以是非易失性，所述存储介质存储有至少一可执行指令，该计算机可执行指令可执行上述任意方法实施例中的虚拟网络的流量采集方法。According to an embodiment of the present application, a storage medium is provided. The storage medium may be volatile or non-volatile. The storage medium stores at least one executable instruction. The computer executable instruction can execute the foregoing Any method embodiment of the virtual network traffic collection method.

Claims

一种虚拟网络的流量采集方法，其中，包括：A method for collecting traffic in a virtual network, which includes:

初始化用户态采集模块和内核采集模块的流量采集的软件运行环境；Initialize the flow collection software operating environment of the user mode collection module and the kernel collection module;

建立通信通道，所述通信通道是所述用户态采集模块和所述内核采集模块之间的数据传输通道；Establishing a communication channel, the communication channel being a data transmission channel between the user mode acquisition module and the kernel acquisition module;

所述用户态采集模块申请大页内存，在所述大页内存上创建环形缓存区，并将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块；The user mode collection module applies for a large page memory, creates a ring buffer area on the large page memory, and transmits the physical address of the large page memory to the kernel collection module through the communication channel;

所述内核采集模块接收所述大页内存的物理地址，并将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址；The kernel collection module receives the physical address of the large page memory, maps the physical address of the large page memory to a linear address in the kernel address space, and obtains the buffer of the ring buffer in the kernel collection module address;

所述内核采集模块采集虚拟交换机接收到的数据包，并根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置；The kernel collection module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer position of the ring buffer;

所述用户态采集模块如果监测到所述环形缓冲区更新，则访问所述环形缓冲区并获取所述数据包，统计流经所述虚拟交换机的数据流量，同时更新所述环形缓冲区的读指针位置。If the user state acquisition module detects that the ring buffer is updated, it accesses the ring buffer and obtains the data packet, counts the data traffic flowing through the virtual switch, and updates the reading of the ring buffer at the same time. Pointer position.
如权利要求1所述的方法，其中，所述通信通道是Netlink通道；The method of claim 1, wherein the communication channel is a Netlink channel;

所述建立通信通道，包括：The establishment of a communication channel includes:

根据套接字创建函数，创建套接字；Create a socket according to the socket creation function;

设置所述套接字的本地地址和目的地址，所述本地地址为所述用户态采集模块的物理地址，所述目的地址为所述内核采集模块的物理地址；Setting a local address and a destination address of the socket, where the local address is the physical address of the user mode acquisition module, and the destination address is the physical address of the kernel acquisition module;

如果第一监听函数监听到所述套接字接收到连接请求，则建立所述通信通道。If the first monitoring function monitors that the socket receives a connection request, the communication channel is established.
如权利要求1所述的方法，其中，所述通信通道是管道通道；The method of claim 1, wherein the communication channel is a pipe channel;

所述建立通信通道，包括：The establishment of a communication channel includes:

在所述用户态采集模块，根据创建函数建立所述管道通道；In the user state acquisition module, the pipeline channel is established according to the creation function;

如果第二监听函数监听到所述内核采集模块发送连接请求，则通过所述管道通道连接所述用户态采集模块和所述内核采集模块。If the second monitoring function detects that the core collection module sends a connection request, the user mode collection module and the core collection module are connected through the pipeline channel.
如权利要求1所述的方法，其中，所述将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块，包括：The method of claim 1, wherein the transmitting the physical address of the large page memory to the kernel collection module through the communication channel comprises:

查找与所述通信通道对应的传输函数；Searching for a transmission function corresponding to the communication channel;

根据所述传输函数，将所述大页内存的物理地址传输至所述内核采集模块。According to the transfer function, the physical address of the large page memory is transferred to the kernel collection module.
如权利要求1所述的方法，其中，所述将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址，包括：The method of claim 1, wherein the mapping the physical address of the large page memory to a linear address of the kernel address space and obtaining the buffer address of the ring buffer in the kernel acquisition module comprises :

获取宿主机和虚拟机之间的内存分配关系；Obtain the memory allocation relationship between the host machine and the virtual machine;

根据所述内存分配关系，将所述大页内存的物理地址映射成所述内核地址空间的线性地址；Mapping the physical address of the large page memory to the linear address of the kernel address space according to the memory allocation relationship;

根据所述线性地址、所述大页内存和所述环形缓冲区的地址分配关系，计算所述环形缓冲区的在所述内核采集模块的缓冲区地址。According to the linear address, the address allocation relationship of the large page memory and the ring buffer, the buffer address of the ring buffer in the kernel acquisition module is calculated.
如权利要求1所述的方法，其中，所述内核采集模块采集虚拟交换机接收到的数据包，包括：The method of claim 1, wherein the core collection module collecting data packets received by the virtual switch comprises:

判断所述虚拟交换机的类型，所述类型包括Linux Bridge和OpenvSwitch；Determining the type of the virtual switch, the type including Linux Bridge and OpenvSwitch;

如果所述虚拟交换机的类型为Linux Bridge，则所述内核采集模块注册钩子函数，并根据所述钩子函数切入所述内核采集模块，采集所述虚拟交换机接收到的数据包；If the type of the virtual switch is Linux Bridge, the kernel collection module registers the hook function, switches into the kernel collection module according to the hook function, and collects the data packet received by the virtual switch;

如果所述虚拟交换机的类型为OpenvSwitch，则所述内核采集模块利用流表结构ovsdatapath执行管理操作flow中的采集动作action，采集所述虚拟交换机接收到的数据包。If the type of the virtual switch is OpenvSwitch, the kernel collection module uses the flow table structure ovsdatapath to execute the collection action action in the management operation flow, and collect the data packets received by the virtual switch.
如权利要求1所述的方法，其中，所述根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置，包括：The method according to claim 1, wherein the storing the data packet in the ring buffer according to the buffer address and simultaneously updating the write pointer position of the ring buffer comprises:

获取所述写指针位置；Acquiring the write pointer position;

从所述缓冲区地址中的所述写指针位置开始，在所述环形缓冲区存储所述数据包，并更新所述写指针位置。Starting from the write pointer position in the buffer address, store the data packet in the ring buffer and update the write pointer position.
一种虚拟网络的流量采集装置，其中，包括：A virtual network traffic collection device, which includes:

初始化单元，用于初始化用户态采集模块和内核采集模块的流量采集的软件运行环境；The initialization unit is used to initialize the flow collection software operating environment of the user mode collection module and the core collection module;

通道建立单元，用于建立通信通道，所述通信通道是所述用户态采集模块和所述内核采集模块之间的数据传输通道；A channel establishing unit, configured to establish a communication channel, the communication channel being a data transmission channel between the user mode acquisition module and the kernel acquisition module;

缓存创建单元，用于所述用户态采集模块申请大页内存，在所述大页内存上创建环形缓存区，并将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块；The cache creation unit is used for the user mode collection module to apply for a large page memory, create a ring buffer area on the large page memory, and transmit the physical address of the large page memory to the kernel collection via the communication channel Module

地址获取单元，用于所述内核采集模块接收所述大页内存的物理地址，并将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址；The address acquisition unit is configured to receive the physical address of the large page memory by the kernel acquisition module, map the physical address of the large page memory to a linear address in the kernel address space, and obtain the ring buffer in the The buffer address of the kernel acquisition module;

数据存储单元，用于所述内核采集模块采集虚拟交换机接收到的数据包，并根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置；The data storage unit is used for the kernel collection module to collect the data packet received by the virtual switch, and save the data packet to the ring buffer according to the buffer address, and at the same time update the write pointer of the ring buffer Location;

流量统计单元，用于所述用户态采集模块如果监测到所述环形缓冲区更新，则访问所述环形缓冲区并获取所述数据包，统计流经所述虚拟交换机的数据流量，同时更新所述环形缓冲区的读指针位置。The traffic statistics unit is configured to, if the user state acquisition module detects that the ring buffer is updated, it will access the ring buffer and obtain the data packets, count the data traffic flowing through the virtual switch, and update all the data at the same time. The position of the read pointer of the ring buffer.
一种存储介质，所述存储介质中存储有至少一可执行指令，所述可执行指令使处理器执行如下步骤：A storage medium storing at least one executable instruction, and the executable instruction causes a processor to execute the following steps:

初始化用户态采集模块和内核采集模块的流量采集的软件运行环境；Initialize the flow collection software operating environment of the user mode collection module and the kernel collection module;

建立通信通道，所述通信通道是所述用户态采集模块和所述内核采集模块之间的数据传输通道；Establishing a communication channel, the communication channel being a data transmission channel between the user mode acquisition module and the kernel acquisition module;

所述用户态采集模块申请大页内存，在所述大页内存上创建环形缓存区，并将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块；The user mode collection module applies for a large page memory, creates a ring buffer area on the large page memory, and transmits the physical address of the large page memory to the kernel collection module through the communication channel;

所述内核采集模块接收所述大页内存的物理地址，并将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址；The kernel collection module receives the physical address of the large page memory, maps the physical address of the large page memory to a linear address in the kernel address space, and obtains the buffer of the ring buffer in the kernel collection module address;

所述内核采集模块采集虚拟交换机接收到的数据包，并根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置；The kernel collection module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer position of the ring buffer;

所述用户态采集模块如果监测到所述环形缓冲区更新，则访问所述环形缓冲区并获取所述数据包，统计流经所述虚拟交换机的数据流量，同时更新所述环形缓冲区的读指针位置。If the user state acquisition module detects that the ring buffer is updated, it accesses the ring buffer and obtains the data packet, counts the data traffic flowing through the virtual switch, and updates the reading of the ring buffer at the same time. Pointer position.
如权利要求9所述的存储介质，其中，所述通信通道是Netlink通道；8. The storage medium of claim 9, wherein the communication channel is a Netlink channel;

所述建立通信通道，包括：The establishment of a communication channel includes:

根据套接字创建函数，创建套接字；Create a socket according to the socket creation function;

设置所述套接字的本地地址和目的地址，所述本地地址为所述用户态采集模块的物理地址，所述目的地址为所述内核采集模块的物理地址；Setting a local address and a destination address of the socket, where the local address is the physical address of the user mode acquisition module, and the destination address is the physical address of the kernel acquisition module;

如果第一监听函数监听到所述套接字接收到连接请求，则建立所述通信通道。If the first monitoring function monitors that the socket receives a connection request, the communication channel is established.
如权利要求9所述的存储介质，其中，所述通信通道是管道通道；8. The storage medium of claim 9, wherein the communication channel is a pipe channel;

所述建立通信通道，包括：The establishment of a communication channel includes:

在所述用户态采集模块，根据创建函数建立所述管道通道；In the user state acquisition module, the pipeline channel is established according to the creation function;

如果第二监听函数监听到所述内核采集模块发送连接请求，则通过所述管道通道连接所述用户态采集模块和所述内核采集模块。If the second monitoring function detects that the core collection module sends a connection request, the user mode collection module and the core collection module are connected through the pipeline channel.
如权利要求9所述的存储介质，其中，所述将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块，包括：9. The storage medium according to claim 9, wherein the transmitting the physical address of the large page memory to the kernel collection module through the communication channel comprises:

查找与所述通信通道对应的传输函数；Searching for a transmission function corresponding to the communication channel;

根据所述传输函数，将所述大页内存的物理地址传输至所述内核采集模块。According to the transfer function, the physical address of the large page memory is transferred to the kernel collection module.
如权利要求9所述的存储介质，其中，所述将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址，包括：8. The storage medium according to claim 9, wherein the mapping the physical address of the large page memory to the linear address of the kernel address space, and obtaining the buffer address of the ring buffer in the kernel acquisition module, include:

获取宿主机和虚拟机之间的内存分配关系；Obtain the memory allocation relationship between the host machine and the virtual machine;

根据所述内存分配关系，将所述大页内存的物理地址映射成所述内核地址空间的线性地址；Mapping the physical address of the large page memory to the linear address of the kernel address space according to the memory allocation relationship;

根据所述线性地址、所述大页内存和所述环形缓冲区的地址分配关系，计算所述环形缓冲区的在所述内核采集模块的缓冲区地址。According to the linear address, the address allocation relationship of the large page memory and the ring buffer, the buffer address of the ring buffer in the kernel acquisition module is calculated.
如权利要求9所述的存储介质，其中，所述内核采集模块采集虚拟交换机接收到的数据包，包括：9. The storage medium of claim 9, wherein the core collection module collecting data packets received by the virtual switch comprises:

判断所述虚拟交换机的类型，所述类型包括Linux Bridge和OpenvSwitch；Determining the type of the virtual switch, the type including Linux Bridge and OpenvSwitch;

如果所述虚拟交换机的类型为Linux Bridge，则所述内核采集模块注册钩子函数，并根据所述钩子函数切入所述内核采集模块，采集所述虚拟交换机接收到的数据包；If the type of the virtual switch is Linux Bridge, the kernel collection module registers the hook function, switches into the kernel collection module according to the hook function, and collects the data packet received by the virtual switch;

如果所述虚拟交换机的类型为OpenvSwitch，则所述内核采集模块利用流表结构ovsdatapath执行管理操作flow中的采集动作action，采集所述虚拟交换机接收到的数据包。If the type of the virtual switch is OpenvSwitch, the kernel collection module uses the flow table structure ovsdatapath to execute the collection action action in the management operation flow, and collect the data packets received by the virtual switch.
如权利要求9所述的存储介质，其中，所述根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置，包括：8. The storage medium according to claim 9, wherein the storing the data packet in the ring buffer according to the buffer address and simultaneously updating the write pointer position of the ring buffer comprises:

获取所述写指针位置；Acquiring the write pointer position;

从所述缓冲区地址中的所述写指针位置开始，在所述环形缓冲区存储所述数据包，并更新所述写指针位置。Starting from the write pointer position in the buffer address, store the data packet in the ring buffer and update the write pointer position.
一种计算机设备，包括：处理器、存储器、通信接口和通信总线，所述处理器、所述存储器和所述通信接口通过所述通信总线完成相互间的通信；A computer device includes: a processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface communicate with each other through the communication bus;

所述存储器用于存放至少一可执行指令，所述可执行指令使所述处理器执行如下步骤：The memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform the following steps:

初始化用户态采集模块和内核采集模块的流量采集的软件运行环境；Initialize the flow collection software operating environment of the user mode collection module and the kernel collection module;

建立通信通道，所述通信通道是所述用户态采集模块和所述内核采集模块之间的数据传输通道；Establishing a communication channel, the communication channel being a data transmission channel between the user mode acquisition module and the kernel acquisition module;

所述用户态采集模块申请大页内存，在所述大页内存上创建环形缓存区，并将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块；The user mode collection module applies for a large page memory, creates a ring buffer area on the large page memory, and transmits the physical address of the large page memory to the kernel collection module through the communication channel;

所述内核采集模块接收所述大页内存的物理地址，并将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址；The kernel collection module receives the physical address of the large page memory, maps the physical address of the large page memory to a linear address in the kernel address space, and obtains the buffer of the ring buffer in the kernel collection module address;

所述内核采集模块采集虚拟交换机接收到的数据包，并根据所述缓冲区地址将所述数据包保存至所述环形缓冲区，同时更新所述环形缓冲区的写指针位置；The kernel collection module collects the data packet received by the virtual switch, and saves the data packet to the ring buffer according to the buffer address, and at the same time updates the write pointer position of the ring buffer;

所述用户态采集模块如果监测到所述环形缓冲区更新，则访问所述环形缓冲区并获取所述数据包，统计流经所述虚拟交换机的数据流量，同时更新所述环形缓冲区的读指针位置。If the user state acquisition module detects that the ring buffer is updated, it accesses the ring buffer and obtains the data packet, counts the data traffic flowing through the virtual switch, and updates the reading of the ring buffer at the same time. Pointer position.
如权利要求16所述的计算机设备，其中，所述通信通道是Netlink通道；The computer device according to claim 16, wherein the communication channel is a Netlink channel;

所述建立通信通道，包括：The establishment of a communication channel includes:

根据套接字创建函数，创建套接字；Create a socket according to the socket creation function;

设置所述套接字的本地地址和目的地址，所述本地地址为所述用户态采集模块的物理地址，所述目的地址为所述内核采集模块的物理地址；Setting a local address and a destination address of the socket, where the local address is the physical address of the user mode acquisition module, and the destination address is the physical address of the kernel acquisition module;

如果第一监听函数监听到所述套接字接收到连接请求，则建立所述通信通道。If the first monitoring function monitors that the socket receives a connection request, the communication channel is established.
如权利要求16所述的计算机设备，其中，所述通信通道是管道通道；The computer device according to claim 16, wherein the communication channel is a pipe channel;

所述建立通信通道，包括：The establishment of a communication channel includes:

在所述用户态采集模块，根据创建函数建立所述管道通道；In the user state acquisition module, the pipeline channel is established according to the creation function;

如果第二监听函数监听到所述内核采集模块发送连接请求，则通过所述管道通道连接所述用户态采集模块和所述内核采集模块。If the second monitoring function detects that the core collection module sends a connection request, the user mode collection module and the core collection module are connected through the pipeline channel.
如权利要求16所述的计算机设备，其中，所述将所述大页内存的物理地址通过所述通信通道传输至所述内核采集模块，包括：The computer device according to claim 16, wherein the transmitting the physical address of the large page memory to the kernel collection module through the communication channel comprises:

查找与所述通信通道对应的传输函数；Searching for a transmission function corresponding to the communication channel;

根据所述传输函数，将所述大页内存的物理地址传输至所述内核采集模块。According to the transfer function, the physical address of the large page memory is transferred to the kernel collection module.
如权利要求16所述的计算机设备，其中，所述将所述大页内存的物理地址映射成内核地址空间的线性地址，并获取所述环形缓冲区在所述内核采集模块的缓冲区地址，包括：The computer device according to claim 16, wherein the mapping the physical address of the large page memory to the linear address of the kernel address space, and obtaining the buffer address of the ring buffer in the kernel acquisition module, include:

获取宿主机和虚拟机之间的内存分配关系；Obtain the memory allocation relationship between the host machine and the virtual machine;

根据所述内存分配关系，将所述大页内存的物理地址映射成所述内核地址空间的线性地址；Mapping the physical address of the large page memory to the linear address of the kernel address space according to the memory allocation relationship;

根据所述线性地址、所述大页内存和所述环形缓冲区的地址分配关系，计算所述环形缓冲区的在所述内核采集模块的缓冲区地址。According to the linear address, the address allocation relationship of the large page memory and the ring buffer, the buffer address of the ring buffer in the kernel acquisition module is calculated.