CN111935663B

CN111935663B - Sensor data stream processing method, device, medium and electronic equipment

Info

Publication number: CN111935663B
Application number: CN202010694064.1A
Authority: CN
Inventors: 丁小华; 严石伟; 蒋楠
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2020-07-17
Filing date: 2020-07-17
Publication date: 2022-01-25
Anticipated expiration: 2040-07-17
Also published as: CN111935663A

Abstract

The embodiment of the application provides a method, a device, a medium and an electronic device for processing a sensor data stream. The processing method comprises the following steps: receiving task information sent by a task issuing process, wherein the task information comprises identification information of a target sensor data stream and an operation task aiming at the target sensor data stream; carrying out validity check on the target sensor data stream according to the identification information of the target sensor data stream to obtain a check result; generating a processing instruction aiming at the target sensor data stream according to the operation task and the verification result; and sending the processing instruction to a production service process of the sensor data stream, wherein the processing instruction is used for instructing the production service process to process the target sensor data stream. According to the technical scheme of the embodiment of the application, the operation on the sensor data stream can be realized on the premise of not restarting the consumption service process and the production service process, the operation and maintenance cost of the system is reduced, and the management efficiency of the sensor data stream is improved.

Description

Sensor data stream processing method, device, medium and electronic equipment

Technical Field

The present application relates to the field of computer and communication technologies, and in particular, to a method, an apparatus, a medium, and an electronic device for processing a sensor data stream.

Background

With the high development of the technology of the internet of things, the method has important application significance for the efficient management of the sensor data stream. Particularly, as the number of internet of things devices in the internet of things increases and the service requirement in the internet of things is complicated, the management frequency of sensor data streams also increases. In the related art, when configuration change needs to be performed on the sensor data stream, a service process related to the sensor data stream needs to be restarted, which may cause frequent restarting of the service process, increase system operation and maintenance costs, and also cause inefficient management of the sensor data stream.

Disclosure of Invention

Embodiments of the present application provide a method, an apparatus, a medium, and an electronic device for processing a sensor data stream, so that an operation on the sensor data stream can be implemented at least to a certain extent on the premise of not restarting a consumption service process and a production service process, thereby reducing a system operation and maintenance cost and improving management efficiency of the sensor data stream.

Other features and advantages of the present application will be apparent from the following detailed description, or may be learned by practice of the application.

According to an aspect of an embodiment of the present application, there is provided a method for processing a sensor data stream, including: receiving task information sent by a task issuing process, wherein the task information comprises identification information of a target sensor data stream and an operation task aiming at the target sensor data stream; carrying out validity check on the target sensor data stream according to the identification information of the target sensor data stream to obtain a check result; generating a processing instruction aiming at the target sensor data stream according to the operation task and the verification result; and sending the processing instruction to a production service process of the sensor data stream, wherein the processing instruction is used for instructing the production service process to process the target sensor data stream.

According to an aspect of an embodiment of the present application, there is provided a method for processing a sensor data stream, including: receiving a processing instruction aiming at a target sensor data stream sent by a consumption service process, wherein the processing instruction is generated by the consumption service process according to a validity check result of the target sensor data stream and an operation task aiming at the target sensor data stream sent by a task issuing process; and processing the target sensor data stream according to the processing instruction.

In some embodiments of the present application, based on the foregoing solution, a shared memory is created between the consumption service process and a production service process of a sensor data stream, where the shared memory is used to store the sensor data stream produced by the production service process, and the consumption service process is used to consume the sensor data stream in the shared memory; the processing method of the sensor data stream further comprises the following steps: and if the queue length of the shared memory is monitored to reach a set threshold value, sending the sensor data stream which is not stored in the shared memory queue to the consumption service process through a communication line between the sensor data stream and the consumption service process.

In some embodiments of the present application, based on the foregoing solution, sending a sensor data stream that is not stored in a shared memory queue to the consumption service process through a communication line with the consumption service process, includes: and sending the sensor data streams which are not stored in the shared memory queue to the consumption service process through the communication line according to the sequence of the service priorities of the sensor data streams from high to low, wherein the service priority of the sensor data stream of the real-time service type is higher than that of the sensor data stream of the non-real-time service type.

According to an aspect of an embodiment of the present application, there is provided a processing apparatus for a sensor data stream, including: the first receiving unit is configured to receive task information sent by a task issuing process, wherein the task information comprises identification information of a target sensor data stream and an operation task aiming at the target sensor data stream; the verification unit is configured to verify the legality of the target sensor data stream according to the identification information of the target sensor data stream to obtain a verification result; a generating unit configured to generate a processing instruction for the target sensor data stream according to the operation task and the verification result; a sending unit configured to send the processing instruction to a production service process of a sensor data stream, where the processing instruction is used to instruct the production service process to process the target sensor data stream.

In some embodiments of the present application, based on the foregoing scheme, the checking unit is configured to: calling the production service process to inquire whether the target sensor data stream exists or not according to the identification information of the target sensor data stream; or

And calling the production service process to inquire whether the target sensor data stream exists or not according to the identification information of the target sensor data stream, and calling the production service process to check whether the stream state of the target sensor data stream is abnormal or not under the condition that the target sensor data stream exists.

In some embodiments of the present application, based on the foregoing scheme, the operation task includes an addition task; the generation unit is configured to: if the verification result shows that the target sensor data stream does not exist, generating a processing instruction for creating the target sensor data stream; and if the verification result shows that the target sensor data stream exists and the stream state of the target sensor data stream is abnormal, generating a processing instruction for deleting the target sensor data stream.

In some embodiments of the present application, based on the foregoing solution, a shared memory corresponding to the target sensor data stream is created between the consumption service process and the production service process of the sensor data stream; the processing device of the sensor data stream further comprises: the first control unit is configured to close the shared memory corresponding to the target sensor data stream in the consumption service process when the verification result indicates that the target sensor data stream exists and the flow state of the target sensor data stream is abnormal.

In some embodiments of the present application, based on the foregoing solution, the processing device of the sensor data stream further includes: the second control unit is configured to start a monitoring thread for the target sensor data stream after the generation unit generates the processing instruction for creating the target sensor data stream, wherein the monitoring thread is used for inquiring and monitoring the stream state of the target sensor data stream.

In some embodiments of the present application, based on the foregoing scheme, the generating unit is configured to: if the operation task comprises a deletion task, generating a processing instruction for deleting the target sensor data stream under the condition that the verification result shows that the target sensor data stream exists; if the operation task comprises a modification task, generating a processing instruction for modifying the target sensor data stream under the condition that the verification result indicates that the target sensor data stream exists; and if the operation task comprises an inquiry task, generating a processing instruction for inquiring the target sensor data stream under the condition that the verification result indicates that the target sensor data stream exists.

In some embodiments of the present application, based on the foregoing solution, a shared memory is created between the production service process and a consumption service process of the sensor data stream, where the shared memory is used to store the sensor data stream produced by the production service process, and the consumption service process is used to consume the sensor data stream in the shared memory; the processing device of the sensor data stream further comprises: and the third control unit is configured to start a corresponding consuming thread in the consuming service process to consume the sensor data stream in the shared memory according to an indication signal of the production service process, wherein the indication signal is generated by the production service process based on a queue length in the shared memory, and the queue length and the number of the consuming threads started by the indication signal are in a positive correlation relationship.

In some embodiments of the present application, based on the foregoing solution, the third control unit is configured to: registering the information of the consumption thread of the consumption service process into a thread table in the shared memory; and starting the corresponding consuming thread to consume the sensor data stream in the shared memory according to the flag bit set by the production service process for each consuming thread in the thread table, wherein the value of the flag bit is used for indicating whether the consuming thread is started or not.

In some embodiments of the present application, based on the foregoing scheme, the first receiving unit is further configured to: receiving a sensor data stream sent by the production service process through a communication line, wherein the sensor data stream sent by the communication line is sent by the production service process after the queue length of the shared memory reaches a set threshold value; the processing device of the sensor data stream further comprises: and the persistence unit is configured to perform persistence processing on the sensor data stream transmitted through the communication line.

According to an aspect of an embodiment of the present application, there is provided a processing apparatus for a sensor data stream, including: the second receiving unit is configured to receive a processing instruction aiming at a target sensor data stream, which is sent by a consumption service process, wherein the processing instruction is generated by the consumption service process according to a validity check result of the target sensor data stream and an operation task aiming at the target sensor data stream, which is sent by a task issuing process; a processing unit configured to process the target sensor data stream according to the processing instruction.

In some embodiments of the present application, based on the foregoing solution, if the processing instruction is a processing instruction for creating the target sensor data stream, the processing unit is configured to: if the processing resources occupied by the existing sensor data stream do not exceed the resource threshold, a decoding object corresponding to the target sensor data stream is created; and acquiring the target sensor data stream, decoding the acquired target sensor data stream through the decoding object, and storing data obtained by decoding into a shared memory between the consumption service process and the production service process of the sensor data stream.

In some embodiments of the present application, based on the foregoing solution, the processing unit is further configured to: and if the decoding of the acquired target sensor data stream by the decoding object fails, identifying the stream state of the target sensor data stream as an abnormal state.

In some embodiments of the present application, based on the foregoing solution, a shared memory is created between the consumption service process and a production service process of a sensor data stream, where the shared memory is used to store the sensor data stream produced by the production service process, and the consumption service process is used to consume the sensor data stream in the shared memory; the processing unit is further configured to: generating an indication signal based on the monitored queue length in the shared memory, wherein the indication signal is used for indicating that the corresponding consumption thread in the consumption service process is started to consume the sensor data stream in the shared memory, and the queue length and the number of the consumption threads started by the indication signal are in positive correlation.

In some embodiments of the present application, based on the foregoing solution, the processing unit is configured to: setting a flag bit of each consumption thread contained in a thread table in the shared memory based on the monitored queue length in the shared memory, wherein the information of the consumption thread of the consumption service process is registered in the thread table, and the value of the flag bit is used for indicating whether to start the consumption thread.

In some embodiments of the present application, based on the foregoing solution, the processing unit is further configured to: and if the queue length of the shared memory is monitored to reach a set threshold value, sending the sensor data stream which is not stored in the shared memory queue to the consumption service process through a communication line between the sensor data stream and the consumption service process.

In some embodiments of the present application, based on the foregoing solution, the processing unit is configured to: and sending the sensor data streams which are not stored in the shared memory queue to the consumption service process through the communication line according to the sequence of the service priorities of the sensor data streams from high to low, wherein the service priority of the sensor data stream of the real-time service type is higher than that of the sensor data stream of the non-real-time service type.

According to an aspect of embodiments of the present application, there is provided a computer-readable medium, on which a computer program is stored, which, when being executed by a processor, implements the method of processing a sensor data stream as described in the above embodiments.

According to an aspect of an embodiment of the present application, there is provided an electronic device including: one or more processors; a storage device for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement a method of processing a stream of sensor data as described in the embodiments above.

According to an aspect of embodiments herein, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method for processing the sensor data stream provided in the various alternative embodiments described above.

In the technical solutions provided in some embodiments of the present application, a consumption service process may receive task information sent by a task issuing process, where the task information includes identification information of a target sensor data stream and an operation task for the target sensor data stream, then the consumption service process performs validity check on the target sensor data stream according to the identification information of the target sensor data stream, and then generates a processing instruction for the target sensor data stream according to a validity check result and the operation task included in the task information, and sends the processing instruction to a production service process of the sensor data stream to instruct the production service process to process the target sensor data stream. It can be seen that in the technical solution of the embodiment of the present application, the task issuing process may directly issue task information to the consumption service process, the consumption service process sends a corresponding processing instruction to the production service process after completing the validity check, and the production service process may process the sensor data stream based on the processing instruction, so as to implement the operation on the sensor data stream on the premise of not restarting the consumption service process and the production service process, that is, implement the hot plug of the sensor, reduce the system operation and maintenance cost, and improve the management efficiency of the sensor data stream.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application. It is obvious that the drawings in the following description are only some embodiments of the application, and that for a person skilled in the art, other drawings can be derived from them without inventive effort. In the drawings:

FIG. 1 shows a system architecture diagram for processing a camera video stream;

FIG. 2 is a diagram illustrating an application scenario of the technical solution of the embodiment of the present application in smart retail;

FIG. 3 shows an application scenario diagram of the technical solution of the embodiment of the present application in smart security;

FIG. 4 shows a flow diagram of a method of processing a sensor data stream according to an embodiment of the present application;

FIG. 5 shows a flow diagram of a method of processing a sensor data stream according to an embodiment of the present application;

FIG. 6 illustrates an architecture diagram of a camera data stream management system according to an embodiment of the present application;

FIG. 7 illustrates a process flow diagram for flow scheduling logic of the algorithmic service shown in FIG. 6;

FIG. 8 is a flow chart illustrating a process of the streaming service shown in FIG. 6;

FIG. 9 is a flow diagram illustrating the logic of the algorithm service shown in FIG. 6 in conjunction with the streaming service to perform blocking processing;

FIG. 10 is a flowchart illustrating a process for a shared memory based established signal synchronization mechanism according to an embodiment of the present application;

FIG. 11 shows a block diagram of a processing device of a sensor data stream according to an embodiment of the present application;

FIG. 12 shows a block diagram of a processing device of a sensor data stream according to an embodiment of the present application;

FIG. 13 illustrates a schematic structural diagram of a computer system suitable for use in implementing the electronic device of an embodiment of the present application.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the subject matter of the present application can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known methods, devices, implementations, or operations have not been shown or described in detail to avoid obscuring aspects of the application.

The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.

The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.

It should be noted that: reference herein to "a plurality" means two or more. "and/or" describe the association relationship of the associated objects, meaning that there may be three relationships, e.g., A and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.

The Internet of Things (The Internet of Things, referred to as IOT for short) means that various devices and technologies such as various information sensors, radio frequency identification technologies, global positioning systems, infrared sensors, laser scanners and The like are used to realize ubiquitous connection of objects and people, and intelligent sensing, identification and management of objects and processes are realized. The internet of things is an information bearer based on the internet, a traditional telecommunication network and the like, and all common physical objects which can be independently addressed form an interconnected network.

The camera is a commonly used internet of things device, and after the camera collects a video stream, the collected video stream generally needs to be processed by using a Computer Vision technology (CV for short), so as to achieve the purposes of face recognition, image processing and the like. The computer vision technology is a science for researching how to make a machine "see", and in particular, it refers to that a camera and a computer are used to replace human eyes to make machine vision of identifying, tracking and measuring target, and further make image processing, so that the computer processing becomes an image more suitable for human eye observation or transmitted to an instrument for detection. As a scientific discipline, computer vision research-related theories and techniques attempt to build artificial intelligence systems that can capture information from images or multidimensional data.

Computer vision technologies generally include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content/behavior recognition, three-dimensional object reconstruction, 3D technologies, virtual reality, augmented reality, synchronous positioning, map construction, and other technologies, and also include common biometric technologies such as face recognition and fingerprint recognition.

After a camera acquires a video stream, a streaming service process (hereinafter referred to as streaming service) is usually required to acquire an original frame of the video stream, and an algorithm service process (hereinafter referred to as algorithm service) is used to process the original frame of the video stream. Specifically, after the streaming service finishes acquiring the real-time video stream original frame of the camera, the algorithm service is decoupled through shared memory communication, and the algorithm service can quickly acquire the video stream original frame from the shared memory to perform algorithm SDK (Software Development Kit) processing, such as face tracking processing based on the acquired video stream.

In a technical solution, a system architecture for processing a camera video stream is shown in fig. 1, where a streaming service and an algorithm service pull down all camera configuration information that needs to be currently managed from a configuration center service when starting up, then the streaming service initializes N decoding threads and N shared memories according to N paths of camera configuration information (in this embodiment, it is assumed that N paths of cameras need to be currently managed), a decoding thread corresponding to each camera performs decoding processing on an acquired video stream original frame, and then stores the decoded frame into a shared memory corresponding to the camera, and a KEY (KEY) of the shared memory is linearly related to a camera ID. The algorithm service is attached to the shared memory according to the camera ID, and video stream frame data from the stream taking service is acquired from the shared memory to perform algorithm flow processing.

In the system architecture shown in fig. 1, the camera configuration only functions at the moment when the streaming service and the algorithmic service are started, and during the operation of the streaming service and the algorithmic service, the streaming state of the camera video stream acquired by the streaming service is unknown. When the camera configuration needs to be added, deleted, and modified due to business requirements, decoration, and the like, the system architecture shown in fig. 1 can be implemented only by restarting the streaming service and the algorithm service to reload the camera configuration, and in this case, if the camera is frequently changed, continuously restarting the streaming service and the algorithm service will increase the system operation and maintenance cost, and also affect the normal operation of other cameras. Meanwhile, in the system Architecture shown in fig. 1, data transmission between the streaming service and the algorithmic service can only be realized through a communication mode of a shared memory, and the algorithmic service cannot acquire the state of each camera video stream, so that it is also impossible to determine whether an abnormal stream exists, and the abnormal stream generally occupies a large amount of resources such as FFMPEG (a set of open source computer programs that can be used to record, convert digital audio and video, and convert them into streams), CUDA (computer Unified Device Architecture), shared memory, and the like, and if a recovered resource is not found to be cleared in time, a large amount of invalid occupation of system resources is caused.

In addition, in the system architecture shown in fig. 1, the streaming service and the algorithm service only have a unique communication mode of the shared memory, the streaming service is used as a producer process, and the algorithm service is used as a consumer process, which may cause the situation that the shared memory buffer is full during a peak period, but based on the system architecture shown in fig. 1, this information cannot be obtained, so that the problem that the shared memory is full cannot be solved, and further, video frame data of the camera is lost.

Based on the problem of the system architecture shown in fig. 1, the embodiment of the present application proposes that the task issuing process pulls information such as camera configuration from the configuration center service, then, the task information is sent to a consumption service process (the consumption service process can also be called as an algorithm service process), after the algorithm service process judges the legality based on the task information, the production service process (the production service process can also be called as a streaming service process) can be informed to carry out operations such as adding, deleting, changing and the like of different camera video streams through GRPC (Google Remote Procedure Call) communication, so that timely online, offline and configuration updating of related cameras can be realized under the condition that the consumption service process and the production service process are not restarted, and further, hot plugging of the cameras can be realized, so that the reliable and stable operation of an online system is ensured. Meanwhile, in the embodiment of the application, the task issuing process can also issue the streaming state query task, and the consumption service process can establish the monitoring thread to poll the query streaming state, so that the operation task of deleting the streaming can be issued when the streaming state is abnormal, the video stream of the camera can be deleted in time, the invalid occupation of resources such as FFMPEG, CUDA and shared memory by the invalid video stream is avoided, and the resource utilization rate is improved. In addition, in the embodiment of the application, a synchronization mechanism based on signals can be introduced into the shared memory, when the production service process finds that the shared memory queue is too long, the production service process can synchronize the shared memory queue to the consumption service process through the signal mechanism, the consumption service process can further reduce the length of the shared memory queue through a persistent queue fast consumption scheme, and the production service process can also increase a stream taking interval so as to avoid the problem of a large number of frame loss caused by the blockage of the shared memory queue.

Fig. 2 is a diagram illustrating an application scenario of the technical solution of the embodiment of the present application in smart retail.

As shown in fig. 2, in an embodiment of the present application, a task issuing process, a consumption service process, and a production service process may be included in the management system for smart retail. Cameras are installed in each goods area of a retail site, and a management system meets retail related business requirements through acquisition and algorithm analysis of videos of the cameras. The task issuing process can pull the camera configuration information from the configuration center service and then send the task information to the consumption service process; the consumption service process is mainly used for carrying out validity check according to task information sent by the task issuing process, sending instructions such as addition, deletion, modification, query and the like to the production service process, and meanwhile, the consumption service process is also used for consuming camera data streams for algorithm analysis or other purposes; the production service process is mainly used for acquiring the camera data stream and performing actions such as adding, deleting, modifying, inquiring and the like on the camera data stream according to the instruction sent by the consumption service process.

In the application scenario shown in fig. 2, if the task information sent by the task issuing process is the camera data stream with the added number 0053, after receiving the task information, the consumption service process may check whether a corresponding camera data stream exists according to the number 0053, and if not, send an instruction for creating the camera data stream with the number 0053 to the production service process, so that the production service process may add the camera data stream with the number 053, and then the production service process puts the obtained camera data stream with the number 0053 into the shared memory for consumption by the consumption service process. In the process, the production service process and the consumption service process are kept in normal running states and do not need to be restarted.

Fig. 3 shows an application scenario diagram of the technical solution in the embodiment of the application in smart security.

As shown in fig. 3, in an embodiment of the present application, the management system for smart security may include a task issuing process, a consumption service process, and a production service process. Cameras are installed in all areas (such as all areas of an exhibition hall) of a security site, and a management system meets related security business requirements through acquisition and algorithm analysis of videos of the cameras. The task issuing process can pull the camera configuration information from the configuration center service and then send the task information to the consumption service process; the consumption service process is mainly used for carrying out validity check according to task information sent by the task issuing process, sending instructions such as addition, deletion, modification, query and the like to the production service process, and meanwhile, the consumption service process is also used for consuming camera data streams for algorithm analysis or other purposes; the production service process is mainly used for acquiring the camera data stream and performing actions such as adding, deleting, modifying, inquiring and the like on the camera data stream according to the instruction sent by the consumption service process.

In the application scenario shown in fig. 3, if the task information issued by the task issuing process is the camera data stream with the deletion number 0053, after receiving the task information, the consumption service process may check whether a corresponding camera data stream exists according to the number 0053, and if so, send an instruction to delete the camera data stream with the deletion number 0053 to the production service process, and then the production service process may delete the camera data stream with the number 053. In the process, the production service process and the consumption service process are kept in normal running states and do not need to be restarted.

It should be noted that fig. 2 and fig. 3 only schematically show service performances of the technical solution of the embodiment of the present application in a specific application scenario. The technical scheme of the embodiment of the application can be applied to intelligent retail and intelligent security, and can also be applied to intelligent communities, intelligent education, intelligent agriculture or other application scenes needing to manage the sensor data stream.

It should be further noted that the production service process and the consumption service process in the embodiment of the present application may be disposed in a server. The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud computing services. In addition, the production service process and the consumption service process can be arranged in the same server or different servers.

The sensor data stream in the embodiment of the application can be a video stream collected by a camera, a temperature data stream collected by a temperature sensor, a humidity data stream collected by a humidity sensor and the like, or can also be a data stream collected by a terminal device, such as a user pulse data stream collected by an intelligent bracelet and the like. The terminal device may be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, or the like, but is not limited thereto. The terminal and the server may be directly or indirectly connected through wired or wireless communication, and the application is not limited herein.

The implementation details of the technical solution of the embodiment of the present application are set forth in detail below:

FIG. 4 illustrates a flow diagram of a method of processing a sensor data stream, which may be performed by a computing device, which may be a server, of a consumer service process involving the sensor data stream, according to one embodiment of the present application. Referring to fig. 4, the processing method of the sensor data stream at least includes steps S410 to S440, which are described in detail as follows:

in step S410, task information sent by the task issuing process is received, where the task information includes identification information of the target sensor data stream and an operation task for the target sensor data stream.

In an embodiment of the application, the task issuing process may be a service issuing process of a service layer, the task issuing process may pull configuration information of the sensor from the configuration center service, and may send task information to the consumption service process according to service needs. The task information may be, for example, an operation task for addition, deletion, modification, query, etc. of the target sensor data stream.

In an embodiment of the application, a task information configuration interface may be displayed in a visual user interface, a user may perform instruction input or drag and configure a visual component on the configuration interface, and then a task issuing process may determine a target sensor data stream to be processed and an operation task for the target sensor data stream according to an operation configuration of the user. For example: and the user inputs a deleting instruction and the sensor data stream ID on the task information configuration interface, and then the task issuing process generates task information containing the sensor data stream ID and the deleting operation and sends the task information to the consumption service process.

In step S420, according to the identification information of the target sensor data stream, validity check is performed on the target sensor data stream to obtain a check result.

In an embodiment of the present application, the legality checking on the target sensor data stream may be to invoke a production service process to query whether the target sensor data stream already exists according to the identification information of the target sensor data stream.

In an embodiment of the present application, the legality check on the target sensor data stream may be to invoke a production service process to query whether the target sensor data stream already exists according to the identification information of the target sensor data stream, and if the target sensor data stream already exists, further invoke the production service process to check whether the stream status of the target sensor data stream is abnormal.

It should be noted that, after acquiring the target sensor data stream, the production service process may perform decoding processing on the acquired target sensor data stream, and if the decoding of the acquired target sensor data stream fails, the stream status of the target sensor data stream may be identified as an abnormal status.

In an embodiment of the present application, since the consumption service process can acquire the flow state of the sensor data stream by invoking the production service process (for example, by using GRPC communication), the consumption service process can delete the sensor data stream with the abnormal flow state in time when determining that the flow state of the sensor data stream is abnormal, so as to reduce resource occupation of an invalid sensor data stream.

In step S430, a processing instruction for the target sensor data stream is generated based on the operation task included in the verification result and the task information.

In one embodiment of the present application, the operation task for the target sensor data stream may be an addition task, in which case, if it is determined that the target sensor data stream does not exist according to the validity check result of the target sensor data stream, a processing instruction to create the target sensor data stream may be generated; if it is determined that the target sensor data stream already exists according to the validity check result of the target sensor data stream and the stream state of the target sensor data stream is abnormal, a processing instruction for deleting the target sensor data stream may be generated.

In an embodiment of the application, a shared memory corresponding to a target sensor data stream is created between a consumption service process and a production service process of the sensor data stream, and in this case, if it is determined that the target sensor data stream already exists according to a validity check result of the target sensor data stream and a stream state of the target sensor data stream is abnormal, the shared memory corresponding to the target sensor data stream in the consumption service process may be closed.

In one embodiment of the present application, after the consumption service process generates the processing instruction for creating the target sensor data stream, the generated processing instruction for creating the target sensor data stream may be sent to the production service process, so that the production service process creates the target sensor data stream. Meanwhile, the consumption service process may start a monitoring thread for the target sensor data stream, where the monitoring thread is configured to query and monitor a stream state of the target sensor data stream, for example, the monitoring thread periodically performs polling to obtain the stream state of the target sensor data stream.

In one embodiment of the present application, the operation task may also be a deletion task, and then in the case where it is determined that the target sensor data stream exists according to the validity check result of the target sensor data stream, a processing instruction for deleting the target sensor data stream may be generated.

In one embodiment of the present application, the operation task may also be a modification task, and then in case it is determined that the target sensor data stream already exists according to the validity check result of the target sensor data stream, a processing instruction for modifying the target sensor data stream may be generated.

In one embodiment of the present application, the operation task may also be a query task, and then in the case that it is determined that the target sensor data stream exists according to the validity check result of the target sensor data stream, a processing instruction for querying the target sensor data stream may be generated. The processing instruction to query the target sensor data stream may be a processing instruction to query a flow state of the target sensor data stream.

Continuing with fig. 4, in step S440, the generated processing instruction is sent to the production service process of the sensor data stream, and the processing instruction is used to instruct the production service process to process the target sensor data stream.

In one embodiment of the present application, if the processing instruction is a processing instruction to create a target sensor data stream, then the production service process may add the target sensor data stream; if the processing instruction is a delete target sensor data stream processing instruction, the production service process may delete the target sensor data stream; if the processing instruction is a processing instruction that modifies a target sensor data stream, then the production service process may modify the target sensor data stream; if the processing instruction is a processing instruction that queries a target sensor data stream, the production service process may query the target sensor data stream and may return the query result to the consumption service process.

In one embodiment of the present application, optionally, the consumption service process may send the generated processing instruction to the production service process of the sensor data stream through GRPC communication.

In one embodiment of the application, a shared memory is created between a production service process and a consumption service process of a sensor data stream, the shared memory is used for storing the sensor data stream produced by the production service process, and the consumption service process is used for consuming the sensor data stream in the shared memory. Because the production service process can sense the length of the shared memory queue, the production service process can generate an indication signal based on the queue length in the shared memory, and the consumption service process can start a corresponding consumption thread in the consumption service process to consume the sensor data stream in the shared memory according to the indication signal. Optionally, the queue length of the shared memory and the number of consumption threads started by the indication signal indicate form a positive correlation relationship, so that more consumption threads can be started to consume the sensor data stream when the queue length of the shared memory is longer, and further, the blocking of a shared memory buffer area can be reduced, and the loss of sensor data is avoided.

In an embodiment of the present application, the consumption service process may register information of the consumption threads in a thread table located in the shared memory at initialization, in this case, the production service process may set a flag bit of each consumption thread in the thread table based on a queue length in the shared memory, where a value of the flag bit is used to indicate whether to start the consumption thread, and the consumption service process may further determine whether to start a corresponding consumption thread by reading the thread table in the shared memory and according to the flag bit of each consumption thread in the thread table. For example, if the flag bit of a certain consuming thread has a value of 1, it indicates that the consuming thread is not started; if the flag bit of a consuming thread has a value of 0, it indicates that the consuming thread needs to be started.

In an embodiment of the present application, the production service process may further directly send the sensor data stream to the consumption service process through the communication line, for example, when the queue length of the shared memory reaches a set threshold (e.g., a maximum value or is about to reach the maximum value), the production service process directly sends the sensor data stream to the consumption service process through GRPC communication, and then the consumption service process may perform persistence processing on the sensor data stream sent through the communication line. Alternatively, the consuming service process may initiate a persistence thread to persist the sensor data stream sent over the communication line and store it in a database, such as a LEVELDB (a high-efficiency KV database).

In an embodiment of the application, the consumption service process may further set a lifetime for the sensor data stream, and may continuously poll whether the SDK processing is idle (for example, polling by a persistent thread), and once the SDK processing is idle, read data stored in the database, determine whether the data is expired according to the set lifetime, and if the data is not expired, start performing the SDK algorithm processing.

Fig. 4 is an illustration from the perspective of a consumption service process, and details of implementation of the technical solution of the embodiment of the present application from the perspective of a production service process are described below in conjunction with fig. 5:

FIG. 5 illustrates a flow diagram of a method of processing a sensor data stream, which may be performed by a computing device, which may be a server, that incorporates a production service process, according to one embodiment of the present application. Referring to fig. 5, the method for processing the sensor data stream at least includes steps S510 to S520, which are described in detail as follows:

in step S510, a processing instruction for the target sensor data stream sent by the consumption service process is received, where the processing instruction is generated by the consumption service process according to the validity check result of the target sensor data stream and the operation task for the target sensor data stream sent by the task issuing process.

In one embodiment of the present application, the processing instructions for the target sensor data stream may be processing instructions to create the target sensor data stream, processing instructions to delete the target sensor data stream, processing instructions to modify the target sensor data stream, processing instructions to query the target sensor data stream, and the like. The process of generating the processing instruction by the consuming service process refers to the technical solution of the foregoing embodiment, and is not described in detail.

In step S520, the target sensor data stream is processed according to the processing instruction for the target sensor data stream.

In an embodiment of the present application, if the processing instruction is a processing instruction for creating a target sensor data stream, the production service process may first determine whether a processing resource occupied by an existing sensor data stream exceeds a resource threshold, and if the processing resource does not exceed the resource threshold, may create a decoding object corresponding to the target sensor data stream; and then acquiring a target sensor data stream, decoding the acquired target sensor data stream through the decoding object, and storing data obtained by decoding into a shared memory between a consumption service process and a production service process of the sensor data stream so as to facilitate the consumption service process to acquire the sensor data stream from the shared memory for consumption.

In one embodiment of the present application, if decoding of the acquired target sensor data stream by the decoding object fails, the stream status of the target sensor data stream is identified as an abnormal status. In this case, the consumption service process may obtain the sensor data stream in the abnormal state by calling the production service process, and then may delete the sensor data stream in the abnormal state in time, so as to avoid invalid occupation of various resources by the sensor data stream in the abnormal state, and improve the resource utilization rate.

In an embodiment of the present application, the production service process may further directly send the sensor data stream to the consumption service process through the communication line, for example, when the queue length of the shared memory reaches a set threshold (e.g., a maximum value or is about to reach the maximum value), the production service process directly sends the sensor data stream to the consumption service process through GRPC communication, and then the consumption service process may perform persistence processing on the sensor data stream sent through the communication line.

In one embodiment of the present application, the production service process may send the sensor data streams that are not stored in the shared memory queue to the consumption service process through the communication line in an order from high to low in the business priority of the sensor data streams. For example, for real-time traffic types and non-real-time traffic types, the traffic priority of the sensor data stream of the real-time traffic type is higher than the traffic priority of the sensor data stream of the non-real-time traffic type.

The technical solutions of the embodiments of the present application have been described with respect to a consumption service process and a production service process, respectively. The task issuing process can directly issue task information to the consumption service process, the consumption service process sends a corresponding processing instruction to the production service process after the validity check is completed, the production service process can process the sensor data stream based on the processing instruction, and then the operation on the sensor data stream can be realized on the premise of not restarting the consumption service process and the production service process, namely, the hot plug of the sensor is realized, the system operation and maintenance cost is reduced, and the management efficiency of the sensor data stream is improved.

For the purpose of visually demonstrating the implementation process of the embodiment of the present application, in the following description of the embodiment, the sensor data stream is exemplarily used as the camera data stream, and the technical solution of the embodiment of the present application is described by taking the processing of the camera data stream as an example. It is to be understood that the following exemplary embodiments are not intended to limit the scope of the application, its function, or uses.

FIG. 6 illustrates an architecture diagram of a camera data flow management system according to one embodiment of the present application. As shown in fig. 6, the camera data stream management system includes a configuration center service 601 mainly used for storing camera information, a business layer issuing service 602 mainly used for issuing management tasks, an algorithm service 603 mainly used for consuming camera data streams (algorithm service is a consumption service process in the above embodiment), a stream fetching service 604 mainly used for acquiring camera data streams (stream fetching service is a production service process in the above embodiment), a shared memory between the algorithm service 603 and the stream fetching service 604, and a LEVELDB mainly used for storing persistent data.

In an embodiment of the present application, the service layer issuing service 602 may pull the camera position information from the configuration center service 601, and then issue task information including camera information to the algorithm service 603 according to service requirements, such as adding one camera, deleting one camera, modifying one camera, or querying one camera.

After receiving the task information issued by the service layer issuing service 602, the algorithm service 603 performs validity judgment on the camera data stream corresponding to the camera data stream identifier included in the task information by using the call flow scheduling logic, for example, judges whether a corresponding camera data stream already exists, if a corresponding camera data stream already exists, judges whether the flow state is abnormal, and then notifies the flow taking service 604 of a processing instruction determined according to the validity judgment result and the task information through GRPC communication, so that the flow taking service 604 calls the flow management logic to perform corresponding operations such as addition, deletion, modification, check and the like.

After acquiring the camera data stream, the streaming service 604 stores the camera data stream in the shared memory, and the algorithm service 603 consumes the stored data from the shared memory. During this process, the shared memory may be jammed. In this case, a signal synchronization mechanism established based on the shared memory may be used to trigger the blocking processing logic to reduce the blocking of the data queue in the shared memory. The signal synchronization mechanism established based on the shared memory may be that the streaming service 604 notifies the algorithm service 603 that the queue of the current shared memory is longer through the shared memory to trigger the algorithm service 603 to increase the consumption rate of the camera data stream, for example, a larger number of consumption threads are started to consume the camera data stream in the shared memory.

It should be noted that the embodiment is only an exemplary illustration, and should not limit the function and the scope of the disclosure.

Fig. 7 shows a process flow diagram of the flow scheduling logic of the algorithmic service shown in fig. 6, and this embodiment is described by taking the task of adding a camera data flow as an example.

As shown in fig. 7, the flow scheduling logic of the algorithm service 603 mainly includes scheduling of the flow processing module and the flow state query module, and the flow shown in fig. 7 mainly includes the following steps:

step S701, the service layer issuing service issues task information such as operation types, camera information, stream IDs and the like to the algorithm service.

Step S702, the algorithm service checks the parameter according to the task information issued by the service layer issuing service, and initializes the state of the flow scheduling flow to init; then, calling the GRPC interface of searchstream to indicate a stream fetching service to inquire whether the data stream of the camera already exists, and if so, executing step S703; if not, step S704 is performed.

Step S703, if the flow already exists, the flow state query module determines whether the flow state is normal. If the flow state is normal, a processing result of the information being processed by the flow is constructed and returned to the service layer for issuing service. If the flow state is abnormal, the shared memory resource of the algorithm service is closed, the deletesream GRPC interface is called to indicate the flow fetching service to delete the flow so as to recycle the resource, and the processing result of flow error information is constructed so as to inform the business layer that the flow exists but the state is abnormal.

Step S704, if the flow does not exist, the add camera data flow logic is executed by calling the GRPC interface addstream. If the calling is successful, the state of the stream scheduling process is set to be running through the stream processing module, namely, state is equal to run, the shared memory is connected to obtain data from the shared memory for the algorithm side SDK to carry out stream processing, and a processing result is returned to the service layer issuing logic. Meanwhile, a monitoring thread can be started to inquire the state of the stream at regular intervals, and if the state is abnormal, a deletesream interface is immediately called to delete the stream, so that resources are timely recovered.

When the deletstream interface is called to delete a stream, whether the process state of the stream scheduling process is an initial state or not is judged, if not, the algorithm side performs stream cleaning, cancels connection with the shared memory, and cancels data acquisition from the shared memory.

Fig. 8 shows a process flow diagram of the streaming service shown in fig. 6.

As shown in fig. 8, the flow adding logic 801 (i.e., addstream), the flow querying logic 802 (i.e., searchstream), and the flow deleting logic 803 (i.e., deleterteam) of the streaming service are mainly described in this embodiment, and the functions of adding a camera data stream, querying a camera data stream, and deleting a camera data stream are respectively completed.

In the flow adding logic 801, after receiving a call from the algorithmic service to the addstream interface, the streaming service determines whether the current number of streaming connections exceeds the resource limit according to resource usage of a current CPU (Central Processing Unit), a memory, a CUDA, a network, and the like. If the resource limit is exceeded, returning addition failure information to the algorithm service; if the resource limit is not exceeded, the FFMPEG decoded object is created and an execution thread is started to return information of successful addition to the algorithm service. After creating an FFMPEG decoding object, the streaming service clears up historical resources, decodes the video frame rate, video bandwidth and video height to obtain a video frame rate, video bandwidth and video height, initializes the shared memory according to the size Of the video frame, then decodes the video frame rate, Pushes (PUSH) the decoded frame into a data queue corresponding to the data stream Of the camera, and sets a stream ending state according to an End Of File (EOF) Of the stream. And in the decoding process, if the decoding fails, setting a loss state. Meanwhile, the stream fetching service can also clear the orphan stream, namely clear the stream which is not cleared for a long time and is caused by the exiting of the algorithm process.

In the flow query logic 802, after receiving a call from the algorithm service to the searchstream interface, the stream fetching service searches whether the stream ID exists in the cache stream. If the ID exists, inquiring the flow state, and further returning the flow to the algorithm service as the information of the collection task existence and the flow state; if the ID does not exist, the information that the flow does not exist is returned to the algorithm service.

In the stream deletion logic 803, after receiving the call of the deleteream interface from the algorithm service, the stream fetching service clears the relevant cache of the stream, sets a stream ending state, exits from an execution thread, destroys an FFMPEG decoding object, clears resources, and returns stream deletion information to the algorithm service.

Fig. 9 is a flow chart showing the logic of blocking processing performed by the algorithmic service in cooperation with the streaming service shown in fig. 6, including the following steps:

in step S901, the streaming service initializes the signal shared memory, which refers to a signal type shared memory.

In step S902, the algorithm service registers all the consumption thread IDs in the signal shared memory.

Step S903, the streaming service survives data in the shared memory queue, and puts the generated data frame PUSH into the shared memory queue. The consuming thread in the algorithm service thus fetches (POP) data from the shared memory queue for processing by the algorithm SDK.

Step S904, the streaming service monitors the queue length of the shared memory queue during the data production process. If the queue length is detected to be larger than the length threshold, updating a flag bit corresponding to the consumption thread ID registered in the signal shared memory, so as to send a synchronous signal to the algorithm service and inform that the queue length of the algorithm service shared memory queue is larger than the length threshold.

Step S905, the streaming service continuously determines whether the shared memory queue is full, and if it is detected that the shared memory queue is full, sends a PUSH failure frame (i.e., a frame that is not stored in the shared memory queue) to an algorithm service (preferentially sending a camera data frame of a real-time service type) through the GRPC, and increases a frame skipping interval to reduce the production speed.

For example, the streaming service extracts a data frame from a camera data frame and stores the data frame into the shared memory in a manner of extracting a video frame every 3 frames, and if a queue of the shared memory is full, a frame skipping interval may be increased, for example, a manner of extracting a video frame every 5 frames is changed.

Step S906, after receiving the synchronization signal, the algorithm service starts to block the logic processing: and starting a fast POP shared memory queue of the persistent thread and monitoring GRPC requests. For frames that are rapidly POP's and frames received via GRPC, the algorithm service determines the camera type in which the frames are located. Setting survival time for the camera data frame of the real-time service type; real-time camera frames are preferentially persisted in a persistence process.

In step S907, the persistent thread started by the algorithm service periodically queries whether the SDK processing flow is idle. And when the SDK processing flow is idle, reading a frame in the LEVELDB, if the read frame is a camera data frame of a real-time service type, judging whether the frame is expired, if so, giving up the frame for the SDK processing, and if not, giving up the frame for the SDK processing.

Fig. 10 is a flowchart illustrating a processing of a shared memory based signal synchronization mechanism according to an embodiment of the present application, in which the streaming service in the embodiment shown in fig. 6 notifies the algorithm service that a queue of the current shared memory is longer through the shared memory to trigger the algorithm service to increase a consumption rate of the camera data stream.

Referring to fig. 10, the method mainly includes the following steps:

step S1001, a shared memory of the streaming service initialization signal type is fetched.

Step S1002, when the algorithm service is started, registers all the consumption thread IDs to the consumption thread ID table in the shared memory.

Step S1003, the streaming service sets the bit position corresponding to the subscript of the consumption thread ID table in the bitmap (bitmap) stored in the shared memory to 1 according to the consumption thread ID table, and the setting of 1 in this embodiment indicates that the corresponding consumption thread is not started.

In step S1004, the streaming service starts to produce data in the shared memory queue.

Step S1005, the streaming service monitors the queue length of the shared memory during the data production process, determines whether the queue length of the shared memory reaches a threshold, and if the queue length of the shared memory reaches the threshold, executes step S1006; otherwise, the step S1004 is returned to continue to produce data in the shared memory queue.

Step S1006, the streaming service scans bitmap, and extracts N consumption thread ID table subscripts according to N bits of 1. Wherein, N can be configured in advance according to requirements.

Step S1007, the stream fetching service acquires a consuming thread ID according to the consuming thread ID table, sends a signal to the algorithm service, and sets the bit position corresponding to the consuming thread ID to 0, where the setting of 0 in this embodiment indicates that the corresponding consuming thread is started. And the algorithm service can start the corresponding consumption thread to quickly POP the shared memory queue.

The technical scheme of the embodiment of the application is suitable for the current mainstream hardware platform comprising a PC (personal computer), a server and the like, and the sensor (such as a camera) hot plug function is provided, so that the sensor information can be quickly, flexibly and nondestructively updated in various application occasions such as markets, shops and the like. Meanwhile, the technical scheme of the embodiment of the application can also realize blocking processing, and ensures reliable transmission of data. In addition, due to the fact that the sensor data stream with the abnormal stream state can be deleted in time, occupation of invalid resources can be reduced, and the resource utilization rate is improved.

Embodiments of the apparatus of the present application are described below, which may be used to perform the method for processing the sensor data stream in the above-described embodiments of the present application. For details which are not disclosed in the embodiments of the apparatus of the present application, please refer to the embodiments of the sensor data stream processing method described above in the present application.

FIG. 11 illustrates a block diagram of a processing apparatus of a sensor data stream, which may be disposed within a computing device, which may be a server, of a consuming service process involving the sensor data stream, according to an embodiment of the present application.

Referring to fig. 11, a processing apparatus 1100 for sensor data stream according to an embodiment of the present application includes: a first receiving unit 1102, a checking unit 1104, a generating unit 1106 and a transmitting unit 1108.

The first receiving unit 1102 is configured to receive task information sent by a task issuing process, where the task information includes identification information of a target sensor data stream and an operation task for the target sensor data stream; the checking unit 1104 is configured to perform validity checking on the target sensor data stream according to the identification information of the target sensor data stream to obtain a checking result; the generating unit 1106 is configured to generate a processing instruction for the target sensor data stream according to the operation task and the verification result; the sending unit 1108 is configured to send the processing instruction to a production service process of the sensor data stream, where the processing instruction is used to instruct the production service process to process the target sensor data stream.

In some embodiments of the present application, based on the foregoing solution, the checking unit 1104 is configured to: calling the production service process to inquire whether the target sensor data stream exists or not according to the identification information of the target sensor data stream; or

In some embodiments of the present application, based on the foregoing scheme, the operation task includes an addition task; the generating unit 1106 is configured to: if the verification result shows that the target sensor data stream does not exist, generating a processing instruction for creating the target sensor data stream; and if the verification result shows that the target sensor data stream exists and the stream state of the target sensor data stream is abnormal, generating a processing instruction for deleting the target sensor data stream.

In some embodiments of the present application, based on the foregoing solution, a shared memory corresponding to the target sensor data stream is created between the consumption service process and the production service process of the sensor data stream; the processing device 1100 for sensor data stream further comprises: the first control unit is configured to close the shared memory corresponding to the target sensor data stream in the consumption service process when the verification result indicates that the target sensor data stream exists and the flow state of the target sensor data stream is abnormal.

In some embodiments of the present application, based on the foregoing solution, the processing device 1100 for sensor data stream further includes: the second control unit is configured to start a monitoring thread for the target sensor data stream after the generation unit generates the processing instruction for creating the target sensor data stream, wherein the monitoring thread is used for inquiring and monitoring the stream state of the target sensor data stream.

In some embodiments of the present application, based on the foregoing scheme, the generating unit 1106 is configured to: if the operation task comprises a deletion task, generating a processing instruction for deleting the target sensor data stream under the condition that the verification result shows that the target sensor data stream exists; if the operation task comprises a modification task, generating a processing instruction for modifying the target sensor data stream under the condition that the verification result indicates that the target sensor data stream exists; and if the operation task comprises an inquiry task, generating a processing instruction for inquiring the target sensor data stream under the condition that the verification result indicates that the target sensor data stream exists.

In some embodiments of the present application, based on the foregoing solution, a shared memory is created between the production service process and a consumption service process of the sensor data stream, where the shared memory is used to store the sensor data stream produced by the production service process, and the consumption service process is used to consume the sensor data stream in the shared memory; the processing device 1100 for sensor data stream further comprises: and the third control unit is configured to start a corresponding consuming thread in the consuming service process to consume the sensor data stream in the shared memory according to an indication signal of the production service process, wherein the indication signal is generated by the production service process based on a queue length in the shared memory, and the queue length and the number of the consuming threads started by the indication signal are in a positive correlation relationship.

In some embodiments of the present application, based on the foregoing scheme, the first receiving unit 1102 is further configured to: receiving a sensor data stream sent by the production service process through a communication line, wherein the sensor data stream sent by the communication line is sent by the production service process after the queue length of the shared memory reaches a set threshold value; the processing device of the sensor data stream further comprises: and the persistence unit is configured to perform persistence processing on the sensor data stream transmitted through the communication line.

FIG. 12 illustrates a block diagram of a sensor data stream processing apparatus according to an embodiment of the present application, which may be disposed in a computing device, which may be a server, of a production service process involving a sensor data stream.

Referring to fig. 12, a processing apparatus 1200 for sensor data stream according to an embodiment of the present application includes: a second receiving unit 1202 and a processing unit 1204.

The second receiving unit 1202 is configured to receive a processing instruction for a target sensor data stream sent by a consumption service process, where the processing instruction is generated by the consumption service process according to a validity check result of the target sensor data stream and an operation task for the target sensor data stream sent by a task issuing process; the processing unit 1204 is configured to process the target sensor data stream according to the processing instructions.

In some embodiments of the present application, based on the foregoing solution, if the processing instruction is a processing instruction for creating the target sensor data stream, the processing unit 1204 is configured to: if the processing resources occupied by the existing sensor data stream do not exceed the resource threshold, a decoding object corresponding to the target sensor data stream is created; and acquiring the target sensor data stream, decoding the acquired target sensor data stream through the decoding object, and storing data obtained by decoding into a shared memory between the consumption service process and the production service process of the sensor data stream.

In some embodiments of the present application, based on the foregoing solution, the processing unit 1204 is further configured to: and if the decoding of the acquired target sensor data stream by the decoding object fails, identifying the stream state of the target sensor data stream as an abnormal state.

In some embodiments of the present application, based on the foregoing solution, a shared memory is created between the consumption service process and a production service process of a sensor data stream, where the shared memory is used to store the sensor data stream produced by the production service process, and the consumption service process is used to consume the sensor data stream in the shared memory; the processing unit 1204 is further configured to: generating an indication signal based on the monitored queue length in the shared memory, wherein the indication signal is used for indicating that the corresponding consumption thread in the consumption service process is started to consume the sensor data stream in the shared memory, and the queue length and the number of the consumption threads started by the indication signal are in positive correlation.

In some embodiments of the present application, based on the foregoing solution, the processing unit 1204 is configured to: setting a flag bit of each consumption thread contained in a thread table in the shared memory based on the monitored queue length in the shared memory, wherein the information of the consumption thread of the consumption service process is registered in the thread table, and the value of the flag bit is used for indicating whether to start the consumption thread.

In some embodiments of the present application, based on the foregoing solution, the processing unit 1204 is further configured to: and if the queue length of the shared memory is monitored to reach a set threshold value, sending the sensor data stream which is not stored in the shared memory queue to the consumption service process through a communication line between the sensor data stream and the consumption service process.

In some embodiments of the present application, based on the foregoing solution, the processing unit 1204 is configured to: and sending the sensor data streams which are not stored in the shared memory queue to the consumption service process through the communication line according to the sequence of the service priorities of the sensor data streams from high to low, wherein the service priority of the sensor data stream of the real-time service type is higher than that of the sensor data stream of the non-real-time service type.

It should be noted that the computer system 1300 of the electronic device shown in fig. 13 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.

As shown in fig. 13, a computer system 1300 includes a Central Processing Unit (CPU)1301 that can perform various appropriate actions and processes, such as performing the methods described in the above embodiments, according to a program stored in a Read-Only Memory (ROM) 1302 or a program loaded from a storage portion 1308 into a Random Access Memory (RAM) 1303. In the RAM 1303, various programs and data necessary for system operation are also stored. The CPU 1301, the ROM 1302, and the RAM 1303 are connected to each other via a bus 1304. An Input/Output (I/O) interface 1305 is also connected to bus 1304.

The following components are connected to the I/O interface 1305: an input portion 1306 including a keyboard, a mouse, and the like; an output section 1307 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage portion 1308 including a hard disk and the like; and a communication section 1309 including a Network interface card such as a LAN (Local Area Network) card, a modem, or the like. The communication section 1309 performs communication processing via a network such as the internet. A drive 1310 is also connected to the I/O interface 1305 as needed. A removable medium 1311 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 1310 as necessary, so that a computer program read out therefrom is mounted into the storage portion 1308 as necessary.

In particular, according to embodiments of the application, the processes described above with reference to the flow diagrams may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising a computer program for performing the method illustrated by the flow chart. In such embodiments, the computer program may be downloaded and installed from a network via communications component 1309 and/or installed from removable media 1311. The computer program executes various functions defined in the system of the present application when executed by a Central Processing Unit (CPU) 1301.

It should be noted that the computer readable medium shown in the embodiments of the present application may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM), a flash Memory, an optical fiber, a portable Compact Disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with a computer program embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. Each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in the embodiments of the present application may be implemented by software, or may be implemented by hardware, and the described units may also be disposed in a processor. Wherein the names of the elements do not in some way constitute a limitation on the elements themselves.

As another aspect, the present application also provides a computer-readable medium, which may be contained in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by an electronic device, cause the electronic device to implement the method described in the above embodiments.

It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the application. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present application.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains.

It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. A processing method of a sensor data stream, wherein the processing method is executed by a consumption service process of the sensor data stream, a shared memory is created between the consumption service process and a production service process of the sensor data stream, the shared memory is used for storing the sensor data stream produced by the production service process, and the consumption service process is used for consuming the sensor data stream in the shared memory, and the processing method comprises the following steps:

receiving task information sent by a task issuing process, wherein the task information comprises identification information of a target sensor data stream and an operation task aiming at the target sensor data stream;

carrying out validity check on the target sensor data stream according to the identification information of the target sensor data stream to obtain a check result;

generating a processing instruction aiming at the target sensor data stream according to the operation task and the verification result;

sending the processing instruction to a production service process of a sensor data stream, wherein the processing instruction is used for instructing the production service process to process the target sensor data stream;

wherein, the processing method further comprises: according to the indication signal of the production service process, starting a corresponding consumption thread in the consumption service process to consume the sensor data stream in the shared memory, wherein the indication signal is generated by the production service process based on the queue length in the shared memory, and the queue length and the number of the consumption threads started by the indication signal are in positive correlation.

2. The method of processing sensor data stream of claim 1, wherein performing a validity check on the target sensor data stream according to the identification information of the target sensor data stream comprises:

calling the production service process to inquire whether the target sensor data stream exists or not according to the identification information of the target sensor data stream; or

3. The method of processing a sensor data stream of claim 2, wherein the operational task comprises an add task; generating a processing instruction for the target sensor data stream according to the operation task and the verification result, wherein the processing instruction comprises:

if the verification result shows that the target sensor data stream does not exist, generating a processing instruction for creating the target sensor data stream;

and if the verification result shows that the target sensor data stream exists and the stream state of the target sensor data stream is abnormal, generating a processing instruction for deleting the target sensor data stream.

4. The method for processing the sensor data stream according to claim 3, wherein a shared memory corresponding to the target sensor data stream is created between a consumption service process and the production service process of the sensor data stream;

the processing method of the sensor data stream further comprises the following steps: and if the verification result shows that the target sensor data stream exists and the stream state of the target sensor data stream is abnormal, closing the shared memory corresponding to the target sensor data stream in the consumption service process.

5. The method of processing a sensor data stream of claim 3, wherein after generating the processing instructions to create the target sensor data stream, the method further comprises:

and starting a monitoring thread aiming at the target sensor data stream, wherein the monitoring thread is used for inquiring and monitoring the stream state of the target sensor data stream.

6. The method for processing the sensor data stream according to claim 2, wherein generating a processing instruction for the target sensor data stream according to the operation task and the verification result comprises:

if the operation task comprises a deletion task, generating a processing instruction for deleting the target sensor data stream under the condition that the verification result shows that the target sensor data stream exists;

if the operation task comprises a modification task, generating a processing instruction for modifying the target sensor data stream under the condition that the verification result indicates that the target sensor data stream exists;

and if the operation task comprises an inquiry task, generating a processing instruction for inquiring the target sensor data stream under the condition that the verification result indicates that the target sensor data stream exists.

7. The method for processing the sensor data stream according to claim 1, wherein the starting of the corresponding consuming thread to consume the sensor data stream in the shared memory according to the indication signal of the production service process comprises:

registering the information of the consumption thread of the consumption service process into a thread table in the shared memory;

and starting the corresponding consuming thread to consume the sensor data stream in the shared memory according to the flag bit set by the production service process for each consuming thread in the thread table, wherein the value of the flag bit is used for indicating whether the consuming thread is started or not.

8. The method of processing a sensor data stream of claim 1, further comprising:

receiving a sensor data stream sent by the production service process through a communication line, wherein the sensor data stream sent by the communication line is sent by the production service process after the queue length of the shared memory reaches a set threshold value;

the sensor data stream transmitted via the communication line is subjected to a persistence process.

9. A processing method of a sensor data stream is characterized in that the processing method is executed by a production service process of the sensor data stream, a shared memory is created between the production service process and a consumption service process of the sensor data stream, the shared memory is used for storing the sensor data stream produced by the production service process, and the consumption service process is used for consuming the sensor data stream in the shared memory, and the processing method comprises the following steps:

receiving a processing instruction aiming at a target sensor data stream sent by a consumption service process, wherein the processing instruction is generated by the consumption service process according to a validity check result of the target sensor data stream and an operation task aiming at the target sensor data stream sent by a task issuing process;

processing the target sensor data stream according to the processing instruction;

wherein, the processing method further comprises: generating an indication signal based on the monitored queue length in the shared memory, wherein the indication signal is used for indicating that the corresponding consumption thread in the consumption service process is started to consume the sensor data stream in the shared memory, and the queue length and the number of the consumption threads started by the indication signal are in positive correlation.

10. The method for processing the sensor data stream according to claim 9, wherein if the processing instruction is a processing instruction for creating the target sensor data stream, processing the target sensor data stream according to the processing instruction includes:

if the processing resources occupied by the existing sensor data stream do not exceed the resource threshold, a decoding object corresponding to the target sensor data stream is created;

and acquiring the target sensor data stream, decoding the acquired target sensor data stream through the decoding object, and storing data obtained by decoding into a shared memory between the consumption service process and the production service process of the sensor data stream.

11. The method of processing a sensor data stream of claim 10, further comprising:

and if the decoding of the acquired target sensor data stream by the decoding object fails, identifying the stream state of the target sensor data stream as an abnormal state.

12. The method of processing sensor data streams of claim 9, wherein generating an indication signal based on the monitored queue length in the shared memory comprises:

setting a flag bit of each consumption thread contained in a thread table in the shared memory based on the monitored queue length in the shared memory, wherein the information of the consumption thread of the consumption service process is registered in the thread table, and the value of the flag bit is used for indicating whether to start the consumption thread.

13. A processing apparatus for a sensor data stream, wherein the processing apparatus is disposed in a consumption service process of the sensor data stream, a shared memory is created between the consumption service process and a production service process of the sensor data stream, the shared memory is used for storing the sensor data stream produced by the production service process, and the consumption service process is used for consuming the sensor data stream in the shared memory, and the processing apparatus comprises:

the first receiving unit is configured to receive task information sent by a task issuing process, wherein the task information comprises identification information of a target sensor data stream and an operation task aiming at the target sensor data stream;

the verification unit is configured to verify the legality of the target sensor data stream according to the identification information of the target sensor data stream to obtain a verification result;

a generating unit configured to generate a processing instruction for the target sensor data stream according to the operation task and the verification result;

a sending unit, configured to send the processing instruction to a production service process of a sensor data stream, where the processing instruction is used to instruct the production service process to process the target sensor data stream;

and the third control unit is configured to start a corresponding consuming thread in the consuming service process to consume the sensor data stream in the shared memory according to an indication signal of the production service process, wherein the indication signal is generated by the production service process based on a queue length in the shared memory, and the queue length and the number of the consuming threads started by the indication signal are in a positive correlation relationship.

14. A processing apparatus for a sensor data stream, the processing apparatus being disposed in a production service process of the sensor data stream, a shared memory being created between the production service process and a consumption service process of the sensor data stream, the shared memory being used for storing the sensor data stream produced by the production service process, and the consumption service process being used for consuming the sensor data stream in the shared memory, the processing apparatus comprising:

the second receiving unit is configured to receive a processing instruction aiming at a target sensor data stream, which is sent by a consumption service process, wherein the processing instruction is generated by the consumption service process according to a validity check result of the target sensor data stream and an operation task aiming at the target sensor data stream, which is sent by a task issuing process;

a processing unit configured to process the target sensor data stream according to the processing instruction;

wherein the processing unit is further configured to: generating an indication signal based on the monitored queue length in the shared memory, wherein the indication signal is used for indicating that the corresponding consumption thread in the consumption service process is started to consume the sensor data stream in the shared memory, and the queue length and the number of the consumption threads started by the indication signal are in positive correlation.

15. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out a method of processing a sensor data stream according to any one of claims 1 to 12.

16. An electronic device, comprising:

one or more processors;

storage means for storing one or more programs which, when executed by the one or more processors, cause the electronic device to implement the method of processing a sensor data stream of any of claims 1 to 12.