CN111108470B - Whole wall redisplay method and device for distributed splicing system and computer equipment - Google Patents
Whole wall redisplay method and device for distributed splicing system and computer equipment Download PDFInfo
- Publication number
- CN111108470B CN111108470B CN201980003346.8A CN201980003346A CN111108470B CN 111108470 B CN111108470 B CN 111108470B CN 201980003346 A CN201980003346 A CN 201980003346A CN 111108470 B CN111108470 B CN 111108470B
- Authority
- CN
- China
- Prior art keywords
- node
- spliced
- display
- picture
- wall
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1423—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display
- G06F3/1446—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display display composed of modules, e.g. video walls
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The embodiment of the invention relates to a method and a device for displaying back the whole wall of a distributed splicing system and computer equipment, wherein the method comprises the following steps: acquiring node layout information of the spliced wall; the node layout information is the node positions of output pictures formed by connecting a plurality of spliced screens; acquiring a echoing code stream of each node on the splicing wall; decoding the echo code stream to obtain a node code stream; acquiring a display area, selecting a node code stream corresponding to the display area based on the node layout information and the display area, and splicing and cutting the selected node code stream to obtain a redisplay picture; and outputting a playback picture. The display picture of the whole wall back display method of the distributed splicing system is consistent with the display area of the spliced wall; the effect of the spliced wall redisplay cannot be deteriorated due to the increase of the number of spliced screens of the spliced wall, and the high-definition redisplay and the local high-definition redisplay of the spliced wall are realized. The problem of the picture that current spliced wall shows exist can not all show, cause the display frame to make mistakes, lead to the picture quality poor that shows is solved.
Description
Technical Field
The invention relates to the technical field of spliced wall playback, in particular to a method and a device for full wall playback of a distributed splicing system and computer equipment.
Background
With the development of science and the rapid increase of social informatization, the visualization demand of information is rapidly expanded, the realization difficulty of high-end visualization is increasingly high, the information quantity displayed by a single traditional display device can not meet the requirements of customers, and the method is particularly suitable for places such as monitoring centers, command centers and dispatching centers. Therefore, the wall splicing system becomes an indispensable foundation for the visualization field.
The existing wall splicing system adopts a distributed splicing system, and the distributed splicing system consists of a series of nodes, is suitable for high concurrency, is easy to expand and the like, and is widely applied to visual products. The function of the whole wall back display is to synchronously display the current picture on the spliced wall on other operation ends or a display, so that the picture content on the spliced wall can be visually seen conveniently during operation.
Since the resolution of the tiled wall is very high, if the entire wall is displayed back, the high resolution is also displayed, and the general display device has no way to support.
At present, the common practice in the visualization industry is to take the subcode stream with low resolution according to the display signal on the spliced wall, and then display the subcode stream by reducing a certain proportion according to the overlapping and arranging sequence on the large wall. The method of splicing the walls can reduce the pressure of the display end of the whole wall back display, but the display effect is greatly reduced because the subcode stream is taken, and if the number of signals is large or the subcode stream is not supported by the signals, the decoding pressure of the display end is still too large to influence the quality effect of the display picture of the display equipment. And because the splicing wall mode is to acquire the subcode stream from the signal source end and display the subcode stream according to the arrangement sequence of the splicing walls, the subcode stream is not the playback of the splicing walls in nature, and once a fault occurs on the splicing walls, the content of the picture displayed by the playback of the whole splicing walls is easy to be inconsistent.
Therefore, in view of the above situation, how to display all the displayed pictures on the mosaic wall without being affected by the number of windows of the mosaic wall becomes an important technical problem to be solved by those skilled in the art.
Disclosure of Invention
The embodiment of the invention provides a method and a device for displaying back a whole wall of a distributed splicing system and computer equipment, which are used for solving the technical problem that the displayed picture quality is poor due to the fact that the displayed picture is wrong because the displayed pictures of the existing splicing wall cannot be completely displayed.
In order to achieve the above object, the embodiments of the present invention provide the following technical solutions:
a whole-wall display method of a distributed splicing system comprises the following steps:
s1, acquiring node layout information of a spliced wall; the node layout information is the node positions of output pictures formed by connecting a plurality of spliced screens;
s2, acquiring a echoing code stream of each node on the splicing wall;
s3, decoding the echo code stream to obtain a node code stream;
s4, acquiring a display area, selecting the node code stream corresponding to the display area based on the node layout information and the display area, and splicing and cutting the selected node code stream to obtain a back display picture;
and S5, outputting the display picture.
Preferably, the step of generating the echo code stream of each node includes:
s21, each node synchronizes time with a time synchronization server;
s22, calculating the frame taking time of the node according to the current system time of the spliced wall;
s23, capturing a display picture of the node of the current system time;
and S24, coding the display picture to obtain the echo code stream.
Preferably, in the process of generating the echo code stream of the display picture, the time for all the nodes to take frames for the first time is aligned to an integer second, then the nodes are taken frames at intervals of corresponding time, and the display picture of the nodes is captured according to the frame taking time. Wherein the corresponding time is 1000/echoing frame rate, and the echoing frame rate is based on the setting required by the mobile terminal.
Preferably, at the same frame-fetching time, all the nodes on the spliced wall capture the display frame.
Preferably, the step of selecting the node code stream at the same frame fetching time and splicing and cutting the selected node code stream includes:
s41, acquiring a picture display area on the spliced wall based on the display area, and acquiring each spliced display area and each spliced picture display area of the spliced area based on the spliced area of the spliced wall and the picture display area;
s42, carrying out zooming processing on each spliced display area to obtain a same zooming picture corresponding to the spliced picture display area in size specification;
s43, transmitting the zooming picture to the splicing area corresponding to the zooming picture to obtain the spliced echoed picture.
The invention also provides a whole-wall display device of the distributed splicing system, which comprises:
the node information acquisition unit is used for acquiring node layout information of the spliced wall; the node layout information is the node positions of output pictures formed by connecting a plurality of spliced screens;
the back-display code stream acquiring unit is used for acquiring the back-display code stream of each node on the splicing wall;
the decoding unit is used for decoding the echo code stream to obtain a node code stream;
the display unit is used for acquiring a display area, selecting the node code stream corresponding to the display area according to the node layout information and the display area, and splicing and cutting the selected node code stream to obtain a back display picture;
and the control unit is used for outputting the display picture.
Preferably, the acquisition echo code stream unit comprises a node time synchronization subunit, a calculation subunit, a collection subunit and a coding subunit;
the node time synchronization subunit is used for time synchronization of each node and the time synchronization server;
the calculating subunit is configured to calculate a frame fetching time of the node according to the current system time of the spliced wall;
the acquisition subunit is used for capturing a display picture of the node at the current system time;
and the coding subunit is used for coding the display picture to obtain the echo code stream.
Preferably, in the unit for obtaining the echo code stream, the time for all the nodes to perform frame fetching for the first time is aligned to an integer second, then the nodes are subjected to frame fetching at intervals of corresponding time, and the display frames of the nodes are grabbed according to the frame fetching time. Wherein the corresponding time is 1000/echoing frame rate, and the echoing frame rate is based on the setting required by the mobile terminal.
Preferably, the display unit splices and cuts the node code stream selected at the same frame-fetching time, and the display unit comprises an area division subunit, a zooming subunit and a splicing subunit; the display unit comprises an area dividing subunit, a zooming subunit and a splicing subunit;
the area dividing subunit is configured to obtain a picture display area on the spliced wall based on the display area, and obtain each spliced display area and each spliced picture display area of the spliced area based on the spliced area of the spliced wall and the picture display area;
the zooming subunit is used for zooming each spliced display area to obtain the same zoomed images corresponding to the spliced display areas in the size specification;
and the splicing subunit is used for splicing the pictures with the same frame taking time together according to the zooming picture and the node layout information to obtain the spliced echoed picture.
The invention also provides computer equipment, which comprises a processor and a memory;
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is used for executing the whole wall playback method of the distributed splicing system according to the instructions in the program codes.
According to the technical scheme, the embodiment of the invention has the following advantages:
1. the method for displaying the whole wall of the distributed splicing system back obtains the node layout information of the spliced wall; the node layout information is the node positions of output pictures formed by connecting a plurality of spliced screens; acquiring a echoing code stream of each node on the splicing wall; decoding the echo code stream to obtain a node code stream; acquiring a display area, selecting a node code stream corresponding to the display area based on the node layout information and the display area, and splicing and cutting the selected node code stream to obtain a redisplay picture; and outputting a playback picture. The display picture of the whole wall back display method of the distributed splicing system is consistent with the display area of the spliced wall; the redisplay effect of the spliced wall is not deteriorated due to the increase of the number of spliced screens of the spliced wall; high-definition playback and local high-definition playback of the spliced wall can be realized. The technical problem that the displayed pictures of the existing splicing wall cannot be completely displayed, so that the displayed pictures are wrong, and the displayed pictures are poor in quality is solved;
2. the whole-wall playback device of the distributed splicing system acquires node layout information of a spliced wall through a node information acquisition unit, and acquires the playback code stream of each node by adopting an acquisition playback code stream unit; the decoding unit decodes the echo code stream of each node to obtain a node code stream; and the display unit splices and cuts the node code stream according to the display area and the node layout information to obtain a spliced and cut back display picture. The display picture of the whole wall back display device of the distributed splicing system is consistent with the display area of the spliced wall; the redisplay effect of the spliced wall is not deteriorated due to the increase of the number of spliced screens of the spliced wall; high-definition playback and local high-definition playback of the spliced wall can be realized. The technical problem that the displayed pictures of the existing splicing wall cannot be completely displayed, so that the displayed pictures are wrong, and the displayed pictures are poor in quality is solved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without inventive exercise.
Fig. 1 is a flowchart illustrating steps of a method for displaying a whole wall of a distributed splicing system according to an embodiment of the present invention.
Fig. 2 is a schematic structural diagram of a spliced wall of the method for displaying back a whole wall of a distributed splicing system according to the embodiment of the invention.
Fig. 3 is a flowchart of steps of generating a playback code stream by the method for full-wall playback of a distributed splicing system according to the embodiment of the present invention.
Fig. 4 is a flowchart illustrating steps of obtaining a playback picture by the method for displaying a whole wall of a distributed splicing system according to the embodiment of the present invention.
Fig. 5 is a frame diagram of the whole wall playback device of the distributed splicing system according to the embodiment of the present invention.
Detailed Description
In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the embodiments described below are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the application provides a method, a device and computer equipment for displaying back the whole wall of a distributed splicing system. The method can also adopt a playback server to decode and splice the playback pictures of each splicing node and then directly output the decoded and spliced playback pictures to display equipment, and can also recode the playback pictures into a path of IP video to output through a network, so as to solve the technical problems that the displayed pictures of the existing splicing wall cannot be completely displayed, so that the displayed pictures are wrong, and the displayed picture quality is poor.
The first embodiment is as follows:
fig. 1 is a flowchart illustrating steps of a method for displaying a whole wall of a distributed splicing system according to an embodiment of the present invention, and fig. 2 is a schematic diagram illustrating a structure of a spliced wall of the method for displaying a whole wall of a distributed splicing system according to an embodiment of the present invention. In the present embodiment, the case of the mosaic wall composed of 18 mosaic screens in 3 rows and 6 columns is described.
As shown in fig. 1 and fig. 2, an embodiment of the present invention provides a method for displaying back a whole wall of a distributed splicing system, including the following steps:
s1, acquiring node layout information of a spliced wall; the node layout information is the node positions of output pictures formed by connecting a plurality of spliced screens;
s2, acquiring a echoing code stream of each node on the splicing wall;
s3, decoding the echo code stream to obtain a node code stream;
s4, acquiring a display area, selecting the node code stream corresponding to the display area based on the node layout information and the display area, and splicing and cutting the selected node code stream to obtain a back display picture;
and S5, outputting the display picture.
In step S1 of the embodiment of the present invention, the joined screen is composed of a plurality of joined screens, each joined screen corresponds to one node, the playback server is connected to the joined screen, the playback server obtains the node corresponding to each joined screen on the joined screen, and the position layout of each node constitutes the node layout information.
It should be noted that in this embodiment, the mosaic wall is composed of 18 mosaic screens in total in 3 rows and 6 columns, that is, 18 nodes corresponding to the 18 mosaic screens are obtained, and the display screen is output by the 18 nodes, where the screen area for each node is different.
In step S2 of the embodiment of the present invention, the playback server and the splicing wall may be connected by a wire harness, the playback server and the splicing wall may also be connected by wireless (wireless communication protocols such as wifi, internet, bluetooth, etc.), and the playback server obtains the playback code stream of each node.
It should be noted that, in this embodiment, the echo code stream corresponding to 18 nodes is obtained from the echo server, that is, there are 18 echo code streams.
In step S3 of the embodiment of the present invention, according to the echo code stream obtained in step S2, the echo server decodes the echo code stream to obtain a node code stream corresponding to the echo code stream.
It should be noted that 18 echo code streams can be decoded one by one, so as to obtain 18 node code streams corresponding to 18 nodes; or calculating according to the display area, if the echo code stream is not in the current display area, the echo code stream may not be decoded to save system resources.
In step S4 of the embodiment of the present invention, the redisplay server obtains the display area, where the display area is an area where the redisplay screen is displayed on the splicing wall, and selects the node code stream according to the node layout information on the splicing wall and the display area, so as to obtain all the node code streams corresponding to the display area, and the redisplay server performs splicing and cutting on all the node code streams, so as to obtain the redisplay screen. The display area is a picture area of the spliced wall selected during playback output, and can be the whole area of the spliced wall or a sub-area of the spliced wall. That is, the display area to be displayed on the spliced wall selected by the mobile terminal may be the area of the whole spliced wall or a sub-area of the spliced wall.
It should be noted that, as shown in fig. 2, in this embodiment, a is an area that needs to be redisplayed currently, and it can be seen from fig. 2 that the display area a relates to nine nodes, including 2, 3, 4, 8, 9, 10, 14, 15, and 16, the node code streams of the nine nodes need to be decoded, and after decoding, the node code streams are spliced according to the layout of the display area a, and after splicing, the node code streams are cut into a picture matched with the display area a as the redisplayed picture of the whole wall. The display area a is displayed on the spliced wall as a local high-definition back display, and the whole-wall high-definition back display can be considered as a case where the display area a covers the whole spliced wall. Wherein the display area A is arbitrarily zoomed and moved within the range of the spliced wall. Splicing and shearing the node code stream are based on the node code stream acquired at the same time, and the quality of the back display picture after splicing and shearing is ensured. The node layout information may be an arrangement order of nine nodes 2, 3, 4, 8, 9, 10, 14, 15, and 16 in the display area a, for example, the nodes 2, 3, and 4 constitute a row order, and the nodes 2, 8, and 14 constitute a column order.
In step S5 of the embodiment of the present invention, the redisplay server transmits the spliced and cut redisplay picture to the mobile terminal or the display device, and displays the redisplay picture on the display screen of the mobile terminal or the display device, or the redisplay server may encode the redisplay picture into a path of IP video and output the IP video via a network.
The mobile terminal and the display device may be any devices capable of displaying pictures, such as a computer, an iPad, and a mobile phone.
The whole wall back display method of the distributed splicing system provided by the invention comprises the steps of obtaining node layout information of a splicing wall; the node layout information is the node positions of output pictures formed by connecting a plurality of spliced screens; acquiring a echoing code stream of each node on the splicing wall; decoding the echo code stream to obtain a node code stream; acquiring a display area, selecting a node code stream corresponding to the display area based on the node layout information and the display area, and splicing and cutting the selected node code stream to obtain a redisplay picture; outputting a back display picture, wherein the display picture of the whole wall back display method of the distributed splicing system is consistent with the display area of the splicing wall; the redisplay effect of the spliced wall is not deteriorated due to the increase of the number of spliced screens of the spliced wall; high-definition playback and local high-definition playback of the spliced wall can be realized. The technical problem that the displayed pictures of the existing splicing wall cannot be completely displayed, so that the displayed pictures are wrong, and the displayed pictures are poor in quality is solved.
Fig. 3 is a flowchart of steps of generating a playback code stream by the method for full-wall playback of a distributed splicing system according to the embodiment of the present invention.
As shown in fig. 3, in the embodiment of the present invention, the step of generating the echo code stream of each node includes:
s21, each node synchronizes time with a time synchronization server;
s22, calculating the frame taking time of the node according to the current system time of the spliced wall;
s23, capturing a display picture of the node of the current system time;
and S24, coding the display picture to obtain the echo code stream.
It should be noted that each node generates a playback code stream of one of the spliced screens in the spliced wall, the playback server splices all of the spliced screens, and can take the entire playback code stream of the spliced wall, and if part of the spliced screens are spliced, take the local high-definition playback.
In step S21 of the embodiment of the present invention, the display time of each node on the mosaic wall is synchronized with the display time of the time-tick server, that is, the display time of each node on the mosaic wall is the same as the display time of each node.
It should be noted that, because the spliced wall is composed of a plurality of independent spliced screens, in order to ensure that the entire wall playback code stream of the spliced wall is not torn, synchronous processing needs to be performed, and time synchronization is a basis for synchronous processing, so as to ensure that the time of the nodes of the spliced screens on the spliced wall is uniform. The time synchronization server can be a GPS network time synchronization server, wherein the GPS network time synchronization server takes GPS signals as time sources, can select servers of time sources such as Beidou, CDMA and B codes, embeds internationally popular NTP/SNTP protocol, and synchronizes all computers, controllers and other devices in the network.
In step S22 of the embodiment of the present invention, the time for the node to take the frame is calculated based on the current system time of the joined screen. In the process of generating the echo code stream of the display picture, the time for all the nodes to take frames for the first time is aligned to an integer second, then the nodes are taken frames at intervals of corresponding time, and the display picture of the nodes is captured according to the frame taking time. The corresponding time is 1000/echoing frame rate, and the echoing frame rate is set based on the mobile terminal.
It should be noted that the node calculates the frame-fetching time of the node according to the current system time of the spliced wall. And calculating the frame taking of the node according to the frame taking result, wherein the frame taking result comprises a first frame and other frames because the redisplay picture is not displayed at a certain moment. In order to ensure the synchronization of the echoed pictures, all the nodes need to perform frame fetching at the same time, and the frame fetching time of the first frame can be aligned to be started when the whole number of seconds. Specifically, the alignment to the beginning of an integer second may allow the program executing the method for displaying the entire wall of the distributed splicing system to be paused for a period of time, where the length of pause (time unit is millisecond) is t:
t＝1000-x/1000
and x is the current system time of the spliced wall.
Starting from the first frame, the node is framed at corresponding time intervals, for example, every 33.3 milliseconds if a 30-frame-per-second picture is to be output. And after the first frame fetching is finished, the node is fetched into other frames. All the nodes are aligned to integer seconds according to the first frame, and then each frame is calculated according to the corresponding time of the frame rate, so that the frame taking time of each node is in one-to-one correspondence, and the quality of the display picture is ensured.
In step S23 of the embodiment of the present invention, at the time when the node performs frame fetching, the display frames of all the nodes on the mosaic wall are captured.
It should be noted that capturing the display screen of the node at the current time (i.e., the frame capturing time) needs to be performed through a hardware platform or an operating system. For example, the hua shi hai si platform has a WBC module, and a picture displayed on a device screen can be written back to a channel in a memory through the WBC module, and then the channel is encoded, so that a playback code stream is obtained.
In step S24 of the embodiment of the present invention, the display picture is encoded to obtain a playback code stream.
It should be noted that, the ratio of the captured display picture to the current display picture is generally 1:1 by default, and since the size specification of the mosaic wall is often larger than that of the display picture, if the display pictures are all displayed back according to the ratio of 1:1, the decoding workload of the back display server is large, the display picture is firstly scaled, and then is encoded to obtain the back display code stream. The zooming processing does not affect the quality of the redisplayed image after splicing of the displayable image, and the redisplayed image can still display a high-definition image.
Fig. 4 is a flowchart illustrating steps of obtaining a playback picture by the method for displaying a whole wall of a distributed splicing system according to the embodiment of the present invention.
As shown in fig. 4, the step of selecting the node code stream at the same frame fetching time, and splicing and cutting the selected node code stream includes:
s41, acquiring a picture display area on the spliced wall based on the display area, and acquiring each spliced display area and each spliced picture display area of the spliced area based on the spliced area of the spliced wall and the picture display area;
s42, carrying out zooming processing on each spliced display area to obtain a same zooming picture corresponding to the spliced picture display area in size specification;
s43, the zooming picture splices the pictures with the same frame taking time together according to the node layout information to obtain the spliced echo picture.
In the step of obtaining the cut and spliced echoed pictures, firstly, a picture display area is obtained on the spliced wall based on the display area, and each spliced display area and each spliced picture display area of the spliced area are obtained based on the spliced area of the spliced wall and the picture display area, that is, the spliced area is composed of a plurality of spliced screens. The picture display area is composed of a plurality of spliced display areas. Secondly, zooming each spliced display area to obtain the same zoomed images corresponding to the display area size specification of the spliced images, and finally splicing the images with the same frame taking time together according to the node layout information to obtain the spliced echoed images.
As shown in fig. 2, the display area a and the picture display area are areas overlapping with the display area a on the mosaic wall. The splicing area is formed by splicing each splicing screen. And obtaining each spliced display area and each spliced picture display area on the spliced area corresponding to the picture display area. Specifically, in the present embodiment, the screen display area is formed by seamlessly splicing nine tiled screen display areas, each of the tiled screen display areas has two coordinates, one is an image area coordinate, i.e., a coordinate of the tiled screen display area in the node screen area of the tiled wall, and the other is a display coordinate (x1, y12, w1, h1), i.e., a coordinate (x2, y2, w2, h2) of the tiled display area in the display area a. One step of displaying the display picture in the spliced picture display area is as follows: the first step is to set up a space for storing image pictures, and the second step is to zoom the image pictures of the display area of the spliced picture with (x1, y12, w1 and h1) coordinates to w2 x h2 to obtain zoomed image pictures; the third step is to transfer the scaled image frames onto the tiled display area of (x2, y2, w2, h2) coordinates. And repeating the second step and the third step, and processing the image pictures on the nine spliced picture display areas to obtain the spliced echoed pictures.
Example two:
fig. 5 is a frame diagram of the whole wall playback device of the distributed splicing system according to the embodiment of the present invention.
As shown in fig. 5, an embodiment of the present invention provides a full-wall display device of a distributed splicing system, including:
a node information obtaining unit 10, configured to obtain node layout information of a spliced wall; the node layout information is the node positions of output pictures formed by connecting a plurality of spliced screens;
an echo code stream obtaining unit 20, configured to obtain an echo code stream of each node on the spliced wall;
the decoding unit 30 is configured to decode the echo code stream to obtain a node code stream;
the display unit 40 is configured to acquire a display area, select the node code stream corresponding to the display area according to the node layout information and the display area, and splice and cut the selected node code stream to obtain a back display picture;
and a control unit 50 for outputting the display screen.
In the embodiment of the present invention, the node layout information acquired by the node information acquiring unit 10 is a composition of each mosaic of the mosaic wall.
It should be noted that the spliced wall is composed of a plurality of spliced screens, each spliced screen corresponds to one node, the playback server is connected with the spliced wall, the playback server acquires the node corresponding to each spliced screen on the spliced wall, and the playback server obtains the node layout information of the spliced wall. In this embodiment, the mosaic wall is composed of 18 mosaic screens in 3 rows and 6 columns, that is, 18 nodes corresponding to the 18 mosaic screens are obtained, and the display screen is output by the 18 nodes, where the screen area of each node is different.
In the embodiment of the present invention, the unit 20 for obtaining the echo code stream is connected to the splicing wall through the echo server, and the echo server obtains the echo code streams of all nodes on the splicing wall based on the node layout information.
It should be noted that, in this embodiment, the echo code stream corresponding to 18 nodes is obtained from the echo server, that is, there are 18 echo code streams.
In this embodiment of the present invention, the decoding unit 30 decodes the echo code stream through the echo server to obtain a node code stream corresponding to the echo code stream. And the echoing server can decode the 18 echoing code streams one by one to obtain 18 node code streams corresponding to the 18 nodes.
In the embodiment of the present invention, the display unit 40 obtains the display area, the node layout information on the splicing wall, and the display area, the redisplay server selects the node code stream to obtain all the node code streams corresponding to the display area, and the redisplay server performs splicing and cutting on all the node code streams to obtain the redisplay picture. And the display area refers to an area of the redisplay picture displayed on the spliced wall.
It should be noted that, as shown in fig. 2, in this embodiment, a is an area that needs to be redisplayed currently, and it can be seen from fig. 2 that the display area a relates to nine nodes, including 2, 3, 4, 8, 9, 10, 14, 15, and 16, the node code streams of the nine nodes need to be decoded, and after decoding, the node code streams are spliced according to the layout of the display area a, and after splicing, the node code streams are cut into a picture matched with the display area a as the redisplayed picture of the whole wall. The display area a is displayed on the spliced wall as a local high-definition back display, and the whole-wall high-definition back display can be regarded as a case that the display area a covers the spliced wall. Wherein the display area A is arbitrarily zoomed and moved within the range of the spliced wall. The display area is a picture area of the spliced wall selected during playback output, and can be the whole area of the spliced wall or a sub-area of the spliced wall. That is, the display area to be displayed on the spliced wall selected by the mobile terminal may be the area of the whole spliced wall or a sub-area of the spliced wall.
In the embodiment of the present invention, the control unit 50 controls the playback server to transmit the spliced and cut display picture to the mobile terminal or the display device to implement picture display, and may also control the playback server to encode the playback picture into a path of IP video and output the IP video through a network.
The mobile terminal and the display device may be any devices capable of displaying pictures, such as a computer, an iPad, and a mobile phone.
The whole wall back display device of the distributed splicing system obtains the node layout information of the spliced wall through the node information obtaining unit, and obtains the back display code stream of each node by adopting the back display code stream obtaining unit; the decoding unit decodes the echo code stream of each node to obtain a node code stream; and the display unit splices and cuts the node code stream according to the display area and the node layout information to obtain a spliced and cut back display picture. The display picture of the whole wall back display device of the distributed splicing system is consistent with the display area of the spliced wall; the redisplay effect of the spliced wall is not deteriorated due to the increase of the number of spliced screens of the spliced wall; high-definition playback and local high-definition playback of the spliced wall can be realized. The technical problem that the displayed pictures of the existing splicing wall cannot be completely displayed, so that the displayed pictures are wrong, and the displayed pictures are poor in quality is solved.
In the embodiment of the present invention, the echo code stream acquiring unit 20 includes a node time synchronization subunit 21, a calculation subunit 22, a collection subunit 23, and a coding subunit 24;
the node time synchronization subunit 21 is configured to time-pair each node with a time-pair server;
the computation subunit 22 is configured to compute a frame taking time of the node according to the current system time of the spliced wall;
the acquisition subunit 23 is configured to capture a display picture of the node at the current system time;
the coding subunit 24 is configured to code the display picture to obtain the echo code stream.
In this embodiment of the present invention, the time synchronization in the node time synchronization subunit 21 means that the display time of each node on the mosaic wall is synchronized with the display time of the time synchronization server, that is, the display time of each node on the mosaic wall is the same as the display time of the time synchronization server.
It should be noted that, because the spliced wall is composed of a plurality of independent spliced screens, in order to ensure that the entire wall playback code stream of the spliced wall is not torn, synchronous processing needs to be performed, and time synchronization is a basis for synchronous processing, so as to ensure that the time of the nodes of the spliced screens on the spliced wall is uniform.
In the embodiment of the present invention, the calculation of the time for the node to take the frame by the calculation subunit 22 is performed based on the current system time of the joined wall. In the process of generating the echo code stream of the display picture, the time for all the nodes to take frames for the first time is aligned to an integer second, then the nodes are taken frames at intervals of corresponding time, and the display picture of the nodes is captured according to the frame taking time. The corresponding time is 1000/echoing frame rate, and the echoing frame rate is set based on the mobile terminal.
It should be noted that the node calculates the frame-fetching time of the node according to the current system time of the spliced wall. And calculating the frame taking of the node according to the frame taking result, wherein the frame taking result comprises a first frame and other frames because the redisplay picture is not displayed at a certain moment. In order to ensure the synchronization of the echoed pictures, all the nodes need to perform frame fetching at the same time, and the frame fetching time of the first frame can be aligned to be started when the whole number of seconds. Specifically, the alignment to the beginning of an integer second may allow the program executing the method for displaying the entire wall of the distributed splicing system to be paused for a period of time, where the length of pause (time unit is millisecond) is t:
t＝1000-x/1000
and x is the current system time of the spliced wall.
Starting from the first frame, the node is framed at corresponding time intervals, for example, every 33.3 milliseconds if a 30-frame-per-second picture is to be output. And after the first frame fetching is finished, the node is fetched into other frames. All the nodes are aligned to integer seconds according to the first frame, and then each frame is calculated according to the corresponding time of the frame rate, so that the frame taking time of each node is in one-to-one correspondence, and the quality of the display picture is ensured.
In the embodiment of the present invention, the display frames of all the nodes on the mosaic wall are captured at the time when the node performs frame fetching in the collecting subunit 23. Capturing a display picture of the node at the current time (i.e. the frame capturing time), wherein the capturing of the display picture of the node needs to be performed through a hardware platform or an operating system. For example, the hua shi hai si platform has a WBC module, and a picture displayed on a device screen can be written back to a channel in a memory through the WBC module, and then the channel is encoded, so that a playback code stream is obtained.
In the embodiment of the present invention, the encoding subunit 24 encodes the display picture captured in the capturing subunit 23. The captured display picture is generally in a default ratio of 1:1 to the current display picture, because the size specification of the spliced wall is often larger than that of the display picture, if the display pictures are displayed back according to the ratio of 1:1, the decoding workload of the back display server is large, the display picture is firstly zoomed, and then the back display code stream is obtained by encoding. The zooming processing does not affect the quality of the redisplayed image after splicing of the displayable image, and the redisplayed image can still display a high-definition image.
In the embodiment of the present invention, the display unit 40 includes an area dividing subunit 41, a scaling subunit 42, and a splicing subunit 43;
the area dividing subunit 41 is configured to obtain a picture display area on the spliced wall based on the display area, and obtain each spliced display area and each spliced picture display area of the spliced area based on the spliced area of the spliced wall and the picture display area;
the zooming subunit 42 is configured to perform zooming processing on each of the tiled display areas to obtain a zoomed image with the same size specification of the tiled display area;
and the splicing subunit 43 is configured to splice the frames with the same frame fetching time together according to the zoom frame and the node layout information, so as to obtain the spliced playback frame.
In this embodiment of the present invention, in the display unit 40 according to this embodiment of the present invention, first, the area dividing subunit 41 obtains the picture display area, and based on the splicing area of the splicing wall and the picture display area, that is, the splicing area is composed of a plurality of splicing screens, each splicing display area and each splicing picture display area of the splicing area are obtained. The picture display area is composed of a plurality of spliced display areas. Secondly, zooming each spliced display area through the zooming subunit 42 to obtain a same zoomed picture corresponding to the spliced picture display area in size specification; finally, the frames with the same frame-taking time are spliced together according to the node layout information by the zoom frame of the splicing subunit 43, so as to obtain the spliced echo frame.
As shown in fig. 2, the display area a and the picture display area are areas overlapping with the display area a on the mosaic wall. The splicing area is formed by splicing each splicing screen. And obtaining each spliced display area and each spliced picture display area on the spliced area corresponding to the picture display area. Specifically, in the present embodiment, the screen display area is formed by seamlessly splicing nine tiled screen display areas, each of the tiled screen display areas has two coordinates, one is an image area coordinate, i.e., a coordinate of the tiled screen display area in the node screen area of the tiled wall, and the other is a display coordinate (x1, y12, w1, h1), i.e., a coordinate (x2, y2, w2, h2) of the tiled display area in the display area a. One step of displaying the display picture in the spliced picture display area is as follows: the first step is to set up a space for storing image pictures, and the second step is to zoom the image pictures of the display area of the spliced picture with (x1, y12, w1 and h1) coordinates to w2 x h2 to obtain zoomed image pictures; the third step is to transfer the scaled image frames onto the tiled display area of (x2, y2, w2, h2) coordinates. And repeating the second step and the third step, and processing the image pictures on the nine spliced picture display areas to obtain the spliced echoed pictures.
Example three:
the embodiment of the invention provides computer equipment, which comprises a processor and a memory;
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is used for executing the whole wall playback method of the distributed splicing system according to instructions in the program codes.
It should be noted that the processor is configured to execute the steps in the embodiment of the whole wall display method of the distributed splicing system according to the instructions in the program code, such as the steps S1 to S5 shown in fig. 1. Alternatively, the processor, when executing the computer program, implements the functions of the modules/units in the above-described device embodiments, such as the functions of the units 10 to 50 shown in fig. 5.
Illustratively, a computer program may be partitioned into one or more modules/units, which are stored in a memory and executed by a processor to accomplish the present application. One or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution of a computer program in a terminal device.
The terminal device may be a desktop computer, a notebook, a palm computer, a cloud server, or other computing devices. The terminal device may include, but is not limited to, a processor, a memory. Those skilled in the art will appreciate that the terminal device is not limited and may include more or fewer components than those shown, or some components may be combined, or different components, e.g., the terminal device may also include input output devices, network access devices, buses, etc.
The Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The storage may be an internal storage unit of the terminal device, such as a hard disk or a memory of the terminal device. The memory may also be an external storage device of the terminal device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the terminal device. Further, the memory may also include both an internal storage unit of the terminal device and an external storage device. The memory is used for storing computer programs and other programs and data required by the terminal device. The memory may also be used to temporarily store data that has been output or is to be output.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (6)
1. A method for displaying back the whole wall of a distributed splicing system is characterized by comprising the following steps:
s1, acquiring node layout information of a spliced wall; the node layout information is the node positions of output pictures formed by connecting a plurality of spliced screens;
s2, acquiring a echoing code stream of each node on the splicing wall;
s3, decoding the echo code stream to obtain a node code stream;
s4, acquiring a display area, selecting the node code stream corresponding to the display area based on the node layout information and the display area, and splicing and cutting the selected node code stream to obtain a back display picture;
s5, outputting a display picture;
the step of generating the echo code stream of each node comprises the following steps:
s21, each node synchronizes time with a time synchronization server;
s22, calculating the frame taking time of the node according to the current system time of the spliced wall;
s23, capturing a display picture of the node of the current system time;
s24, coding the display picture to obtain the echo code stream;
the steps of selecting the node code stream at the same frame-fetching time, and splicing and cutting the selected node code stream comprise:
s41, acquiring a picture display area on the spliced wall based on the display area, and acquiring each spliced display area and each spliced picture display area of the spliced area based on the spliced area of the spliced wall and the picture display area;
s42, carrying out zooming processing on each spliced display area to obtain a same zooming picture corresponding to the spliced picture display area in size specification;
s43, the zooming picture splices the pictures with the same frame taking time together according to the node layout information to obtain the spliced echo picture.
2. The method according to claim 1, wherein in the process of generating the echo code stream of the display screen, the time for all the nodes to fetch frames for the first time is aligned to an integer second, then the nodes are fetched at intervals of corresponding time, and the display screen of the nodes is captured according to the time for fetching the frames;
wherein the corresponding time is 1000/echoing frame rate, and the echoing frame rate is set based on the mobile terminal.
3. The method for displaying the entire wall of the distributed splicing system according to claim 1, wherein all the nodes on the splicing wall capture the display image at the same frame capturing time.
4. The utility model provides a distributing type mosaic system whole wall shows device back which characterized in that includes:
the node information acquisition unit is used for acquiring node layout information of the spliced wall; the node layout information is the node positions of output pictures formed by connecting a plurality of spliced screens;
the back-display code stream acquiring unit is used for acquiring the back-display code stream of each node on the splicing wall;
the decoding unit is used for decoding the echo code stream to obtain a node code stream;
the display unit is used for acquiring a display area, selecting the node code stream corresponding to the display area according to the node layout information and the display area, and splicing and cutting the selected node code stream to obtain a back display picture;
a control unit for outputting a display screen;
the code stream acquiring and back displaying unit comprises a node time synchronization subunit, a calculating subunit, a collecting subunit and a coding subunit;
the node time synchronization subunit is used for time synchronization of each node and the time synchronization server;
the calculating subunit is configured to calculate a frame fetching time of the node according to the current system time of the spliced wall;
the acquisition subunit is used for capturing a display picture of the node at the current system time;
the coding subunit is configured to code the display picture to obtain the echo code stream;
the display unit splices and cuts the node code stream selected at the same frame taking time, and comprises an area division subunit, a zooming subunit and a splicing subunit;
the area dividing subunit is configured to obtain a picture display area on the spliced wall based on the display area, and obtain each spliced display area and each spliced picture display area of the spliced area based on the spliced area of the spliced wall and the picture display area;
the zooming subunit is used for zooming each spliced display area to obtain the same zoomed images corresponding to the spliced display areas in the size specification;
and the splicing subunit is used for splicing the pictures with the same frame taking time together according to the zooming picture and the node layout information to obtain the spliced echoed picture.
5. The device according to claim 4, wherein in the unit for obtaining a playback code stream, all the nodes are aligned to an integer second for the first time of frame fetching, and then are fetched at intervals of corresponding time, and display pictures of the nodes are fetched according to the fetching time;
wherein the corresponding time is 1000/echoing frame rate, and the echoing frame rate is set based on the mobile terminal.
6. A computer device comprising a processor and a memory;
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is used for executing the whole wall echoing method of the distributed splicing system according to any one of claims 1 to 3 according to instructions in the program code.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2019/127789 WO2021127964A1 (en) | 2019-12-24 | 2019-12-24 | Whole-wall echo display method and device for distributed combination system, and computer apparatus |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111108470A CN111108470A (en) | 2020-05-05 |
| CN111108470B true CN111108470B (en) | 2021-12-07 |
Family
ID=70426438
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201980003346.8A Active CN111108470B (en) | 2019-12-24 | 2019-12-24 | Whole wall redisplay method and device for distributed splicing system and computer equipment |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111108470B (en) |
| WO (1) | WO2021127964A1 (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112561929B (en) * | 2020-12-09 | 2022-10-25 | 威创集团股份有限公司 | Splicing screen cutting and zooming method and device, electronic device and storage medium thereof |
| CN112866784A (en) * | 2020-12-31 | 2021-05-28 | 威创集团股份有限公司 | Large-screen local playback control method, control system, equipment and storage medium |
| CN113014964B (en) * | 2021-03-04 | 2023-04-28 | 深圳市东微智能科技股份有限公司 | Method, device and system for generating spliced wall back display code stream and readable storage medium |
| CN113157232A (en) * | 2021-04-26 | 2021-07-23 | 青岛海信医疗设备股份有限公司 | Multi-screen splicing display system and method |
| CN114281232B (en) * | 2021-12-03 | 2024-05-14 | 威创集团股份有限公司 | Display interface control method and related device for distributed spliced screen |
| CN114302086B (en) * | 2021-12-24 | 2023-04-21 | 威创集团股份有限公司 | Method, device and system for recording and playing back full-screen ultrahigh-resolution picture of display wall |
| CN117499564A (en) * | 2023-11-21 | 2024-02-02 | 北京视睿讯科技有限公司 | Video back display method, system, terminal and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102184723A (en) * | 2011-03-28 | 2011-09-14 | 广东威创视讯科技股份有限公司 | Echoing device and method for multi-screen splicing processing system |
| CN104333739A (en) * | 2014-10-28 | 2015-02-04 | 广东威创视讯科技股份有限公司 | Echoing device and echoing method for spliced wall system |
| CN110045942A (en) * | 2019-04-30 | 2019-07-23 | 河南泰德信息技术有限公司 | A kind of echo method and apparatus of distribution splicing device |
| CN110581828A (en) * | 2018-06-08 | 2019-12-17 | 成都鼎桥通信技术有限公司 | Time synchronization method and device for private network cluster terminal |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104284098B (en) * | 2013-07-01 | 2017-11-14 | 杭州海康威视数字技术股份有限公司 | The processing method and system of video data |
| KR101882016B1 (en) * | 2017-10-13 | 2018-07-25 | 주식회사 제노시스 | Video wall system control device using a plurality of decoder devices |
| CN110581962B (en) * | 2018-06-07 | 2021-03-19 | 杭州海康威视数字技术股份有限公司 | Television wall image redisplay method, service board and distributed television wall system |
-
2019
- 2019-12-24 WO PCT/CN2019/127789 patent/WO2021127964A1/en active Application Filing
- 2019-12-24 CN CN201980003346.8A patent/CN111108470B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102184723A (en) * | 2011-03-28 | 2011-09-14 | 广东威创视讯科技股份有限公司 | Echoing device and method for multi-screen splicing processing system |
| CN104333739A (en) * | 2014-10-28 | 2015-02-04 | 广东威创视讯科技股份有限公司 | Echoing device and echoing method for spliced wall system |
| CN110581828A (en) * | 2018-06-08 | 2019-12-17 | 成都鼎桥通信技术有限公司 | Time synchronization method and device for private network cluster terminal |
| CN110045942A (en) * | 2019-04-30 | 2019-07-23 | 河南泰德信息技术有限公司 | A kind of echo method and apparatus of distribution splicing device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111108470A (en) | 2020-05-05 |
| WO2021127964A1 (en) | 2021-07-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111108470B (en) | Whole wall redisplay method and device for distributed splicing system and computer equipment | |
| CN107018370B (en) | Display method and system for video wall | |
| US20160291917A1 (en) | Screen splicing system and video data stream processing method | |
| CN111078070B (en) | PPT video barrage play control method, device, terminal and medium | |
| CN110581962B (en) | Television wall image redisplay method, service board and distributed television wall system | |
| CN111107411B (en) | Distributed cross-node video synchronization method and system | |
| EP2914040B1 (en) | Terminal and synchronization control method thereof | |
| CN103491317B (en) | Three-dimensional figure and image multi-screen synchronous broadcasting method, device and system | |
| CN104822070B (en) | Multi-path video stream playback method and device | |
| US9588610B2 (en) | Display device, display system and control method thereof for achieving synchronized interaction with terminal device | |
| CN110519645B (en) | Video content playing method and device, electronic equipment and computer readable medium | |
| US20130187832A1 (en) | Display apparatus and method for controlling the same | |
| CN103369411A (en) | Multiscreen video playing system dynamically generating scaled video, and related device | |
| CN108156511B (en) | Method and system for realizing double-screen or multi-screen splicing display | |
| CN101662611A (en) | Template display method in video conference system | |
| WO2021207979A1 (en) | Video processing device and system | |
| CN112866784A (en) | Large-screen local playback control method, control system, equipment and storage medium | |
| CN112583821A (en) | Display method, display system, electronic device, and computer-readable storage medium | |
| CN113015007A (en) | Video frame insertion method and device and electronic equipment | |
| CN113596571B (en) | Screen sharing method, device, system, storage medium and computer equipment | |
| CN111045770A (en) | Method, first terminal, device and readable storage medium for remote exhibition | |
| US20160078664A1 (en) | Image synthesizing apparatus and method | |
| EP3289582A1 (en) | Managing display data | |
| CN110989959A (en) | Screen splicing display method and device | |
| CN104469478A (en) | Information processing method and device and electronic devices |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |