CN111932663B - Parallel drawing method based on multi-level asymmetric communication management - Google Patents
Parallel drawing method based on multi-level asymmetric communication management Download PDFInfo
- Publication number
- CN111932663B CN111932663B CN202010798412.XA CN202010798412A CN111932663B CN 111932663 B CN111932663 B CN 111932663B CN 202010798412 A CN202010798412 A CN 202010798412A CN 111932663 B CN111932663 B CN 111932663B
- Authority
- CN
- China
- Prior art keywords
- data
- parallel
- server
- client
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 239
- 238000004891 communication Methods 0.000 title claims abstract description 77
- 230000008569 process Effects 0.000 claims abstract description 164
- 238000004088 simulation Methods 0.000 claims abstract description 30
- 230000003993 interaction Effects 0.000 claims abstract description 10
- 238000009877 rendering Methods 0.000 claims description 47
- 238000004364 calculation method Methods 0.000 claims description 25
- 238000007726 management method Methods 0.000 claims description 20
- 230000015572 biosynthetic process Effects 0.000 claims description 14
- 238000003786 synthesis reaction Methods 0.000 claims description 14
- 230000002452 interceptive effect Effects 0.000 claims description 9
- 238000004422 calculation algorithm Methods 0.000 claims description 8
- 238000013144 data compression Methods 0.000 claims description 6
- 238000007906 compression Methods 0.000 claims description 3
- 230000006835 compression Effects 0.000 claims description 3
- 230000000977 initiatory effect Effects 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 abstract description 12
- 238000012545 processing Methods 0.000 description 14
- 238000005516 engineering process Methods 0.000 description 12
- 238000005286 illumination Methods 0.000 description 8
- 238000012800 visualization Methods 0.000 description 8
- 230000005540 biological transmission Effects 0.000 description 7
- 238000013461 design Methods 0.000 description 6
- 230000003068 static effect Effects 0.000 description 5
- 238000001308 synthesis method Methods 0.000 description 5
- 230000001360 synchronised effect Effects 0.000 description 4
- 238000013500 data storage Methods 0.000 description 3
- 238000013079 data visualisation Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012856 packing Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/005—General purpose rendering architectures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2111/00—Details relating to CAD techniques
- G06F2111/10—Numerical modelling
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computer Graphics (AREA)
- General Engineering & Computer Science (AREA)
- Processing Or Creating Images (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a parallel drawing method based on multi-level asymmetric communication management, which comprises the following steps: s1, reading numerical simulation result data in parallel; s2, starting drawing of a client interaction process; s3, the server side performs parallel drawing in a drawing process; s4, the server side transmits the drawing result to the client side, and the like; the method and the device reduce the inter-process communication times in the parallel visual computing and drawing process, realize inter-process load balance, improve the data loading efficiency and improve the drawing efficiency.
Description
Technical Field
The invention relates to the field of scientific visualization, in particular to a parallel rendering method based on multi-level asymmetric communication management.
Background
With the continuous development of huge computer technology, the computational accuracy of scientific numerical simulation is continuously improved, the grid scale is continuously increased, the generated scientific data volume reaches PB or even EB magnitude, and how to effectively analyze and understand the large-scale scientific data becomes an important problem for scientists at present. Scientific calculation visualization is one of effective methods for helping scientists analyze and understand large-scale scientific data, and refers to a theory, a method and a technology which apply computer graphics and image processing technology to convert output data of scientific calculation and data generated by observation and test in other fields into graphics and images, and finally display the graphics and the images on a screen for interactive processing. For the visualization processing of the large-scale scientific calculation data, the application requirements of the large-scale scientific data visualization cannot be met only by the expansion of the storage space of a single computer and the improvement of the calculation speed. Therefore, the introduction of the parallel processing technology into the visualization algorithm becomes an inevitable choice, and the parallel visualization processing by using a massively parallel computer becomes an effective means for analyzing the massive scientific data at present. Based on the distributed parallel computer cluster system, the computing power and the image drawing function of a plurality of computers can be fully developed, and the distributed parallel computer cluster system has better expandability and strong flexibility. The parallel rendering technology introduces a parallel computing theory and a method into the field of graphic processing, and is an important means for realizing high-performance graphic computation. Traditionally, the graphics processing tasks are distributed in a pipeline-organized fashion, i.e. in such a way that the re-distribution of the graphics processing tasks takes place at the stage of the graphics pipeline, the parallel drawing system structure is divided into three types of sort-first, sort-middle and sort-last.
In the existing scheme, for the application requirements of large-scale complex scene drawing, patent CN 101986710A proposes a parallel drawing system based on sort-last architecture, which is composed of a PC cluster system, and includes a fusion node and several drawing nodes, the nodes are connected through a local area network, the drawing nodes are responsible for drawing respective scenes, the fusion node is responsible for summarizing the scenes formed by the drawing nodes and corresponding pixel depth images, and finally fusing to form final image output according to image depth information. Patent CN 103136399A proposes a radiometric parallel rendering system of an indoor scene, and the patent provides a radiometric parallel rendering method facing the indoor scene: firstly, calculating a parameterized texture map of an object by each drawing node, and sending the parameterized texture map to a file server; then, a parallel drawing manager of the main control node sorts according to the size of the parameterized texture of each object, and direct illumination drawing tasks are distributed to a plurality of drawing nodes according to the size of the texture; calculating a direct illumination texture map of the object by each drawing node, and sending the direct illumination texture map to a file server; finally, calculating the calculation cost of each illumination texture on each drawing node by the size of the direct illumination texture of the parallel drawing manager object of the main control node and combining the hardware and network information of each drawing node, and sequencing and distributing the calculation cost to a plurality of drawing nodes; and the drawing node calculates an indirect illumination texture map of the object and sends the indirect illumination texture map to the file server. Patent CN 103699354A presents a three-dimensional graphics rendering method based on parallel rendering technology, which includes: and constructing a working environment of the parallel rendering system, determining primitive attribution by the user service node, determining primitive attribution by the rendering node, executing a rendering task by the rendering node, extracting pixel data and sending the pixel data to the image service node, receiving the pixel data by the image service node, executing image splicing, outputting the image data to the projection equipment and displaying the image data.
The above solution is closer to the technology of the present invention, and the patent CN 101986710A solves the problem of parallel drawing between drawing nodes, but does not solve the problem of returning the final drawing image to the service processing system and the user terminal. Patent CN 103136399A proposes a parallel rendering system including a main control node, rendering task sub-nodes, a task manager and a file server, which can be applied to an indoor-facing scene in virtual reality, however, in the method, a parameterization task is allocated to a rendering node through the parallel rendering manager of the main control node, and a direct illumination rendering task is allocated to the rendering node by the parallel rendering manager of the main control node. Patent CN 103699354A uses an image service node to receive pixel data and perform image stitching. Due to the characteristics of a domestic super-computing structure and a remote operation starting mode applicable to the method, when the methods of the patents CN 103136399A and CN 103699354A are applied to the hardware system in the drawing process, the communication between a main control node (service node) and a drawing node becomes a performance bottleneck, and the drawing efficiency is influenced; and the above scheme cannot realize interactive exploration on numerical simulation data.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a parallel drawing method based on multi-level asymmetric communication management, which reduces the inter-process communication times in the parallel visual calculation and drawing process, realizes inter-process load balance, improves the data loading efficiency and improves the drawing efficiency.
The purpose of the invention is realized by the following technical scheme: a parallel drawing method based on multi-level asymmetric communication management comprises the following steps:
s1, reading numerical simulation result data in parallel;
s2, starting drawing of a client interaction process;
s3, the server side drawing process executes parallel drawing;
and S4, the server side transmits the drawing result to the client side.
Further, the method comprises the following steps:
s11, dividing numerical simulation result data into data blocks according to grid connection relation, then distributing the data blocks to different processes, wherein the difference of the number of the data blocks processed among the processes is at most 1 block;
s12, when the binary file in the tecplot format is read, the offset and the length of any block of data are obtained at the header part of the file, the file is divided according to the information to carry out reading tasks, and each process can obtain the number of all blocks and the header information of each block of grid;
s13, each process calculates the byte length of the data part of each block according to the acquired header information, then the offset of the data part of each block can be acquired through the acquired byte length of each block and the pointer position of the file, communication among the processes is reduced through the calculation of the offset, finally different reading positions can be set for different processes to realize data division, and parallel data file reading operation of each process is realized.
Further, comprising the steps of:
s21, the client obtains the size and position information of the drawing window and sends the information to the No. 0 process of the server; the client acquires the position of a renderer, camera information and renderer information related to a camera transformation matrix and sends the renderer information to a No. 0 process of the server;
s22, after receiving the data from the step S21, the server-side process No. 0 broadcasts the data to other drawing satellite processes, and the satellite processes update the renderer information after receiving the information;
and S23, the client initiates a remote procedure call through the socket, and the server starts to execute the image drawing task after receiving the procedure call.
Further, comprising the steps of:
s31, when the process 0 of the server starts to draw, initiating remote process call through MPI to call the drawing tasks of other processes of the server;
s32, converting the numerical simulation result data into displayable primitives or images by the graph renderers of all the drawing processes, and transmitting the displayable primitives or images to the data compression module;
s33, the data compression module calls a compression algorithm to compress the image into small data blocks, and the small data blocks are submitted to the communication module and sent to other drawing nodes;
s34, the drawing node responsible for synthesis receives the compressed data from the communication module and restores the compressed data into an image;
and S35, combining the data by the combining node according to the image depth information to obtain a combined complete simulation data image.
Further, the method comprises the following steps:
s41, after the server finishes drawing, the server main process sends the synthesized picture data to the client;
s42, the client draws the received image data in the rendering window;
s43, recovering all process states of the client and the server, and waiting for user interactive operation or finishing operation.
Further, the data communication in step S21 is realized through a socket data communication interface.
Further, the data communication in step S22 is realized through an MPI data communication interface.
Further, the server host process in step S41 sends the synthesized picture data to the client through the socket communication interface.
The beneficial effects of the invention are:
(1) The invention is improved based on the existing parallel rendering method, improves rendering efficiency facing to a supercomputing system, fully utilizes high-speed interconnection of a high-performance computer system to perform image synthesis, and transmits synthesized image data based on a low-speed interconnection system, so that the data transmission quantity in a low-speed network can be effectively reduced, communication resources are fully utilized, and the real-time interaction requirement is met; the method has the advantages that the parallel data are efficiently read in, the inter-process communication times in the parallel visual computing and drawing process are reduced, the inter-process load balance is realized, the data loading efficiency is improved, the C/S mode parallel drawing method is realized, and the multi-level asymmetric communication method capable of simultaneously processing the inside of the server and the communication between the client and the server is realized.
(2) The invention designs a parallel data file reading method, which is characterized in that a grid data file is divided into a plurality of subdata, and the offset of each data part is obtained according to the byte length of each block and the pointer position of the file, so that the parallel file reading is realized; meanwhile, the data loading efficiency is improved through the static load balance design among the processes, and the drawing efficiency is further improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a block diagram of a supercomputing system to which embodiment 1 of the present invention is applied;
FIG. 2 is a flow chart of parallel rendering and synchronous rendering according to the present invention;
FIG. 3 is a schematic diagram of a binary file structure in a apply format;
FIG. 4 is a method for dividing parallel IO file data portions according to the present invention.
Detailed Description
The technical solutions of the present invention are further described in detail below with reference to the accompanying drawings, but the scope of the present invention is not limited to the following. All of the features disclosed in this specification, or all of the steps of a method or process so disclosed, may be combined in any combination, except combinations where mutually exclusive features and/or steps are present.
As shown in fig. 1 to 4, a parallel rendering method based on multi-level asymmetric communication management includes the steps of:
s1, reading in numerical simulation result data in parallel;
s2, starting drawing of a client interaction process; after data is read in, or a user executes interactive control, such as operations of image zooming, rotation and the like, drawing operation needs to be executed on a data file, an image is displayed on a client, the data file is on a server at the moment, and in order to reduce data transmission to the Ethernet, the client only needs to acquire drawn and synthesized image data.
S3, the server side performs parallel drawing in a drawing process; in order to improve the efficiency of parallel rendering to the maximum extent, a tree synthesis method is adopted in the process of drawing at the server side. And the tree synthesis method carries out pairwise grouping synthesis on all intermediate result images until all the intermediate result images are synthesized into a complete result image. If n processes participate in the synthesis process, the number of images to be synthesized is n, and the communication times of the tree synthesis mode is log 2 n, time overhead of
Wherein t is l Is the image transmission time, t r For image read back time, t d Time is calculated for image synthesis.
S4, the server side transmits the drawing result to the client side; the server is responsible for parallel drawing, and draws the simulation data distributed on each process into a complete image. The client is responsible for displaying the drawing result, so the drawing result needs to be synchronously transmitted to the client.
Further, the step S1 specifically includes the following sub-steps:
s11, dividing numerical simulation result data into data blocks according to grid connection relation, then distributing the data blocks to different processes, wherein the difference of the number of the processed data blocks among the processes is at most 1 block;
and redistributing the data blocks according to the grid connection relation, wherein the data blocks owned by each process are required to be compact in space in order to reduce the inter-process communication times in the parallel visual computing process. For data read-in of m data blocks with n processes, the ith process is divided into { b } i×m/n ,b i×m/n+1 ,b i×m/n+2 And 8230, the data blocks are divided into the data blocks which are as equal as possible by the processes, the difference of the number of the processed blocks among the processes is at most 1 block, and static load balance is realized.
S12, when the binary file in the tecplot format is read, the offset and the length of any block of data are obtained at the header part of the file, the file is divided according to the information to carry out reading tasks, and each process can obtain the number of all blocks and the header information of each block of grid;
when a binary file is read, as shown in fig. 3, the structure of a commonly used Tecplot format binary file is opened in a stream form, and the offset and length of any block of data of the file are obtained in the header part of the file, so that the file is divided according to the information to perform a reading task. The header information and the data storage part of the data block are separated, and the number of bytes occupied by the header information is small. All processes can read the header information of all blocks at the same time, namely, each process can acquire the number of all blocks and the header information of each grid, and the communication among the processes is reduced through the calculation of the offset, so that the communication overhead of the high-speed interconnection system at the server end is reduced.
S13, each process calculates the byte length of the data part of each block according to the acquired header information, then the offset of the data part of each block can be acquired through the acquired byte length of each block and the pointer position of the file, communication among the processes is reduced through the calculation of the offset, finally different reading positions can be set for different processes to realize data division, and parallel data file reading operation of each process is realized.
Each process calculates the byte length of the data part of each block according to the acquired header information, then the offset of the data part of each block can be acquired through the acquired byte length of each block and the pointer position of the file, finally, different reading positions can be set for different processes, the offset calculation is shown in fig. 4, and parallel data file reading operation of each process is realized according to data division realized on the basis of the data division.
Further, step S2 includes the steps of:
s21, the client obtains the size and position information of the drawing window and sends the information to the No. 0 process of the server; the client acquires the position of a renderer, camera information and renderer information related to a camera transformation matrix and sends the renderer information to a No. 0 process of the server; data communication is realized through a socket data communication interface.
S22, after receiving the data from the step S21, the server-side process No. 0 broadcasts the data to other drawing satellite processes, the satellite processes update renderer information after receiving the information, and at the moment, data communication is achieved through an MPI data communication interface;
and S23, the client initiates a remote procedure call through the socket, and the server starts to execute the image drawing task after receiving the procedure call.
Further, comprising the steps of:
s31, when the process 0 of the server starts to draw, initiating remote process call through MPI to call the drawing tasks of other processes of the server;
s32, converting the numerical simulation result data into displayable primitives or images by the graph renderers of all the drawing processes, and transmitting the primitives or images to the data compression module;
s33, the data compression module calls a compression algorithm to compress the image into small data blocks, and the small data blocks are submitted to the communication module and sent to other drawing nodes;
s34, the drawing node responsible for synthesis receives the compressed data from the communication module and restores the compressed data into an image;
and S35, combining the data by the combining node according to the image depth information to obtain a combined complete simulation data image.
Further, comprising the steps of:
s41, after the server finishes drawing, the server main process sends the synthesized picture data to the client;
s42, the client draws the received image data in a rendering window;
s43, recovering all process states of the client and the server, and waiting for user interactive operation or finishing operation.
In step S41, the server host process transmits the synthesized picture data to the client via the socket communication interface.
A parallel data file reading method comprises the following steps:
s11, dividing numerical simulation result data into data blocks according to grid connection relation, then distributing the data blocks to different processes, wherein the difference of the number of the processed data blocks among the processes is at most 1 block;
and redistributing the data blocks according to the grid connection relation, wherein the data blocks owned by each process are required to be compact in space in order to reduce the inter-process communication times in the parallel visual computing process. For data read-in of m data blocks with n processes, the ith process is divided into { b } i×m/n ,b i×m/n+1 ,b i×m/n+2 And 8230, the data blocks are divided into the data blocks which are as equal as possible by the processes, the difference of the number of the processed blocks among the processes is at most 1 block, and static load balance is realized.
S12, when the binary file is read, the offset and the length of any block of data are obtained at the header part of the file, the file is divided according to the information to carry out reading tasks, each process can obtain the number of all blocks and the header information of each grid, and the communication among the processes is reduced through the calculation of the offset;
when a binary file is read, as shown in fig. 2, the structure of a commonly used Tecplot format binary file is opened in a stream form, and the offset and length of any block of data of the file are obtained in the header part of the file, so that the file is divided according to the information to perform a reading task. The header information and the data storage part of the data block are separated, and the number of bytes occupied by the header information is small. All processes can read the header information of all blocks at the same time, namely, each process can acquire the header information of all the block numbers and each grid, and the communication among the processes is reduced through the calculation of the offset, so that the communication overhead of a high-speed interconnection system at a server end is reduced.
S13, each process calculates the byte length of the data part of each block according to the acquired header information, then the offset of the data part of each block can be acquired through the acquired byte length of each block and the pointer position of the file, communication among the processes is reduced through the calculation of the offset, finally different reading positions can be set for different processes to realize data division, and parallel data file reading operation of each process is realized.
Each process calculates the byte length of the data part of each block according to the acquired header information, then the offset of the data part of each block can be acquired through the acquired byte length of each block and the pointer position of the file, finally, different reading positions can be set for different processes, the offset calculation is shown in fig. 3, and the parallel data file reading operation of each process is realized according to the data division realized on the basis of the data division.
Example 1
The visualization technology applies a computational graphics correlation method and a graphic processing and displaying technology, converts a large amount of discrete data obtained by numerical simulation into a graphic image or animation form, and vividly displays a numerical simulation result, so that the characteristics and essence in the data can be visually and accurately displayed, and scientific researchers are helped to analyze rules in the data. For visualization of large-scale data, a single computer is difficult to process. Based on the distributed parallel computer cluster system, the computing power and the image drawing function of a plurality of computers can be fully developed, and the distributed parallel computer cluster system has better expandability and strong flexibility. The parallel rendering technology introduces a parallel computing theory and a parallel computing method into the field of graphics processing, divides rendering tasks, distributes the rendering tasks to different rendering resources for parallel rendering, and synthesizes rendering results, so that the problems of data distribution and image centralized display of data on a computer cluster can be solved.
The related papers and patent documents implement a parallel rendering method applicable to a certain computer architecture, and for the currently mainstream super computer system, the architecture is as shown in fig. 1. In the image drawing process, a parallel drawing task needs to be started in a mode of remotely sending jobs by a client on the super-computation cluster system, the drawing result on the drawing node needs to be sent to a local client for displaying, and user interaction input and interaction control are responded; from the architecture of a supercomputer, a service processing system is connected with a drawing node in an ethernet mode (the bandwidth is hundreds of mega or giga per second), and a high-speed interconnection (such as Intel InfiniBand, the bandwidth is tens of GB per second) is used between the drawing nodes, which are hundreds of times or even thousands of times different. The complexity of supercomputer system architecture and data visualization make existing graphical rendering methods unsuitable for current supercomputer system architectures.
Aiming at the defects of the prior art, the embodiment 1 of the invention designs a parallel drawing method based on multi-level asymmetric communication management, and realizes efficient parallel drawing suitable for the current super computer system by effectively dividing data and managing different communication networks of the super computer system.
In this embodiment, a parallel rendering method based on multi-level asymmetric communication management includes:
s1, reading numerical simulation result data in parallel;
s2, starting drawing of a client interaction process;
s3, the server side drawing process executes parallel drawing;
and S4, the server side transmits the drawing result to the client side.
To facilitate understanding and implementation of the present embodiment for those skilled in the art, as described below, in step S1, during the parallel visualization calculation and rendering process, the large-scale simulation data is first divided into data blocks, then allocated to different processes, the data is read into the memory through the parallel IO operation, and the data is exchanged between the processes according to the algorithm flow. In the embodiment, a data division method divided according to grids is adopted, and the whole flow field is divided into a plurality of data blocks in a balanced manner within a spatial range; meanwhile, the data blocks are redistributed according to the grid connection relation, so that the data blocks owned by each process are ensured to be compact in space as much as possible, the inter-process communication times in the parallel visual computing and drawing process are reduced, and the inter-process load balance is realized.
In step S2, after the data is read in, or the user performs interactive control, such as image scaling, rotation, and the like, the data file needs to be rendered, the image is displayed on the client, and the data file is on the server at this time.
In step S3, in order to improve the efficiency of parallel rendering to the maximum extent, a tree synthesis method is adopted in the server side drawing process. And the tree-shaped synthesis method carries out pairwise grouping synthesis on all the intermediate result images until all the intermediate images are synthesized into a complete result image. Supposing that n processes participate in the synthesis process, the number of the images to be synthesized is nThe number of communication times of the tree synthesis method is log 2 n, time overhead of
Wherein t is l Is the image transfer time, t r For image read back time, t d Time is calculated for image synthesis.
In step S4, the server side transmits the drawing result to the client side; the server is responsible for parallel drawing, and draws the simulation data distributed on each process into a complete image. The client is responsible for displaying the drawing result, so the drawing result needs to be synchronously transmitted to the client.
The embodiment is improved based on the existing parallel drawing method, the drawing efficiency facing the supercomputing system is improved, specifically, the traditional parallel drawing technology can solve the problems of image synthesis and the like, however, in the current mainstream supercomputing system, the traditional method is used for drawing images, and the communication between the main control node and the drawing node becomes a performance bottleneck. In contrast, the method of combining parallel rendering with synchronous rendering is adopted, so that the high-speed interconnection of the high-performance computer system can be fully utilized for image synthesis, and the synthesized image data is transmitted based on the low-speed interconnection system, so that the data transmission quantity in the low-speed network can be effectively reduced, the communication resources are fully utilized, and the real-time interaction requirement is met. Compared with the current parallel rendering algorithm, the parallel rendering algorithm based on asymmetric multilevel communication management provided in this embodiment has the following advantages in design:
(1) And reading in data of numerical simulation results in parallel, dividing large-scale simulation data into data blocks firstly in the process of parallel visual calculation and drawing, then distributing the data blocks to different processes, reading the data into a memory through parallel IO operation, and exchanging the data among the processes according to an algorithm flow. The invention adopts a data division method divided according to grids, and the whole flow field can be divided into a plurality of data blocks in a balanced manner within a space range; meanwhile, the data blocks are redistributed according to the grid connection relation, the data blocks owned by each process are guaranteed to be compact in space as much as possible, the inter-process communication times in the parallel visual computing and drawing process are reduced, and inter-process load balancing is achieved.
(2) The method comprises the following steps that multi-level asymmetric communication management is carried out, a computing task is started in a mode that a client sends a job remotely on a supercomputing cluster system, for a parallel drawing computing task, parallel visual computing and a parallel drawing task need to be completed on a remote cluster, a multi-process parallel program of a server is assisted through a high-speed interconnection system, and a message transmission mechanism is adopted for interprocess communication; and the connection between the client and the server and the visual data transmission link are low-speed network and Socket communication interfaces based on a TCP/IP protocol. The drawing result of the server needs to be sent to the local client for display and the user interactive input is responded at the same time. In the calculation process, firstly, a client side is required to be connected with a server side, and then the server can respond to a request of a user to execute a corresponding drawing task; the response mode is realized by adopting a remote process calling mode, the operation process of the client is sent to the main process of the server in a parameter packing mode, and after the main process analyzes the operation process, the calculation of other processes is started according to the task division condition; after all processes complete image synthesis, the main process has a complete drawing result; and finally, sending the main process picture file to a client for display through data synchronization operation.
The method provided by the embodiment can be used for realizing the parallel drawing and the synchronous drawing of the images by fully utilizing the current super computer system structure, greatly improving the image drawing and transmission efficiency, meeting the requirement of real-time interaction, realizing the cooperation of the operation management and the data communication of the system structure facing the super computer, and realizing the coordination management of two drawing methods of the parallel drawing and the synchronous drawing based on the system structure of high-speed interconnection among the computing nodes and low-speed interconnection between the main control node and the drawing nodes so as to support interactive exploration.
Example 2
The invention can be implemented as a parallel data file reading method by a person skilled in the art, and the whole flow field can be divided into a plurality of data blocks in a space range by a data division method divided according to grids; meanwhile, the redistribution of the data blocks is carried out according to the grid connection relation, so that the data blocks owned by each process are ensured to be compact in space as much as possible, the inter-process communication times in the parallel visual computing and drawing process are reduced, the inter-process load balance is realized, and the method is realized by the following steps:
s11, dividing numerical simulation result data into data blocks according to grid connection relation, then distributing the data blocks to different processes, wherein the difference of the number of the processed data blocks among the processes is at most 1 block;
s12, when the binary file is read, the offset and the length of any block of data of the binary file are obtained at the header part of the file, the file is divided according to the information to carry out reading tasks, each process can obtain the number of all blocks and the header information of each block of grid, and the communication among the processes is reduced through the calculation of the offset;
s13, each process calculates the byte length of the data part of each block according to the acquired header information, then the offset of the data part of each block can be acquired through the acquired byte length of each block and the pointer position of the file, communication among the processes is reduced through the calculation of the offset, finally different reading positions can be set for different processes to realize data division, and parallel data file reading operation of each process is realized.
To facilitate the understanding and enabling those of ordinary skill in the art to practice the present invention, further description is provided below,
in step S11, dividing the numerical simulation result data into data blocks according to the grid connection relation, then distributing the data blocks to different processes, wherein the difference of the number of the data blocks processed among the processes is at most 1 block; and redistributing the data blocks according to the grid connection relation, wherein the data blocks owned by each process are required to be compact in space in order to reduce the inter-process communication times in the parallel visual computing process. For data read-in of m data blocks with n processes, the ith process is divided into { b } i×m/n ,b i×m/n+1 ,b i×m/n+2 \8230; } dataAnd blocks are used for ensuring that each process is divided into data blocks which are as equal as possible, the difference of the number of the processed blocks among the processes is at most 1 block, and static load balance is realized.
In step S12, when reading the binary file, by obtaining the offset and length of any block of data in the header portion of the file, and dividing the file according to these information to perform a reading task, each process can obtain the number of all blocks and the header information of each block of mesh, and communication between processes is reduced by calculating the offset; when a binary file is read, as shown in fig. 2, the structure of a commonly used Tecplot format binary file is opened in a stream form, and the offset and length of any block of data of the file are obtained in the header part of the file, so that the file is divided according to the information to perform a reading task. The header information and the data storage part of the data block are separated, and the number of bytes occupied by the header information is small. All processes can read the header information of all blocks at the same time, namely, each process can acquire the header information of all the block numbers and each grid, and the communication among the processes is reduced through the calculation of the offset, so that the communication overhead of a high-speed interconnection system at a server end is reduced.
In step S13, each process calculates the byte length of the data portion of each block according to the acquired header information, then the offset of the data portion of each block can be acquired by the acquired byte length of each block and the pointer position of the file, communication between processes is reduced by calculating the offset, and finally different reading positions can be set for different processes to implement data division, thereby implementing parallel data file read-in operation of each process. Each process calculates the byte length of the data part of each block according to the acquired header information, then the offset of the data part of each block can be acquired through the acquired byte length of each block and the pointer position of the file, finally, different reading positions can be set for different processes, the offset calculation is shown in fig. 3, and the parallel data file reading operation of each process is realized according to the data division realized on the basis of the data division.
The embodiment designs a parallel data file reading method, which includes the steps of dividing a grid data file into a plurality of subdata, and acquiring the offset of each data part according to the acquired byte length of each block and the pointer position of the file, so that reading of the parallel file is realized; meanwhile, the data loading efficiency is improved through static load balance design among the processes.
Example 3
The module, device, apparatus and system, and the method for using the module, device, apparatus and system, which are implemented by any step in example 1 or any step in example 2, or any combination of multiple steps, are all within the scope of protection of this embodiment.
Features of the various embodiments of the invention may be interchanged or combined. The functionality of the present invention, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention or a part thereof which substantially contributes to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
Claims (7)
1. A parallel drawing method based on multi-level asymmetric communication management is characterized by comprising the following steps:
s1, reading numerical simulation result data in parallel; step S1, comprising the substeps of:
s11, dividing numerical simulation result data into data blocks according to grid connection relation, then distributing the data blocks to different processes, wherein the difference of the number of the data blocks processed among the processes is at most 1 block;
s12, when the binary file in the tecplot format is read, the offset and the length of any block of data are obtained at the header part of the file, the file is divided according to the information to carry out reading tasks, and each process can obtain the number of all blocks and the header information of each block of grid;
s13, each process calculates the byte length of the data part of each block according to the acquired header information, then the offset of the data part of each block can be acquired through the acquired byte length of each block and the pointer position of the file, the communication among the processes is reduced through the calculation of the offset, and finally different reading positions can be set for different processes to realize data division, so that the parallel data file reading operation of each process is realized;
s2, starting drawing of a client interaction process;
s3, the server side drawing process executes parallel drawing;
and S4, the server side transmits the drawing result to the client side.
2. The parallel rendering method based on multi-level asymmetric communication management as claimed in claim 1, comprising the steps of:
s21, the client obtains the size and position information of the drawing window and sends the information to the No. 0 process of the server; the client acquires the position of a renderer, camera information and renderer information related to a camera transformation matrix and sends the renderer information to a No. 0 process of the server;
s22, after receiving the data from the step S21, the server-side process No. 0 broadcasts the data to other drawing satellite processes, and the satellite processes update the renderer information after receiving the information;
and S23, the client initiates a remote procedure call through the socket, and the server starts to execute the image drawing task after receiving the procedure call.
3. The parallel rendering method based on multi-level asymmetric communication management as claimed in claim 1, comprising the steps of:
s31, when the progress 0 of the server starts to draw, initiating remote process call through MPI to call the drawing tasks of other progresses of the server;
s32, converting the numerical simulation result data into displayable primitives or images by the graph renderers of all the drawing processes, and transmitting the primitives or images to the data compression module;
s33, the data compression module calls a compression algorithm to compress the image into small data blocks, and the small data blocks are submitted to the communication module and sent to other drawing nodes;
s34, the drawing node responsible for synthesis receives the compressed data from the communication module and restores the compressed data into an image;
and S35, combining the data by the combining node according to the image depth information to obtain a combined complete simulation data image.
4. The parallel rendering method based on multi-level asymmetric communication management as claimed in claim 1, comprising the steps of:
s41, after the server finishes drawing, the server main process sends the synthesized picture data to the client;
s42, the client draws the received image data in the rendering window;
s43, recovering all process states of the client and the server, and waiting for user interactive operation or finishing operation.
5. The parallel drawing method based on multilevel asymmetric communication management according to claim 2, wherein the data communication in step S21 is implemented by a socket data communication interface.
6. The parallel rendering method based on multi-level asymmetric communication management as claimed in claim 2, wherein the step S22 data communication is implemented by MPI data communication interface.
7. The parallel drawing method based on multi-level asymmetric communication management as claimed in claim 4, wherein in step S41, the server host process sends the synthesized picture data to the client through a socket communication interface.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010798412.XA CN111932663B (en) | 2020-08-11 | 2020-08-11 | Parallel drawing method based on multi-level asymmetric communication management |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010798412.XA CN111932663B (en) | 2020-08-11 | 2020-08-11 | Parallel drawing method based on multi-level asymmetric communication management |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111932663A CN111932663A (en) | 2020-11-13 |
| CN111932663B true CN111932663B (en) | 2022-10-18 |
Family
ID=73307321
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010798412.XA Active CN111932663B (en) | 2020-08-11 | 2020-08-11 | Parallel drawing method based on multi-level asymmetric communication management |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111932663B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113254241B (en) * | 2021-06-25 | 2021-09-21 | 中国空气动力研究与发展中心计算空气动力研究所 | Data cross-process transmission method applied to CFD high-performance computation |
| CN113253965B (en) * | 2021-06-25 | 2021-10-29 | 中国空气动力研究与发展中心计算空气动力研究所 | Mass data multi-view-port visual interaction method, system, equipment and storage medium |
| CN114463476B (en) * | 2022-04-11 | 2022-06-28 | 中国空气动力研究与发展中心计算空气动力研究所 | Visual parallel rendering method, device and system and storage medium |
| CN117473798B (en) * | 2023-12-26 | 2024-05-14 | 国家超级计算天津中心 | Simulation project management method, device, equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2098995A2 (en) * | 2008-03-04 | 2009-09-09 | Agfa Healthcare | System for real-time volume rendering on thin clients via a render server. |
| JP2012209379A (en) * | 2011-03-29 | 2012-10-25 | Dainippon Screen Mfg Co Ltd | Pattern drawing method, pattern drawing device, and computer program |
| CN106803231A (en) * | 2016-12-29 | 2017-06-06 | 东北大学 | A kind of figure parallel drawing method based on sort first load balances |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10249073B2 (en) * | 2015-07-21 | 2019-04-02 | Intel Corporation | Distributed frame buffer and API for scalable parallel rendering |
-
2020
- 2020-08-11 CN CN202010798412.XA patent/CN111932663B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2098995A2 (en) * | 2008-03-04 | 2009-09-09 | Agfa Healthcare | System for real-time volume rendering on thin clients via a render server. |
| JP2012209379A (en) * | 2011-03-29 | 2012-10-25 | Dainippon Screen Mfg Co Ltd | Pattern drawing method, pattern drawing device, and computer program |
| CN106803231A (en) * | 2016-12-29 | 2017-06-06 | 东北大学 | A kind of figure parallel drawing method based on sort first load balances |
Non-Patent Citations (3)
| Title |
|---|
| "Parallel In Situ Coupling of Simulation with a Fully Featured Visualization System";Brad Whitlock等;《Eurographics Symposium on Parallel Graphics and Visualization》;20110410;第101-109页第1-7节 * |
| 一种自适应可视化软件结构;杨帆等;《***仿真学报》;20060831;第18卷;第333-335,339页 * |
| 面向科学计算可视化的两级并行数据读取加速方法;石刘等;《计算机研究与发展》;20170430;第54卷(第04期);第844-854页第1-3节 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111932663A (en) | 2020-11-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111932663B (en) | Parallel drawing method based on multi-level asymmetric communication management | |
| Doerr et al. | CGLX: a scalable, high-performance visualization framework for networked display environments | |
| Staadt et al. | A survey and performance analysis of software platforms for interactive cluster-based multi-screen rendering | |
| US20110004827A1 (en) | Scalable, Cross-Platform Method for Multi-Tile Display Systems | |
| CN110706341A (en) | High-performance rendering method and device of city information model and storage medium | |
| CN112669428A (en) | BIM (building information modeling) model rendering method based on server and client cooperation | |
| CN103679813B (en) | The construction method of three-dimensional digital earth and three-dimensional digital earth system | |
| Eilemann et al. | Equalizer 2.0–convergence of a parallel rendering framework | |
| Ogasa et al. | Visualization technology for the K computer | |
| Jensen et al. | An efficient system for collaboration in tele-immersive environments | |
| CN113253965B (en) | Mass data multi-view-port visual interaction method, system, equipment and storage medium | |
| Bues et al. | Towards a scalable high performance application platform for immersive virtual environments | |
| Nonaka et al. | Hybrid hardware-accelerated image composition for sort-last parallel rendering on graphics clusters with commodity image compositor | |
| Liu et al. | Design and Implementation of Distributed Rendering System | |
| Eilemann et al. | The Equalizer parallel rendering framework | |
| Altenhofen et al. | Rixels: Towards secure interactive 3d graphics in engineering clouds | |
| Mwalongo | Interactive web-based visualization | |
| Schmauder et al. | Distributed visual analytics on large-scale high-resolution displays | |
| Kranzlmuller et al. | Interactive virtual reality on the grid | |
| CN102622198A (en) | Method and system for visualizing data | |
| Sigitov et al. | Enabling global illumination rendering on large, high-resolution displays | |
| Bundulis et al. | Infiniviz: Taking quake 3 arena on a large-scale display system to the next level | |
| Bolívar et al. | An architecture approach for 3D render distribution using mobile devices in real time | |
| Stoll et al. | Evaluating multi-port frame buffer designs for a mesh-connected multicomputer | |
| Li et al. | MSPR: a retained-mode-based multiscreen parallel rendering system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |