Background technology
Meso-scale meteorology is the important branch quickly grown in modern weather science, the air mesoscale studied by it
Motion, is related to the discovery and prevention of the important diastrous weather in region.One side therein be by meso-scale model, it is right
Study of Meso Scale Weather process carries out deep simulation study and forecast experiments.With developing rapidly for computer technology in recent years, in
Yardstick numerical model is reached its maturity.Mesoscale Numerical weather forecasting pattern has many, such as MM5 computing systems (Mesoscale
5) Model is the relatively broad pattern of current domestic and international application.
The amount of calculation that marine numerical simulation is carried out with MM5 is generally large, it is therefore desirable to carry out on HPCC
Parallel computation.Now with mode of operation in, user generally needs to be connected to high-performance calculation collection using remote character terminal
Group's system, and under command line environment, execution sequence of operations can just complete this numerical simulation, generally comprise following several
Individual flow process:
Prepare the data file of the executable file and its needs of MM5.These files generally need to be placed on the complete of cluster
In the shared storage of office, user generally uses the instruments such as FTP, SFTP and uploads these files, in most of the cases, user to cluster
Also need to modify these files with the text edit tool.
If there is job scheduler in computing cluster system, user needs to write a script text using text editor
Part, wherein fill in that the computing resource quantity of application is needed, and the order of operation MM5 programs, then using job scheduling order
Submit operation to.If no job scheduler, user need by operation MM5 programs enter number of passes and on which node transport
The information such as row pass to the process initiation device of MPI, such as mpirun, mpiexec etc. by way of command line parameter.By MPI
Process initiation device start MM5 programs carrying out parallel computation.
After the completion of program operation, destination file is downloaded using instruments such as FTP, SFTP and check calculating to user's local computer
As a result.
There is following defect in the method for existing operation MM5 calculation procedures:
(1) it is big using difficulty.Want to smoothly complete the calculating task of MM5 programs, user generally requires very familiar a lot
The computer literacy and instrument of meter specialty, such as (SuSE) Linux OS basic operation, Shell scripts, conventional text editing
Use etc. with character handling implement, the use of MPI environment, job scheduling software.
(2) execution efficiency is low.Want smoothly to submit the calculating task of a MM5 program to, need to perform multiple flow processs, need
Complicated loaded down with trivial details operation, so as to the submission for causing parallel computation task is wasted time and energy, inefficiency are performed in command line environment.
(3) error probability is big.For most of domestic consumer, complicated loaded down with trivial details operation is performed in command line environment,
The probability for human operational error occur is very big.It is especially for the unfamiliar user of operating process, equal in the links for performing
It is likely to encounter problem.
(4) security risk is high.Traditional operational approach is needed to domestic consumer's open command row terminal access authority, for
For the manager of HPCC, there is the risk of safety to domestic consumer's open command row terminal.The mistake of user
Operation may affect the stable operation of system, hacker to be also possible to avail oneself of the opportunity to get in.
For the problem in correlation technique, effective solution is not yet proposed at present.
The content of the invention
For the problem in correlation technique, the present invention proposes that a kind of user's centering yardstick Numerical weather forecasting that reduces calculates system
System MM5 submits the Web method for packing and Web package systems of operation to.
The technical scheme is that what is be achieved in that:
On the one hand, there is provided a kind of Web service encapsulation side for mesoscale Numerical weather forecasting computing system MM5
Method, including:
The information related to task is received by Web service interface;
Useful information is extracted by script generator according to the received information related to task and job script is generated
File;
The job script file for being generated is submitted to into computing cluster system;
The computing cluster system calls the mesoscale Numerical weather forecasting to calculate system according to the job script file
Unite MM5 to perform corresponding operation.
In an alternative embodiment, it is before the step of receiving the information related to task by Web service interface, described
Web method for packing also includes:Corresponding application configuration file is obtained according to the application request for receiving;And, answered according to described
Corresponding Web service is generated with the ruuning situation of configuration file and mesoscale Numerical weather forecasting computing system MM5
Interface.
In an alternative embodiment, the information related to task includes one below or combination in any:The mesoscale
Numerical weather forecasting computing system MM5 place path, the catalogue of required input file, perform order, required calculate node, every
The calculating time of the CPU core calculation of individual calculate node and memory size and execution task.
In an alternative embodiment, useful information is extracted by script generator according to the received information related to task
And before generating job script, the Web method for packing also includes:Described in detection, whether the information related to task is legal, inspection
When surveying to be, to perform and useful information is extracted by script generator according to the received information related to task and operation is generated
The step of script;When detecting no, generate illegal information and be displayed on the Web service interface.
In an alternative embodiment, the job script for being generated is submitted to into the meteorological calculating forecast system of the mesoscale numerical value
To perform operation, including:The job script file for being generated is submitted in job scheduling queue, is adjusted by job scheduling system
Spend to perform operation.
In an alternative embodiment, before the information related to task is received by Web service application interface, detection is described
The queuing message of computing cluster system, including:Queue name, total check figure, the operation number in queue and available calculate node can be used
Number.
In an alternative embodiment, before the information related to task being received by Web service interface, detect the magnetic of user
Disk quota and charging amount.
Provide in another aspect a kind of Web service encapsulation system for mesoscale Numerical weather forecasting computing system MM5
System, including:
Web application modules, for receiving the information related to task by Web service interface;
Script generation module, for according to the received information retrieval useful information related to task and generating operation
Script file;
Module is submitted in operation to, for the job script for being generated file is submitted to computing cluster system;So as to the meter
Calculate group system to call mesoscale Numerical weather forecasting computing system MM5 to perform accordingly according to the job script file
Operation.
In an alternative embodiment, the Web application modules include:Command Line Parsing module, for according to the application for receiving
The corresponding application configuration file of acquisition request;And, Web service interface generation module, for according to the application configuration file
And the ruuning situation of mesoscale Numerical weather forecasting computing system MM5 generates corresponding Web service interface.
In an alternative embodiment, the Web service package system also includes detection module, described with task phase for detecting
Whether the information of pass is legal, and when being detected as being, the script generation module is carried according to the received information related to task
Take useful information and generate job script file;When detecting no, generate illegal information and be displayed in Web service circle
On face.
Specific embodiment
Below in conjunction with the accompanying drawings the embodiment of the present invention is further described.
Fig. 1 is that the Web service for mesoscale Numerical weather forecasting computing system MM5 according to an embodiment of the invention is sealed
The flow chart of dress method.As shown in figure 1, the Web service method for packing includes:
S101, receives the information related to task by Web service interface.
Alternatively, before step S101, the Web service method for packing can also be included according to the application request for receiving
Obtain corresponding application configuration file;According to the application configuration file and the mesoscale Numerical weather forecasting computing system
The ruuning situation of MM5 generates corresponding Web service interface.
In embodiments of the present invention, for each of mesoscale Numerical weather forecasting computing system MM5 is applied, can all have one
Individual corresponding application configuration file.The application configuration file includes:Corresponding page items initial value, Command Line Parsing can root
Command Line Parsing is carried out according to the ruuning situation of computing cluster system and the practical situation of application, corresponding five application page is generated,
It is exactly corresponding Web service interface.
Alternatively, the information related to task can include:It is located in the mesoscale Numerical weather forecasting computing system
Path, the catalogue of required input file and perform order.
Alternatively, before step S101, simultaneously or after can also receive and will perform by the Web service interface
The related computing resource information of operation, the computing resource information includes:CPU core calculation, memory size and calculating time.
Alternatively, user can also be chosen whether using Virtual network computer (Virtual by Web service interface
Network Computing, VNC) function or CheckPoint functions (i.e. breakpoint again computing function).Alternatively, user may be used also
With by Web service interface submit to computing cluster system in node, single node core number, network speed agreement, performs order, apply
Required input file etc..
S102, extracts useful information by script generator according to the received information related to task and generates operation
Script file.
The job script file for being generated is submitted to computing cluster system by S103, so as to the computing cluster system root
Call mesoscale Numerical weather forecasting computing system MM5 to perform corresponding operation according to the job script file.
In an alternative embodiment, the Web method for packing also included before step S102:It is related to task described in detection
Information it is whether legal, when being detected as being, execution step S103;When detecting no, generate illegal information and be displayed in institute
State on Web service interface.
In an alternative embodiment, computing cluster system includes job scheduling system, and step S103 includes:By the work for being generated
During industry script file submits the job scheduling queue of computing cluster system to, it is scheduled for calling by job scheduling system described
Mesoscale Numerical weather forecasting computing system MM5 is performing corresponding operation.
In an alternative embodiment, before step S101, Web method for packing also includes:Detect the computing cluster system
Queuing message, including:Queue name, total check figure, the operation number in queue and available calculate node number can be used.
In an alternative embodiment, before the information related to task being received by Web service interface, detect the magnetic of user
Disk quota and charging amount.
Fig. 2 is that the Web service for mesoscale Numerical weather forecasting computing system MM5 according to embodiments of the present invention is encapsulated
The schematic block diagram of system.As schematically shown in Figure 2, the Web service package system includes:Web application modules 10, for being taken by Web
Business interface receives the information related to task;Script generation module 20, for according to the received information related to task
Extract useful information and generate job script file;And, module 30 is submitted in operation to, for the job script file that will be generated
Submit to computing cluster system;So that the computing cluster system calls the mesoscale numerical value according to the job script file
Weather forecast computing system MM5 is performing corresponding operation.
In an alternative embodiment, the information related to task includes one below or combination in any:The mesoscale
Numerical weather forecasting computing system MM5 place path, the catalogue of required input file, perform order, required calculate node, every
The calculating time of the CPU core calculation of individual calculate node and memory size and execution task.
In an alternative embodiment, the Web application modules include:Command Line Parsing module, for according to the application for receiving
The corresponding application configuration file of acquisition request;And, Web service interface generation module, for according to the application configuration file
And the ruuning situation of mesoscale Numerical weather forecasting computing system MM5 generates corresponding Web service interface.
In an alternative embodiment, the Web service package system also includes detection module, described with task phase for detecting
Whether the information of pass is legal, and when being detected as being, the script generation module is carried according to the received information related to task
Take useful information and generate job script file;When detecting no, generate illegal information and be displayed in Web service circle
On face.
In an alternative embodiment, operation submits to module that the job script file for being generated is submitted to computing cluster system bag
Include:The job script file for being generated is submitted in the job scheduling queue of computing cluster system, is carried out by job scheduling system
Job scheduling performs corresponding operation to call mesoscale Numerical weather forecasting computing system MM5.
In an alternative embodiment, the detection module is additionally operable to receive the letter related to task in Web service application interface
Before breath, the queuing message of the computing cluster system is detected, including:Queue name, total check figure, the operation number in queue can be used
With available calculate node number.
In an alternative embodiment, the Web service package system also includes amount computing module, for being taken by Web
Before business interface receives the information related to task, the Disk Quotas and charging amount of user are detected, so that Web service encapsulation is
According to the Disk Quotas and charging amount of user, system determines whether that the operation for the user provides Web packing services.Further,
Computing cluster system can be also used for according to the Disk Quotas of user and distribute corresponding resource for the user.
The present invention is encapsulated by Web, submits to flow process to be hidden in the operation of mesoscale Numerical weather forecasting computing system MM5
Bottom, there is provided to the Web page of one simple, intuitive of user, user only needs to execution shirtsleeve operation in Web page just can be real
Existing data file preparation, computing resource application, operation submission, result such as check at the entire work flow.Encapsulated by Web, can be significantly
The use threshold of user is reduced, service efficiency is improved, reduces error probability.
Its advantage can be embodied in the following aspects:
(1) reduce using difficulty.User no longer need to be grasped (SuSE) Linux OS basic operation, Shell scripts,
Conventional text editing and character handling implement, the calling of MPI environment, job scheduling software etc. are unrelated with its specialty background to be known
Know and instrument, can just be easily accomplished the work such as preparation, the execution of mesoscale Numerical weather forecasting computing system MM5 calculating task,
Greatly reduce and use threshold.
(2) improve work efficiency.The order line of very complicated operates Hide All in bottom, holds in bottom automatic batch
OK, the substantial amounts of manual operation time can be saved, service efficiency is improved.
(3) improve the success rate that task is submitted to.In the embodiment of the present invention, the parameter that user can be submitted in web interface and
Data carry out various validity checkings.Such as, check whether the computing resource of user's application exceedes its authority, the ginseng of user input
Whether number is legal, and whether the input file form of user is correct etc., if it is determined that making mistake, can be given in web interface and accordingly carrying
Show.The success rate of parallel task submission can be greatly improved by these validity checkings.
(4) sophisticated functions can easily be realized.Some when mesoscale Numerical weather forecasting computing system MM5 is run are complicated
Or Premium Features, the selection of such as calculating network, start cpu process binding etc., if needing to specify multiple by command line mode
Miscellaneous parameter, it is cumbersome, it is larger using difficulty.After carrying out Web encapsulation, user is only needed to by some Option Boxs on interface
Can be achieved with these functions.
(5) improve the safety in utilization of HPCC system.Operation is submitted to be encapsulated in the task of domestic consumer
After Web, what user performed is all enclosed operation, and largely avoiding affects the user of system stability or safety illegal
Operation.
Presently preferred embodiments of the present invention is the foregoing is only, not to limit the present invention, all essences in the present invention
Within god and principle, any modification, equivalent substitution and improvements made etc. should be included within the scope of the present invention.