CN109885584A - The implementation method and terminal device of distributed data analyzing platform - Google Patents
The implementation method and terminal device of distributed data analyzing platform Download PDFInfo
- Publication number
- CN109885584A CN109885584A CN201910080165.7A CN201910080165A CN109885584A CN 109885584 A CN109885584 A CN 109885584A CN 201910080165 A CN201910080165 A CN 201910080165A CN 109885584 A CN109885584 A CN 109885584A
- Authority
- CN
- China
- Prior art keywords
- plan
- logic
- pig
- data
- distributed data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Devices For Executing Special Programs (AREA)
Abstract
The present invention is suitable for data analysis technique field, provide the implementation method and terminal device of a kind of distributed data analyzing platform, this method comprises: the query analysis text that setting is write using Pig Latin language, it is the Pig logic plan accorded with comprising logical relationship operations by the query analysis text resolution, convert the Pig logic plan to the logic executive plan of memory computation model, the logic executive plan is mapped as physics plan, to solve in the prior art in the large-scale data epoch, offline batch application scene is suitable only for using traditional distributed data analyzing processing platform, it can not carry out calculating in real time and interactive mode calculates, the problem for causing the process efficiency of Data Analysis Services lower.
Description
Technical field
The invention belongs to the implementation method of data analysis technique field more particularly to a kind of distributed data analyzing platform and
Terminal device.
Background technique
With the continuous development of Internet technology, the data on internet are increased with exponential speed, additionally, due to number
According to the diversity in source, the isomerism of data format is caused, for example the data of existing structure there are non-structured data again,
Even there are also partly-structured datas.In big data era, the mode of traditional database and operating system locally stored file
Obviously not applicable, traditional one-of-a-kind system oneself through incompetent so huge data analysis task, and it is traditional based on decimal
The analysis based on big data quantity is also far below according to the accuracy of analysis of the algorithm of amount as a result, in addition single Open Framework can not
Complete, general data analysis solution is provided.Traditional distributed data analyzing processing platform is normally based on
What MapReduce was realized, however Hadoop MapReduc is suitable only for offline batch application scene, with big data field
Fast development, calculates in real time and interactive calculating scene is more and more common, and the defect of MapReduce gradually exposes.
Summary of the invention
In view of this, the embodiment of the invention provides a kind of implementation methods of distributed data analyzing platform and terminal to set
It is standby, to solve in the prior art in the large-scale data epoch, using traditional distributed data analyzing processing platform be suitable only for from
Line batch application scene can not carry out calculating in real time and interactive mode calculates, cause the process efficiency of Data Analysis Services lower
The problem of.
The first aspect of the embodiment of the present invention provides a kind of implementation method of distributed data analyzing platform, comprising:
The query analysis text write using Pig Latin language is set；
It is the Pig logic plan accorded with comprising logical relationship operations by the query analysis text resolution；
Convert the Pig logic plan to the logic executive plan of memory computation model；
The logic executive plan is mapped as physics plan, each node of the physics in the works includes an execution
Method, the execution method is for inquiry or analysis distribution formula data intensive data.
In one embodiment, described by the query analysis text resolution is the Pig logic accorded with comprising logical relationship operations
Plan, comprising:
According to the syntax rule of Pig and syntax parsing tool, the query analysis text is subjected to syntax parsing, construction is taken out
As syntax tree；
The abstract syntax tree is traversed, the basic component units of inquiry or analysis are obtained；
The Pig logic plan comprising logical relationship operations symbol is generated according to the basic component units.
In one embodiment, it is generated according to the basic component units comprising described in logical relationship operations symbol described
After Pig logic plan, further includes:
According to preset rules, optimize the Pig logic plan.
In one embodiment, the logic executive plan for converting the Pig logic plan to memory computation model, packet
It includes:
The Pig logic plan is converted to using conversion class the logic executive plan of memory computation model, the conversion
Class comprises at least one of the following: Pig Translation Visitor, Logical Plan Translation Visitor,
Expression Plan Translation Visitor and Nested Plan Translation Visitor.
It is in one embodiment, described that the logic executive plan is mapped as physics plan, comprising:
A series of logic is converted to after the logic executive plan is optimized executes step；
Logic execution step is converted into physics plan.
In one embodiment, which is characterized in that in the query analysis text that the setting is write using Pig Latin language
Before, further includes:
Acquire structural data, semi-structured data and unstructured data；
The structural data, the semi-structured data and the unstructured data of acquisition are stored.
In one embodiment, the acquisition structural data, semi-structured data and unstructured data, comprising:
According to preset data collection model, by data acquisition interface to structural data, semi-structured data and non-knot
Structure data are acquired.
The second aspect of the embodiment of the present invention provides a kind of realization device of distributed data analyzing platform, comprising:
Setup module, for the query analysis text for using Pig Latin language to write to be arranged；
Parsing module, for being the Pig logic plan accorded with comprising logical relationship operations by the query analysis text resolution；
Task Switching module, for converting the Pig logic plan to the logic executive plan of memory computation model；
Query optimization module, for the logic executive plan to be mapped as physics plan, the physics in the works every
A node includes an execution method, and the execution method is for inquiry or analysis distribution formula data intensive data.
The third aspect of the embodiment of the present invention provides a kind of terminal device, comprising: memory, processor and is stored in
In the memory and the computer program that can run on the processor, when the processor executes the computer program
Realize the step as described in the implementation method of above-mentioned distributed data analyzing platform.
The fourth aspect of the embodiment of the present invention provides a kind of computer readable storage medium, comprising: the computer can
It reads storage medium and is stored with computer program, realize when the computer program is executed by processor such as above-mentioned distributed data point
Analyse step described in the implementation method of platform.
Existing beneficial effect is the embodiment of the present invention compared with prior art: by setting using Pig Latin language
The query analysis text resolution is the Pig logic plan accorded with comprising logical relationship operations by the query analysis text write, will
The Pig logic plan is converted into the logic executive plan of memory computation model, and the logic executive plan is mapped as physics
Pig language, can be eventually converted into the operation to distributed data collection of memory Computational frame by plan, and the embodiment of the present invention will
The simple Pig language of form is combined with the efficient memory computation model of operation, so that the process of Data Analysis Services is more square
Just efficiently.
Detailed description of the invention
It to describe the technical solutions in the embodiments of the present invention more clearly, below will be to embodiment or description of the prior art
Needed in attached drawing be briefly described, it should be apparent that, the accompanying drawings in the following description is only of the invention some
Embodiment for those of ordinary skill in the art without any creative labor, can also be according to these
Attached drawing obtains other attached drawings.
Fig. 1 is a kind of implementation process signal of the implementation method of distributed data analyzing platform provided in an embodiment of the present invention
Figure；
Fig. 2 is that the implementation process of the implementation method of another distributed data analyzing platform provided in an embodiment of the present invention is shown
It is intended to；
Fig. 3 is the exemplary diagram of the realization device of distributed data analyzing platform provided in an embodiment of the present invention；
Fig. 4 is the schematic diagram of terminal device provided in an embodiment of the present invention.
Specific embodiment
In being described below, for illustration and not for limitation, the tool of such as particular system structure, technology etc is proposed
Body details, to understand thoroughly the embodiment of the present invention.However, it will be clear to one skilled in the art that there is no these specific
The present invention also may be implemented in the other embodiments of details.In other situations, it omits to well-known system, device, electricity
The detailed description of road and method, in case unnecessary details interferes description of the invention.
In order to illustrate technical solutions according to the invention, the following is a description of specific embodiments.
The embodiment of the present invention provides a kind of realization of distributed data analyzing platform, as shown in Figure 1, this method includes following
Step:
Step 101, the query analysis text that setting is write using Pig Latin language.
The query analysis text write using Pig Latin language, and it is uploaded to the distributed data in the present embodiment point
Analyse platform.
Optionally, before step 101, further includes:
Acquire structural data, semi-structured data and unstructured data.Optionally, mould is acquired according to preset data
Type is acquired structural data, semi-structured data and unstructured data by data acquisition interface.In the present embodiment
When being acquired to data, using a kind of general data collecting model, and in order to cope with different Data Concurrent amounts, provide
In addition data acquisition interface based on differing transmission protocols provides data buffer storage and distribution function to the data by verifying
Energy.
Further, to the structural data of acquisition, the semi-structured data and the unstructured data into
Row storage.A variety of storage modes are combined in the present embodiment.
It step 102, is the Pig logic plan accorded with comprising logical relationship operations by the query analysis text resolution.
Optionally, the distributed data analyzing platform in the present embodiment carries out the solution of program language using class SQL resolver
The query analysis text resolution of upload is the Pig logic plan comprising logical relationship operations symbol, then to the Pig by analysis
Logic plan is tentatively optimized.
It is further alternative, as shown in Fig. 2, step 102 may include following sub-step.
Step 1021, according to the syntax rule of Pig and syntax parsing tool, the query analysis text is subjected to grammer solution
Analysis constructs abstract syntax tree.
Optionally, parse () function in the GruntParser class of Pig platform is according to PigScriptParser.jj text
The generation of part completion abstract syntax tree.
Step 1022, the abstract syntax tree is traversed, the basic component units of inquiry or analysis are obtained.
Step 1023, the Pig logic plan comprising logical relationship operations symbol is generated according to the basic component units.
Optionally, parse () method of the QueryParserDriverde of PigServer can traverse parsing abstract syntax
It sets and returns to logic plan, specific logic executive plan is then generated by LogicalPlanGenera-tor class.
Further, according to preset rules, optimize the Pig logic plan.It is initialized by Pig Server
HExecution Engine, and it is combined into different optimisation strategies for different situations, then by HExecu-tion
Engine.compile (Logical Plan, Properties) optimizes logic plan.
Step 103, the Pig logic plan is converted to the logic executive plan of memory computation model.
Optionally, this step is the core of distributed data analyzing platform, includes multiple customized conversion classes, institute
It states conversion class to comprise at least one of the following: Pig Translation Visitor, Logical Plan Translation
Visitor, Expression Plan Translation Visitor and Nested Plan Translation Visitor.
Step 104, the logic executive plan is mapped as physics plan, each node of the physics in the works includes
One execution method, the execution method is for inquiry or analysis distribution formula data intensive data.
Optionally, logic executive plan is optimized, such as converts pipelining for continuous mapping and executes, it will be more
A operation is merged into a step etc..Computing engines are handled based on large-scale data in distributed data analyzing platform in this way
Logic plan is just switched to series of steps by frame, and each step is made of multiple tasks, and wherein task is at large-scale data
It manages the smallest working cell in computing engines frame, then converts specific physics plan for these logic steps, i.e., it will be excellent
Logic executive plan after change is mapped to physical operations class, these operation classes are all the operations of distributed data collection, finally by object
Reason plan is transferred to execute node execution.Specific DAGScheduler (the oriented nothing for executing logic and giving memory computation model completely
Ring figure scheduler) cutting is gone, cutting operation is multiple operating process, and Internal reforming is multiple tasks collection, by Task
Scheduler (task dispatcher) completes the scheduling of task, and corresponding node is finally transferred to be executed.
As shown in Fig. 2, step 104 specifically includes following sub-step:
Step 1041, a series of logic is converted to after the logic executive plan being optimized executes step.
Step 1042, logic execution step is converted into physics plan.
Further, each node of the physics in the works includes that an execute () executes method, the side of execution
The operation of one or more distributed data collection can be performed for inquiry or analysis distribution formula data intensive data in method, these
The operation of distributed data collection is exactly the method for data query analysis.
The embodiment of the present invention provides a kind of implementation method of distributed data analyzing platform, uses Pig by setting
The query analysis text that Latin language is write；It is that the Pig accorded with comprising logical relationship operations is patrolled by the query analysis text resolution
Collect plan；Convert the Pig logic plan to the logic executive plan of memory computation model；The logic executive plan is reflected
Penetrate as physics plan, each node of the physics in the works includes an executions method, the execution method for inquiry or
Analysis distribution formula data intensive data.
Linguistic level-the PigLatin of traditional Data Analysis Platform Pig is handled into computing engines frame with based on big data
It is integrated, the present embodiment calculates the linguistic level-PigLatin of traditional Data Analysis Platform Pig with based on big data processing
The advantages of engine framework is integrated, and Hadoop MapReduce has both been remained has good fault-tolerance and scalability etc., together
When overcome defect of the MapReduce in terms of iterative calculating and interactive calculating.By the simple Pig language of form and operation
Efficient memory computation model combines, so that the process of Data Analysis Services is more convenient efficiently, while providing complete number
According to analysis platform, tradition is saved using model and does some cumbersome operations when data processing.
It should be understood that the size of the serial number of each step is not meant that the order of the execution order in above-described embodiment, each process
Execution sequence should be determined by its function and internal logic, the implementation process without coping with the embodiment of the present invention constitutes any limit
It is fixed.
The embodiment of the present invention provides a kind of realization device of distributed data analyzing platform, as shown in figure 3, the device packet
It includes: setup module 301, parsing module 302, Task Switching module 303, query optimization module 304.
Setup module 301, for the query analysis text for using Pig Latin language to write to be arranged.
Further, described device is also used to acquire structural data, semi-structured data and unstructured data；To adopting
The structural data, the semi-structured data and the unstructured data of collection are stored.
Parsing module 302, by being based on the Pig logic accorded with comprising logical relationship operations by the query analysis text resolution
It draws.
Optionally, the parsing module 302 is also used to syntax rule and syntax parsing tool according to Pig, looks into described
It askes analysis text and carries out syntax parsing, construct abstract syntax tree；The abstract syntax tree is traversed, inquiry is obtained or is analyzed basic
Component units；The Pig logic plan comprising logical relationship operations symbol is generated according to the basic component units.According to default
Rule optimizes the Pig logic plan.
Task Switching module 303, based on the logic execution for converting the Pig logic plan to memory computation model
It draws.
Optionally, the Task Switching module 303 is also used to convert the Pig logic plan to using conversion class interior
The logic executive plan of computation model is deposited, the conversion class comprises at least one of the following: Pig Translation Visitor,
Logical Plan Translation Visitor, Expression Plan Translation Visitor and Nested
Plan Translation Visitor。
Query optimization module 304, for the logic executive plan to be mapped as physics plan, the physics is in the works
Each node includes an execution method, and the execution method is for inquiry or analysis distribution formula data intensive data.
The query optimization module 304 is also used to be converted to after optimizing the logic executive plan a series of
Logic executes step；Logic execution step is converted into physics plan.
The embodiment of the present invention provides a kind of realization device of distributed data analyzing platform, by setup module, for setting
Set the query analysis text write using Pig Latin language；Parsing module, for being packet by the query analysis text resolution
The Pig logic plan of the symbol containing logical relationship operations；Task Switching module, by converting the Pig logic plan to based on memory
Calculate the logic executive plan of model；Query optimization module can be incited somebody to action for the logic executive plan to be mapped as physics plan
Pig language is eventually converted into the operation to distributed data collection of memory Computational frame, by the simple Pig language of form and operation
Efficient memory computation model combines, so that the process of Data Analysis Services is more convenient efficiently.
Fig. 4 is the schematic diagram for the terminal device that one embodiment of the invention provides.As shown in figure 4, the terminal of the embodiment is set
Standby 4 include: processor 401, memory 402 and are stored in the memory 402 and can run on the processor 401
Computer program 403, such as the realization program of distributed data analyzing platform.The processor 401 executes the computer
The step in the implementation method embodiment of above-mentioned distributed data analyzing platform, such as step shown in FIG. 1 are realized when program 403
101 to 104.The processor 401 realizes the function of each module in above-mentioned each Installation practice when executing the computer program 403
Can, such as the function of module 301 to 304 shown in Fig. 3.
Illustratively, the computer program 403 can be divided into one or more modules, one or more of
Module is stored in the memory 402, and is executed by the processor 401, to complete the present invention.It is one or more of
Module can be the series of computation machine program instruction section that can complete specific function, and the instruction segment is for describing the computer
Implementation procedure of the program 403 in the realization device or terminal device 4 of the distributed data analyzing platform.For example, described
Computer program 403 can be divided into setup module 301, parsing module 302, Task Switching module 303, query optimization module
304, each module concrete function is as shown in figure 3, this is no longer going to repeat them.
The terminal device 4 can be the calculating such as desktop PC, notebook, palm PC and cloud server and set
It is standby.The terminal device may include, but be not limited only to, processor 401, memory 402.It will be understood by those skilled in the art that
Fig. 4 is only the example of terminal device 4, does not constitute the restriction to terminal device 4, may include more more or fewer than illustrating
Component, perhaps combines certain components or different components, for example, the terminal device can also include input-output equipment,
Network access equipment, bus etc..
Alleged processor 401 can be central processing unit (Central Processing Unit, CPU), can also be
Other general processors, digital signal processor (Digital Signal Processor, DSP), specific integrated circuit
(Application Specific Integrated Circuit, ASIC), ready-made programmable gate array (Field-
Programmable Gate Array, FPGA) either other programmable logic device, discrete gate or transistor logic,
Discrete hardware components etc..General processor can be microprocessor or the processor is also possible to any conventional processor
Deng.
The memory 402 can be the internal storage unit of the terminal device 4, for example, terminal device 4 hard disk or
Memory.The memory 402 is also possible to the External memory equipment of the terminal device 4, such as is equipped on the terminal device 4
Plug-in type hard disk, intelligent memory card (Smart Media Card, SMC), secure digital (Secure Digital, SD) card,
Flash card (Flash Card) etc..Further, the memory 402 can also have been deposited both the inside including the terminal device 4
Storage unit also includes External memory equipment.The memory 402 is for storing the computer program and the terminal device 4
Other required programs and data.The memory 402, which can be also used for temporarily storing, have been exported or will export
Data.
It is apparent to those skilled in the art that for convenience of description and succinctly, only with above-mentioned each function
Can unit, module division progress for example, in practical application, can according to need and by above-mentioned function distribution by different
Functional unit, module are completed, i.e., the internal structure of described device is divided into different functional unit or module, more than completing
The all or part of function of description.Each functional unit in embodiment, module can integrate in one processing unit, can also
To be that each unit physically exists alone, can also be integrated in one unit with two or more units, it is above-mentioned integrated
Unit both can take the form of hardware realization, can also realize in the form of software functional units.In addition, each function list
Member, the specific name of module are also only for convenience of distinguishing each other, the protection scope being not intended to limit this application.Above system
The specific work process of middle unit, module, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.
In the above-described embodiments, it all emphasizes particularly on different fields to the description of each embodiment, is not described in detail or remembers in some embodiment
The part of load may refer to the associated description of other embodiments.
Those of ordinary skill in the art may be aware that list described in conjunction with the examples disclosed in the embodiments of the present disclosure
Member and algorithm steps can be realized with the combination of electronic hardware or computer software and electronic hardware.These functions are actually
It is implemented in hardware or software, the specific application and design constraint depending on technical solution.Professional technician
Each specific application can be used different methods to achieve the described function, but this realization is it is not considered that exceed
The scope of the present invention.
In embodiment provided by the present invention, it should be understood that disclosed device/terminal device and method, it can be with
It realizes by another way.For example, device described above/terminal device embodiment is only schematical, for example, institute
The division of module or unit is stated, only a kind of logical function partition, there may be another division manner in actual implementation, such as
Multiple units or components can be combined or can be integrated into another system, or some features can be ignored or not executed.Separately
A bit, shown or discussed mutual coupling or direct-coupling or communication connection can be through some interfaces, device
Or the INDIRECT COUPLING or communication connection of unit, it can be electrical property, mechanical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme
's.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit
It is that each unit physically exists alone, can also be integrated in one unit with two or more units.Above-mentioned integrated list
Member both can take the form of hardware realization, can also realize in the form of software functional units.
If the integrated module/unit be realized in the form of SFU software functional unit and as independent product sale or
In use, can store in a computer readable storage medium.Based on this understanding, the present invention realizes above-mentioned implementation
All or part of the process in example method, can also instruct relevant hardware to complete, the meter by computer program
Calculation machine program can be stored in a computer readable storage medium, the computer program when being executed by processor, it can be achieved that on
The step of stating each embodiment of the method.Wherein, the computer program includes computer program code, the computer program
Code can be source code form, object identification code form, executable file or certain intermediate forms etc..Computer-readable Jie
Matter may include: can carry the computer program code any entity or device, recording medium, USB flash disk, mobile hard disk,
Magnetic disk, CD, computer storage, read-only memory (ROM, Read-Only Memory), random access memory (RAM,
Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium etc..It should be noted that described
The content that computer-readable medium includes can carry out increasing appropriate according to the requirement made laws in jurisdiction with patent practice
Subtract, such as does not include electric carrier signal and electricity according to legislation and patent practice, computer-readable medium in certain jurisdictions
Believe signal.
Embodiment described above is merely illustrative of the technical solution of the present invention, rather than its limitations；Although referring to aforementioned reality
Applying example, invention is explained in detail, those skilled in the art should understand that: it still can be to aforementioned each
Technical solution documented by embodiment is modified or equivalent replacement of some of the technical features；And these are modified
Or replacement, the spirit and scope for technical solution of various embodiments of the present invention that it does not separate the essence of the corresponding technical solution should all
It is included within protection scope of the present invention.
Claims (10)
1. a kind of implementation method of distributed data analyzing platform characterized by comprising
The query analysis text write using Pig Latin language is set；
It is the Pig logic plan accorded with comprising logical relationship operations by the query analysis text resolution；
Convert the Pig logic plan to the logic executive plan of memory computation model；
The logic executive plan is mapped as physics plan, each node of the physics in the works includes the side of execution
Method, the execution method is for inquiry or analysis distribution formula data intensive data.
2. the implementation method of distributed data analyzing platform as described in claim 1, which is characterized in that described by the inquiry
Analyzing text resolution is the Pig logic plan accorded with comprising logical relationship operations, comprising:
According to the syntax rule of Pig and syntax parsing tool, the query analysis text is subjected to syntax parsing, constructs abstract language
Method tree；
The abstract syntax tree is traversed, the basic component units of inquiry or analysis are obtained；
The Pig logic plan comprising logical relationship operations symbol is generated according to the basic component units.
3. the implementation method of distributed data analyzing platform as claimed in claim 2, which is characterized in that described according to
Basic component units generate after the Pig logic plan comprising logical relationship operations symbol, further includes:
According to preset rules, optimize the Pig logic plan.
4. the implementation method of distributed data analyzing platform as claimed in claim 3, which is characterized in that described by the Pig
Logic plan is converted into the logic executive plan of memory computation model, comprising:
The Pig logic plan is converted to using conversion class the logic executive plan of memory computation model, the conversion class packet
Include following at least one: Pig Translation Visitor, Logical Plan Translation Visitor,
Expression Plan Translation Visitor and Nested Plan Translation Visitor.
5. the implementation method of distributed data analyzing platform as claimed in claim 4, which is characterized in that described by the logic
Executive plan is mapped as physics plan, comprising:
A series of logic is converted to after the logic executive plan is optimized executes step；
Logic execution step is converted into physics plan.
6. the implementation method of the distributed data analyzing platform as described in any one of claims 1 to 5, which is characterized in that
Before the query analysis text that the setting uses Pig Latin language to write, further includes:
Acquire structural data, semi-structured data and unstructured data；
The structural data, the semi-structured data and the unstructured data of acquisition are stored.
7. the implementation method of distributed data analyzing platform as claimed in claim 6, which is characterized in that the acquisition structuring
Data, semi-structured data and unstructured data, comprising:
According to preset data collection model, by data acquisition interface to structural data, semi-structured data and unstructured
Data are acquired.
8. a kind of realization device of distributed data analyzing platform characterized by comprising
Setup module, for the query analysis text for using Pig Latin language to write to be arranged；
Parsing module, for being the Pig logic plan accorded with comprising logical relationship operations by the query analysis text resolution；
Task Switching module, for converting the Pig logic plan to the logic executive plan of memory computation model；
Query optimization module, for the logic executive plan to be mapped as physics plan, each section of the physics in the works
Point includes an execution method, and the execution method is for inquiry or analysis distribution formula data intensive data.
9. a kind of terminal device, including memory, processor and storage are in the memory and can be on the processor
The computer program of operation, which is characterized in that the processor realizes such as claim 1 to 7 when executing the computer program
The step of any one the method.
10. a kind of computer readable storage medium, the computer-readable recording medium storage has computer program, and feature exists
In when the computer program is executed by processor the step of any one of such as claim 1 to 7 of realization the method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910080165.7A CN109885584A (en) | 2019-01-28 | 2019-01-28 | The implementation method and terminal device of distributed data analyzing platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910080165.7A CN109885584A (en) | 2019-01-28 | 2019-01-28 | The implementation method and terminal device of distributed data analyzing platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109885584A true CN109885584A (en) | 2019-06-14 |
Family
ID=66927117
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910080165.7A Pending CN109885584A (en) | 2019-01-28 | 2019-01-28 | The implementation method and terminal device of distributed data analyzing platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109885584A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111027196A (en) * | 2019-12-03 | 2020-04-17 | 南方电网科学研究院有限责任公司 | Simulation analysis task processing method and device for power equipment and storage medium |
| CN112818663A (en) * | 2021-01-15 | 2021-05-18 | 北京有竹居网络技术有限公司 | Processing method for language model, text generation method, text generation device and medium |
| CN113515285A (en) * | 2020-04-10 | 2021-10-19 | 北京沃东天骏信息技术有限公司 | Method and device for generating real-time calculation logic data |
| CN116594977A (en) * | 2023-05-10 | 2023-08-15 | 阿里巴巴达摩院(杭州)科技有限公司 | Distributed processing system for remote sensing data |
| CN116775276A (en) * | 2023-05-10 | 2023-09-19 | 阿里巴巴达摩院(杭州)科技有限公司 | Distributed computing method, device, equipment and storage medium |
-
2019
- 2019-01-28 CN CN201910080165.7A patent/CN109885584A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111027196A (en) * | 2019-12-03 | 2020-04-17 | 南方电网科学研究院有限责任公司 | Simulation analysis task processing method and device for power equipment and storage medium |
| CN111027196B (en) * | 2019-12-03 | 2023-04-28 | 南方电网科学研究院有限责任公司 | Simulation analysis task processing method and device for power equipment and storage medium |
| CN113515285A (en) * | 2020-04-10 | 2021-10-19 | 北京沃东天骏信息技术有限公司 | Method and device for generating real-time calculation logic data |
| CN112818663A (en) * | 2021-01-15 | 2021-05-18 | 北京有竹居网络技术有限公司 | Processing method for language model, text generation method, text generation device and medium |
| CN116594977A (en) * | 2023-05-10 | 2023-08-15 | 阿里巴巴达摩院(杭州)科技有限公司 | Distributed processing system for remote sensing data |
| CN116775276A (en) * | 2023-05-10 | 2023-09-19 | 阿里巴巴达摩院(杭州)科技有限公司 | Distributed computing method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109885584A (en) | The implementation method and terminal device of distributed data analyzing platform | |
| CN107480789B (en) | Efficient conversion method and device of deep learning model | |
| Dayal et al. | Flexpath: Type-based publish/subscribe system for large-scale science analytics | |
| CN105512162B (en) | A kind of flow data real-time intelligentization processing frame based on Storm | |
| CN109709811A (en) | A kind of edge calculations controller and its working method towards intelligence manufacture | |
| CN108932588B (en) | Hydropower station group optimal scheduling system with separated front end and rear end and method | |
| CN109542399A (en) | Software development methodology, device, terminal device and computer readable storage medium | |
| CN107632845B (en) | Integrated knowledge cloud service method and system for multi-axis milling of runner blade | |
| WO2015094269A1 (en) | Hybrid flows containing a continuous flow | |
| Yamashina et al. | crecomp: Automated design tool for ros-compliant fpga component | |
| CN101944113B (en) | Cloud computing system-based data acquisition system | |
| CN105468756A (en) | Design and implementation method of mass data processing system | |
| CN111159897A (en) | Target optimization method and device based on system modeling application | |
| CN108769065A (en) | Rapid safety product docking system based on protocol conversion technology | |
| CN102103519A (en) | Method for optimizing Web service execution based on scripts | |
| Ritter et al. | Hardware accelerated application integration processing: Industry paper | |
| Eisenhauer et al. | A type system for high performance communication and computation | |
| US9870397B2 (en) | Processing multi-way theta join queries involving arithmetic operators on MapReduce | |
| CN105164667B (en) | Modification analysis stream | |
| CN110442753A (en) | A kind of chart database auto-creating method and device based on OPC UA | |
| CN116775041B (en) | Real-time decision engine implementation method based on stream calculation and RETE algorithm | |
| CN103530132A (en) | Method for transplanting CPU (central processing unit) serial programs to MIC (microphone) platform | |
| CN104239630A (en) | Simulation scheduling system for supporting test design | |
| Li et al. | A fast big data collection system using MapReduce framework | |
| AU2017269259A1 (en) | Data driven invocation of real time wind market forecasting analytics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190614 |