CN110286921A - A kind of distributed big data platform CDH method of automation installation - Google Patents
A kind of distributed big data platform CDH method of automation installation Download PDFInfo
- Publication number
- CN110286921A CN110286921A CN201910568151.XA CN201910568151A CN110286921A CN 110286921 A CN110286921 A CN 110286921A CN 201910568151 A CN201910568151 A CN 201910568151A CN 110286921 A CN110286921 A CN 110286921A
- Authority
- CN
- China
- Prior art keywords
- node
- configuration
- installation
- cdh
- service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000009434 installation Methods 0.000 title claims abstract description 63
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000007689 inspection Methods 0.000 claims abstract description 18
- 230000008569 process Effects 0.000 claims abstract description 12
- 230000007613 environmental effect Effects 0.000 claims abstract description 6
- 238000002360 preparation method Methods 0.000 claims abstract description 6
- 230000002159 abnormal effect Effects 0.000 claims description 14
- 238000012986 modification Methods 0.000 claims description 12
- 230000004048 modification Effects 0.000 claims description 12
- 238000011835 investigation Methods 0.000 claims description 7
- 238000012360 testing method Methods 0.000 claims description 7
- 238000013475 authorization Methods 0.000 claims description 5
- 230000003068 static effect Effects 0.000 claims description 5
- 238000012795 verification Methods 0.000 claims description 5
- 230000009286 beneficial effect Effects 0.000 abstract 1
- 241001269238 Data Species 0.000 description 1
- 241000282326 Felis catus Species 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/61—Installation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Stored Programmes (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a kind of automations to install distributed big data platform CDH method, specifically includes the following steps: S10: environmental preparation；It specifically refers to: the basic environment of each node server being checked, installation resource is got out；S20: script configuration；It is required according to configuration file, the self-defined information configuration of essential information and user to all node servers finishes；S30: installation and deployment；Configure cluster basic environment and big data running environment according to the operational process of perform script, specifically include infrastructure service configuration, build NTP service, SSH exempt from key log in, java environment, installation CDH management console service and service for checking credentials state；S40: result inspection；It logs in CDH management console and checks result.The beneficial effects of the present invention are: the present invention can effectively realize the automation installation of CDH management console；The present invention can make the process of big data cluster building have more high efficiency, reduce fault rate.
Description
Technical field
The present invention relates to computer big data technical fields, are a kind of distributed big datas of automation installation specifically
Platform CDH method.
Background technique
With the rapid development of information age, big data using more and more extensive, but because of the difference of installation system environment
It is different, it produces and many complicated operations of distributed big data platform is installed, number of clusters is more, and cost of labor is higher.
Since distributed big data platform needs to use computer that can be extending transversely, number of clusters from 3 to up to a hundred, on
Thousand have, and data are more, and the probability of human error is higher, and spent cost of labor is higher, urgent need to resolve manual operation
Bring time cost and maloperation problem.
Summary of the invention
The purpose of the present invention is to provide a kind of automations to install distributed big data platform CDH method, effectively reduces
The probability of error reduces cost of labor and time cost.
The present invention is achieved through the following technical solutions:
A kind of distributed big data platform CDH method of automation installation, specifically includes the following steps:
S10: environmental preparation；It specifically refers to: the basic environment of each node server being checked, installation resource is got out；
S20: script configuration；It is required according to configuration file, the customized letter of essential information and user to all node servers
Breath configuration finishes；
S30: installation and deployment；Cluster basic environment and big data running environment are configured according to the operational process of perform script, specifically
Including infrastructure service configuration, build NTP service, SSH exempt from key log in, java environment, installation CDH management console service and
Service for checking credentials state；
S40: result inspection；It logs in CDH management console and checks result.
Further, in order to preferably realize the present invention, in the step S10 to the basic environment of each node server into
Row checks, specifically includes:
Step S11: judge whether the operating system version of each node server meets the requirements；
If it is not, then needing to reinstall operating system；
Step S12: system architecture inspection to each node server operating system judges whether operating system is that x86_64 is
System framework；
Whether step S13: checking the IP network of each node server operating system, test between each node and be connected to；
Step S14: the configuration file, installation kit and perform script of each node server are checked.
Further, in order to preferably realize the present invention, the step S20 is specifically included:
Step S21: IP address, the port SSH, root user password, the user's big data platform peace of all node servers are checked
The self-defined information of dress is configured in deployment file；
If it is not, the place modification for needing not to be arranged rationally for prompt；
Step S22: check whether the configuration information of all node servers is correct and whether self-defined information configuration is reasonable；
If it is not, then needing to adjust for abnormal and unreasonable message.
Further, in order to preferably realize the present invention, the step S30 specifically includes stand-alone configuration installation and matches with online
Set installation: stand-alone configuration installation specifically includes the following steps:
Step S31: for all nodes according to the corresponding relationship of script configuration information, the IP that each server is arranged is static IP, setting
NETWORK service is restarted after the completion；
Step S32: the entitled corresponding system of machine of each server is arranged according to the corresponding relationship of script configuration information in all nodes
One format, and name in sequence；
Step S33: all nodes modify hosts file according to the corresponding relationship of configuration file, and each node IP and right is added
The machine name answered；
Step S34: confirm that mutually each node can be connected to, and there is no packet loss phenomenon, guarantee the normal of network；
Step S35: all nodes variable setting: specifically refer to: modify all node swappiness value and
The value of transparent_hugepage；All node adjustment are configured to booting self-starting modification；
Step S36: the firewall and selinux of all nodes are closed；Cancel booting self-starting.
Further, in order to preferably realize the present invention, the online configuration installation is specifically referred to: being matched according to existing process
Cluster running environment is set, including NTP service, SSH exempt from key and log in, JAVA environment, install CDH management console service, execute
It finishes, verifies each service state operation；Specifically includes the following steps:
Step S311: the running environment of Servers-all is configured；
Step S312: all node server mutual trusts；Judge whether all node verification mutual trusts succeed, passes through SSH to each section
Point, verifies whether configuration successful；
If it is not, the problem of then checking individual server SSH abnormal state reason, checking permission；
Step S313: check whether all nodes are Chinese time zones；If it is not, being then modified to Chinese time zone；
Step S314: the source yum of all node configuration systems, using management node as HTTP server, remaining all node configuration；
Whether the source yum for verifying all nodes configures correctly；
Step S315: all node installations build NTP service, and the time server for configuring all nodes is same Time Service
Device, and verify whether success；
If it is not, then check individual server abnormal cause, investigation whether NTP component install failure or inactive exception；
Step S316: installation mysql service；
Step S317: installation foundation operation service component；
Step S318: starting serviced component；It specifically refers to: CDH management console service is started according to the configuration management node,
And server is added in booting self-starting；According to the inspection service for checking credentials state；By log inspection and port test,
Verifying starts successfully.
Further, in order to preferably realize the present invention, the step S316 is specifically included:
According to the configuration management node installation mysql database；
Mysql configuration file is modified according to the configuration management node；
Mysql is arranged according to the configuration management node to service as booting self-starting；
According to the configuration management node initializing mysql data, creation access user, authorization access；
Verify the access state of mysql database.
Further, in order to preferably realize the present invention, the step S317 is specifically included:
According to all node installation java running environment of configuration；
According to all node installation mysql drive environments of configuration；
Console service is managed according to the configuration management node installation CDH；
The metadatabase used according to the configuration CDH management console service.
Further, in order to preferably realize the present invention, the step S14 is specifically referred to:
After judging whether the configuration file of the management node modifies IP user, password, modification according to current cluster case of machines
Machine name；
Judge under the management node catalogue with the presence or absence of the configuration file and installation source.
Compared with prior art, the present invention have the following advantages that and the utility model has the advantages that
(1) present invention can effectively realize the automation installation of CDH management console；
(2) present invention can make the process of big data cluster building have more high efficiency, reduce fault rate.
Specific embodiment
The present invention is described in further detail below with reference to embodiment, embodiments of the present invention are not limited thereto.
Embodiment 1:
The present invention is achieved through the following technical solutions, and a kind of distributed big data platform CDH method of automation installation specifically includes
Following steps:
S10: environmental preparation；It specifically refers to: the basic environment of each node server being checked, installation resource is got out；Institute
It states node serve and basic environment includes, the system version of each node server, system digit, network environment；
The basic environment of each node server is checked in the step S10, is specifically included:
Step S11: judge whether the operating system version of each node server meets the requirements；The preferred operating system version
For Red Hat Enterprise Linux 6/CentOS6.x family release；
If it is not, then needing to reinstall operating system；
Step S12: system architecture inspection to each node server operating system judges whether operating system is that x86_64 is
System framework；Since big data platform only supports the system architecture of x86_64, this is checked using uname-r order, checks output
It whether include x86_64.
Whether step S13: checking the IP network of each node server operating system, test between each node and connect
It is logical；If it is not, then needing to server network environment inspection；
Step S14: the configuration file, installation kit and perform script of each node server are checked；It specifically refers to:
After judging whether the configuration file of the management node modifies IP user, password, modification according to current cluster case of machines
Machine name；
Judge under the management node catalogue with the presence or absence of the configuration file and installation source.
S20: script configuration；It is required according to configuration file, the essential information and user to all node servers are made by oneself
Adopted information configuration finishes；
S30: installation and deployment；Cluster basic environment and big data running environment are configured according to the operational process of perform script, specifically
Including infrastructure service configuration, build NTP service, SSH exempt from key log in, java environment, installation CDH management console service and
Service for checking credentials state；
S40: result inspection；It logs in CDH management console and checks result.
The other parts of the present embodiment are same as the previously described embodiments, and so it will not be repeated.
Embodiment 2:
The present embodiment advanced optimizes on the basis of the above embodiments, and the step S20 is specifically included:
Step S21: IP address, the port SSH, root user password, the user's big data platform peace of all node servers are checked
The self-defined information of dress is configured in deployment file；
If it is not, the place modification for needing not to be arranged rationally for prompt；
Step S22: check whether the configuration information of all node servers is correct and whether self-defined information configuration is reasonable；
If it is not, then needing to adjust for abnormal and unreasonable message.
The other parts of the present embodiment are same as the previously described embodiments, and so it will not be repeated.
Embodiment 3:
The present embodiment advanced optimizes on the basis of the above embodiments, and the step S30 specifically includes stand-alone configuration installation
Install with online configuration: the stand-alone configuration installation specifically refers to: configuring the basic environment of Servers-all, specifically includes following
Step:
Step S31: for all nodes according to the corresponding relationship of script configuration information, the IP that each server is arranged is static IP, setting
NETWORK service is restarted after the completion；If it is not, checking server network, again instauration net according to the abnormal conditions of respective nodes
Network；If so, entering step S32；
Step S32: the entitled corresponding system of machine of each server is arranged according to the corresponding relationship of script configuration information in all nodes
One format, and name in sequence；
Step S33: all nodes modify hosts file according to the corresponding relationship of configuration file, and each node IP and right is added
The machine name answered；
Step S34: confirm that mutually each node can be connected to, and there is no packet loss phenomenon, guarantee the normal of network；If cannot connect
It is logical, then the problem of checking individual server network condition, check hardware；If can be connected to, S35 is entered step；
Step S35: all nodes variable setting: specifically refer to: modify all node swappiness value and
The value of transparent_hugepage；All node adjustment are configured to booting self-starting modification；It specifically refers to:
The value for modifying all node swappiness is arranged it and changes the setting, general/proc/sys/vm/ at runtime
Swappiness is set as 0；
It is booting self-starting for all node adjustment configurations, is set in addition/etc/sysctl.conf with saving this after restart
It sets, which will not lose cat>>/etc/sysctl.conf<<EOF of EOF vm.swappiness=0 after restarting；
All node adjustment configurations, modify the value of transparent_hugepage, execution comes into force
echo never > /sys/kernel/mm/transparent_hugepage/defrag；
It is booting self-starting for all nodes adjustment configuration, and starting script is added, with the echo that permanently comes into force " echo never >
/sys/kernel/mm/transparent_hugepage/defrag" >>/etc/rc.local；
Step S36: the firewall and selinux of all nodes are closed；Cancel booting self-starting；It specifically refers to:
Close the firewall of all nodes, service iptables save
service iptables stop ；
chkconfig iptables off ；
service ip6tables save ；
service ip6tables stop；
chkconfig ip6tables off；
Preferably, further includes:
The selinux of all nodes is closed,
setenforce 0 sed -i.bak 's/SELINUX=enforcing/SELINUX=disabled/' /etc/
sysconfig/selinux；
sed -i.bak 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config；
All nodes execute reboot and restart；
The other parts of the present embodiment are same as the previously described embodiments, and so it will not be repeated.
Embodiment 4:
The present embodiment advanced optimizes on the basis of the above embodiments, and the online configuration installation specifically refers to: according to both
There is process to configure cluster running environment, including NTP service, SSH exempt from key and log in, JAVA environment, install CDH management console clothes
Business, is finished, and verifies each service state operation；Specifically includes the following steps:
Step S311: the running environment of Servers-all is configured；
Step S312: all node server mutual trusts；It specifically refers to: each node private key is copied to each server, join
Examine the following SSH-copy-id-i root@bigdata-a-001 of order；
Each server described herein includes the machine, needs to configure master in/etc/hosts when carrying out private key copy
Machine name and IP corresponding relationship.
Judge whether all node verification mutual trusts succeed, by SSH to each node, verifies whether configuration successful；
If it is not, the problem of then checking individual server SSH abnormal state reason, checking permission；If configuration successful enters step
S313；
Step S313: check whether all nodes are Chinese time zones；If it is not, being then modified to Chinese time zone；It specifically refers to:
Check whether all nodes are Chinese time zones, returned the result by date inspection, if includes CST；
If it is not, being then modified to Chinese time zone ln-sf/usr/share/zoneinfo/Asia/Shanghai/etc/
localtime；
Step S314: the source yum of all node configuration systems, using management node as HTTP server, remaining all node configuration；
Whether the source yum for verifying all nodes configures correctly；
All node verification yum states check whether output is normal by using yum search, to verify whether the source yum matches
It sets normal；
If it is not, then check configuration repo file under the yum.repos.d file of the source yum whether mistake；
Step S315: all node installations build NTP service, and the time server for configuring all nodes is same Time Service
Device, and verify whether success；
If it is not, then check individual server abnormal cause, investigation whether NTP component install failure or inactive exception；
Step S316: installation mysql service；It specifically includes
According to the configuration management node installation mysql database；
Mysql configuration file is modified according to the configuration management node；The mysql configuration file includes mysql port numbers, sets
Set ignorecase, character set encoding；
Mysql is arranged according to the configuration management node to service as booting self-starting；
According to the configuration management node initializing mysql data, creation access user, authorization access；Verify mysql database
Access state；Specifically refer to: management node initialization data library includes creation CDH management console metadatabase, and according to
The customized telnet user name creation remote access user of user, authorization remote access；
All node verification database connection status；
If it is not, then checking individual server abnormal cause, whether investigation database starts or user remotely accesses authorization and loses
It loses；
If so, all node installation java environments and installation database connection, yum-y install oracle-
j2sdk1.7 mysql-connector-java；
If it is not, then checking individual server abnormal cause, investigation is with the presence or absence of execution rights concerns；
If so, management node installation CDH manages console service；
If it is not, then checking management node server exception reason, investigation whether there is the problem of missing relies on；
If so, the metadatabase that management node configuration CDH management console service uses；
If it is not, then checking management node server mysql database, check database connection with the presence or absence of abnormal；
Step S317: installation foundation operation service component；
Step S318: starting serviced component；It specifically refers to: CDH management console service is started according to the configuration management node,
And server is added in booting self-starting；According to the inspection service for checking credentials state；By log inspection and port test,
Verifying starts successfully.
Management node server starts CDH and manages console service；
If it is not, then checking management node server exception log, investigation is abnormal with the presence or absence of database connection；
If so, verifying starts successfully by log inspection and port test.
The other parts of the present embodiment are same as the previously described embodiments, and so it will not be repeated.
Embodiment 5:
The present embodiment advanced optimizes on the basis of the above embodiments, and current most of servers are all LINUX servers,
It supports shell script, and the present embodiment provides a kind of automations under LINUX server based on shell script to install distribution
Formula big data platform CDH method, " shell script " technology is sequencing script, is a kind of computer program and text file, interior
Appearance is made of a series of shell-command, is literal translated via UNIX SHELL and is operated after its content.It is perceived as a kind of scripting language
It designs, function mode is suitable with interpreted languages, and the role of command line interpreter is played the part of by UNIX SHELL, reading
After shell script, shell-command therein is sequentially run, exports result later.System can be carried out using shell script
Management, file operation etc..
S10, environmental preparation；
It is required that server has been mounted with Red Hat Enterprise Linux 6/CentOS6.x operating system, and grasp
Making system digit must be x86_64, and system IP network is normal, and firewall has been switched off, the RPM installation kit in the source yum on
Pass to management node.
During specific implementation, step S10 " environmental preparation " needs to complete workflow below, installs x86_64
Red Hat Enterprise Linux 6/CentOS6.x operating system, manually perform order close firewall, check net
Whether network normal, guarantee all nodes each other can SSH access, the RPM installation kit in the source yum uploads to management node server
Under specified directory.
S20, script configuration；
It is required that completing the modification of configuration file according to the case where current all nodes and the remarks of configuration file.
In specific implementation procedure, step S20 " configuration script " needs to complete workflow below, according to all nodes
Actual conditions, according to the remarks of configuration file, IP, root password, port numbers and the big data for filling in current each node are flat
Inside the customized new static ip address and machine name for needing to change to configuration script file of platform, configuration file and foot is executed
Originally management node specified directory is uploaded to.
S30, installation and deployment；
In specific implementation procedure, step S30 is needed in management node perform script, can be obtained inside configuration file current
The essential information of server, verifies the basic environment of system, if encountering exception can prompt to modify, finishes to operation maintenance personnel adjustment
Later, script will continue to execute, and the operation such as static IP and modification machine name can be configured according to process, can be according to both after completion
There is process installation configuration cluster running environment, including NTP service, SSH exempt from key and log in, JAVA environment, install CDH management control
Platform service etc., is finished, proofing state.
As a result S40 is checked, log in CDH management console interface inspection state；
In the embodiment of the present application, step S40 " result inspection " needs to log in CDH management console, verifies its state.
In specific implementation procedure, step S40 only needs to log in CDH management console, and whether the service checked starts, verify
Whether state is normal, and the process that distributed big data platform CDH is installed in this automation finishes.
The other parts of the present embodiment are same as the previously described embodiments, and so it will not be repeated.
The above is only presently preferred embodiments of the present invention, not does limitation in any form to the present invention, it is all according to
According to technical spirit any simple modification to the above embodiments of the invention, equivalent variations, protection of the invention is each fallen within
Within the scope of.
Claims (8)
1. distributed big data platform CDH method is installed in a kind of automation, it is characterised in that: specifically includes the following steps:
S10: environmental preparation；It specifically refers to: the basic environment of each node server being checked, installation resource is got out；
S20: script configuration；It is required according to configuration file, the customized letter of essential information and user to all node servers
Breath configuration finishes；
S30: installation and deployment；Cluster basic environment and big data running environment are configured according to the operational process of perform script, specifically
Including infrastructure service configuration, build NTP service, SSH exempt from key log in, java environment, installation CDH management console service and
Service for checking credentials state；
S40: result inspection；It logs in CDH management console and checks result.
2. distributed big data platform CDH method is installed in a kind of automation according to claim 1, it is characterised in that: institute
It states in step S10 and the basic environment of each node server is checked, specifically include:
Step S11: judge whether the operating system version of each node server meets the requirements；
If it is not, then needing to reinstall operating system；
Step S12: system architecture inspection to each node server operating system judges whether operating system is that x86_64 is
System framework；
Whether step S13: checking the IP network of each node server operating system, test between each node and be connected to；
Step S14: the configuration file, installation kit and perform script of each node server are checked.
3. distributed big data platform CDH method is installed in a kind of automation according to claim 2, it is characterised in that: institute
Step S20 is stated to specifically include:
Step S21: IP address, the port SSH, root user password, the user's big data platform peace of all node servers are checked
The self-defined information of dress is configured in deployment file；
If it is not, the place modification for needing not to be arranged rationally for prompt；
Step S22: check whether the configuration information of all node servers is correct and whether self-defined information configuration is reasonable；
If it is not, then needing to adjust for abnormal and unreasonable message.
4. distributed big data platform CDH method is installed in a kind of automation according to claim 1, it is characterised in that: institute
State step S30 and specifically include stand-alone configuration installation and online configuration installation: stand-alone configuration installation specifically includes the following steps:
Step S31: for all nodes according to the corresponding relationship of script configuration information, the IP that each server is arranged is static IP, setting
NETWORK service is restarted after the completion；
Step S32: the entitled corresponding system of machine of each server is arranged according to the corresponding relationship of script configuration information in all nodes
One format, and name in sequence；
Step S33: all nodes modify hosts file according to the corresponding relationship of configuration file, and each node IP and right is added
The machine name answered；
Step S34: confirm that mutually each node can be connected to, and there is no packet loss phenomenon, guarantee the normal of network；
Step S35: all nodes variable setting: specifically refer to: modify all node swappiness value and
The value of transparent_hugepage；All node adjustment are configured to booting self-starting modification；
Step S36: the firewall and selinux of all nodes are closed；Cancel booting self-starting.
5. distributed big data platform CDH method is installed in a kind of automation according to claim 4, it is characterised in that: institute
It states online configuration installation to specifically refer to: cluster running environment being configured according to existing process, including NTP service, SSH are exempted from key and stepped on
Land, JAVA environment, installation CDH manage console service, are finished, and verify each service state operation；It specifically includes following
Step:
Step S311: the running environment of Servers-all is configured；
Step S312: all node server mutual trusts；Judge whether all node verification mutual trusts succeed, passes through SSH to each section
Point, verifies whether configuration successful；
If it is not, the problem of then checking individual server SSH abnormal state reason, checking permission；
Step S313: check whether all nodes are Chinese time zones；If it is not, being then modified to Chinese time zone；
Step S314: the source yum of all node configuration systems, using management node as HTTP server, remaining all node configuration；
Whether the source yum for verifying all nodes configures correctly；
Step S315: all node installations build NTP service, and the time server for configuring all nodes is same Time Service
Device, and verify whether success；
If it is not, then check individual server abnormal cause, investigation whether NTP component install failure or inactive exception；
Step S316: installation mysql service；
Step S317: installation foundation operation service component；
Step S318: starting serviced component；It specifically refers to: CDH management console service is started according to the configuration management node,
And server is added in booting self-starting；According to the inspection service for checking credentials state；By log inspection and port test,
Verifying starts successfully.
6. distributed big data platform CDH method is installed in a kind of automation according to claim 5, it is characterised in that: institute
Step S316 is stated to specifically include:
According to the configuration management node installation mysql database；
Mysql configuration file is modified according to the configuration management node；
Mysql is arranged according to the configuration management node to service as booting self-starting；
According to the configuration management node initializing mysql data, creation access user, authorization access；
Verify the access state of mysql database.
7. distributed big data platform CDH method is installed in a kind of automation according to claim 6, it is characterised in that: institute
Step S317 is stated to specifically include:
According to all node installation java running environment of configuration；
According to all node installation mysql drive environments of configuration；
Console service is managed according to the configuration management node installation CDH；
The metadatabase used according to the configuration CDH management console service.
8. distributed big data platform CDH method is installed in a kind of automation according to claim 7, it is characterised in that: institute
Step S14 is stated to specifically refer to:
After judging whether the configuration file of the management node modifies IP user, password, modification according to current cluster case of machines
Machine name；
Judge under the management node catalogue with the presence or absence of the configuration file and installation source.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910568151.XA CN110286921B (en) | 2019-06-27 | 2019-06-27 | CDH method for automatically installing distributed big data platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910568151.XA CN110286921B (en) | 2019-06-27 | 2019-06-27 | CDH method for automatically installing distributed big data platform |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110286921A true CN110286921A (en) | 2019-09-27 |
| CN110286921B CN110286921B (en) | 2023-11-10 |
Family
ID=68019236
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910568151.XA Active CN110286921B (en) | 2019-06-27 | 2019-06-27 | CDH method for automatically installing distributed big data platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110286921B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111142887A (en) * | 2019-12-27 | 2020-05-12 | 焦点科技股份有限公司 | Automatic CDH installation method |
| CN112631643A (en) * | 2019-10-08 | 2021-04-09 | 华晨宝马汽车有限公司 | Comprehensive operation and maintenance management method, system, equipment and medium |
| CN112836220A (en) * | 2021-02-07 | 2021-05-25 | 浪潮云信息技术股份公司 | Cloud center environment inspection method |
| CN113434157A (en) * | 2021-06-30 | 2021-09-24 | 青岛海尔科技有限公司 | Shell script-based front-end engineering deployment method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104317610A (en) * | 2014-10-11 | 2015-01-28 | 福建新大陆软件工程有限公司 | Method and device for automatic installation and deployment of hadoop platform |
| CN107563153A (en) * | 2017-08-03 | 2018-01-09 | 华子昂 | A kind of PacBio microarray dataset IT architectures based on Hadoop structures |
| CN107682209A (en) * | 2017-11-10 | 2018-02-09 | 青岛萨纳斯智能科技股份有限公司 | A kind of SDP big datas automatically dispose monitor supervision platform |
| CN109710281A (en) * | 2018-12-28 | 2019-05-03 | 中科曙光国际信息产业有限公司 | The installation method and device of big data platform |
| US20190146839A1 (en) * | 2017-11-14 | 2019-05-16 | Bank Of America Corporation | Distributed data platform resource allocator |
-
2019
- 2019-06-27 CN CN201910568151.XA patent/CN110286921B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104317610A (en) * | 2014-10-11 | 2015-01-28 | 福建新大陆软件工程有限公司 | Method and device for automatic installation and deployment of hadoop platform |
| CN107563153A (en) * | 2017-08-03 | 2018-01-09 | 华子昂 | A kind of PacBio microarray dataset IT architectures based on Hadoop structures |
| CN107682209A (en) * | 2017-11-10 | 2018-02-09 | 青岛萨纳斯智能科技股份有限公司 | A kind of SDP big datas automatically dispose monitor supervision platform |
| US20190146839A1 (en) * | 2017-11-14 | 2019-05-16 | Bank Of America Corporation | Distributed data platform resource allocator |
| CN109710281A (en) * | 2018-12-28 | 2019-05-03 | 中科曙光国际信息产业有限公司 | The installation method and device of big data platform |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112631643A (en) * | 2019-10-08 | 2021-04-09 | 华晨宝马汽车有限公司 | Comprehensive operation and maintenance management method, system, equipment and medium |
| CN112631643B (en) * | 2019-10-08 | 2024-05-24 | 华晨宝马汽车有限公司 | Comprehensive operation and maintenance management method, system, equipment and medium |
| CN111142887A (en) * | 2019-12-27 | 2020-05-12 | 焦点科技股份有限公司 | Automatic CDH installation method |
| CN112836220A (en) * | 2021-02-07 | 2021-05-25 | 浪潮云信息技术股份公司 | Cloud center environment inspection method |
| CN113434157A (en) * | 2021-06-30 | 2021-09-24 | 青岛海尔科技有限公司 | Shell script-based front-end engineering deployment method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110286921B (en) | 2023-11-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110286921A (en) | A kind of distributed big data platform CDH method of automation installation | |
| US20090307763A1 (en) | Automated Test Management System and Method | |
| CN115437925A (en) | Functional automation test tool for operating system | |
| US11966880B2 (en) | Policies and controls for building and validating database software in a shared management environment | |
| CN110007932B (en) | Automatic deployment method of large data processing system based on local area network | |
| WO2023050110A1 (en) | Method for implementing automatic password change having fault tolerance mechanism for cloud host and cloud bastion host | |
| US20230125904A1 (en) | Recommendation system for building and validating database software in a shared management environment | |
| Cisco | Release Notes for the Catalyst 2950 Switch Ciso IOS Release 12.1(6)EA2c | |
| Cisco | Initial Installation | |
| Cisco | Troubleshooting | |
| Cisco | Release Notes for the Catalyst 2950 Switch Cisco IOS Release 12.1(6)EA2b | |
| Cisco | Release Notes for the Catalyst 2950 Switch Cisco IOS Release 12.1(6)EA2a | |
| Cisco | Release Notes for the Catalyst 2950 Switch Cisco IOS Release 12.1(6)EA2 | |
| WO2020258175A1 (en) | Method for automatically installing distributed big data platform cdh | |
| CN112631897A (en) | Method for quickly debugging linux kernel | |
| JP2016115182A (en) | Virtual equipment test device, virtual equipment test method and virtual equipment test program | |
| WO2023069500A1 (en) | Techniques for building and validating database software in a shared management environment | |
| Choi et al. | Cisco Router Upgrading Playbook | |
| Han et al. | Network security experiment simulation platform based on CTSL | |
| Fröhlingsdorf | Continuous Integration in CFMGR | |
| CN117421012A (en) | Cluster visual installation method, device, equipment and medium | |
| Kuusisto | Centralized data center provisioning and configuration in managed service provider environment | |
| CN118410477A (en) | Password modification method, device, computer program product, equipment and medium | |
| Younker | A Build for Every Check-In | |
| CN116400935A (en) | Cross-platform deployment system and method based on domestic platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |