CN104363095A - Method for establishing hadoop identity authentication mechanism - Google Patents
Method for establishing hadoop identity authentication mechanism Download PDFInfo
- Publication number
- CN104363095A CN104363095A CN201410645216.3A CN201410645216A CN104363095A CN 104363095 A CN104363095 A CN 104363095A CN 201410645216 A CN201410645216 A CN 201410645216A CN 104363095 A CN104363095 A CN 104363095A
- Authority
- CN
- China
- Prior art keywords
- hadoop
- principal
- hdfs
- mapred
- keytab
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Storage Device Security (AREA)
Abstract
The invention discloses a method for establishing a hadoop identity authentication mechanism. The method comprises the following steps that a KDC server is established; a hafs principal, a mapred principal and an HTTP principal are established for each node in a hadoop cluster; a hafs.keytab file including the hdfs principals and the HTTP principals is established; a mapred.keytab file including the mapred principals and the HTTP principals is established. According to the method for establishing the hadoop identity authentication mechanism, the problems that in the prior art, hadoop access control is not safe, and cluster nodes can be simulated are solved.
Description
Technical field
The present invention relates to field of identity authentication, particularly relate to the ID authentication mechanism construction method of a kind of hadoop.
Background technology
Large data age is that hadoop provides many chances.Hadoop relies on its good autgmentability in mass data, efficiently read-write and disposal ability, receives attention.But in the behind of numerous opportunity, hadoop is also faced with a lot of challenge.How to ensure that the fail safe of hadoop has just become a topic can't steer clear of naturally, do not have access control, the data stored wherein can arbitrarily be accessed even performs misoperations such as revising, can cause a lot of potential risks.Therefore, the access control of hadoop, just becomes a very important demand.
Do not containing in the hadoop environment of authentication, user and hadoop distributed file system (HDFS) or M/R do not need to verify when carrying out mutual, malicious user can disguise oneself as real user or server invasion hadoop cluster, malice submits mapreduce operation to, change JobTracker state, the data etc. on amendment HDFS.
Current HDFS adds the purview certification of file and catalogue, but these can only shield to accidental loss of data.Malicious user can disguise oneself as other users easily to revise authority, causes priority assignation to perform practically no function, can not play safety guarantee to hadoop cluster.
Kerberos agreement is mainly used in the identity verify (Authentication) of computer network, be characterized in that user only need input an authentication information and the bill (ticket-grantingticket) of this checking acquisition just can be relied on to access multiple service, i.e. SSO (Single Sign On).Owing to establishing shared key between each Client and Service, this agreement is made to have suitable fail safe.
Summary of the invention
The invention provides the ID authentication mechanism construction method of a kind of hadoop, with solving, hadoop access control in prior art is dangerous, and clustered node may by the problem of pretending to be.
In order to solve the problems of the technologies described above, the invention provides the ID authentication mechanism construction method of a kind of hadoop, comprising the following steps: build key distribution center (KDC) server; Be respectively all nodes in hadoop cluster and create hdfs principal, mapred principal and HTTP principal; Create the hdfs.keytab file comprising hdfs principal and HTTP principal; Create the mapred.keytab file comprising mapred principal and HTTP principal.
Further, the ID authentication mechanism of described hadoop is completed by kerberos agreement.
Further, the ID authentication mechanism construction method of described hadoop, further comprising the steps of: to use the hdfs.keytab file after merging and mapred.keytab file acquisition certificate, and dispose hdfs.keytab file and mapred.keytab file, can access to make hdfs and mapred user.
The ID authentication mechanism construction method of hadoop provided by the present invention is the ID authentication mechanism reliable, efficient and simple to operate that hadoop cluster provides based on kerberos agreement, guarantee malicious user cannot disguise oneself as real user or server invasion hadoop cluster, submit mapreduce operation to, change JobTracker state, the malicious operation such as the data on amendment HDFS, greatly ensure that the secure safety of hadoop cluster.
Accompanying drawing explanation
The flow chart of the ID authentication mechanism construction method of the hadoop provided according to present pre-ferred embodiments is provided.
Embodiment
Below in conjunction with drawings and the specific embodiments mode, the present invention is described in further detail.
Be described for following existing environment: an available system bag yum source (local, network); The hadoop cluster environment of available three nodes, and the host name of three nodes is respectively: node01.test.com, node02.test.com, node03.test.com.
As shown in Figure 1, concrete steps are described below.
In step S1, the installation kit needed for kerberos service is installed, and revises associated profile, build key distribution center (KDC) server.Specifically, all node installation krb5-workstation, krb5-workstation and dependence bag thereof, KDC node installs krb5-server in addition.
In step S2, creating new principal database is that hadoop cluster uses.Wherein, principal represents the primary entity (such as, client server) participating in kerberos certification.
In step S3, the relevant configuration of change kerberos, creates the keeper of kerberos telemanagement.Specific as follows: change configuration file/etc/krb5.conf; Change configuration file/var/kerberos/krb5kdc/kdc.conf; Change configuration file/var/kerberos/krb5kdc/kadm5.acl; And the same directory that general/etc/krb5.conf copies other nodes to is replaced.
In step S4, be respectively all nodes in hadoop cluster and create principal.Wherein, establishment hdfs principal, mapred principal and HTTP principal is comprised.
Wherein, the mode creating hdfs principal is as follows:
#kadmin.local
Kadmin:addprinc-randkey hdfs/node01.test.com@HADOOP
Kadmin:addprinc-randkey hdfs/node02.test.com@HADOOP
Kadmin:addprinc-randkey hdfs/node03.test.com@HADOOP
The mode creating mapred principal is as follows:
#kadmin.local
Kadmin:addprinc-randkey mapred/node01.test.com@HADOOP
Kadmin:addprinc-randkey mapred/node02.test.com@HADOOP
Kadmin:addprinc-randkey mapred/node03.test.com@HADOOP
The mode creating HTTP principal is as follows:
#kadmin.local
Kadmin:addprinc-randkey HTTP/node01.test.com@HADOOP
Kadmin:addprinc-randkey HTTP/node02.test.com@HADOOP
Kadmin:addprinc-randkey HTTP/node03.test.com@HADOOP
In step S5, create corresponding keytab.In this, create the hdfs.keytab file comprising hdfs principal and HTTPprincipal, and create the mapred.keytab file comprising mapred principal and HTTP principal.
Create the hdfs.keytab file comprising hdfs principal and HTTP principal, as follows:
kadmin:xst-norandkey-k hdfs.keytab hdfs/node01.test.com@HADOOPHTTP/node01.test.com@HADOOP
kadmin:xst-norandkey-k hdfs.keytab hdfs/node02.test.com@HADOOPHTTP/node02.test.com@HADOOP
kadmin:xst-norandkey-k hdfs.keytabhdfs/node03.test.com@HADOOPHTTP/node03.test.com@HADOOP
Create the mapred.keytab file comprising mapred principal and HTTP principal, as follows:
kadmin:xst-norandkey-k mapred.keytabmapred/node01.test.com@HADOOP HTTP/node01.test.com@HADOOP
kadmin:xst-norandkey-k mapred.keytabmapred/node02.test.com@HADOOP HTTP/node02.test.com@HADOOP
kadmin:xst-norandkey-k mapred.keytabmapred/node03.test.com@HADOOP HTTP/node03.test.com@HADOOP
In step S6, use the keytab after merging to obtain certificate, mode is as follows:
#kinit–k–t hdfs.keytab hdfs/node01.inspur.com@HADOOP
#kinit–k–t mapred.keytab mapred/node01.inspur.com@HADOOP
In step S7, dispose kerberos keytab file, make it can be accessed by hdfs and mapred user.Specifically, under the hdfs.keytab generated in step S5 is moved to $ HADOOP_HOME/conf, under the mapred.keytab generated in step S5 is moved to $ HADOOP_HOME/conf
In step S8, add kerberos self-starting and restart service.
In step S9, stop all services of hadoop cluster.
In step S10, change hadoop associated profile, makes hadoop cluster use kerberos agreement to carry out authentication.
Specifically, add in $ HADOOP_HOME/core-site.xml or revise following content:
<property>
<name>hadoop.security.authentication</name>
<value>kerberos</value>
</property>
<property>
<name>hadoop.security.authorization</name>
<value>true</value>
</property>
Following content is added in $ HADOOP_HOME/conf/hdfs-site.xml:
<property>
<name>dfs.namenode.keytab.file</name>
<value>$HADOOP_HOME/conf/hdfs.keytab</value>
</property>
<property>
<name>dfs.namenode.kerberos.principal</name>
<value>hdfs/_HOST@HADOOP</value>
</property>
<property>
<name>dfs.namenode.kerberos.internal.spnego.principal</name>
<value>HTTP/_HOST@HADOOP</value>
</property>
<property>
<name>dfs.secondary.namenode.keytab.file</name>
<value>$HADOOP_HOME/conf/hdfs.keytab</value>
</property>
<property>
<name>dfs.secondary.namenode.kerberos.principal</name>
<value>hdfs/_HOST@HADOOP</value>
</property>
<property>
<name>dfs.secondary.namenode.kerberos.internal.spnego.principal</name>
<value>HTTP/_HOST@HADOOP</value>
</property>
<property>
<name>dfs.datanode.keytab.file</name>
<value>$HADOOP_HOME/conf/hdfs.keytab</value>
</property>
<property>
<name>dfs.datanode.kerberos.principal</name>
<value>hdfs/_HOST@HADOOP</value>
</property>
<property>
<name>dfs.datanode.kerberos.internal.spnego.principal</name>
<value>HTTP/_HOST@HADOOP</value>
</property>
Following content is added in $ HADOOP_HOME/conf/mapred-site.xml:
<property>
<name>mapreduce.jobtracker.kerberos.principal</name>
<value>mapred/_HOST@HADOOP</value>
</property>
<property>
<name>mapreduce.jobtracker.kerberos.https.principal</name>
<value>host/_HOST@HADOOP</value>
</property>
<property>
<name>mapreduce.jobtracker.keytab.file</name>
<value>$HADOOP_HOME/conf/mapred.keytab</value>
</property>
<property>
<name>mapreduce.tasktracker.kerberos.principal</name>
<value>mapred/_HOST@HADOOP</value>
</property>
<property>
<name>mapreduce.tasktracker.kerberos.https.principal</name>
<value>host/_HOST@HADOOP</value>
</property>
<property>
<name>mapreduce.tasktracker.keytab.file</name>
<value>$HADOOP_HOME/conf/mapred.keytab</value>
</property>
And under copying $ HADOOP_HOME core-site.xml, hdfs-site.xml, mapred-site.xml, hdfs.keytab, mapred.keytab to the corresponding catalogue of other nodes.
In step 11, restart hadoop cluster, whether checking all functions are normal.If start successfully, then the ID authentication mechanism of hadoop successfully constructs.
More than show and describe general principle of the present invention and principal character and advantage of the present invention.The present invention is not restricted to the described embodiments; what describe in above-described embodiment and specification just illustrates principle of the present invention; without departing from the spirit and scope of the present invention, the present invention also has various changes and modifications, and these changes and improvements all fall in the claimed scope of the invention.
Claims (3)
1. an ID authentication mechanism construction method of hadoop, is characterized in that, comprise the following steps:
Build key distribution center (KDC) server;
Be respectively all nodes in hadoop cluster and create hdfs principal, mapred principal and HTTP principal;
Create the hdfs.keytab file comprising hdfs principal and HTTP principal;
Create the mapred.keytab file comprising mapred principal and HTTP principal.
2. the ID authentication mechanism construction method of hadoop as claimed in claim 1, is characterized in that: the ID authentication mechanism of described hadoop is completed by kerberos agreement.
3. the ID authentication mechanism construction method of hadoop as claimed in claim 1, it is characterized in that, further comprising the steps of: to use the hdfs.keytab file after merging and mapred.keytab file acquisition certificate, and dispose hdfs.keytab file and mapred.keytab file, can access to make hdfs and mapred user.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410645216.3A CN104363095A (en) | 2014-11-12 | 2014-11-12 | Method for establishing hadoop identity authentication mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410645216.3A CN104363095A (en) | 2014-11-12 | 2014-11-12 | Method for establishing hadoop identity authentication mechanism |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104363095A true CN104363095A (en) | 2015-02-18 |
Family
ID=52530323
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410645216.3A Pending CN104363095A (en) | 2014-11-12 | 2014-11-12 | Method for establishing hadoop identity authentication mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104363095A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106295384A (en) * | 2015-05-21 | 2017-01-04 | ***通信集团重庆有限公司 | A kind of big data platform access control method, device and certificate server |
| CN106375323A (en) * | 2016-09-09 | 2017-02-01 | 浪潮软件股份有限公司 | Method for carrying out kerberos identity authentication in multi-tenant mode |
| WO2017206960A1 (en) * | 2016-06-03 | 2017-12-07 | 中兴通讯股份有限公司 | Data transmission method, data transfer client and data transfer executor |
| CN107483491A (en) * | 2017-09-19 | 2017-12-15 | 山东大学 | The access control method of distributed storage under a kind of cloud environment |
| CN112540830A (en) * | 2020-12-21 | 2021-03-23 | 广州华资软件技术有限公司 | Method for simultaneously supporting multiple Kerberos authentication in single JVM process |
| CN113377454A (en) * | 2021-06-23 | 2021-09-10 | 浪潮云信息技术股份公司 | Method for realizing Flink dynamic connection Kerberos authentication component |
| CN114745130A (en) * | 2022-04-02 | 2022-07-12 | 杭州玳数科技有限公司 | Authentication method and device for multiple KDC data sources |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102457555A (en) * | 2010-10-28 | 2012-05-16 | 中兴通讯股份有限公司 | Security system and method for distributed storage |
-
2014
- 2014-11-12 CN CN201410645216.3A patent/CN104363095A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102457555A (en) * | 2010-10-28 | 2012-05-16 | 中兴通讯股份有限公司 | Security system and method for distributed storage |
Non-Patent Citations (3)
| Title |
|---|
| XJC2694，: ""hadoop添加kerberos认证"", 《HTTP://BLOG.CHINAUNIX.NET/UID-1838361-ID-3243243.HTML》 * |
| 王超，: ""Hadoop框架下的身份认证技术研究"", 《中国优秀硕士学位论文全文数据库-信息科技辑》 * |
| 范学辉，: ""基于HDFS架构的云存储访问控制机制的研究与设计"", 《中国优秀硕士学位论文全文数据库-信息科技辑》 * |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106295384A (en) * | 2015-05-21 | 2017-01-04 | ***通信集团重庆有限公司 | A kind of big data platform access control method, device and certificate server |
| CN106295384B (en) * | 2015-05-21 | 2020-04-10 | ***通信集团重庆有限公司 | Big data platform access control method and device and authentication server |
| WO2017206960A1 (en) * | 2016-06-03 | 2017-12-07 | 中兴通讯股份有限公司 | Data transmission method, data transfer client and data transfer executor |
| CN107465644A (en) * | 2016-06-03 | 2017-12-12 | 中兴通讯股份有限公司 | Data transmission method, data transmission client and data transmission actuator |
| CN107465644B (en) * | 2016-06-03 | 2021-02-23 | 中兴通讯股份有限公司 | Data transmission method, data transmission client and data transmission executor |
| CN106375323A (en) * | 2016-09-09 | 2017-02-01 | 浪潮软件股份有限公司 | Method for carrying out kerberos identity authentication in multi-tenant mode |
| CN107483491A (en) * | 2017-09-19 | 2017-12-15 | 山东大学 | The access control method of distributed storage under a kind of cloud environment |
| CN112540830A (en) * | 2020-12-21 | 2021-03-23 | 广州华资软件技术有限公司 | Method for simultaneously supporting multiple Kerberos authentication in single JVM process |
| CN113377454A (en) * | 2021-06-23 | 2021-09-10 | 浪潮云信息技术股份公司 | Method for realizing Flink dynamic connection Kerberos authentication component |
| CN114745130A (en) * | 2022-04-02 | 2022-07-12 | 杭州玳数科技有限公司 | Authentication method and device for multiple KDC data sources |
| CN114745130B (en) * | 2022-04-02 | 2023-12-08 | 杭州玳数科技有限公司 | Authentication method and device for multi-KDC data source |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104363095A (en) | Method for establishing hadoop identity authentication mechanism | |
| US10911428B1 (en) | Use of metadata for computing resource access | |
| US8769642B1 (en) | Techniques for delegation of access privileges | |
| CN109643242A (en) | Safe design and framework for multi-tenant HADOOP cluster | |
| CN111919417A (en) | System, method and apparatus for implementing super communities and community sidechains for distributed ledger technology with consensus management in a cloud-based computing environment | |
| CN113711536A (en) | Extracting data from a blockchain network | |
| CN104769908A (en) | LDAP-based multi-tenant in-cloud identity management system | |
| CN108092945B (en) | Method and device for determining access authority and terminal | |
| CN107409129B (en) | Use the authorization in accesses control list and the distributed system of group | |
| US20200342136A1 (en) | Localization of did-related claims and data | |
| CN110245843B (en) | Information management method and related device based on blockchain | |
| US11153293B1 (en) | Identity information linking | |
| CN112910904A (en) | Login method and device of multi-service system | |
| Zhao et al. | Attribute-based access control scheme for data sharing on hyperledger fabric | |
| Jeong et al. | Multilateral personal portfolio authentication system based on hyperledger fabric | |
| You et al. | [Retracted] Research and Design of Docker Technology Based Authority Management System | |
| CN117397205A (en) | Booting trust for a decentralised identifier | |
| Li et al. | A 4D‐Role Based Access Control Model for Multitenancy Cloud Platform | |
| Lee et al. | A keystone-based virtual organization management system | |
| CN103581200A (en) | Method and system for achieving fast circulation of structural file among multiple levels of safety domains | |
| CN114297598B (en) | User permission processing method and device | |
| Siebach | The Abacus: A New Approach to Authorization | |
| Musliyana et al. | Integrated Email Management System Based Google Application Programming Interface Using OAuth 2.0 Authorization Protocol | |
| CN106027535A (en) | Campus network security authentication system and method | |
| CN106445892B (en) | Document assembly method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20150218 |