Big Data and Cloud Tips: How to setup password-less ssh to the slaves?
Rant about Big Data, Cloud and related technologies.
For setting up Hadoop on a cluster of machines, the master should be able to do a password-less ssh to start the daemons on all the slaves.
Class MR - master starts TaskTracker and the DataNode on all the slaves.
MRv2 (next generation MR) - master starts NodeManager and the DataNode on all the slaves.
Here are the steps to setup password-ssh.
Ensure that port 22 is open on all the slave (`telnet slave-hostname 22` should connect).
1) Install openssh-client on the master
sudo apt-get install openssh-client
2) Install openssh-server on all the slaves
sudo apt-get install openssh-server
3) Generate the ssh key
ssh-keygen -t rsa -P "" -f ~/.ssh/id_rsa
4) Copy the key to all the slaves (replace username appropriately as the user starting the Hadoop daemons). Will be prompted for the password.
ssh-copy-id -i $HOME/.ssh/id_rsa.pub username@slave-hostname
5) If the master also acts a slave (`ssh localhost` should work without a password)
cat $HOME/.ssh/id_rsa.pub && $HOME/.ssh/authorized_keys
If hdfs/mapreduce are run as
then the steps (3,4 and 5) have to be repeated for all the users.
How to test ?
1) Run `ssh user@slave-hostname`. It should get connected without prompting for a password.
Subscribe to:
Total Pageviews
Blog Archive
My Blog List
This originally came up in an earlier blog comment but it’s an interesting
question and one not necessarily one restricted to the changes driven by
deep le...
I’m writing this post from my hotel room in Tokyo while doing my best to
fight jet lag! I’m here to speak at JAWS Days and Startup Day, and to meet
with so...
My name is Carolyn Duby. I am a Solutions Engineer, a cybersecurity and
data-science expert, and the president of Women @ Hortonworks. My career as
We are excited to announce the general availability of Cloudera Altus SDK
for Java to programmatically leverage the Altus platform-as-a service for
This article titled "Daten müssen strategischer Teil des Gesch?fts werden"
appeared in German last week in the "IT und Datenproduktion" column of
Wirtsch...
@StackOverflow
Subscribe To
Follow by Email
Getting Started
CSV is the most familiar way of storing the data. In this blog I will try to compare the performance aspects of the ORC and the Parquet for...
Besides the JobCounter
and the TaskCounter
counters which Hadoop framework maintains, it's also possible to define custom counters
Also don't forget to do check another entry on how to get some interesting facts from Twitter using R here . And also this entr...
In the previous blog , we looked at on converting the CSV format into Parquet format using Hive. It was a matter of creating a regular tabl...
Apache Hive
is a client side library providing a table like abstraction on top of the data in HDFS for data processing. Hive jobs are co...
For those who wanted to get started with Hadoop as with any other technology there are a couple of steps involved from downloading/install...
There might be a requirement to pass additional parameters to the mapper and reducers, besides the the inputs which they process. Lets say ...
For setting up Hadoop on a cluster of machines, the master should be able to do a password-less ssh to start the daemons on all the slaves....
In the earlier blog entries, we have looked into how install Oozie here
and how to do the Click Stream analysis using Hive and Pig here ...
Many a times the events from the applications have to be analyzed to know more about the customer behavior for recommendations or to figu...SSH原理与运用（一）：远程登录 - 阮一峰的网络日志
SSH原理与运用（一）：远程登录
SSH是每一台Linux电脑的标准配置。
随着Linux设备从电脑逐渐扩展到手机、外设和家用电器，SSH的使用范围也越来越广。不仅程序员离不开它，很多普通用户也每天使用。
SSH具备多种功能，可以用于很多场合。有些事情，没有它就是办不成。本文是我的学习笔记，总结和解释了SSH的常见用法，希望对大家有用。
虽然本文内容只涉及初级应用，较为简单，但是需要读者具备最基本的"Shell知识"和了解"公钥加密"的概念。如果你对它们不熟悉，我推荐先阅读和。
=======================================
SSH原理与运用
作者：阮一峰
一、什么是SSH？
简单说，SSH是一种网络协议，用于计算机之间的加密登录。
如果一个用户从本地计算机，使用SSH协议登录另一台远程计算机，我们就可以认为，这种登录是安全的，即使被中途截获，密码也不会泄露。
最早的时候，互联网通信都是明文通信，一旦被截获，内容就暴露无疑。1995年，芬兰学者Tatu Ylonen设计了SSH协议，将登录信息全部加密，成为互联网安全的一个基本解决方案，迅速在全世界获得推广，目前已经成为Linux系统的标准配置。
需要指出的是，SSH只是一种协议，存在多种实现，既有商业实现，也有开源实现。本文针对的实现是，它是自由软件，应用非常广泛。
此外，本文只讨论SSH在Linux Shell中的用法。如果要在Windows系统中使用SSH，会用到另一种软件，这需要另文介绍。
二、最基本的用法
SSH主要用于远程登录。假定你要以用户名user，登录远程主机host，只要一条简单命令就可以了。
　　$ ssh user@host
如果本地用户名与远程用户名一致，登录时可以省略用户名。
　　$ ssh host
SSH的默认端口是22，也就是说，你的登录请求会送进远程主机的22端口。使用p参数，可以修改这个端口。
　　$ ssh -p 2222 user@host
上面这条命令表示，ssh直接连接远程主机的2222端口。
三、中间人攻击
SSH之所以能够保证安全，原因在于它采用了公钥加密。
整个过程是这样的：（1）远程主机收到用户的登录请求，把自己的公钥发给用户。（2）用户使用这个公钥，将登录密码加密后，发送回来。（3）远程主机用自己的私钥，解密登录密码，如果密码正确，就同意用户登录。
这个过程本身是安全的，但是实施的时候存在一个风险：如果有人截获了登录请求，然后冒充远程主机，将伪造的公钥发给用户，那么用户很难辨别真伪。因为不像https协议，SSH协议的公钥是没有证书中心（CA）公证的，也就是说，都是自己签发的。
可以设想，如果攻击者插在用户与远程主机之间（比如在公共的wifi区域），用伪造的公钥，获取用户的登录密码。再用这个密码登录远程主机，那么SSH的安全机制就荡然无存了。这种风险就是著名的（Man-in-the-middle attack）。
SSH协议是如何应对的呢？
四、口令登录
如果你是第一次登录对方主机，系统会出现下面的提示：
　　$ ssh user@host
　　The authenticity of host 'host (12.18.429.21)' can't be established.
　　RSA key fingerprint is 98:2e:d7:e0:de:9f:ac:67:28:c2:42:2d:37:16:58:4d.
　　Are you sure you want to continue connecting (yes/no)?
这段话的意思是，无法确认host主机的真实性，只知道它的公钥指纹，问你还想继续连接吗？
所谓"公钥指纹"，是指公钥长度较长（这里采用RSA算法，长达1024位），很难比对，所以对其进行MD5计算，将它变成一个128位的指纹。上例中是98:2e:d7:e0:de:9f:ac:67:28:c2:42:2d:37:16:58:4d，再进行比较，就容易多了。
很自然的一个问题就是，用户怎么知道远程主机的公钥指纹应该是多少？回答是没有好办法，远程主机必须在自己的网站上贴出公钥指纹，以便用户自行核对。
假定经过风险衡量以后，用户决定接受这个远程主机的公钥。
　　Are you sure you want to continue connecting (yes/no)? yes
系统会出现一句提示，表示host主机已经得到认可。
　　Warning: Permanently added 'host,12.18.429.21' (RSA) to the list of known hosts.
然后，会要求输入密码。
　　Password: (enter password)
如果密码正确，就可以登录了。
当远程主机的公钥被接受以后，它就会被保存在文件$HOME/.ssh/known_hosts之中。下次再连接这台主机，系统就会认出它的公钥已经保存在本地了，从而跳过警告部分，直接提示输入密码。
每个SSH用户都有自己的known_hosts文件，此外系统也有一个这样的文件，通常是/etc/ssh/ssh_known_hosts，保存一些对所有用户都可信赖的远程主机的公钥。
五、公钥登录
使用密码登录，每次都必须输入密码，非常麻烦。好在SSH还提供了公钥登录，可以省去输入密码的步骤。
所谓"公钥登录"，原理很简单，就是用户将自己的公钥储存在远程主机上。登录的时候，远程主机会向用户发送一段随机字符串，用户用自己的私钥加密后，再发回来。远程主机用事先储存的公钥进行解密，如果成功，就证明用户是可信的，直接允许登录shell，不再要求密码。
这种方法要求用户必须提供自己的公钥。如果没有现成的，可以直接用ssh-keygen生成一个：
　　$ ssh-keygen
运行上面的命令以后，系统会出现一系列提示，可以一路回车。其中有一个问题是，要不要对私钥设置口令（passphrase），如果担心私钥的安全，这里可以设置一个。
运行结束以后，在$HOME/.ssh/目录下，会新生成两个文件：id_rsa.pub和id_rsa。前者是你的公钥，后者是你的私钥。
这时再输入下面的命令，将公钥传送到远程主机host上面：
　　$ ssh-copy-id user@host
好了，从此你再登录，就不需要输入密码了。
如果还是不行，就打开远程主机的/etc/ssh/sshd_config这个文件，检查下面几行前面"#"注释是否取掉。
　　RSAAuthentication yes
　　PubkeyAuthentication yes
　　AuthorizedKeysFile .ssh/authorized_keys
然后，重启远程主机的ssh服务。
　　// ubuntu系统
　　service ssh restart
　　// debian系统
　　/etc/init.d/ssh restart
六、authorized_keys文件
远程主机将用户的公钥，保存在登录后的用户主目录的$HOME/.ssh/authorized_keys文件中。公钥就是一段字符串，只要把它追加在authorized_keys文件的末尾就行了。
这里不使用上面的ssh-copy-id命令，改用下面的命令，解释公钥的保存过程：
　　$ ssh user@host 'mkdir -p .ssh && cat && .ssh/authorized_keys' & ~/.ssh/id_rsa.pub
这条命令由多个语句组成，依次分解开来看：（1）"$ ssh user@host"，表示登录远程主机；（2）单引号中的mkdir .ssh && cat && .ssh/authorized_keys，表示登录后在远程shell上执行的命令：（3）"$ mkdir -p .ssh"的作用是，如果用户主目录中的.ssh目录不存在，就创建一个；（4）'cat && .ssh/authorized_keys' & ~/.ssh/id_rsa.pub的作用是，将本地的公钥文件~/.ssh/id_rsa.pub，重定向追加到远程文件authorized_keys的末尾。
写入authorized_keys文件后，公钥登录的设置就完成了。
==============================================
关于shell远程登录的部分就写到这里，下一次接着介绍。
学习编程其实就是学高级语言，即那些为人类设计的计算机语言。
去年，比特币暴涨，其他币也像雨后春笋一样冒出来，已经有1000多种了。
比特币（bitcoin）诞生于2008年的一篇论文。
区块链（blockchain）是眼下的大热门，新闻媒体大量报道，宣称它将创造未来。Access denied | www.cyberciti.biz used Cloudflare to restrict access
Please enable cookies.
What happened?
The owner of this website (www.cyberciti.biz) has banned your access based on your browser's signature (3fa5dd7e7d346df0-ua98).How to SSH to an iPhone (or iPad)
You may know already that
has the same underlying unix architecture as , and because of this you can SSH into an iPhone or iPad just like you would connect to any other Mac or unix based machine.
As it turns out for us nerdier users, this ability is disabled without a jailbreak, so in order to SSH into your iOS device you will need to jailbreak first. How to do this varies on what version of iOS your hardware is using, but you can find our
or just look for the .
To clarify, this is a guide on how to setup the ability to SSH into your iPhone or iPad from another machine, if you are just looking for an SSH client for the iPhone or iPad,
is by far the best, and costs $7.99.
How to Setup SSH and then Connect with SSH to an iPhone or iPad
As previously mentioned, the first thing you need to do is jailbreak, that is out of the scope of this article but it’s easy to do. After your iPhone, iPad, or iPod touch is jailbroken, proceed with the following:
Step 1) From the iOS Device:
Launch Cydia and search for and install OpenSSH (it’s in the networking section on Cydia) – you won’t see anything on your Springboard because this runs in the background
After OpenSSH is downloaded and installed, tap onto “Settings” and then onto “Wi-Fi”
Tap on the arrow next to the WiFi router you are connected to, this will bring up the wireless network settings
Make note of the IP Address visible on the first screen, as an example we’ll say it’s 192.168.1.103
Step 2) SSH from your Mac or Windows PC:
Launch the Terminal in Mac OS X, or PuTTY for Windows users
Type the following at the command line:
ssh root@192.168.1.103
Remember to use the IP address you found in the prior step on your iPhone
Wait a minute or two while the SSH encryption keys are generated, accept them when (if) asked – this delay only occurs the first time you ssh from a computer to the iOS device
When asked for a password, use “alpine” but without the quotes, this is the default password for all iOS devices
Step 3) Change the Default iOS Passwords:
You will now be connected to your iPhone or iPad via SSH. The first thing you’re going to want to do is change the default passwords, otherwise anyone on the network could theoretically connect to your iPhone, iPad, or iPod touch. This is just a matter of typing the following command:
Provide a new password and then confirm it when asked.
Now you’ll want to change the ‘mobile’ ID password to be safe, this is basically the same procedure:
passwd mobile
Enter a new password and confirm it.
The video below walks through the process of changing the root passwords. This is very easy and only takes a moment.
If you plan on connecting to your iOS device frequently, you may want to set a manual DHCP IP address so that it doesn’t change on you, and then
so you don’t have to type the entire connection string again.
SSH into your iPhone from the iPhone (or iPad to itself, etc) ie: Connect to localhost:
For connecting to localhost from your iOS device, you just need to have an SSH or Terminal client on the iPhone itself. Again, I’ll , but there are other options out there.
SFTP to the iPhone or iPad:
Transferring files to the iPhone or iPad is just a matter of using SFTP after OpenSSH has been installed and is running. You’ll use the same IP address, login, and password as connecting with SSH, just from an ftp client instead of the Terminal. A few good free FTP clients are CyberDuck for Mac, or Filezilla for Mac, Windows, and Linux.
Enjoy this tip? Subscribe to the OSXDaily newsletter to get more of our great Apple tips, tricks, and important news delivered to your inbox! Enter your email address below:
Related articles:
Posted by: William Pearson in , , ,
Leave a ReplyAccess denied | linuxconfig.org used Cloudflare to restrict access
Please enable cookies.
What happened?
The owner of this website (linuxconfig.org) has banned your access based on your browser's signature (3fa5de0bda47935a-ua98).}