· software-development

rsyncing to an AWS instance

I wanted to try running some of the machine learning algorithms that Jen and I have been playing around with on a beefier machine so I thought spinning up an AWS instance would be the best way to do that.

I built the JAR with the appropriate algorithms on my machine and then wanted to copy it up onto an AWS instance.

I could have used scp but I quite like the progress bar that you can get with rsync and since the JAR had somehow drifted to a size of 47MB the progress bar was useful.

When I provisioned the machine I created a public/private key pair and I was able to ssh into the machine like this:

ssh -l ubuntu -i ~/Downloads/machinenursery.pem ec2-54-242-108-142.compute-1.amazonaws.com

I needed to tell rsync to use the pen file which I initially tried to do with the following command:

rsync --progress 'ssh -i /Users/markhneedham/Downloads/machinenursery.pem' -avz target/ ubuntu@ec2-54-242-108-142.compute-1.amazonaws.com:machinenursery

It seemed to ignore the pem file and I got a permission denied error when I ran this:

Permission denied (publickey).
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at /SourceCache/rsync/rsync-42/rsync/io.c(452) [sender=2.6.9]

Eventually came across an article which explained a way around the problem using RSH instead of SSH:

rsync --progress --rsh 'ssh -i /Users/markhneedham/Downloads/machinenursery.pem' -avz target/ ubuntu@ec2-54-242-108-142.compute-1.amazonaws.com:machine nursery

As I understand it RSH isn’t secure but all I’m transferring is a JAR file so it didn’t seem like too much of an issue.

I’m sure there must be a way to transfer this file using SSH but I’ve tried all the different flags and I can’t figure it out so if you know how to please let me know!.

  • LinkedIn
  • Tumblr
  • Reddit
  • Google+
  • Pinterest
  • Pocket