rsyncing to an AWS instance
I wanted to try running some of the machine learning algorithms that Jen and I have been playing around with on a beefier machine so I thought spinning up an AWS instance would be the best way to do that.
I built the JAR with the appropriate algorithms on my machine and then wanted to copy it up onto an AWS instance.
I could have used scp but I quite like the progress bar that you can get with rsync and since the JAR had somehow drifted to a size of 47MB the progress bar was useful.
When I provisioned the machine I created a public/private key pair and I was able to ssh into the machine like this:
ssh -l ubuntu -i ~/Downloads/machinenursery.pem ec2-54-242-108-142.compute-1.amazonaws.com
I needed to tell rsync to use the pen file which I initially tried to do with the following command:
rsync --progress 'ssh -i /Users/markhneedham/Downloads/machinenursery.pem' -avz target/ firstname.lastname@example.org:machinenursery
It seemed to ignore the pem file and I got a permission denied error when I ran this:
Permission denied (publickey). rsync: connection unexpectedly closed (0 bytes received so far) [sender] rsync error: unexplained error (code 255) at /SourceCache/rsync/rsync-42/rsync/io.c(452) [sender=2.6.9]
Eventually came across an article which explained a way around the problem using RSH instead of SSH:
rsync --progress --rsh 'ssh -i /Users/markhneedham/Downloads/machinenursery.pem' -avz target/ email@example.com:machine nursery
As I understand it RSH isn't secure but all I'm transferring is a JAR file so it didn't seem like too much of an issue.
I'm sure there must be a way to transfer this file using SSH but I've tried all the different flags and I can't figure it out so if you know how to please let me know!.