As an internet user you need to download and upload data in form of files everyday. Also, as an mobile and/or web app developer you must have used FTP (File Transfer Protocol) to transfer data from one machine to another or vice-versa. FTP tools like Filezilla, WinSCP, Cyberduck help you do these transfers.
But, if you have sufficient access to two servers or machines, a quickest way to transfer data can be to use SCP. SCP is an acronym for Secure Copy Protocol. It is a command line utility, available by default on all Linux based machines, that facilitate to securely copy files and directories between two locations.
Prerequisites
In this example, to transfer data between two server using SCP you MUST have SSH access to both servers.
Making transfers
Copying a folder from another server to current folder using keypair
scp -r -i /Local/Machine/path_to_key_file user@server_ip_address:/backup/script ./
Copy from local to remote machine while remote machine requires ssh keypair
scp -r -i /path/to/ssh/keypair_file ./script inimist@ip_dest_server:/folder_to_copy_to/
Transferring between two remote servers (ssh user)
scp ubuntu@xxx.xxx.xxx.xxx:/home/ubuntu/filename.zip ubuntu@zzz.zzz.zzz.zzz:/home/ubuntu/
Transferring between two remote servers (using keypair)
scp -i "/path/to/key/keyfilename.pem" ubuntu@ip_of_source_server:/home/ubuntu/backup/filename.zip user@ip_of_source_server:/path/to/folder/
Setting credentials via config file
I prefer to set server credentials in a common config file so that I do not have to specify username, ip address and private key file path each time I run a command. This way all I have to specify is the alias for a given server or both servers.
Hint: You can skip this step if you prefer to use the traditional user@hostname combination to point to a server. You can also use private key or prefer to include file path using the -i option aka identity key while running command line. As in:
ssh admin@xxx.xxx.xxx.xxx -i /User/inimist/.ssh/server_private_key
Note: I will also demonstrate transferring using full command with username@hostname with and without private key.
Creating a common config file
nano ~/scp_config
If you want to make transfer between two distant servers, create two entries as shown below. If you use ssh key pair to login to any of the servers include full path to private key file as the IdentityFile directive. See Also : How to use SSH key pair to login to remote machine or server
Host src
HostName server1.com
User root
CertificateFile /path/to/privateKey/server1.pem
Host dest
HostName server2.com
User root
CertificateFile /path/to/privateKey/server2.pemNote: In some cases (as in mac) you may need to use IdentityFile instead of CertificateFile.
Transfers using config file
Transferring between two remote servers
Transferring while staying on third machine
Run the following command to transfer a folder between two remote servers while keeping yourself on third machine for example in local machine or laptop.
scp -r -3 -F ~/scp_config src:/path/to/source/folder dest:/path/to/destination/folder/
Explaining Options
- -r – Recursive. Required to copy folders or directories.
- -3 – Using the
-3option means that the transfer will not occur directly between server1 and server2, and will be passing through your current machine. In this case, the certificates needs to be stored in your local machine. In example above it is stored in User’s default home directory, hence referring to as ~.Omitting -3 means that the transfer will occur directly between server1 and server2, without passing through your machine. In that case, the server2’s certificate needs to be stored on server1 i.e. on the source server and referenced with a full path to the certificate. - -F – Path to the configuration file
- src – The name of the source server connection we set in our configuration file
- dest – The name of the destination server connection we set in our configuration file
Transferring while staying on source machine
- Run the following command to transfer a folder from
server1toserver2while staying on the server1.scp -r -F ~/scp_config /path/to/source/folder dest:/path/to/destination/folder/
Note: In this case it is assumed that you have created
scp_configfile on source server have placed the private key for server2 on server1 itself and have set path to it(private key) inscp_configfile
- Run the following command to transfer a folder from
Transferring to local machine by staying on local machine
Run the following command to transfer a folder from a remote server to local machine or laptop.
scp -r -3 -F ~/scp_config src:/path/to/source/folder /local/path/to/folder/
Transferring from local machine to remote server
Run the following command to transfer a folder from local machine to a remote server.
scp -r -3 -F ~/scp_config /local/path/to/folder/ dest:/path/to/source/folder
Transferring by direct commands
Transferring from remote to local by staying on local
You can trasfer by specifying full command with private key
scp -i /Users/inimist/.ssh/server_private_key admin@4serverhostname:~/sample.txt /Users/inimist/dumps/
Transferring from one remote to another by staying on local
Similarly you can transfer from one server to another
scp -i /Users/inimist/.ssh/server1_private_key user@server_1_hostname:/path_to_file -i /Users/inimist/.ssh/server2_private_key user@server_2_hostname:/destination/folder/on/server2/
Hint: Use -r option to transfer folders.
Troubleshooting
- You may need to use passwords to login to one or both servers.
- You may need to use private keys to login to one or both servers.
- You may need to set readonly permissions on the private key files. Ex:
chmod 400 privateKey.pem
- You may need to use
sudoto copy sensitive/system folders or files.
See Also : How to use SSH key pair to login to remote server
Conclusion
In this articles we have learned about transferring files via SCP tool. We can transfer file between two server while staying on local machine or we can transfer directly from one remote server to another remote server. We need to have SSH access to both servers to run the scp command to transfer files.
