How to copy files and folders between two servers via SCP

As an internet user you need to download and upload data in form of files everyday. Also, as an mobile and/or web app developer you must have used FTP (File Transfer Protocol) to transfer data from one machine to another or vice-versa. FTP  tools like Filezilla, WinSCP, Cyberduck help you do these transfers.

But, if you have sufficient access to two servers or machines, a quickest way to transfer data can be to use SCP.  SCP is an acronym for Secure Copy Protocol. It is a command line utility, available by default on all Linux based machines, that facilitate to securely copy files and directories between two locations.

Prerequisites

In this example, to transfer data between two server using SCP you MUST have SSH access to both servers.

Making transfers

Transferring between two remote servers (ssh user)

scp ubuntu@xxx.xxx.xxx.xxx:/home/ubuntu/filename.zip ubuntu@zzz.zzz.zzz.zzz:/home/ubuntu/

Transferring between two remote servers (using keypair)

scp -i "/path/to/key/keyfilename.pem" ubuntu@ip_of_source_server:/home/ubuntu/backup/filename.zip user@ip_of_source_server:/path/to/folder/

Setting credentials via config file

I prefer to set server credentials in a common config file so that I do not have to specify username, ip address and private key file path each time I run a command. This way all I have to specify is the alias for a given server or both servers.

Hint: You can skip this step if you prefer to use the traditional user@hostname combination to point to a server. You can also use private key or prefer to include file path using the -i option aka identity key while running command line. As in:

ssh admin@xxx.xxx.xxx.xxx -i /User/inimist/.ssh/server_private_key

Note: I will also demonstrate transferring using full command with username@hostname with and without private key.

Creating a common config file

nano ~/scp_config

If you want to make transfer between two distant servers, create two entries as shown below. If you use ssh key pair to login to any of the servers include full path to private key file as the IdentityFile directive. See Also : How to use SSH key pair to login to remote machine or server

Host src
    HostName server1.com
    User root
    CertificateFile /path/to/privateKey/server1.pem

Host dest
    HostName server2.com
    User root
    CertificateFile /path/to/privateKey/server2.pem

Note: In some cases (as in mac) you may need to use IdentityFile instead of CertificateFile.

Transfers using config file

1. Transferring between two remote servers

   1. Transferring while staying on third machine

      Run the following command to transfer a folder between two remote servers while keeping yourself on third machine for example in local machine or laptop.

      scp -r -3 -F ~/scp_config src:/path/to/source/folder dest:/path/to/destination/folder/

      Explaining Options

      • -r – Recursive. Required to copy folders or directories.
      • -3 – Using the -3 option means that the transfer will not occur directly between server1 and server2, and will be passing through your current machine. In this case, the certificates needs to be stored in your local machine. In example above it is stored in User’s default home directory, hence referring to as ~.Omitting -3 means that the transfer will occur directly between server1 and server2, without passing through your machine. In that case, the server2’s certificate needs to be stored on server1 i.e. on the source server and referenced with a full path to the certificate.
      • -F – Path to the configuration file
      • src – The name of the source server connection we set in our configuration file
      • dest – The name of the destination server connection we set in our configuration file

   2. Transferring while staying on source machine

      1. Run the following command to transfer a folder from server1 to server2 while staying on the server1.

         scp -r -F ~/scp_config /path/to/source/folder dest:/path/to/destination/folder/

         Note: In this case it is assumed that you have created scp_config file on source server have placed the private key for server2 on server1 itself and have set path to it(private key) in scp_config file

2. Transferring to local machine by staying on local machine

   Run the following command to transfer a folder from a remote server to local machine or laptop.

   scp -r -3 -F ~/scp_config src:/path/to/source/folder /local/path/to/folder/

3. Transferring from local machine to remote server

   Run the following command to transfer a folder from local machine to a remote server.

   scp -r -3 -F ~/scp_config /local/path/to/folder/ dest:/path/to/source/folder

Transferring by direct commands

1. Transferring from remote to local by staying on local

   You can trasfer by specifying full command with private key

   scp -i /Users/inimist/.ssh/server_private_key admin@4serverhostname:~/sample.txt /Users/inimist/dumps/

2. Transferring from one remote to another by staying on local

   Similarly you can transfer from one server to another

   scp -i /Users/inimist/.ssh/server1_private_key user@server_1_hostname:/path_to_file -i /Users/inimist/.ssh/server2_private_key user@server_2_hostname:/destination/folder/on/server2/

   Hint:  Use -r option to transfer folders.

Troubleshooting

• You may need to use passwords to login to one or both servers.
• You may need to use private keys to login to one or both servers.
• You may need to set readonly permissions on the private key files. Ex:

  chmod 400 privateKey.pem

• You may need to use sudo to copy sensitive/system folders or files.
  See Also : How to use SSH key pair to login to remote server

Conclusion

In this articles we have learned about transferring files via SCP tool. We can transfer file between two server while staying on local machine or we can transfer directly from one remote server to another remote server. We need to have SSH access to both servers to run the scp command to transfer files.

Share on Facebook

Tweet

Follow us

Save