Moving a very large site: when FTP is too slow and what to use instead

On a site with many thousands of files FTP is slow because every file is a separate request, and the time goes on opening and closing connections. The way out is to move one big file: compress everything into an archive, send it over a connection that resumes if it drops, and unpack it on the other side. If both ends have SSH, rsync does the same job with no archive at all.

The methods, side by side

Method Good when Weak spot
One archive (.tar.gz or .zip) + SFTP You have the file manager or the Terminal on the old side. Needs space for the archive and for the unpacked files.
FTP file by file A few files, or one large file. Very slow with thousands of small files.
rsync over SSH Both ends have SSH. It resumes where it stopped and, on later runs, sends only what changed. Needs SSH on both sides. See copying files to another server with rsync, safely.
Ask us to do it You would rather not handle it yourself. Talk to us first, to say what it is and where it is. See our migration page.

Step by step with an archive

1 Clean up before you copy. Cache, logs and old copies only add to what has to travel. See the limits nobody advertises: inodes, processes and memory.
2 Export the database separately, already compressed. In phpMyAdmin use the custom method with compression; if the database is big, see exporting a database with phpMyAdmin and, on the other side, importing a large database over SSH.
3 Compress the site folder into one archive. In File Manager use Compress. In the Terminal: tar -czf site.tar.gz public_html. Include hidden files, or the .htaccess stays behind.
4 Check the archive is whole before you move it: tar -tzf site.tar.gz lists the contents and fails if it is damaged.
5 Move it over SFTP, which resumes if the connection drops. See SFTP instead of FTP: when you can, and why it is better and day to day in FileZilla. If the old side has no SFTP, use its file manager to download it.
6 Unpack on the new side and check: the number of files and the size should match (du -sh shows a folder’s size).
7 Test before changing the DNS: testing a migrated site before you change the DNS.

With rsync, if both ends have SSH

On our shared hosting SSH uses port 2299 and depends on the account having command-line access (not all do: see the cPanel Terminal). Run it on the new side, pulling from the old one, swapping the values for yours:

rsync -avz --partial -e "ssh -p PORT" user@old-server:public_html/ ~/public_html/

You can repeat the command as often as you like: repeats only send what changed. So the good tactic is one big pass days beforehand and one small pass on moving day.

Do not leave the archive where everyone can reach it. A site archive includes configuration files with passwords. Never put it in public_html under an easy-to-guess name, and delete it from both sides when you finish.
Do the sums on space. During extraction the archive and the files coexist: you need room for both, and enough file allowance (inodes) for all of them. If space is short, unpack in parts or ask us before you start.
On moving day, freeze changes. Put the old site in maintenance, export the database a second time, import it, and only then change the DNS. The full order is in how to migrate a site without downtime.

Is your site big and you would rather not risk it? Tell us where it is today and how much it weighs, and we will tell you how we would do it.

Open a support ticket

SEE ALSO

Moving a site by hand: files by FTP and the database

Importing a large database over SSH

Copying files to another server with rsync, safely

How to migrate a site without downtime

RECOMMENDED PRODUCT

Web hosting with cPanel

Domain and SSL included, daily backups and the panel you already know. from R118.80/mo (3-year plan, with coupon)

See plans
  • 0 Users Found This Useful
Was this answer helpful?

Related Articles

How to migrate a site without downtime Changing hosting provider does not have to take your site offline. The trick is simple: set everything up here before po...
Internal migration: moving sites between plans Changing plan inside MozOut - from Hospedagem Pro to Plus, say - is far simpler than migrating from another provider. In...
How to generate zip and sql files to migrate a WordPress site A WordPress site is always two halves: the files (theme, plugins, images) and the database (posts, pages, settings). To...
How to restore your data with JetBackup Every account here is backed up automatically, once a day, in the early hours of the morning - the last 30 daily backups...
Making and keeping your own backup, and testing that it works Restoring a copy of ours is covered in how to restore your data with JetBackup. This article is about the other half, an...
The site moved but something is broken: what each symptom points to After a move, each error almost always has one typical cause. Instead of redoing everything, read the symptom, go to the...