I run a number of websites and know from experience that the more you keep, the harder they are to keep track of.
The situation is multiplied when you need to keep copies and website mirrors elsewhere.
A casual perusal of the internet came up with a number of open source options. I prefer to use FTP over ssh, so popular solutions like rsync were out.
The I came across the aptly named Sitecopy, which allows for FTP style file transfers.
Here's how to get it installed and running on a mirror server..
Sitecopy can be downloaded from the Sitecopy website
Go to the download directory and extract the tarball..
tar -zxvf sitecopy-0.16.3.tar.gz
(configure the package for your system)
(compile the package..)
(install programs, data files and documentation)
1. Create a storage directory
. Sitecopy will use this directory to contain a file which keeps track of each remote website you synchronise with.
Login as root
to change to root's
From the /root
mkdir -m 700 .sitecopy
(note hidden dir - starts with a dot)
2. Configuration is performed via the run control file, .sitecopyrc
. This file contains a set of site definitions. A unique name is assigned to every site definition, which is used on the command line to refer to the site. Create an rcfile
, which will store information about the sites to administer..
From the /root
(note hidden file - starts with a dot)
chmod 600 .sitecopyrc
file and .sitecopy
directory must both be accessible only by root
- Sitecopy will not run otherwise.
Once this is done, edit the .sitecopyrc
file to enter the required site details. Here is an example of my own settings which allow sitecopy to connect to this website via FTP and do the business..
Explanation of above settings..
site nobluescreens.com - an arbitrary name give to this profile
server andrewmallett.net - name of server containing the website
remote /www/nobluescreens - name of remote directory to use
local /usr/local/apache/htdocs/nobluescreens - name of local directory to use
username yourusername - FTP username
password yourpassword - FTP password
url http://www.nobluescreens.com - url of website
protocol ftp - default protocol
ftp showquit - show quit status at end of FTP session
ftp usecwd - only upload in current dir, change dir for other uploads
http expect - can speed up the file transfer
permissions ignore - ignores file permissions
nodelete - don't delete anything on the remote server
exclude *.html - ignore files ending in .html
For more details on these settings, run man sitecopy
Note that Sitecopy can be used to keep local sites
or remote sites
updated, either from the local webserver
or from the remote webserver
My own requirement is to mirror www.NoBlue
Screens.com, by pulling these file down to a server (called Franga - don't ask) sitting on my teaching network at TAFE Tasmania. So the example shown here will reflect this functionality. Again check Sitecopy's man
page for more settings.
Although it's not strictly necessary, I like to create a local copy manually of the website material I wish to mirror. So I copy all the material from NoBlue
Screens.com to Franga's Apache directory /usr//local/apache/htdocs/nobluescreens
Next change to root's home directory and then run the following two commands in order..
sitecopy --fetch nobluescreens.com
sitecopy --synch nobluescreens.com
It's not strictly necessary to do this from /root
, but I like to as I can quickly get to the config file .sitecopyrc
which also lives there. Handy in the early stages of fine tuning.
command tells Sitecopy to make a note of all files and directories on the remote server, as specified in .sitecopyrc
and the synch
command pulls down any files which are different, overwriting the local files
. Again note I am mirroring NBS from Franga Server, so I want any changes to be propagated to this mirror server on the teaching network.
And that's about it. You can't run synch
without running fetch
first. I like to automate this job so it runs every night when things are quiet. So I have created a script and a Cron
job to take care of this..
/usr/local/bin/sitecopy --fetch nobluescreens.com
/usr/local/bin/sitecopy --synch nobluescreens.com
..and add the cron
#min hr day month weekday command
00 22 * * * /root/sitesynch.sh
Note that cron
runs as csh
, not bash
, so it's necessary to include the full path to the sitecopy
executable (top) and the script (bottom).
So now at 10pm every day, my website should be mirrored down to the teaching server. Test the script first before running it as a cron
job and then test the cron
job during daylight hours so you can be sure it works properly. Results are reported in root