Skip to: Navigation | Content | Sidebar | Footer


Weblog Entry

Changing Web Servers

October 25, 2005

I’ll soon be going through the almost-always-painful process of switching hosts on a site I’m working on. I’ve become much better at planning for these over the years, but the complexity of sites I build has added more variables to the mix that I need to account for which probably negates any advantage I thought I had.

There is still an uncomfortably large amount of guesswork and finger-crossing leading up to the switch. Inevitably, some setting or other doesn’t get transferred which makes for a day or two of debugging after the fact.

Here’s my working checklist of things that need to happen. Any you’d add?

Transfer files.
All the .html, .php, .jpg, etc. files sitting on the old server need to move to the new server. In the past I’d have used an FTP program to take care of the job, but thanks to rsync I can much more quickly mirror the two servers. At the moment I transfer the data from the old server to my local hard drive temporarily, and then re-upload to the new server. I suspect I can skip the middle step of my local system and do a direct transfer, but it never hurts to have the data in more than one spot anyway.
Transfer databases.
All MySQL databases need to move as well; though I’m generally not a big fan of it, PHPMyAdmin is a bit easier to wrangle than the command line (sometimes), and has a one-click import/export function.
Change absolute paths.
Some of my custom PHP relies on absolute server paths for include files and the like file output. As much as possible I’ve been declaring the path as a constant in global file, so it makes for a somewhat easier migration; except that the path to the global file itself is usually absolute, so unfortunately that does mean a bunch of find and replacing.
Verify permissions.
I’ve been using PHP’s ability to write files to the server in a function that grabs EXIF metadata from images and caches that in static text files. Problem being, PHP’s file management honours file permissions, and I’ve yet to de-tangle which combination of permission/ownership I need to assign the script and write directory in order to securely write these files; to actually make it work the way I’d expect, I’ve simply assigned 777 to the directory and treat the cached data as disposable. There must be a better way, but I haven’t found it. The upshot for the migration is namely that I need to ensure these permissions remain unchanged. That’s the other advantage to using rsync over FTP, it should do the work for me.
Verify passwords.
I’m hoping the directories I’ve password-protected using .htaccess and .htpasswd files are using relative paths. But in case not, I’ll need to check if the directories that need to be protected are such.

Luckily I haven’t installed any PHP modules or done any extensive customization of Apache, so assuming the new host has a similar setup, that shouldn’t become a problem.

And I have a static IP on the new server, which allows me to test ahead of time, instead of waiting for DNS to propagate before I go into mad panic mode to fix the things that get busted from the switch.

The main question mark for me is mail; assuming I set up the user accounts ahead of time, the transfer shouldn’t interrupt regular mail delivery (aside from the switch in server settings). But I’m not nearly well-versed enough in mail software to know what’ll happen for sure until after I throw the switch.

Again, any glaring omissions would be more than welcomed. I’m hoping I’ve accounted for all I need to, but general advice is always appreciated.


Joe says:
October 25, 01h

A hint about PHP directory paths I’ve found useful.

To construct relative file paths in PHP which are relative to the current script and not PHP’s current working directory (ie the original script that started the process), try doing something like this:

include(dirname(__FILE__).’/../somepath/file.php’)

Basically __FILE__ gives you the asolute path to the current script, and then dirname() gives you the directory name. From there, you can do as many ‘../’ as you need to get to the proper directory in a predictable manner.

October 25, 02h

Re: Change absolute paths.
You can use relative paths reliably with the dirname(__FILE__) construct. It will always give the full path to the file currently being parsed. From there, you can navigate the FS with ../ etc.

Also, as has been said, you can put the new IP in your HOSTS file to test easily, or set up a small DNS server in your home LAN.

(Speaking of server issues: Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.)

October 25, 03h

I’ve had a similar experience recently.

If you move from MySQL 4.0 to MySQL 4.1, you should now that the format of timestamp fields (‘20051026123000’) has changed. It is now similar to datetime fields (‘2005-10-26 12:30:00’).

The conversion will be automatic when you upload your sql dump… But if you use scripts that expect the old format, there’ll be some head scratching. Well not if you know it in advance!

Stefan says:
October 25, 03h

Regarding the PHP-Includes: What I find very useful is to use (if possible, it depends on the Apache configuration) PHP directives in the htaccess files. Add for example this in the htaccess:

php_value include_path /path/to/the/directory

and then you can always use something like this:

include(‘inc/include_me.php’)

As I said, this is not always possible, but it’s worth to check out - just one change in the htaccess file instead of s&r stuff :)

October 25, 04h

I just switched servers last weekend and wrote a little post about moving mysql databases from server to server.

http://addictedtonew.com/archives/95/moving-mysql-databases-from-server-to-server/

I would second Shaun (#14) and Jeff (#8). Use ssh for the database moves.

6
Name says:
October 25, 05h

I’ve found http://www.ozerov.de/bigdump.php (a utility to import large files into mySQL helpful when the server in question did not allow full access, and the exports were too large for phpmyadmin.

October 25, 06h

I would second Stefan’s suggestion of modifying PHPs include-path dynamically. But use it wisely, like only adding one core folder which contains basically the most important file - the application core, which deals with grabbing the configuration file(s) and setting everything up for the auxiliary files.

It can also be useful when you want to use PEAR-libraries which aren’t installed on your server. I usually have a ‘lib’ folder for 3rd party libraries in my applications, which gets added to the include-path (i.e. PEAR uses relative paths for includes, assuming the PEAR-root-dir is in the include-path).

Regarding different PHP/Apache configurations. Obviously this can easily bite you hard, but I would suggest setting up a local version of your site/app where you let PHP run in a very strict mode (reporting ALL notices/warnings/errors, not using register_globals etc..).

What bit me recently concerning Apache, was the MultiViews option in a directory, which together with mod_negotiation enabled a feature where you could type only the filename without extension of a file, and Apache would return the best match from the directory - rendering RewriteRules useless as Apache would serve the files instead!

October 25, 11h

For the database side of things (if you are using a frequently updated DB or several DB’s) I find replication a good solution.

Until you are sure evrything is over correctly you can keep replicating your MySQL DB. This also means there doesnt have to be one scary instant when you switch over, nor any lost data.

However it does require access to my.cnf and other database permissions.

October 25, 11h

In my experience of moving sites from one server to another, as long as you have corresponding accounts set up for your mail on the new server, you shouldn’t miss anything (or very little, if anything) when the DNS switch takes place on the domain.

The only time that you may have a problem is if someone may be in a location that it takes the propogation longer to get to for some reason.

10
Kenzie Campbell says:
October 25, 11h

I’m not much help, I always end up doing a transfer at the last minute and messing something up. Just curious what host you’re transferring to.

October 25, 11h

A couple thoughts. Since you have the static IP, TEST TEST TEST! This could be your life saver. Too many times, even with static IP, I’ve taken things for granted that bit me in the arse.

As far as email goes, depending on email clients, leave them set to old setting until you’re 100% dns propogates. It’s better to leave the old settings during the interim so when you switch the client settings, it picks everything up from the new, rather than having to clean out the old.

Plus, surely there is a mysql gui for mac. I use Mysql Front for Win, but I think there are a few mac versions (Navicat?).

Also, be sure the new host supports ALL of the software you’re using. One switch broke half of my PHP because they were running in safe mode and didn’t disclose such things.

Jarrod says:
October 25, 11h

I’ve hosted about 20-30 sites and have changed dedicated servers twice in the last year and a half.

The biggest piece of advice I would give is get your new space setup and working by a virtual domain so that you can ensure everything works before changing the DNS. On my dedicated server I setup all 30 sites by IP address and port number for testing and then when I moved the DNS, i did a quick update on all my databases from the current server and it all moved without a problem.

I’d pay the extra money if you need to to have a couple weeks overlap between the move so that you can have everything working before the actual move.

Turnip says:
October 25, 12h

Check the new server is sending the correct charset.

Also, I’m surprised you do includes with absolutes. So long as all the files maintain the same directory structure it should be easy enough to generate the absolutes or just use relative. Have you tried dirname(__FILE__) or similar?

October 25, 12h

Absolute paths: Here’s a solution (which I haven’t used myself…yet) which might make this less painful. Rather than having every file contain the absolute path to one file that sets all your other paths, use a relative path to the master file. For example, in your web root directory, have a file named masterpath.php that sets any other paths. Then in each file, include “masterpath.php” or “../masterpath.php” or “../../masterpath.php” depending on where each file is located.

DNS: If you have to switch DNS servers, try to get the old DNS server to keep working for you for a few extra days to avoid any gaps in service–update that server to point to the new IP address.

Also, be sure to shorten the TTL for your DNS entries down to something short, like a half hour, far enough before making the switch that you won’t have a lot of people caching your old IP address for too long. How far in advance you have to do this depends on your old TTL. For example, if it was 24 hours, you’ll need to shorten the TTL at least 24 hours before switching IP addresses. Otherwise, not only will people have your old IP address cached, they’ll also have your old TTL cached.

October 25, 12h

Unfortunately, I have WAY too much experience changing servers. Lucky for you, that means I have probably made every mistake you can make.

First, with regards to transferring databases, make sure that your MySQL dump files are not too large for phpMyAdmin to handle. What I generally do is generate the .sql dump file in phpMyAdmin on the old server, upload that file as-is to the new server, and then SSH in and do the import through the shell. It is faster, and it doesn’t have the problems with timeouts that phpMyAdmin sometimes does. Of course, I learned the hard way about eight host switches ago that my DB is too large to import through phpMyAdmin, so you may not have that problem.

As for email, what I do is set up my mail program with two email accounts: one with the new settings, and one with the old settings. The one difference is that for the one with the old settings, I do the IP address of the POP and SMTP servers, instead of the name. For example, if your current mailserver is just “mezzoblue.com,” then you replace that with “67.19.16.228” in your mail settings. That way, your program will catch any mail that is caught in limbo, meaning it is sent from someone whose connection hasn’t propagated the DNS change yet, but after your connection already has. Leave both accounts set up for a few days, until you are 100% sure everyone has propagated, and then you delete the old one.

My final little trick, which you may or may not need since you have a static IP: changing the DNS settings on your computer. I do not know how to do it on a Mac, but on a Windows machine, you can go into your TCP/IP settings for your Internet connection and manually set the DNS server. So let’s say the DNS for your new site is NS1.SOMESITE.COM. Simply set that on your local machine, and then when you type in www.mezzoblue.com, it will first look on that server for your site. So once you have your site set up on the new server, you can access it through the actual domain name simply by changing your DNS settings. One thing I can think of where this might be more helpful that the static IP is for cookies or anything on your site. If they are domain-based cookies, they won’t work if you show up on an IP address, I don’t think. (I am not an expert on that, it’s just my guess.)

Anyway, good luck with the new server. I would be interested as well to know who you are switching to. I switched to Dreamhost almost 11 months ago, and it is the first company I have ever been with that I would actually recommend to a friend.

Dave S. says:
October 25, 12h

#4 - “Since you have the static IP, TEST TEST TEST!”

Granted. Static IPs are a life-saver for this, even if (as another commenter mentioned) you have a few weeks or so of overlap/extra charges. It’s worth it.

#6 - “Also, I’m surprised you do includes with absolutes.”

I was wrong in my description, it’s not for includes, it’s for file writing which actually does require a server path.

#7 - “Then in each file, include “masterpath.php” or “../masterpath.php” or “../../masterpath.php” depending on where each file is located.”

Yeah. That’s the ideal. Unfortunately I have a bunch of auto-generated files at a bunch of different directory levels, and I haven’t yet devoted the brainpower to figuring out how to parse the current URL and factor that into the level of ../’s required.

#8 - thanks for the mail tip.

October 25, 12h

$public_path = substr(__FILE__, 0, strpos( __FILE__, ‘public_html/’ ) + 12);
require_once( $public_path . ‘common.php’ );

That little trick has saved me hours of absolute path headache. =]

@Jeff: A “sudo vi /etc/hosts” on a Mac can get you up and running independant of your DNS as long as you have the static IP to work with. If you need a bit more power (such as a DNS server to point to) that’s in the /etc/resolv.conf

For mail, setting up postfix (or the old address) with an auto forwarder can help remove some of the strain. We used RPOP during our server migration so that the new mail server would query the old server by IP and retrieve any messages that were there from remanant MX records.

In the end though, there’s always those first horus of panic when it switches. It’s really easy to forget a chmod, or forget a path statement somewhere. Best of luck!

18
gerben says:
October 25, 12h

Make sure PHP has the same settings. Especially SafeMode and CGI or SAPI mode.
check whether apache modules are present, like ModRewrite (which you probably checked before buying).

The following does same as Jeff’s but migth be a usefull alternative:
also test with
Host: www.mezzoblue.com
added to your HTTP headers by simply adding
123.456.654.321 www.mezzoblue.com
to your local hostfile (C:\WINDOWS\system32\drivers\etc\hosts)
This however is especially usefull if you don’t have a personal IP like you have.

Just a tip. make some (temporary) difference between the old and new version while testing. I sometimes get confused whether I’m working on the old or new version.
Goodluck.

October 25, 12h

/etc/hosts is your friend, if you’re running OSX (or Linux or *BSD or almost any modern flavour of UNIX).

I usually edit the /etc/hosts on my computer to access the new server, this is a method that has the advantage of working even if your domain does not have an IP for itself. Moreover, your scripts on the new host gets to work in real conditions, that is with the real hostname and all.
I do that for ready-for-productions sites at work too, when the client already has a domain name serving a site.
And e-mail too, actually. Figures I had some e-mails left on an old server once, from people whose ISP’s DNS must have taken their very sweet time to update…

As for MySQL, what you might want to check is the charset it runs with, if you’re moving to MySQL 4.1+. That can be done with some voodoo, or the SQL command “show client encoding;”. If you get something else than latin1 and you’re using MySQL 4.0 or older on the previous host, or if with identical MySQL versions you get different encodings, that might be an issue.

Fifteen days ago I moved from a shared hosting setup (TxD… no really, it was getting frustratingly slow and down) to a dedicated server setup using different softwares, you can read the details here: http://zengun.org/weblog/archives/2005/10/transatlantic-move

Good luck!

October 25, 12h

Oh yeah, while I was typing all this, Jakob came up with /etc/hosts before I did. :)

As for chmods, if both your hosts give you ssh access, it shouldn’t be too hard to either do a tarball or (better) do an rsync.

October 25, 12h

I just want to second/emphasize Jeff Snider’s comment on not relying on PhpMyAdmin for a non-trivial database move. With shell access on both servers, using the command line mysqldump/mysql < to export/import will be quicker and far less painful.

October 26, 05h

I just changed server the other day, so while my memory is fresh;

If you use IMAP and not POP3 for your email, remember to backup all your mail on your local hard drive. Normally they would just be stored on the server, and only cached locally. I was really happy that I did this, cause the second the domain transfer got picked up by my ISP, all e-mail in Mail.app disappeared.

I’ve also read somewhere that if the two hosts use the same control panel software, like cPanel, it could be possible to have them talk to eachother, and transfer the whole account, as it is, databases and all.

Good luck.

John says:
October 26, 05h

Another way of making sure your include files don’t get lost…

require_once($_SERVER[‘DOCUMENT_ROOT’] . ‘/common/config_inc.php’);

24
Thomas Tallyce says:
November 03, 03h

I’ve never had the sort of problems you mention with absolute paths, despite the fact I develop on a Windows machine and host live sites on Linux.

1. INCLUDING LIBRARIES

The solution is simple: just set the include_path in your .htaccess (or httpd.conf) file, like so:

php_value include_path /path/to/siteroot/:/path/to/libraries/

or in the case of Windows
php_value include_path “C:/full path/to/siteroot/;C:/path/to/libraries/”

Then when you do

require_once (‘libraryname.php’);

it will just be found. I aim never to put subdirectory paths from one of the include paths in the library include (e.g. ‘subdirectory/to/libraryname.php’)

because instead you can just set the include_path to:

php_value include_path /path/to/siteroot/:/path/to/libraries/:/path/to/siteroot/subdirectory/to/libraryname.php

which can be added to as much as you want.

If your PHP code is reasonably well organised then this should remain short as most of the code should end up in one place, ideally your common libraries area (which of course could be outside the site root).

The upshot of this is simply that, when you move host, the *only* thing you should ever need to change is just this include_path line. This is indeed the only difference between my dev and live sites.

You can even actually keep it exactly the same - if you have httpd.conf on your dev box - by setting the include_path (as above) in your .htaccess file to have the ‘live’ site’s paths. Then in Apache on the dev server, set the dev site’s include_path using php_admin_value (which means it can’t be overriden by php_value directives put in .htaccess). The .htaccess version is thus ignored for the local dev site but not for the live site.

2. READ/WRITE TO FILES

For writable/readable files, e.g. logs, CSVs, etc., just make use of the server’s DOCUMENT_ROOT. For instance

$_SERVER[‘DOCUMENT_ROOT’] defines the root of your site. Sometimes this will have a slash at the end, so it’s sensible to have an auto_prepend_file ‘d library which contains a clean-up line like the following which ensures there is NOT a slash:

$_SERVER[‘DOCUMENT_ROOT’] = ((substr ($_SERVER[‘DOCUMENT_ROOT’], -1) == ‘/’) ? substr ($_SERVER[‘DOCUMENT_ROOT’], 0, -1) : $_SERVER[‘DOCUMENT_ROOT’]);

Now that you know you will have a consistent site root, just write/read files using

$contents = file_get_contents ($_SERVER[‘DOCUMENT_ROOT’] . ‘/path/to/file.csv’);

So thus again, there is no need to hard code any directory paths.