I’ve been doing this phpBB consulting work part time for more than nine years now. Lately I’ve learned a painful lesson that you can never be too careful.
A client with a 1.4M posts database wanted to upgrade from 3.0 to 3.1. I dutifully did what I always do before starting any work: backing up the files and the database.
The files are straightforward to backup. The database on the other hand…
Shared or VPS hosting have some disadvantages: resource limitations that are not always known. It’s more problematic with shared hosting. phpBB admins suffer through these all the time. Database backups don’t finish because of HTTP 500 timeouts, for example. That’s what happened with this client. Not too surprising that with 1.4M posts, the phpBB backup would time out because shared hosting can’t create a backup in the number of CPU seconds that a program will get from the host, allocated in Apache and PHP’s configuration.
So I went into the web host control panel, kicked up phpMyAdmin and used its export utility. With 1.4M posts, the download took a long time, 15 minutes or more. But it downloaded, or at least appeared to download successfully. There it was in my downloads folder, and ungzipped it was 2.4G. I made a number of these over the several days I was working on the project as I tried a number of approaches to upgrade the forum. Kept getting mysterious errors documented here.
No problem if the conversion failed because I had both a file and a database backup. Of course, reloading 1.4M posts and associated data takes a while. First I had to drop the tables. I ended up using a staggered MySQL importer to import everything. It took hours.
But after all that effort when I tried to bring up the board, it complained about missing tables.
I finally looked at the bottom of the database export using the Unix tail command. The export was not complete. It had timed out, but there was no symptom that it had. All my other exports had failed for the same reason, so none of my backups were complete, and there was no clue to let me know something went wrong.
So I’ve spent the better part of a week trying to bring it back. We were at the mercy of the web host, who had some stuff backed up, but inconsistently. Some tables had newer tables than others. Fortunately the posts table looked complete and a copy of the users table looked complete. I wrote a program to recreate the missing topics. So things are back, sort of. Client is not happy but under the circumstances he was lucky to get it back at all. And I encountered an issue I never had before and learned a painful lesson.
It’s for these reasons that I’ve revised my pricing to up my fees on boards with lots of posts and users. This work becomes unprofitable at a “one size fits all” rate. There are too many minefields. Now I have to verify that my backups are actually complete. I can’t rely on software to do its job.
