Occasionally I do something new. This month something new meant helping to rehost a client on Amazon Web Services (AWS). AWS provides cloud computing services, and its EC2 service (EC = Elastic Computing) is probably its most popular service.
Cloud services provide scalable services. Also, you pay for what you use. They can be configured so that if there are spikes in demand the service will become “elastic”, scaling to meet demand.
I did not do this rehosting by myself. The client had another technical guy that set up and configured his AWS EC2 workspace. The forum is very large with 2.6M posts. In addition to rehosting the forum, I had to upgrade the client at the same time to the newest version of phpBB and move his WordPress site. This project literally took months to complete, although I was not working on it all the time.
I don’t know what Amazon Machine Instance (AMI) was set up in this case. I don’t think the AMI chosen was ideal because WordPress had technical issues that required fine tuning EC2 to get things to work. One thing I took away from the project though is that there is a “tax” if you move to AWS. The learning curve is steep and the tools available to you are miniscule.
Some lessons learned:
- AWS doesn’t do any handholding. While you get a console to configure EC2, you don’t get any rich control panel like cPanel or Plesk to allow you to easily do complex things. So there is no File Manager, no phpMyAdmin (unless you want to install it yourself), no easy way to create mailboxes or send email. Instead, you need a UNIX geek. I was given a SSH key file and I had to use that to do my work. This meant doing pretty much everything from the command line.
- Connecting to the AWS workspace was complex since I had to use SSH. The command lines used to connect with SSH were long to type in and easy to get wrong. I had to reference an argument to use the key (.pem) file I was given. When you work from the command line, it’s easy to type something incorrectly. So you often end up typing the command multiple times until you get it right. Closing and opening sessions becomes time consuming and a hassle.
- To access the database, I had to do it from the command line. It took a while to get database credentials and since I had to do things from the command line again I had to type statements very carefully. I had to export and import databases but getting the syntax just right was challenging. I frequently had to go into the database to tweak things, which meant typing a lot of SQL statements. It’s a good thing I am very fluent in SQL. This really slowed me down. Had I had phpMyAdmin, I would have saved hours of time and hassle.
- File permissions were a pain. The default user did not have the permissions to the web root folder, which meant becoming root and granting correct group and file permissions. Then secure FTP would work. These problems kept recurring which made the process quite tedious and time consuming.
- The Apache web server was not configured correctly for WordPress. The AMI was apparently not tuned for WordPress, so it took research and carefully editing of a httpd.conf file to get the settings right. Then the web server had to be restarted from the command line, which is not intuitive, particularly since it had to be done as root.
- Editing files became a pain. There was a lot of this, mostly tweaking forum styles, templates and configuration files. Since I had only the command line, I had to use a command line text editor. I chose nano, but it was still tedious. There is also a lot of command line navigation to get to the right folders where you needed to do stuff. This would have been easier if I had an editor on my computer that worked with SSH and .pem files. I had the former but not that latter, as I use an old copy of Dreamweaver to more easily edit files remotely.
- I often had to become root to do things, like read the error_log file to troubleshoot issues.
- The database import for the forum failed three times. I finally figured out the issue from the obscure error message. I had written two triggers for the client and that required CREATE TRIGGER privileges that were not granted to the default MySQL DBA. I had to snip these lines out of a 2GB+ export file to get the import to succeed. This has never been a problem on other hosts I have worked on.
- Setting up HTTPS was a pain. It too required special permissions to create public and private key files, on the command line only, of course. It took many attempts before it worked and a valid certificate was installed. While we were waiting to test WordPress and the forum, it required using the long URL provided by AWS, which meant changing the configuration of WordPress and phpBB via the database.
I suspect that the wrong AMI was used or that using a better one would have prevented a lot of problems. In any event, the move to AWS turned out to be tricky, time consuming and a large hassle. For my client, it was an expensive endeavor. It turned out that all this work turned into my largest bill ever. In that sense it was good, although I would have been happier if it had spent a lot less time and a lot less of his money.
Now that my client is on AWS though, as long as it is tuned properly there will be some big advantages. Hosting may cost less in the long run, and the service should be more predictable and scalable.
If you take up a project like this, you will definitely need someone who has set up AWS EC2 instances successfully and will work through all these issues with you. Even so he could not do it alone. The migration took four days to complete, although the forum was functional in less than twenty four hours.
For very large forums that need scalability and high reliability, using a cloud service like AWS makes a lot of sense. However, a project like this should not be taken lightly. Most forum owners will probably be much more comfortable on a good shared host, or a good virtual or dedicated server.
