December 2020 work summary

My books aren’t quite closed yet on 2020, but 2020 was at least a good year for me and this business. I am unlikely to see a repeat next year.

Much of the new income was from one commercial client and the work there is slowing down. So I expect less revenue in 2021. But what a nice year it was! I nearly doubled my income from 2019. Normally I’d have spent a good portion of it, but with the pandemic there was no place to spend it. It normally goes toward fancy vacations (like a trip to Ecuador and the Galapagos in 2019), but this year except for a short cruise right before the pandemic hit in earnest, we had no choice but to vacation at home. My phpBB consulting is ideal work during a pandemic, as it is all done from home safely over the Internet. So thank you to all who trusted me with your forums in 2020 and kept me very much in the black.

Otherwise, my work in December was kind of slow because there wasn’t much to bill the one commercial client although curiously they did give me a $250 bonus! I enjoyed the downtime, but I also gave back to the community. Both my digests and filter by country extensions had new releases in December. More importantly I continue to work on a Spamremover extension, which finds spam using the Akismet web service. I had nothing to release yet, but I am getting close to a release.

Other work in December:

  • Worked with a client using my digests extension on Siteground hosting. Subscribers stopped getting digests. I helped the client set the email settings properly to match Siteground’s SMTP server but afterward there were still issues. It turned out that digests weren’t going out because no cron job was set up. It may have disappeared when Siteground moved him to a new server and they didn’t replicate his digests cron job. After finally getting into Siteground with the client’s help, I was able to set up a new phpBB cron which seems to work.
  • Upgraded a board from phpBB 3.3.1 to 3.3.2. Also added markup to display a favicon.
  • Upgraded a major client’s board from phpBB 3.2.2 to 3.3.2. There are many customizations to phpBB 3.2.2 that had to be carried over in phpBB 3.3.2. I first did an upgrade to the test environment to work out any kinks, then a month or so later did a production upgrade to the board. All appears to be okay. I had to change the owner of the /cache/production folder to get rid of a HTTP 500 error. Later, I chased lots of styling issues, mainly on the view topic page and related to mobile display.
  • Issue with too many login attempts during admin login. I reset the number of password attempts using SQL and provided him with the SQL I used. But it didn’t work. I tried to delete the cache manually using FTP but didn’t have permissions in the twig folder. Also, CAPTCHA would not come up even though a message said to solve the CAPTCHA. Had to give my account (and his too) founder permissions. Later I realized there was a simpler solution: use phpBB’s lost password feature. The CAPTCHA issue I figured out was likely a bug in phpBB, so I submitted a problem report. It turns out that if a CAPTCHA is used, allow_url_fopen is false and the CAPTCHA is validated with a post request, the error will occur. A HTTP GET request though won’t have an issue. This is a bug in phpBB that needs fixing.
  • A phpBB 3.2 board that can’t be upgraded had emailing issues. Test emails went out okay. Could not get SMTP to work for sending email. I suggested talking with the client’s web host to see if they are blocking “spammy” looking outgoing emails. Later, I added ACP Add User extension, which apparently was what he really needed.
  • Updated a board from phpBB from 3.3.0 to 3.3.2, and my digests extension from 3.3.1 to 3.3.3. No surprises, just slow FTP and I couldn’t find cPanel credentials that worked. Later, I updated Cleantalk to version 5.7.3.
  • Upgraded forum from phpBB 3.0.x to 3.3.2. No issues during upgrade. Most of the work was trying to replicate the styling to give a similar look. It’s quite close. Created a custom style that inherits from the Elegance SoftBlue style. Copied over two templates. Installed reCaptcha V3 but registrations aren’t allowed. I disabled the contact form. I changed PHP to version 7.4 and set allow_url_fopen to true. Later, I added an extension to add next and previous post buttons inside a topic. I also moved the post edit controls from top right corner to bottom right of the post. I expanded width of content window from what it was before (980px) to the default 1152px.
  • Troubleshooting. Too many SQL connections error. I gave advice on the general issue and what causes it, and suggested Cleantalk’s spam firewall service if the issue is due to malicious robots (to keep spammers off the site), but first to look at traffic for the main domain and see if general increased usage was the cause. As requested, I repaired and optimized all database tables for the board. Last I heard the too many SQL connections problem was still an issue. The client’s hosting is Bluehost. I’ve seen this before on Bluehost. In my opinion, their shared servers do not allow enough resources for reasonable shared usage. I’d avoid hosting with Bluehost if you have a choice.
  • Upgraded a board from version 3.2.2 to 3.3.2. Updated the stop forum spam extension to version 1.3.10 and NavBar search extension to version 3.2.4. I created a custom style “custom” and put style changes there and made it the default. I changed PHP from version 5.6 to 7.4. No issues during upgrade.
  • Troubleshooting. Investigating why images and phpBB Gallery extension was taking up so much space. Primary issue was that the old gallery was copies from a /gallery_old folder. Suggested changing Gallery JPEG compression to 85% to match phpBB’s attachment settings.

May 2020 work summary

May was a reasonably active month full of work, but most of it was for one client, a continuation of the work for this client for several months now. They are not in a particular hurry on their project and neither am I, but they do want it done right, so they’ll take the time to make sure it is done right. We are iterating on the styling in phpBB 3.3 for their current phpBB 3.0 forum with the goal of making it look as close to what they have now as possible. By mid month I felt like the styling was close to being done so stopped work, waiting for a review by the company’s designer. That took nearly two weeks so things were slower toward the last half of the month. I finally got the review last Friday and started doing some more work on it this weekend. Basically we’ll keep iterating through the styling until they are happy, then work on the next phase of the project, which will hopefully involve creating extensions to replicate what they are doing now through their custom modifications. The goal is to see if everything can be done through an extension. That’s similar to my approach with the styling: putting all changes into a custom style that inherits from prosilver, so most changes over time to prosilver will get applied automatically.

I’ve also been working slowly on a spam remover extension using Akismet. There is already an extension that will check new posts for spam and prohibit those, but nothing to clean up old spam. This is a continual issue for some of my clients but there is no way to find these and pull them out reliably. I’m hoping my extension will do this using the Akismet service. As I iterate through it though I keep changing features and trying different technical approaches. Since testing every post on a large board will be very time consuming and probably cause ugly PHP timeouts, I’m trying to figure out a way to do this more asynchronously. I’ll probably use an interface similar to creating a new search index with a screen that refreshes periodically to show progress. This is hard to do in phpBB.

Other work in May:

  • I performed a painful upgrade from phpBB 3.0.8 to 3.3.0. The board used the standard prosilver style, so I just had to add a logo. Sounds easy, right? But there were all sorts of issues with timeouts and things not working optimally. When the basic upgrade was done, I installed my digests extension and tested it. Installed also installed the Board 3 portal extension, which needed a patch to make it work on phpBB 3.3. I also set up reCAPTCHA V2 invisible spambot countermeasure and disabled contact form.
  • I converted an old phpBB 2.0.19 board to phpBB 3.3.0. There were not many posts on this board. The client chose the CleanSilver to replace the old subsilver style. I replaced the phpBB logo with old logo, and made small changes to CSS to show site title and description and to properly size the logo. I disabled the contact form, recreated the search index, and replaced the GD CAPTCHA with reCAPTCHA V2 invisible. I had to create my own account to login. cPanel access wouldn’t work for a while, and I got two different login screens.
  • Upgraded a board from phpBB 3.2.5 to 3.3.0. Eleven extensions were upgraded. Updated the Platinum style to a 3.3 compliant version. A number of extensions had to have their services.yml files edited to work on 3.3.
  • I dad a one hour requirements discussion on Skype with a client. If it goes forward, I will help move a board from a Delphi forum to phpBB through a lot of complex scripts. They can export to YAML to that has to be moved into a database and munged to be usable to phpBB. The basic idea is to import it to phpBB 2, which is relatively simple and if successful upgrade to phpBB 3.3 from there.
  • I upgraded a board from phpBB 3.2.8 to 3.3.0. I applied a private message bug patch. I reapplied their logo, which does not include old Site Lock code in the footer. I upgraded their Cleantalk extension to version 5.7.2.
  • Rehosted phpBB 3.1.6 forum from Bluehost. Lots of issues with Bluehost. Why? Because the Bluehost hosting environment is so tightly optimized that things you would expect, like to make a reliable archive in the file manager or to be able to archive the database easily in phpMyAdmin would time out instead. When I was able to piece together their database from various fragments, I moved it to a virtual machine on my computer to run database cleaner, then moved it to the Mac to upgrade it, then imported the database on Bluehost, with issues of timeout on phpMyAdmin loading the database too! It was all a big pain because Bluehost is a poor host in general, despite their slick advertising. This is probably because it is owned by Endurance International Group, which buys up hosts and squeezes every nickel out of them and their customers. I had to work through their legendary poor technical support to get FTP to work which involve chatting virtually with some guy in India. PHP 5.6 used, changed the config.php file to mysqli. Created an add on domain to do the work. Search index recreated as it was too big to move due to Bluehost’s phpMyAdmin timeout issues, so I had to recreate it manually.
  • Upgraded board from phpBB 3.2.2 to 3.3.0. Updated the Allan-Style SUBSILVER to the latest version and applied a patch to the style that was recommended to use it on phpBB 3.3. I also updated the board rules extension. All looks good.
  • Upgraded a board from phpBB 3.2.9 to 3.3.0. Updated my digests extension to 3.3.1. Upgraded PHP to 7.2 from 5.6. No issues. Client had concerns about emails not being received. There were some items in cPanel’s email deliverability report. Client may not have his phpBB notifications set correctly. Client did not have founder privileges to I granted them.
  • Troubleshooting. The client could not move or edit posts. The issue was that there is no user moderator role defined for him, which became a NEVER blocking permission. Made his role full moderator for all his forums and it seems to fix the problem.

Avoid hosts owned by the Endurance International Group

As I have noted before, since I work with many clients I have developed hopefully informed opinions about many web hosts. Perhaps I should not paint with a broad brush but I do have one suggestion: avoid any web hosts owned by the Endurance International Group.

Web hosting tends to be a low profit business. With so much competition, customers will shop around for the best deal. This results in many hosts offering cheap plans for $5 or $10 per month. Web hosts can hope to find profitability in volume but since there is a lot of competition profitability usually comes from consolidation. The Endurance International Group buys web hosts. It looks like they find profitability through throwing all these companies under one umbrella and one hosting center.

I used to host with Hostgator. I found their support good and their infrastructure above average, yet their pricing was very reasonable. Then they were bought out by the Endurance International Group. Almost immediately afterward their support became crappy and I noticed delays accessing my domain as well as infrastructure related issues. When my hosting contract was over, I was happy to move somewhere else.

When you call these companies for support, you are immediately placed into a third-level support queue. After you finally connect with a human, these brain-dead support people follow scripts that are designed basically to not solve your problem and make you go away. Moreover, I found myself far more knowledgable about hosting and how to solve problems than they were. They could rarely even cover the basics. If you needed real help I found I had to badger for second-tier support.

All this is to keep their costs low since one support center for dozens of companies is obviously cheaper. But it results in inferior service, as evidenced by my experience with hostgator.com.

Endurance International Group own a lot of hosts, most pretty obscure. Among those they purchased you may be familiar with include Hostgator, Bluehost, Hostcentric, iPage and Site5. You can see a full list of the brands they bought on this Wikipedia page.

Given the low margins, the hosting business is likely to continue consolidating. There is certainly a lot of smoke and mirrors in this business. What used to be good hosting can turn into poor hosting pretty quickly when they get acquired. This is true of MediaTemple, at least it’s Grid Service, based on my latest experience since it was bought by GoDaddy. 

My current hosting recommendations, last updated April 2019

Right now my recommended hosts include Dreamhost for most hosting and Rackspace for dedicated and virtual server hosting. If you are considering rehosting, I’ve listed some of their advantages and disadvantages on my rehosting page. Read it over carefully because all hosts have special terms and conditions and limitations. If you choose to host or rehost with Dreamhost, please use my affiliate link. This way I earn a small commission. You will not pay extra.

If you need help moving your website toa new host, I can certainly help. You might want to send me a service inquiry.

It’s quite clear to me though that you are likely to be unhappy with any hosting owned by the Endurance International Group. So avoid.

Session hijacking: what’s (probably) going on

Over the last couple of months I’ve had a number of clients come to me because of mysterious things happening on their forums. Going to a forum they find that they are logged in as someone else and can see things they definitely should not see, such as private messages and forums they don’t have privileges to see. I’ve spent a lot of time trying to figure this out talking to client’s web host support teams and scouting phpbb.com for a solution.

The good news is that this is not due to some deficiencies in phpBB. The bad news is that this is due to the way your web host has configured their servers and it’s affecting phpBB.

phpBB is the #1 forum solution, with something like 70% of the market. But as a percentage of popular software installed on websites, phpBB is tiny, on about 1% of websites. What’s the 800 pound gorilla? It’s WordPress, which runs 27% of websites. So web hosts will meticulously tune their servers to optimize for WordPress, giving short shrift to much of the rest of the open source software out there. Most web hosts now say they are optimized for WordPress and market WordPress-specific hosting. phpBB is being left behind along with lots of other software. Because phpBB gets most of its content from a database to be presented on the fly, more than most open source solutions it is not amenable to static content.

The problem is most acute if you have Bluehost shared hosting. The underlying issue is some software called Varnish, more specifically Varnish HTTP Cache. Varnish helps dynamically driven sites perform more efficiently by caching content in your server’s virtual memory. Web hosts can make more money if they can get more utilization off one one web server. Varnish is one way they keep costs down as it allows them to stuff more websites on one machine.

Varnish is kind of pointless with phpBB since phpBB already has its own cache, which you can find in your forum’s cache folder. Essentially phpBB programs, templates, stylesheets and SQL calls are all compiled into .php programs in the cache folder so they can be executed more quickly. So it’s duplicitous but more importantly interferes with phpBB’s default behavior. So if you have the issue, contact your web host to find if they are using Varnish and if so have them turn it off. As for Bluehost, as of this writing they will tell you they can’t turn it off. You have shared hosting so one size fits all. They will however be happy to move you to their cloud product. Varnish is not installed there, so you won’t be affected. However you may have to pay a higher hosting fee.

It’s unclear if Varnish is the sole cause. Other potential problems may be due to Content Delivery Networks (CDNs). This is most typically CloudFlare, since it is bundled free by most web hosts. CDNs attempt to move content closer to the user by having it fetched from server farms geographically close to the site viewer, thus speeding up page load time. This is usually fine with phpBB since CDNs generally only store static files like images. So a CDN shouldn’t cause issues like this, but if you have a CDN you might want to disable it to see if the problem goes away. Note: the one time you do need to do something with your CDN is when you add a style or significantly change the look of your site. Then it’s a good idea to tell the CDN to delete all its cached content. Otherwise, the experience by end users might be mixed or odd.

I also suspect that ModSecurity may be causing issues like this, but I don’t have enough proof yet. If it is enabled, disabling ModSecurity may make your problem go away. As I blogged recently, disabling ModSecurity in general tends to solve a lot of weird phpBB issues, while it may introduce others by potentially making it easier for your site to propagate malware and viruses.

Where should you be hosting?

It’s not unusual for forum owners to want to rehost. Rehosting though is a big decision. You generally pay for a year or more of hosting up front and you have no assurance that the new host will be better, or even as good, as your current host. In addition, moving a forum to a new host is a pain, which is why a significant part of my business is helping clients move their forums. If you’d like me to help, send me an inquiry.

It does beg the question of where you should move to. Generally the pain level has to be pretty high to move to a new host. It’s often easier to renew what you have or pick a higher level of service with your current host than tackle the time and expense of rehosting.

Hosting is in flux

For the most part you are left to sifting through the general hosting market to figure out a good host. And the hosting market like much in the IT world is in flux. Thus, my recommendations to clients has changed over time. For example, I used to recommend HostGator to my clients and even hosted my sites there too. Then Hostgator became a victim of its own success. It got bought out and is now just another company that is part of the Endurance International Group portfolio. About the time they were bought out, the quality of their hosting declined. I noticed a marked decline in their technical support. Needless to say I don’t recommend Hostgator anymore.

High usage solutions

Certain forums fall into a specialized class of hosting. If you are one of these forums, you are already probably on specialized hosting. Mostly these are highly trafficked forums. To deal with the hundred or thousands of posts per day, you are likely on a dedicated or virtual private server, and are probably paying handsomely for the privilege. If you fall into this category but are on shared hosting, you probably are having issues and need to pay for one of these solutions.

Stick with commodity software

One thing for sure: get generic web hosting. This means you need a cheap LAMP stack: Linux (operating system), Apache (web server, although nginx is acceptable), MySQL or MariaDB (its clone) for the database and PHP for the scripting language. phpBB of course is written in PHP so it must be available. Don’t pay for Windows hosting. It’s more expensive, you don’t need it, it adds complications and you will probably get poorer performance.

My guess is less than 2% of forums fall into the high usage category, which means generally that inexpensive shared hosting is where most forums belong. Okay then, which shared hosting? There are lots of hosting guides on the web, most of dubious value. Working with lots of clients though I can tell you my own personal opinions. The final choice may come down to which services you value the most, such as fast and convenient technical support. As a general rule this is not available for shared hosting.

Here are my current ratings for popular web hosts with notes as applicable. I have no axe to grind and I make no money from these opinions so at least you know they are unbiased.

Shared Hosting

  • Grade A
    • Siteground – No telephone support but chat and ticket support. Nonetheless it is smartly engineered and well thought out with features like automatically managed Let’s Encrypt security certificates.
    • Bluehost – Technical support is a bit slow but you can usually get a hold of someone within half an hour or so. Great support once you get a representative. On par with Siteground. You might want to choose between them based on price or features.
    • MediaTemple (Grid service) – Proprietary control panel (not cPanel or Plesk) but uses all solid state drives. A bit harder to use than cPanel-based sites but much more reliable and fault tolerant than what is typically available, as well as faster-serving due to the solid state drives and the built in Content Delivery Network (CDN). Stay away if you are not particularly technically inclined. Redundancy is built in making it a great choice if you need high availability. This is actually Amazon Web Services under the hood but made much less geeky for us less technical people. Terrific and fast technical support but you have to understand their boundaries of what’s available on the Grid service.
  • Grade B
    • Hostpapa
    • 1and1 – Available in many countries including UK and much of Europe.
  • Grade C
    • Hostgator – See above
    • GoDaddy – Much better than they were a few years ago, decent technical support but sometimes there are frustrating issues with how they have their shared hosting configured. Lately I’ve been having users complain about poor integration with phpBB 3.2.
  • Grade D
    • Web.com – Really poor technical support with Level 1 techs who know very little and work hard to make you just go away. Their web hosting configuration is suboptimal, confusing, nonstandard and often causes problems as a result. If it’s anything beyond the most routine issue they will want to forward you to their Level 2 service for which they will charge a $75 fee.
    • Network Solutions – Part of the same conglomerate that owns web.com. It’s ironic considering Network Solutions used to be the center of the Internet, responsible for maintaining the whole Domain Name system. As a host though they suck and are expensive.
  • Grade F

Virtual Private and Dedicated Servers

For highly trafficked forums only. You basically need to be a system administrator or can hire one to use these solutions. Don’t expect any handholding because you will be lucky if you get any.

  • Grade A
    • MediaTemple – a premium web host worth paying for with terrific technical support
    • Rackspace – services more the business community with prices accordingly, but top notch
  • Grade B
    • Digital Ocean – nice fancy infrastructure with all solid state drive but you are basically on your own. You need to be a techie. Their host control panel can be baffling if you are used to cPanel.
  • Grade C
    • 1and1 – great prices for this class of service, but servers seem to be old and underperforming. Technical support is above average for this tier.

Specialized solutions

  • Amazon Web Services EC2 – only for geeks, but it allows scalable cloud computing. There are AMI (Amazon Machine Instances) for phpBB that you can install.

Obviously I left out lots of hosts as there are hundreds out there. I reference the ones I work with most frequently with clients. Please leave comments about your experiences so others can benefit or avoid mistakes.