Reliable, affordable and high-quality phpBB services and consulting since 2006
This version fixes a bad join between the phpbb_banlist and the phpbb_users table. Details are here.
The version can be downloaded from my selective mass emails page or from the GitHub branch. If downloaded from GitHub, make sure to follow the proper procedures for upgrading extensions and to place the files in the /ext/phpbbservices/selectivemassemails folder.
May was a strange month. Not much work came in, but I was kept plenty busy anyhow. The reason I was busy was because of one large commercial client who I’ve been working with for more than a year now. By the end of the month I had worked 65.5 hours for this client, all at my commercial rate, so it was very profitable work. In addition the work was mostly interesting.
Work performed for this commercial client began with analyzing their shopping cart interface for phpBB 3.0 and trying to make it work on phpBB 3.3.4. When on the view topic page, administrators and moderators see additional fields under the user’s profile that link to their order history so moderators can respond to problems they are having with products they make. To make this work required an extension, tying some logic into some phpBB core and template events and writing some template logic too.
It turned out that another styling extension I wrote for them (to inject their proprietary headers and footers) interfered with this extension, so I had to make changes to that extension to carry this information too. This resulted in a re-think as we were beginning to introduce duplicate code across extensions. Since the styling extension always had to be on, it was the common dependency, so I moved the common code into that extension and adjusted the other extensions to use it the common code as a service. An issue emerged with duplicate post rows on the view topic page that was resolved with the help of kasimi on phpbb.com, who got a nice monetary tip as a result from me. It took a couple of weeks to tediously work through these issues.
Afterward I started next on an integration challenge. The accounting system for this client needs to send push events to synchronize its database with phpBB. For example, if a user’s email address or username changes on the accounting system, it must change in phpBB as well. In addition, the accounting system may issue a ban, which requires users to be banned on the forums as well, either permanent or temporarily. There is also a push event to remove an account, which will require a user’s account in phpBB to be removed as well. First I tried creating an extension with a public facing controller as the command is invoked as a HTTP request using curl. But no forum headers and footers were wanted, so this approach eventually introduced unacceptable problems. So the idea became to write something that works outside of phpBB’s extension system, refactoring code they had written for phpBB 3.0. I’m going with this approach now and have the ban logic working again, as well as user changes needed. Still quite a bit of work to go, and I struggled through a number of issues relating to the need to bypass phpBB’s session logic and cookie usage. This should be solved during June.
As for work for other clients:
Version 3.3.4 of phpBB was released today. It’s a maintenance version with no critical security fixes and no important new features. The only new feature that I could find is that it supports a new image format: WEBP, introduced by Google and supported by most browsers which provides more compact, lossless image rendering than PNG.
Mainly it offers better support for PHP 8. It contains a bug fix that I drew to their attention and an important fix that I stumbled across recently if you have unusual characters in your database name, which kept the database size from displaying and actually wouldn’t let you get into the ACP at all! But if you had this issue, you would have patched it already.
One thing you might notice: the RESET button goes away on a number of forms because, arguably, a RESET button is obsolete. One way to reset a form is simply to refresh the page.
So update to it if you want, but there is probably no tearing hurry for you to do so. You can download it from phpbb.com’s download page, read the announcement, see a summary of the release here and see all the issues addressed here.
I stayed busy in April. The bulk of my work was for a commercial client, work that has been ongoing since February 2020. This work is part of a large project to upgrade their phpBB 3.0.11 board with lots of custom tailoring to phpBB 3.3, keeping the custom logic but moving it into extensions. Since last autumn we’ve been challenged to make a custom OAuth authentication work. In this case the client has its own OAuth server, so it’s a matter of creating an extension that handles this correctly. The good news is by the end of the month we had solved the last major OAuth integration issues. So my next step is to analyze the issues to replicate their moderation changes. The poster’s profile description on the view topic page contains information only seen by moderators including links to their order history. In addition there are two new moderation control panel screens with logic. Presumably this will handled by an extension. So this work tended to keep me busy and fill in time that would otherwise be slow.
Other work I did in April:
(If you decide to buy DreamHost hosting, please use my affiliate link. This way I earn a commission. You won’t pay extra.)
Hosting is changing
The world of hosting has been changing, and often not for the better (at least from the perspective of hosting customers). As I noted in this post, many hosts are getting gobbled up by other firms, usually Wall Street firms looking to maximize profits by consolidating hosts, cutting services and maybe raising prices.
Another big trend is to go virtual. GoDaddy, for example, is moving into Amazon’s cloud. That certainly saves the expense of managing their own hosting centers and paying a hosting staff. More recently, the same is true with Siteground, which I have been using for a few years, which is going into Google’s cloud. Siteground has also raised prices rather substantially, hoping most clients will opt for inertia. For its shared hosting, Siteground keeps an eagle eye on resources you are using and will quickly complain if you go over these resources.
So I’ve been looking for acceptable alternatives, always aware to not to put too much faith in my web host of the moment, since there’s a good chance they’ll get bought out, raise prices, reduce staff or do all of these things. I tend to be irritated by hosts that provide generally incompetent support staff, obscure really important features like limits on disk space or outgoing emails, or who raise prices way beyond the rise in inflation.
A week spent in GoDaddy support hell
I thought GoDaddy had been improving in general. Going virtual wasn’t necessarily bad if the cloud provider can provide greater speed and reliability. But this month I got caught up with a client using GoDaddy which suggested they were reverting to old habits. It might have been because his board was on an old machine that hadn’t transitioned to the cloud yet. The support experience with GoDaddy though was just horrible.
His plan was so old it only supported PHP 5. There was no migration path to PHP 7 except to buy a new plan, and they couldn’t be bothered to help him move his gigabytes of files from the old to the new hosting, or transition the database. Not that we didn’t try. Downloading and uploading files from home is generally a painful, tedious and slow experience. The volume of files was large and the uploading was very tedious because his ISP severely limits upload speeds. It took days.
On the back end I was busy trying to upgrade his board and get that working. The shared hosting timed out exporting a database and timed out uploading it too. I tried all sorts of tricks to get it to work and upgraded the database on my machine since the upgrade scripts would time out due to resource caps. This went on for more than a week.
DreamHost turned out to be reasonably dreamy
Eventually I recommended he get rid of GoDaddy. But where to go? I sent him to DreamHost.
I had a good impression of DreamHost based solely on a seminar I attended at a Boston Wordcamp in 2019. In the seminar the speaker talked about how they fended off lawsuits by the Trump Administration to monitor their servers, and succeeded. I asked my peers about DreamHost and the reviews were generally positive. Still you don’t know until you try.
Was rehosting with DreamHost a wonderful experience? Overall it was pretty good, but it had its quirks. Like most hosts they offer various kinds of hosting, and the client chose the shared hosting plan. Loading his database on DreamHost, it timed out there too with resource limitations, but I was able to work around the problem. His database is nearly a gigabyte in size. Essentially I figured out where it failed (in the middle of populating the phpbb_posts table) and created an extract of the rest of the database. That got the rest of it. I’m just glad I have a text editor that can handle really big files (BBEdit, BTW).
Their tech support was responsive, helpful and reasonably quick. One thing I liked was that they didn’t have explicit quotas on database sizes and overall file sizes. That was important for this customer. Looking through the fine print of the hosting plan, I saw only one red flag: there was a limit of 100 outgoing emails per hour. That didn’t matter for this customer. It looked like a good fit.
Curiously my hosting plan with Siteground was up for renewal, so I took the plunge myself buying their multi-site shared hosting plan. Siteground wanted about $300/year, and this initial 3-year DreamHost contract was about $142, which is a huge savings. This got me more into the weeds.
The good:
The bad:
The weird and unusual:
If you can deal with these oddities, DreamHost may be for you. It is for me, presumably for the next three years at least! It does appear to be a very good value.
The phpBB Group has approved this extension. You can download it and access a FAQ and support forum here.
I kept very busy in March with little downtime. I did make some progress on a new version of my digests extension that should be able to better handle resource timeouts by failing more gracefully and not locking phpBB’s cron process. But mostly paid work got in the way of this other stuff. Here’s a summary of my work in March:
phpBB has lots of features and some are very obscure. I ran into one of these recently for a client. Apparently, it’s possible for users and groups to not have their posts count in their total post count.
Obviously this is not an issue for most boards, and it’s hard to figure out a use case for why you would want to do this. One case is to avoid unearned ranks. phpBB has an optional ranking system which is typically based on the number of posts. Making lots of replies that aren’t really substantive can do this and it’s an issue on some boards. Presumably these boards are not well moderated, as moderators are usually tasked to find and remove these posts.
Whether a post is counted in a user’s total post count or not depends on whether the post_postcount column in the phpbb_posts table is 0 or 1. If it’s 1, it gets counted. If it’s 0, it doesn’t get counted.
For example, you could have a general chat forum and posts in these forums might not want to be counted in a user’s total posts because they are considered trite. Then you could have other forums where the meat of the board’s content is, and posts in these forums could count.
Changing the post count policy
By default, all posts count in a user’s total post count, so if you don’t want posts to count for certain groups or users, you will have to change phpBB’s settings.
This is controlled by a permission. Typically you set this group forum permission. ACP > Forums > Forum based permissions > Group forum permissions.
Next you pick the group, then the forums for the group where the permission will be changed. Next you select the Advanced permissions link and then click on the Post tab. Then set the Increment post counter setting to either Yes or No as desired for the group and the forum. Do this for all the forums whose permission you want to change for the group and save your changes. If you want to change this permission for other groups, repeat.
If you want to remove this permission for a particular user or users only, you can avoid groups by selecting ACP > Forums > Forum based permissions > User forum permissions. In general though I think you’d want to avoid doing this on a user-by-user basis, and do this only for groups.
Changing post counts for old posts
While this works for future posts, it doesn’t affect existing posts. How do you fix these old bogus post counts? As best I can tell there is no extension to help you out. So you have to get down and dirty with your database, using a tool like phpMyAdmin.
Before attempting anything like this, disable your board and back up your database, or at least backup the posts table. This can be done in the ACP on the Maintenance tab.
Generally you need a list of forum_ids and group_ids. A tool like phpMyAdmin makes it simple to look inside your database. Your web host control panel will usually have a tool that runs phpMyAdmin. Make sure you select the database for your board. Your board’s config.php file will provide the database name, if there is any ambiguity.
Database tools like phpMyAdmin are very powerful. It is possible to make large, irreversible changes to the database if you type commands incorrectly.
A forum_id is a number assigned to a forum in the database. You can see a list of forum_ids with this query, which can be executed on the SQL tab for your database in phpMyAdmin. (Change phpbb_ if your table prefix is different):
SELECT forum_id, forum_name FROM phpbb_forums;
Next, you need a similar query to get the group_ids for your groups:
SELECT group_id, group_name FROM phpbb_groups;
With these group_id and forum_id values, you can then write a query to change the post_postcount value in the phpbb_posts table for a particular group and forum. Using the example above, if you wanted to change this flag to 0 (don’t count) for all members in the Registered users group (group_id = 2) to the forum “Your first forum” forum (forum_id = 2), the SQL would be:
UPDATE phpbb_posts p, phpbb_groups g, phpbb_forums f, phpbb_user_group ug
SET post_postcount = 0
WHERE p.forum_id = f.forum_id AND
p.poster_id = ug.user_id AND ug.group_id = 2 AND f.forum_id = 2
If you wanted to set post_postcount to 0 (zero) for all posts made in a forum, the SQL is somewhat simpler. In the case of forum_id = 2 the SQL would be:
UPDATE phpbb_posts p, phpbb_forums f
SET post_postcount = 0
WHERE p.forum_id = f.forum_id AND f.forum_id = 2
Recalculating the post counts
There is one last step: you need to recalculate these statistics:
ACP > General > Resynchronise post counts
Clean up
If you messed up, restore the table you backed up.
Also, if you disabled your board, reenable it.
February was a busy month. Here’s a summary of my activity last month for various clients:
In January, I finally got around to releasing the first draft of my Spam remover extension. This extension finds spam in posts and private messages including very old spam that may be present on your board. It took longer than expected because work for clients comes first (as it pays money) but also because it turned out to be more time intensive than expected. The extension uses Akismet (mostly used to moderate WordPress comments) to make a spam determination and using the service is not necessarily free and requires a license. The most technically challenging part was the finding spam part, as some databases are huge. Based on my testing, you can’t expect Akismet to test more than about four posts per second, which means on larger boards finding all spam may take hours or days. Of course you can narrow the range to a date range. phpBB doesn’t have any great ways to handle long running tasks.
Otherwise, it was somewhat quiet doing work in January. Paid work included these jobs:
