Writing trivial extensions is not too hard

A client recently sought me out. A board user wanted to get email notifications when receiving private messages. These are disabled by default but can be enabled. Most users though won’t dig through the User Control Panel to find the setting: UCP > Board preferences > Edit notification options > Someone sends you a private message > Email.

The client wanted all his board users to have this enabled by default. Not one to ever reinvent the wheel, I searched for an extension. I found this one, in release candidate status, which allows an administrator to tweak notifications for a user on a case by case basis. But that didn’t meet the requirement.

I wasn’t sure how this setting was stored in the database. I did some experimentation by turning it on and off and looking in the phpbb_user_notifications table using phpMyAdmin. This provided a clue on what to do.

Private message email notification setting for user in the database

Essentially if I could add rows like this to this table, I’d effect the change needed. There were two aspects to this:

  • Ensure existing users has this setting set
  • Ensure that new users had this setting applied automatically with registration

If you know Structured Query Language (SQL) pretty well, it wasn’t that hard to figure out the SQL needed to ensure existing users who didn’t have this setting applied had it applied. This amounted to adding rows to the phpbb_user_notifications table for those who didn’t have a row for this permission already. The SQL was little complex but based on what I learned from testing in phpMyAdmin, straightforward and turned out to be:

INSERT INTO phpbb_user_notifications
SELECT ‘notification.type.pm’, 0, user_id, ‘notification.method.email’, 1
FROM phpbb_users
WHERE user_id not in
(SELECT user_id FROM phpbb_user_notifications WHERE item_type = ‘notification.type.pm’ and method = ‘notification.method.email’)
AND user_type = 0;

Having it work automatically with new registrations though was more complicated. Initially I didn’t consider the idea of writing a phpBB extension to do this. Rather, I proposed writing a SQL trigger to do this instead. But then I realized writing an extension would probably not be that big a deal and would have the advantage that if the user was deleted, phpBB’s code would delete these user notifications at the same time.

Writing extensions for phpBB is complex the first few times you try, as I discovered. The phpBB Group has this Wiki you can read to see if you can get your head around it. You need to be able to program in PHP but really you also generally need related web stills in HTML, CSS and Javascript too.

The more extensions I write though, the more I realize a lot of rather simple functionality can be done with rather trivial extensions. There is a skeleton extension you can install that will generate a lot of boilerplate code that makes it relatively easy to add the specialized functionality you need. I placed this on my test board and simply filled out the fields needed. Once installed a link appears on the navigation bar. The most baffling part for new extension authors is figuring out which components to check. Much of this comes from experience, but hovering over the component with your mouse helps explain that the component will do for you.

phpBB Skeleton extension interface

In this case, all I really needed was to check the PHP event listener and service checkboxes. This was for a client so it was not for general release, so I didn’t need to worry about database migrations. I didn’t need to add any Administration Control Panel user interface. It was to happen behind the scenes. When the form is submitted it creates a nice .zip file which can be decompressed and uploaded to your development environment. I enabled the extension.

What I needed to do was to add some logic during user registration. But where? And how? phpBB is all open source, so it’s a matter of reading the code. If you read the code, you realize that at some point the function user_add in /includes/functions_user.php is called and it does the grunt work of adding new users to lots of tables.

Inspecting the user_add function I found inside it a number of events that could be hooked into by my extension. I found the core.user_add_after event at the bottom of the function that looked like it would work. Essentially, after phpBB does all the normal stuff to create a user, you can have it do extra work. The event documentation shows:

/**
* Modify the notifications data to be inserted in the database when a user is added
*
* @event core.user_add_modify_notifications_data
* @var array    user_row            Array of user details submitted to user_add
* @var array    cp_data             Array of Custom profile fields submitted to user_add
* @var array    sql_ary             Array of data to be inserted when a user is added
* @var array    notifications_data  Array of notification data to be inserted when a user is added
* @since 3.2.2-RC1
*/
$vars = array('user_row', 'cp_data', 'sql_ary', 'notifications_data');
extract($phpbb_dispatcher->trigger_event('core.user_add_modify_notifications_data', compact($vars)));

The skeleton extension I used created a main_listener.php program in the /event folder where you can hook in code for events like these. To insert a row into the phpbb_users table, I would need to know the user_id that was created. The event passes a user_row array, so it seemed likely the user_id would be inside of it.

Since I was going to access the database, I needed main_listener.php to be able to use phpBB’s database abstraction layer service. So first I went into the /config/services.yml file for the extension and added this service. This was straightforward. Since I didn’t need to use the language service, I removed the language service too. The file became:

services:
phpbbservices.setemailpmnotifications.listener:
class: phpbbservices\setemailpmnotifications\event\main_listener
arguments:
- '@dbal.conn'
tags:
- { name: event.listener }

Next, to hook the database abstraction layer service into main_listener.php. I removed any references to the language service first, then hooked in the database service. The main_listener class looked in part like this:

public static function getSubscribedEvents()
{
   return [
      'core.user_add_after'           => 'user_add_enable_pm_email_notifications'
   ];
}

/* @var \phpbb\db\driver\factory */
protected $db;

/**
 * Constructor
 *
 * @param \phpbb\language\language $language  Language object
 */
public function __construct(\phpbb\db\driver\factory $db)
{
   $this->db     = $db;
}

The getSubscribedEvents function is where you hook in a custom function to an event, the core.user_add_after event in this case. I told phpBB to look for and execute a function in the program, user_add_enable_pm_email_notifications. Further down in main_listener.php I added a small amount of custom code to do what I wanted done inside this function:

public function user_add_enable_pm_email_notifications($vars)
{
$user_id = $vars['user_id'];
$sql = 'INSERT INTO ' . USER_NOTIFICATIONS_TABLE . "(item_type, item_id, user_id, method, notify)
VALUES ('notification.type.pm', 0, " . (int) $user_id . ", 'notification.method.email', 1)";
$this->db->sql_query($sql);
}

The incoming $vars variable contains a collection of stuff that can be read by my event. It turned out that $vars[‘user_id’] contained the user_id for the next user. Then, based on what I saw in the phpbb_user_notifications table, the SQL INSERT statement became straightforward to write and execute.

I think this demonstrates pretty well how a reasonably experienced web programmer can make a small custom extension to add some missing minor functionality that the phpBB Group did not think was needed.

phpBB 3.3 is a pretty minor minor version

So I’m starting to port my extensions to phpBB 3.3.

I’m surprised by how easy it has been, at least compared with porting my extensions from phpBB 3.1 to 3.2. This is because not much has changed between 3.2 and 3.3.

In moving from 3.1 to 3.2, there were all sorts of changes under the hood that made porting extensions challenging. The way posts were encoded in the phpbb_posts table was changed, which was a big deal. FontAwesome support, which was very limited, was introduced, meaning we could use these in our templates instead of embedding images. A new TWIG library meant that the old way we extension authors inserted dynamic data into pages changed pretty fundamentally. We were asked to use a new template syntax in our templates. Template library calls also changed quite a bit. In short, 3.1 to 3.2 required quite a bit of reengineering, both of the phpBB base code but also of its extensions.

For 3.3, there’s not much upgrading of the database needed. Only one table is altered: phpbb_users, to change the properties of the reset token. The content in the phpbb_extension_groups table is changed. The Flash media group finally goes away (Flash is obsolete) and any non-Flash files in the group are now assigned to the downloadable files group.

As noted, you can use more Emoji codes in 3.3 and can put Emoji in a topic title. But this doesn’t alter the database. I have to test for these in my extensions, but so far it’s not a problem because the existing function I used to display these fields handles it for me. So nothing to do there.

So phpBB 3.3 feels more like an update to phpBB 3.2 than it does a new minor version. On the whole, the new functionality is quite minor. The main thing I have to do is test with PHP 7.4, since phpBB 3.3 now supports PHP 7.1.3 to 7.4 only. Getting my extensions to a stable 3.2 place before upgrading PHP is turning out to be much more work than retrofitting my extensions.

So yea for that.

Using unapproved extensions is dangerous

As more phpBB extensions are developed, they are becoming more popular. Extensions add functionality to phpBB beyond what is available by default. Based on my work with clients, most have extensions installed, so I factor them in when updating or upgrading their forums. They often need to be upgraded as well when a forum is upgraded or updated.

The phpBB Group maintains a database of approved extensions. Both the phpBB Group and me recommend that if you install an extension that you only install extensions downloaded from this database. This is because approved extensions are quality checked by the phpBB Extensions Review Team. The team thoroughly inspects the extension and ensures they adhere to all coding standards as well as use best practices to minimize security issues. An extension typically goes through a number of reviews before it is accepted, if it is accepted. So you can have confidence that if you download an official extension it is of high quality and secure.

If you are not familiar with how to install extensions, the instructions are on the Manage extensions page: ACP > Customise > Manage extensions.

Unapproved extensions fall into two categories:

  • Extensions in development
  • Third-party extensions

Extensions in development

Extensions don’t appear out of nowhere. Like all software, they go through a development process. You can see a list of extensions in development on that forum. The topic title is prefixed by the state of the extension in brackets. Links to the extension for downloading are in the first post. If you have feedback on the extension, you leave it as a post on the topic.

The phpBB group has extension authors self-certify the quality of the extension they are creating. This is similar to other software. The levels from most risky to least risky are:

  • [DEV] – Development – the extension is very recent and is being issued for feedback and to refine features. It should only be used on a test board.
  • [ALPHA] – Alpha – The extension is no longer in development. The feature set is largely set and the code quality has been refined. Traditionally an alpha release has meant that it is to be used “within an organization”. Alpha release testers are expected to provide feedback and significant bugs and security issues may be experienced. “Within the organization” has no meaning with phpBB so it simply indicates it’s out of the principle early development phase. Using it on a live, production board is quite risky and definitely not recommended. A download link is usually provided.
  • [BETA] – Beta – The extension is designed to be used and tested by a larger group of people. There may be significant bugs and security issues. It should not be used on a production forum, but the code quality should be pretty high at this point and most bugs should have been addressed. A download link is required.
  • [RC] – Release candidate – Most of the bugs have been found and fixed. The release candidate could be submitted for formal review for inclusion as an extension if no more issues are discovered as a result of testing. The extension should be stable with no more features anticipated. Using it on a production forum is not recommended, but if you choose to do so anyhow it is likely to work as intended and not show any problems. Release candidates are submitted to the phpBB Group extension review team at the author’s discretion.
  • [CDB] – Customise database. You will see this in the Extensions in development forum. It means that the extension is approved. There should be a link to take you to its official page on phpbb.com. The topic is locked.
  • [ABD] – Abandoned. The extension author abandoned work on the extension. It is not approved, should not be used but some other extension author could take up working on the extension. These are placed in their own abandoned extensions forum.

There may be multiple versions of the extension in each phase. Generally extensions in development start with 0.1 and as an extension reaches Alpha or Beta stage it becomes 1.0. But there is no fixed standard for version numbers other than the PHP Composer guidelines. The extension is usually suffixed by the build quality, ex: 0.1.0-dev. The extension is usually downloaded from GitHub.

Third-party extensions

Third-party extensions are usually developed by commercial companies and typically tie into existing products outside of phpBB. Companies can submit their extensions for review by the phpBB group but usually don’t. This is because a review is time consuming. It can take months to get a review, then multiple issues must be fixed, and the extension resubmitted. This is not agile enough for many companies. In addition, the phpBB Group frowns on software that does not use an open source license. Many third-party extensions are issued with open source licenses but tie into products or services that are not.

When you use one of these extensions, you are assuming significant risk. Obviously, these companies don’t want their reputation besmirched, so generally they will take the time to write a quality extension and possibly adhere to the coding standards for extensions. But since in most cases they aren’t approved extensions, they are risky because they were not reviewed by the phpBB group to ensure their quality. They are typically downloaded from the company’s website or from their GitHub page.

Tapatalk

Tapatalk is a smartphone app that allows you to use the same user interface to access multiple forums, phpBB or otherwise. Prior to phpBB 3.1 the Tapatalk modification was widely used because styles for phpBB were not responsive, i.e. did not resize intelligently for mobile devices. Since phpBB 3.1, approved styles must be responsive, so users can use a browser on their smartphone to access the forum without the hassle of the past. Still, many people like the convenience of using one app to access multiple forums, so Tapatalk developed an extension. It creates an interface between phpBB and the Tapatalk app.

This extension is not approved and likely would never be approved by the phpBB Group. Why? When you install the extension, although an interface is seen in the /ext folder as usual, there is also a /mobiquo folder installed in your forum’s root directory. The software in the /mobiquo folder does most of the work of communicating between phpBB and the Tapatalk app. Tapatalk is available for other forum solutions too, and they use a similar architecture. The /mobiquo folder does all the data munging, so it is unique and proprietary. phpBB’s extension architecture requires that all extensions work within the /ext folder. Since Tapatalk doesn’t do this and its data munging is proprietary, it’s unlikely to ever be approved. It’s clear that Tapatalk developers don’t want to try.

More importantly, Tapatalk injects a major vulnerability in that it can bypass phpBB’s functions that do important work like posting to the database. This makes it dangerous. You should encourage your users to use a mobile browser instead of Tapatalk to access your forum. Ideally, you should disable and uninstall the extension.

Cleantalk

Cleantalk is an antispam service. Cleantalk’s extension for phpBB is approved, but it’s very old (2016 as of this writing). It may be that a newer version of the extension has been submitted for review, but the version on phpBB.com probably won’t work on phpBB 3.2. If it does, it’s missing many features. So as a practical matter, if you use Cleantalk you will want to get its most recent published version off of GitHub. Just bear in mind it’s a version that has not been approved by the phpBB Group, so using it may be risky.

Proprietary style user interfaces

Many proprietary (paid) phpBB styles come with a user interface that makes it easier to customize the style, doing things like changing background colors easily, swapping in different logos, changing fonts, etc. Because these styles are proprietary, they are not free and thus not allowed on the list of approved styles for phpBB. Consequently, extensions bundled with their styles are not approved as well. Using a proprietary style incurs some risk by itself. Using an extension used to manage the style adds additional risk.

New extension under development: selective mass emails

So far I’ve developed two extensions, Smartfeed and Digests. Another one is now under development, Selective mass emails. You can read about it on the topic here which includes a download link.

I wrote Smartfeed and Digests as mods for phpBB 3.0. Digests even has a version for phpBB 2.0. So I pretty much had to turn them into extensions when phpBB 3.1 was introduced. Unfortunately, both are rather complex extensions, so to the extent I had time to work on them, it was a long and complex learning curve. It took about three years to turn digests into an approved extension.

Selective mass emails extends phpBB’s mass email function. Mass emailing is an administrator privilege only. It’s available in the Administration Control Panel: ACP > System > Mass email. The built-in function is pretty basic. It allows administrators to send mass emails to exactly one group or to a list of individual users whose names you have to type in. Only text emails are sent.

I was asked to consider developing this as a paid extension. After looking at the work involved, which was not much (and given that my schedule is not busy at the moment) I decided to develop it for free.

So it’s a simple extension. Moreover, now that I have a couple extensions under my belt, creating a relatively simple extension that is not too complex is pretty straightforward. Now I know what I am doing. So I’m becoming more willing to develop custom extensions since I find them less intimidating. If you have one you want me to consider developing, you can send me a service inquiry and we can discuss my comfort level and a cost range.

The extension will evolve but you can see version 1.0.1 in development mode:

Selective mass emails (version 1.0.1)
Selective mass emails (version 1.0.1)

Basically, the extension refines the mass emails allowed to be sent. By using core and template events built into the phpBB code base, I was able to add fields to the form. The number and placement of these events is also something of a limitation. For example, mass email allows you to send all emails to one group, and only one group. I can’t change that because I can’t tie it to one of the available events.

The extension can be used to do things like send reminder emails to people who haven’t posted in a while, for example. You can also use it to send emails to inactive users, users who have posted more (or less) than a specified number of posts, or have more or less than a specified number of private messages. If you use ranks, it will also allow you to send emails to people with the ranks specified.

As this is a development extension, it should not be used on a real board. At some point I will create a GitHub archive for the extension. You might want to take it for a spin, but on a test forum only for now.

 

Some pricing changes

I’ve upped some of my prices. You can see a list of all my prices on my services page.

I’m trying to recapture the labor involved in answering your questions prior to you hiring me. Not everyone who inquires of my services hires me of course. Typically there is at least an hour of time involved in back and forth emails with clients. This time was not built into my pricing structure, meaning I am effectively discounting my own labor rates by giving away this time for free.

So mostly this amounts to $10 or $20 increases for services I do most of the time, such as updates, upgrades, conversions and extension installations. My basic labor rate is unchanged. Obviously if you choose not to hire me you don’t have to pay anything. But if you do I want the work I do for you to reflect the full amount of my effort. And that typically involves at least an hour of back and forth communications over email, often more, usually over many days, as details about the work are refined.

My prices remain very competitive. If you can find a better deal out there, please go for it.

I also updated the page to discuss developing extensions for clients. In general I don’t do it, but I will refer you to other extension authors I trust. Mostly it’s due to a lack of time as developing extensions takes considerable time. Extension development is quite costly and most clients usually decide they cannot afford to have custom extensions developed anyhow.

Setting up your forum, part four (picking, installing and enabling extensions)

(Read parts 1, 2 and 3.)

Once you have installed your phpBB forum, created the forums you need, installed and configured a style, and set up your user groups, generally the next thing to do is install any needed extensions.

What are extensions?

Extensions apply to phpBB 3.1 and higher. They are somewhat analogous to WordPress plugins and allow you to somewhat easily add additional functionality to phpBB beyond what comes “out of the box”. While not as numerous as the 44,000+ plugins available for WordPress, the number of extensions for phpBB grows regularly over time and should continue to do so. Note: phpBB 3.0 does not support extensions, but does support modifications.

Why are there so few extensions?

One reason there are fewer extensions for phpBB than there are plugins for WordPress is that extensions are a relatively new feature of phpBB. phpBB 3.1 was the first version to support extensions and it is about two years old. Another reason is that extensions are harder to write than WordPress plugins. I should know as I am the author of the Digests and Smartfeed extensions. As of this posting, only my Smartfeed extension has been approved. There is a lot to learn to write an extension and arguably it takes a senior PHP developer to write an extension, as it has a complex framework and a lot of onerous coding standards that must be adhered to. Only approved extensions are released in phpBB’s extension database. Just getting a review can take months, as I discovered.

Official extensions

Do you need to install extensions? If so, which ones? phpBB out of the box is quite feature rich. But there are common things that forum administrators want to do beyond what is available by default. Take for example some of the official phpBB extensions. These are so common that they are developed and maintained by the phpBB group. These include;

  • Pages – creates static pages that convey common information, such as a news page for your site
  • Board Rules – similar to pages, used to convey your policies for use of your forum
  • Auto groups – eases group management by setting up rules for putting people in and taking them out of groups
  • Board announcements – puts a “no way can you miss this” announcement at the top of every page viewed
  • Collapsible forum categories – allows users to collapse and minimise forum categories with a simple click, so they see only categories and forums that interest them
  • Google analytics – Provides a field for an administrator to enter their Google Analytics tracking code, so all web page usage is reported to Google Analytics. This way you can watch usage of your site, see popular pages and gain insight into how your forum is being used inside Google Analytics.

Finding extensions

phpBB’s extensions database contains a list of all approved extensions. Extensions are categorized and searchable but even so it can be hard to find an extension that meets your need. The extension you want may not exist. You can propose it and hope someone will take it up as a project. If someone does, don’t expect it to be something simple to do. At best it takes months to develop, test and get an extension approved.

Installing, updating and deleting extensions

Installing an extension is not as simple as installing a WordPress plugin, at least for phpBB 3.1. This is likely to be more intuitive in future releases of phpBB. Right now you must download the extension, unzip it then upload it to your forum’s ext directory. Here are the steps for installing, updating and removing extensions. An extension is actually a directory inside of another directory. The extensions architecture requires it be placed in a vendor directory. This allows one vendor to write multiple extensions but keep them separate. For example, for my two extensions, Digests is placed in ext/phpbbservices/digests and Smartfeed is placed in /ext/phpbbservices/smartfeed.

Recommended extensions

Let your requirements dictate which extensions you install. Don’t install extensions that you don’t plan to actively use.

Based on extensions I have installed for clients, these extensions tend to be particularly useful:

  • Google analytics (see above)
  • Pages (see above)
  • Share on – allows topics and posts to be easily shared on various popular social media platforms
  • PayPal Donation – allows you to ask for donations via PayPal
  • phpBB Gallery – allows users to have their own photo gallery inside of phpBB
  • Stop forum spam – checks the username, email address and IP of posters against blacklists, to reduce spam posts
  • AJAX shoutbox – Allows users to leave short tweet-like messages when they are on the forum, separate from phpBB’s forum and post system. Users on the forum at the same time can chat this way.
  • Copyright in footer – adds a copyright statement to the bottom of each page if you claim copyright on your forum’s content.
  • Board3 Portal – Provides a portal-like experience for phpBB, allowing sidebars for additional content to be seen on a portal page. With an additional change to your .htaccess file, you can redirect the forum to go to the portal page by default.
  • Sortables CAPTCHA – Provides a new spambot countermeasure where “correct” content must be dragged into an appropriate column. This can be used during registration to add greater assurance that a human is registering.
  • AdSense – Serves a horizontal Google Adsense ad immediately after the first post of a topic on a page. You have to get a Google Adsense account and set up the ads on that site first.
  • Recent topics – display topics with recent posts and replies on the index
  • Smartfeed – This is my extension. It makes your forum accessible to news readers. Since phpBB has an Atom feature built in for public forums, this extension is most useful if you want to support RSS feeds or for users that want to access protected forums securely using their newsreader using services like Feedly. This is done through creating an authentication key using the Smartfeed user interface.

New extensions are regularly added, so make sure to check the current extension database periodically. Many extensions are in development and when approved they will be added and widely used.

Extensions issues with upgrades to phpBB

When you move between one major version of phpBB and another (such as from 3.1 to 3.2) some of your extensions may not work. You can see the versions of phpBB that the extension supports in the Administration Control Panel: ACP > Customize > Extensions Management > Manage extensions. Click on the details link for the extension. If a given extension is critical to your forum, do not upgrade to a new major version until the extension is updated to work for the new major version.