Tag: Spam

Getting rid of old spam is hard

On: By: Mark

Updated November 17, 2021 – My spam remover extension, which uses the Akismet service (may require paying Akismet to use the service) can reliably remove a lot of legacy spam.

phpBB is now pretty good at keeping spam users from registering and posting. In the phpBB 3.0 and 3.1 days, its defenses turned out to be pretty weak. The GD Image spambot countermeasure (still the default) was easily hacked. phpBB has at least added settings to let you tune it better, making it harder to hack. It also started supporting Google’s reCAPTCHA, but the version in phpBB 3.1 was quickly hacked and phpBB was not agile enough to quickly integrate its versions 2 and 3 reCAPTCHAs.

This led to the an inundation of spam on certain forums, mostly bogus spam registrations but also lots of spam posts in some forums. Some administrators countered by requiring all new users to be approved by an administrator. But when inundated with hundreds of these in a short period of time, it’s a hassle to delete them all, or discern the real new users from the spam ones. For a few years, I made quite a bit of money removing spam for clients.

With phpBB 3.2 things slowly got better, at least if administrators used best practices. Best practices were to use reCAPTCHA version 2 “I am not a robot”, or the Question & Answer, providing the questions were sufficiently difficult. A malicious human could still take the time to solve the questions, but these were unusual. There were also a few extensions that could help. The Sortables CAPTCHA was one of the more useful ones.

My go to for years has been the Cleantalk extension, which requires subscribing to their service. But now there is also an Akismet spam extension, which also requires a subscription, which can be free for personal sites.

All this is good at preventing spam, but how do you get rid of months or years of spam posts? That was my dilemma this week working with a client.

The latest version of the Cleantalk extension has a feature that removes spam users and their posts. But I discovered it has a few serious limitations:

  • It bases its judgment based on the IP of the poster. The user’s last IP is stored automatically. It doesn’t examine the post text. Over time, IPs that used to be marked as spam get cleaned up, and when this happens these IPs are no longer flagged, so spam registrations aren’t caught.
  • Its interface for finding these users is slow and can easily time out, which means sometimes it can’t succeed. It also lacks pagination.

Why this particular client ignored this problem for a few years, I don’t know. But cleaning up the database was a big challenge. The only real way to do it is to manually look at every post and flag those that were spam, then use moderator tools to get rid of them.

This was time prohibitive, but if these could be removed presumably people would start posting again and Google would rank the site as legitimate again, bringing in new people.

The next best solution I found was to try to identify when the spam started. This took quite a bit of analysis, but looking in the most posted forum on the board it looked like it started on Feburary 10, 2018. So I used phpBB’s Prune User feature to remove users and their posts that registered after the spam started.

This seems to have gotten rid of the spam. But it also removed accounts of some legitimate users, and their posts as well. Those who had accounts before then were unaffected and their posts remained.

phpBB needs a real solution which so far doesn’t exist.

But I think I have found a solution … if I write the extension. If you are an extension developer, please go ahead and develop it, just tell me so I don’t waste my time.

It turns out that the Akismet, the biggest solution out there and used widely in WordPress to moderate comments, has a Submit Spam API. So in theory, if you pass the needed information to it including the poster’s IP and the post text, it can render a judgment on whether it is spam or not. If these posts can be flagged, they can then be removed.

One possible issue is that the service requires sending it a User Agent string. phpBB does not store this. Perhaps a fake user agent string could be supplied, but would this render a correct judgment? If no, this solution wouldn’t work. Also, it requires an Akismet key to use, which might require some boards to purchase the key. This may be a limiting factor for some.

As I have time I hope to see if this is a viable approach finding and removing spam posts in phpBB.

Cleantalk extension for phpBB can remove spam posts, plus its spam firewall feature is very useful

On: By: Mark

This is an update on an earlier post on removing spam posts.

Removing spam posts is hard because it requires actually reading the post and deciding if the post is spam or not and then using moderator tools to remove these posts. If your forum is overwhelmed with spam posts, this is a Herculean endeavor. Ideally though posts could be “read” by software and it would make the judgment on whether it is spam or not.

The Cleantalk extension for phpBB 3.1.x and 3.2.x can do just this as well as lots of other really cool tricks. My customers love Cleantalk, but the service is not free. However, it is so inexpensive that it easily justifies spending $8/year for the service. You can subscribe on the Cleantalk website. As of this writing, you can try it for free for 7 days. After 7 days, it won’t bring down your forum but it will stop working.

What is Cleantalk?

Cleantalk is essentially a huge database of addresses of known spammer sites. While it’s not perfect, based on the experience of my clients it is about 99% perfect. I originally recommended it as a spam registration solution for my clients. It still does that but is less necessary since phpBB 3.2. This is because since phpBB 3.2, version 2 of Google’s reCaptcha is supported. Unless it gets hacked, as long as you have it properly configured as a spambot countermeasure it should prevent virtually all spam registrations.

However, it has two powerful features that still keep it relevant for phpBB forums.

Cleantalk ACP Interface
Cleantalk ACP Interface

Installing and enabling Cleantalk

Cleantalk is installed like any other extension. While it can be downloaded from phpbb.com, you should download it from its GitHub page. This is because as of this writing the version on phpbb.com does not include the spam firewall feature, and you will probably want to enable this feature. You can access it through the Administration Control Panel: ACP > Extensions > Antispam by Cleantalk. Before you can do much with it you have to enter your Cleantalk key which you can get from their website or by pressing the button in the extension that should retrieve it for you.

Removing spam users and spam posts

As you can see from the image, once the extension is enabled and the key is properly configured there is a prominent Check users for spam button on its page within the Administration Control Panel. If you have lots of users, it may hang. Based on my experience though the next time you go into its interface you will see a list of potential spammers.

As I said, it is not perfect. So I recommend that for users with posts to check these out these users topics to make sure their posts are spam before deleting them. For those you want to delete, check the boxes next to their usernames and then press Delete marked. You can also press Delete all to remove all users and their posts. You may have to go through many pages to delete all spam users and their posts, but this is obviously much faster than doing a visual inspection of all your posts.

Spam firewall

This is a new feature which as of this writing is not available if you download the extension from phpbb.com. It keeps almost all spammers from hitting your site at all. Instead, Cleantalk’s servers grab it first. In the event the user is legitimate, there is a link that will take them to your website.

Why is this useful? Because it reduces the stress on your server by limiting it to legitimate traffic only. It speeds up the performance of your forum and makes it less likely that you will have to pay for the cost of a higher class of hosting to handle your traffic. Isn’t that worth $8 a year?

Stopping contact form spam

Cleantalk has one other useful feature: the ability to stop contact form spam. Of course you can disable the contact form (ACP > General > Contact page settings) and that will solve that issue. Or you can have Cleantalk essentially moderate it for you, passing on only valid contact forms to you. Simply check that option on the extension’s page and submit the form. Somewhat oddly, the phpBB group did not tie the contact form to the spambot countermeasure feature of phpBB. Perhaps that will come in a future release.

In any event for forums that get lots of spam and/or lots of traffic, using the Cleantalk service with the Cleantalk extension for phpBB is a no-brainer providing you know about it. Now you do!

Fixing phpBB emailing problems

On: By: Mark

Note: this post was updated on April 21, 2023.

Having trouble sending out emails such as mass emails or email notifications using phpBB? If so, you are not alone. Sending mass emails is a feature of phpBB (ACP > System > General tasks > Mass email). Users can also sign up for various email notifications in the User Control Panel. For some boards though these email notifications just doesn’t seem to work reliably, and sometimes don’t work at all. Perhaps some but not all of these emails go out.

Even if you don’t send out mass email regularly, phpBB has a number of features that depend on outgoing email servers to send out emails in a timely manner to all intended recipients. It’s been my experience that solving these issues is hard. This is particularly hard on shared hosting where there are quotas on outgoing emails. It’s made more confusing by phpBB’s inelegant interface for dealing with emails.

phpBB hands off email, generally to your host’s email server. Generally you have no idea where this email server is, what its policies are and how to determine if the email server is actually sending the emails. It often requires a support ticket to your web host, who may or may not provide accurate information.

Sorting this out can be very confusing which is why you might want to hire me. In most cases though I will have to work with your web host to fix these problems. Here are some useful techniques that can often solve these issues.

Setting phpBB’s email settings correctly

You can find your forum’s email settings in the Administration Control Panel: ACP > General > Client communications > Email settings. Things to check:

  • Enable board-wide emails. This should be Yes. Obviously if set to No, no emails will go out.
  • SMTP settings > Use SMTP server for email.
    • This is normally set to No. PHP is configured to send emails through your web host’s email server. This makes emailing simple in that you don’t need to know anything about the configuration of the email server. However, sometimes this will bite you. Here’s a list of reasons why using SMTP instead might be better approach.
    • If you have Windows hosting, you usually have to set this to Yes. In addition you will have to configure the SMTP server information fields appropriately using information provided by your web host. Getting SMTP email configured correctly is often challenging, so if you are not sure what your SMTP settings should be, ask your web host. In addition, some web hosts don’t allow their servers to connect with external SMTP email servers. If you see in phpBB’s error log (ACP > Maintenance > Forum logs > Error log) messages like “Could not connect to smtp host : 110 : Connection timed out” this is probably because your host won’t allow external SMTP connections. If they provide an internal SMTP server, that is usually allowed.
  • Email package size. This is the maximum number of emails that will go out in one batch. So if you have 40 emails to send out and this is set to 20, the first 20 go out as a group if possible. What happens to the other 20? They go into a queue. The next time there is traffic on your forum (or the next time a system cron is triggered, if you are set up to use a system cron), phpBB will attempt to send out the next 20 emails in the queue. Set this to 0 and the queue goes away. There are some problems with this approach:
    • It’s slower because it takes time to send out an email to one user at a time, wait for an acknowledgement from the mail server, then send the next. This can cause a significant lag when you submit a post as email notifications go out one by one.
    • It’s possible that this will take so long that the process will time out, something more likely on shared hosting
    • If you hit some sort of outgoing email quota, not all emails will go out and you may not be aware there is a problem
  • Contact email address and return email address. These email addresses should exist as real mailboxes. Your web host control panel should have a feature allowing you to create email accounts. (You can also set up forwarders to send any email these accounts receive to one you will read.) In addition, it is highly advisable that these email addresses use your domain. Why? It lessens the likelihood that the email will be considered spam because:
    • It’s coming from an email address associated with the IP address of your domain
    • Because these email addresses actually exist it suggests the email is legitimate
  • Hide email addresses. If the same email is going to multiple recipients, the email addresses will be BCCed (Blind Carbon Copied) instead of placed on the TO field. Unless your host has a policy of not allowing BCCed emails to be sent, you should leave this set to Yes.

Verify your site name

ACP > General > Board configuration > Board settings. Verify that the Site name field is filled in. Some administrators will blank this out because they don’t want it to appear on their style, as they have substituted a logo for their site name. This is a mistake because outgoing emails, such as email notifications are not distinct. This causes them to look like spam because many emails use a same generic subject, a trigger for helping identifying spam emails. These emails will often be blocked from being sent at all, or placed in a spam folder if they are received by the recipient.

Verify your web host’s email policies

Since most phpBB forums are on shared servers, it’s important to know what limits there are if any for sending out emails. This can often be found buried in the host’s knowledge base, but sometimes it takes a support ticket to know exactly what policies apply. Based on this information you can tune phpBB’s emailing algorithm to match your web host’s policies. Policies often have constraints like:

  • Only X emails can be sent over Y minutes or hours and/or
  • A maximum of Z distinct email addresses can be sent over Y minutes or hours and/or
  • Email FROM addresses must be real email addresses associated with your domain and/or
  • Sending emails using BCC is/is not allowed

Setting the correct email package size

Based on knowing your host’s email policies you can figure out a correct email package size. For example, if a maximum of 25 emails can be sent every 5 minutes, a good email package size would be 25, although it might be better to set it a bit lower, perhaps to 20. If 25 are allowed and you set it to 40, then it’s possible 25 will go out and 15 will not.

Using a system cron

If your email package size is greater than 0, then phpBB will create a queue of emails if necessary. So if you have 25 to go out but specify 20 in a package, 20 will go out and 5 will go into a queue. When do the next five go out? By default they will go out the next time there is web traffic to your forum. (ACP traffic doesn’t count.) So if your board doesn’t get much traffic, it could be hours or days later before the next 5 emails go out. This is not good if your users expect timely email notifications.

A better way is to use a system cron. This helps send emails out on a timely basis plus it allows you to be congruent with your web host’s email policies. Creating a system cron is a bit tricky and may require someone with these skills to set it up. Of course you can hire me to do this for you.

To enable a system cron, go to ACP > General > Server configuration > Server settings. Set Run periodic tasks from system cron to Yes, then program the necessary cron job on your web server. This will disable phpBB’s default cron that depends on board traffic. Instead, depending on the interval you set with the cron job, a system cron built into your web server’s operating system will trigger periodic events that will process the email queue and other phpBB work.

In the above example, if we know the policy is not to permit more than 25 emails per five minutes, a cron job like this could be programmed to run every five minutes:

*/5 * * * * cd /path/to/board; php ./bin/phpbbcli.php cron:run

Using a pseudo-system cron

On shared hosting this often won’t work. This can be because:

  • You don’t know or can’t get the full path to your forum. A relative path won’t work. You need the full path from the server’s root folder. And what you see in tools like your web host control panel may show the full path, as the same machine is used by multiple clients.
  • Your cron tool won’t allow multiple commands. In the above example, the semicolon distinguishes one command from another.

You can still still program a cron however. In this case though you have the cron pretend to be a browser hitting your forum. Something like this might work instead. (Note in this case you should not need to set Run periodic tasks from system cron to Yes. This way board traffic might cause email notifications to go out sooner, but if not the cron job will simulate a human hitting the board, which will do the job.)

*/5 * * * * curl -A=Mozilla/4.0 http://www.yourforum.com/forum/cron.php?cron_type=cron.task.cron_task

However, you might want to change a phpBB setting called queue_interval. This is normally set to 60, for 60 seconds. This means that if there is traffic on your forum a “phpBB cron” won’t be attempted unless at least 60 seconds has elapsed since the last “phpBB cron”. The only case where you might want to change this is if you know your web host has an hourly or daily limit on outgoing emails.

Troubleshooting outgoing email issues

If there are errors sending out emails, phpBB may trap these in its Error log: ACP > Maintenance > Forum logs > Error log

Sometimes though phpBB cannot detect problems. You may find clues in your PHP error log, if one exists and you can read it. Sometimes you will see an error_log file in your phpBB root folder, or in the website’s root folder, or in a /log directory that can be found with the web host’s File Manager. Looking at it or downloading it for examination may show error messages that will point to the root of the problem.

In many cases though you can’t solve them. You will have to get your web host involved. They will have to look at email server logs to see if anything shows there and provide guidance.

Sometimes it helps to look at your email queue, which is the file /cache/production/queue.php. This is a file that contain emails that haven’t been sent yet. If it doesn’t exist, it means there is nothing in the queue. If the queue.php file is very large, it may be that more recent emails are waiting for processing.

The queue.php file is actually a PHP program created by phpBB which is created and maintained dynamically. The email messages are wrapped inside of PHP <?php and ?> tags. If you want, you can carefully use a tool like your web host control panel’s file manager to view the file  The overall size of the file will give you an idea of how big the queue is. The dates of the emails can be seen if you carefully look at the file in the file manager. It helps to look at the top and bottom of the file to get a sense of what’s in the queue. Each email header contains an ISO date string like this:

Date: Mon, 05 Aug 2023 01:35:08 +0000

In this case, it indicates the email was placed into the queue on August 5, 2023 at around 0135 hours GMT.

The queue is processed from top to bottom. The number of emails sent from the queue when a phpBB cron event occurs should not exceed your email package size. The emails sent are removed from the queue. If there are new emails, they are added to the bottom of the file.

Solving emailing issues

If there are no log entries in phpBB’s error log indicating emailing issues, as far as phpBB is concerned the email was successfully sent. However, this may be wrong. If emails are still not being received, there are two likely reasons:

  • Your email server is blocking them from being sent, because they are being flagged as spam. It may not be all outgoing emails, but only certain emails that look like spam that are blocked. Figuring this out usually requires a telephone call with your web host. If you can provide examples of emails that did not go out with the approximate date and time sent, this will help them figure it out.
  • The recipient’s email server is deciding they are spam. Typically they are placed in a spam folder, but it could be they are simply trashed too. Ask users who are complaining to look at their spam folder. Most email programs allow users to create rules to automatically move certain emails in the spam folder into their inbox instead.

Getting off blacklists

There are lots of blacklists out there. These are lists of suspicious email servers that are reportedly sending out spam. This tool can tell you if your domain is blacklisted. If it’s all green, perhaps there is an issue with your web host’s email server not sending them out and thus not triggering the reporting of spam to email blacklists.

If your domain is on one or more blacklists, you should try to get off them. It’s possible you are actually sending spam. For example, malware could have been installed on your server. If so it must be removed. Your web host may require you to scan your site for malware, or to assert there is none on it. They may then stop blocking outgoing email.

You can also appeal to each blacklist operator to remove your domain from its list.

A more effective, long-term strategy is to use Sender Policy Framework (SPF). This is basically a way that you assert “only these email servers are authorized to send email for my domain. Treat email coming from anywhere else as spam”. An SPF record is added to your domain’s DNS record. It looks something like this:

v=spf1 a mx ip4:69.64.153.131 include:_spf.google.com ~all

You can check your domain’s current SPF record here.

To change your record, you may need to ask your web host for the IP of the server your domain uses to send email. You also need to assert any other SMTP servers you use for sending email from the domain. For example, if you use Outlook to reply to email sent to admin@mydomain.com, the SMTP server configured in Outlook should be in this record too.

Setting a DKIM record

Another method provides a way to verify if the email was sent from your domain. This is done with a DomainKeys Identified Mail (DKIM) public key that is sent in the unseen email header. Public key encryption works with two sets of keys: a private key that is not shared, and a public one that anyone can access. Using the protocol, the receiving email server can verify the public key given in the email header with a separate query to your domain. Your domain essentially sends back replies of either “Yes, the public key is valid for this domain” or “No, this public key is invalid for this domain.” This does not necessarily provide assurance that the email is not spam, but it does indicate it was legitimately sent by your domain.

You can check to see if you have published SPF and DKIM records for your domain on the mxtoolbox.com website.

Most web host control panels have an easy-to-use interface for creating these public and private DKIM keys and for managing queries from receiving email servers. They are often created automatically. If you think about it, this approach likely can’t be used for sending emails from your domain with email clients like Microsoft Outlook, which is why a set of SPF records is highly desirable.

Update on dealing with forum spam

On: By: Mark

Some time back I wrote about how to deal with spam forum registrations and spam posts. Since I wrote these posts, spammers have changed tactics and phpBB has a number of new solutions to help address these issues. So a brief update:

  • With the release of phpBB 3.2 Rhea, the phpBB group has integrated Google’s new reCAPTCHA. The old one had been thoroughly hacked. You may have seen this one already on other websites. It asks if you are a human and gives you a checkbox to click on. This is generally all you have to do to “solve” the CAPTCHA. So if you are running phpBB 3.2 you may want to use this Spambot countermeasure as it is simpler than the Question & Answer countermeasure, previously the best solution if it was done right. Since the old reCAPTCHA was eventually defeated by spammers, I suspect this new version will have limited shelf life too, so if you use it keep an eye on it and if it starts failing use something else.
  • Extension developer RMcGirr83 has released a Stop Forum Spam extension. It works on both phpBB 3.1 and 3.2. It works by querying the stopforumspam.com database. This should catch the vast majority of spammers, but it may let a few slip through. If you allow guest posting, it can also be configured to check guest posts.

The Cleantalk service remains an option. It costs $8/year for a website and requires the installation of an extension or modification (depending on your version of phpBB) as well as getting a key from the website to enable it. I have had one client with an issue with it falsely identifying a user as a spammer. I worked with them to address it. Otherwise, my clients have noted no issues and recommend it highly.

 

Removing and preventing spam posts

On: By: Mark

Note: updated January 23, 2023

Note: my spam remover extension is now an approved extension. You may have to pay a fee to Akismet to use it. It can be used to find old spam in your board and remove it. 

Note: this post was updated on February 10, 2019 to bring it up to date.

Note: this post was edited on February 3, 2018 to keep it up to date, due to its popularity.

Note: This post was edited on July 22, 2018 to discuss tools for removing spam posts.

Back in 2015 I promised a subsequent post on removing spam posts from phpBB forums. Before talking about how likely spam posts can be removed let’s first talk about how to prevent them in the first place.

Preventing spam posts

You may want to adopt one or more of these strategies:

  • In 2018, the phpBB group approved the release of the Akismet anti-spam extension. This service uses the popular Akismet service, which is essentially a huge database of IPs and domains that have been reported to have sent out spam. Akismet is primarily used for comments on WordPress blogs, but was tailored by the extension developer to also work with phpBB. While the Akismet service can be free to use, it is not necessarily free. It is free for personal sites and blogs. If your forum is on your personal, noncommercial website, then presumably you can use it for free, although you are encouraged to donate anyhow. The extension though is free to install and use, as is true of all phpBB extensions. When properly enabled, the service will check new registrations and posts and will disallow them if they meet the spam threshold. Note that as of this writing it has no tool to go through existing registrations and posts to find and remove spam.
  • Similar to Akismet is the Cleantalk service. It’s arguably more affordable than Akismet, at least if you don’t qualify for Akismet’s free tier. You pay $8USD a year to subscribe to the service. You will have to download the Cleantalk extension for phpBB. (At this time an extension for 3.1 and 3.2 is available. However I recommend getting the latest version from GitHub, as it has features that may not appear on phpbb.com for months.) Install it, then configure it to check all posts for spam before allowing the post to be posted. As a bonus, it can check for spam in the contact form if that is enabled. There is no CAPTCHA built into the contact form. This is probably the most effective solution currently available. Note: the newer versions of this plugin also have a neat feature called Spam Firewall that can be enabled. It basically keep spambots from hitting your forum in the first place, saving you bandwidth and server resources.
  • Do not allow guests to post. Fortunately, phpBB comes configured this way by default. If you actually want guests to post:
    1. ACP > Users and groups > Group forum permissions
    2. Select the forums that you want guests to post in and submit the forum.
    3. Select the role for guests for each forum. Probably you want Limited access but may prefer to be more expansive with guests and give Standard access. Then click Apply all permissions.
  • Use the phpBB stop forum spam extension. This will check the IP of the poster against a popular known spammer’s database, but only this one list. It’s not foolproof, but it’s probably a 95% solution. Note that this extension works only for guest posts, so a registered user’s IP won’t be checked to see if their post contains spam. One advantage over Akismet and Cleantalk is it never costs any money to use this database. However, the process of checking the database can be slow.
  • Use moderators. Find active and trusted users to help moderate your forums. You can make them global moderators or give them permissions to moderate specific forums only. Moderators also need to learn phpBB’s moderation procedures. In most cases it takes a human to truly identify a spam post.
  • Encourage users to report spam posts. You might want to create an announcement to draw people’s attention to this feature of phpBB. It’s easy not to see it. For every post in the top right corner of the post there is a small button with an exclamation point (!) on it. The user can identify the reason for reporting the post, which can be it is spam. This will flag it for moderators or the administrator to review.
  • By default newly registered users to have their first three posts go through the moderation process before they can post. If you do not have moderators set up, then you as the administrator will have to review and approve these posts. Follow phpBB’s moderation procedures.

Use better registration procedures

If your board is clean of spam, upping your spambot countermeasures can help ensure that no spambots register. A spambot that succeeds in registering can create spam posts.

  • With the release of phpBB 3.2, phpBB can be integrated with the second generation version of reCaptcha. With phpBB 3.3, reCaptcha V3 is supported and should be used instead of reCaptcha V2 if possible. You need to go to the reCaptcha site, select the version of reCaptcha you want with the checkbox Captcha and generate a set of private and public keys for your domain (if you don’t have them already). Then configure the plugin: ACP > General > Board configuration > Spambot countermeasures. Look under Available plugins for reCaptcha and press Configure. Once the keys are entered you have to enable the plugin, which is done on the same page.
  • If you are using phpBB 3.1, the best out-of-box solution is to use the Q&A countermeasure. Make sure the question is not easily retrieved with an “I feel lucky” Google search.

Removing spam posts

The latest version of the Cleantalk extension has tools that can help identify and remove spam users by checking the IP they use with their database. If it matches, you have the option to remove their accounts and with it all their posts. It is possible but unlikely that it will give some false positives, in which case using this approach may delete a lot of legitimate posts. It requires subscribing to their service, which costs $8/year as this is written. For more information, see this blog post. There are some caveats:

  • If you have lots of users, it is likely you will get a timeout. 
  • The IP address database of spammers changes over time. So if you are trying to remove old spam accounts, it may miss them because the IP will no longer be in their database.

Consequently, my spam remover extension is a better option as it uses Akismet’s database, which appears to be a somewhat better database because spam is judged on factors other than the poster’s IP address.

Here are some other much more laborious means of identifying and removing spam posts:

  • An administrator or a forum moderator can manually remove any post he or she judges to be spam. Click on the little X icon in the top right corner of the post. If there is not much spam on your forum, this is generally the quickest approach.
  • If you allow guest to posts, a list of forums, topics and posts that have guest posts is useful. Administrators or moderators could then review these posts and delete them as needed. If you have phpMyAdmin, you can use it to run the following SQL to identify guest posts. (Select the forum’s database and then select the SQL tab.) Make sure you change phpbb_ as the table prefix if your config.php shows you have a different table prefix. The post text may look a little weird, as it is typically stored as HTML (phpBB 3.2) or in BBCode (previous versions), but it can be read.
SELECT f.forum_name, t.topic_title, p.post_subject, p.post_text
 FROM phpbb_forums f, phpbb_topics t, phpbb_posts p, phpbb_users u
 WHERE t.topic_id = p.topic_id and f.forum_id = t.forum_id AND p.poster_id = u.user_id and user_id = 1
 ORDER BY left_id ASC, t.topic_id DESC, post_id ASC
  • If older guest posts were valid but you notice a rash of spam guest posts after a certain time, you can see a list of posts on or after this time. In this example, January 1, 2016 is used.
SELECT f.forum_name, t.topic_title, p.post_subject, p.post_text
 FROM phpbb_forums f, phpbb_topics t, phpbb_posts p, phpbb_users u
 WHERE t.topic_id = p.topic_id and f.forum_id = t.forum_id AND p.poster_id = u.user_id and user_id = 1 AND p.post_time >= unix_timestamp('2016-01-01 00:00:00')
 ORDER BY left_id ASC, t.topic_id DESC, post_id ASC

The query will identify the forum, topic, post subject and post’s text. This query is ordered to present these posts in a way that is ordered the same way it usually is on the forum.

  • phpMyAdmin, which is generally available in your web host control panel, has an export capability. You could, for example, export this list as a comma separated values (CSV) value, import it into a spreadsheet like Excel and pass it out in a more human readable format to moderators for review. They will have to find these posts and delete them manually in phpBB.

Do not delete these using SQL, as you will mess up the topic post counts and possibly the number of topics in a forum count. Manually delete them on the view topic screen instead.

Fixing phpBB spam registration problems

On: By: Mark

(Note: since this post is frequently read, I updated it for phpBB 3.2 on June 28, 2017 and August 1, 2019)

I am frequently sought out to help address issues with spam registrations on phpBB forums. The most typical spam problem I encounter is not spam posts, but spam registrations. The symptoms are lots of bogus usernames in your member list or inactive user list, often strings of random letters and numbers for the username, with a “website” listed in their profile that points to a spam site.

I am happy to fix these for you. You may wish to try some of the solutions below first. The solutions discussed apply principally to phpBB 3.0, 3.1 and 3.2.

Getting rid of existing spam registrations

Most of these “users” don’t bother to complete registration, simply want to leave a spam memberlist link and be gone. Consequently they show up as inactive users.

  • Inactive users can be deleted in the Administration Control Panel. ACP > Users and Groups > Inactive Users. Select the inactive users you believe to be spam registrations and in the little drop down box in the bottom right of the screen select Delete and press Submit. You may have to do this many times for many screens to get rid of them all. (Note the Mark All link at the bottom of the page. This can speed up things.) This is the safest approach.
  • You can also use the Prune Users function: ACP > Users and Groups > Prune users. A user that has not completed registration is not necessarily a spammer. Unfortunately, the Prune User function is not smart enough to be able to examine the profile website field to see if it contains data as this is often the key to filtering out likely spammers. However, if you prune a legitimate inactive user they can always come back to the board and register again. To remove these users it helps to look at your memberlist by date and see if you can figure out when the spam started. In the Prune users utility you can enter this date in the Joined field. Entering 0000-00-00 in the Last Active field essentially is the same as filtering inactive users only. You can select the option to delete posts for these users too but if they haven’t completed registration there should be no posts to delete.
  • You can globally delete these on the backend with SQL but beware: you may also delete legitimate new users that haven’t finished the registration process. Running any raw SQL statement is inherently risky, so backup the database first! Make sure you select the right table prefix (phpbb_ is shown in the examples) for the users table. Your config.php file contains your table prefix, which is usually “phpbb_” as well as the database you are using. If you have more than one application this will distinguish the one containing the data for your forum. You can use a tool (typically phpMyAdmin in your web host control panel) to issue SQL to delete these. Here’s an example of SQL for MySQL that will remove all inactive users with zero posts but with a website URL:
    • For phpBB 3.1 and 3.2:
delete from phpbb_users
 where user_id in
 (select user_id from (select u.user_id from phpbb_users u, phpbb_profile_fields_data p
 where u.user_id = p.user_id
 and user_posts = 0
 and user_type = 1
 and pf_phpbb_website <> '') as u)
    • For phpBB 3.0:
delete from phpbb_users
 where user_id in
 (select user_id from (select user_id from phpbb_users
 where user_posts = 0
 and user_type = 1
 and user_website <> '') as u)

Reducing spam registrations

Since most spam comes in the form of registrations and not spam posts, generally tightening up the registration process can reduce or eliminate these registrations. Many admins require administrator approval for each registration, but this becomes labor intensive. Most CAPTCHAs have been thoroughly hacked and are to be avoided. Here are some better alternatives:

  • Tighten up your spambot countermeasure to something more likely to work, like the question and answer one.
    • ACP > General > Board configuration > Spambot countermeasures
    • Under Available plug ins, select Q&A and press Configure
      • Create a question that is unique to the focus of your forum and won’t be guessed with a Google search. Create as many questions as you want. All must be successfully answered to complete registration.
      • Go back to Spambot countermeasures, select Q&A again and press Submit. This will change the countermeasure.
    • If this stops the spam, it’s probably safe to change it so the admin doesn’t have to approve every registration. An email verification is a good approach. ACP > Board configuration > User registration settings > Account activation > By user (email verification)
    • As a best practice or whenever you start to notice spam registrations again or quarterly change the registration question(s)
  • phpBB 3.2 supports Google’s new version of reCaptcha as a spambot countermeasure.
    • First go to the reCaptcha site and generate the public and private keys you will need for your domain.
    • ACP > General > Board configuration > Spambot countermeasures
    • Under Available plug ins, reCaptcha and press Configure
    • Enter the public and private keys from the reCaptcha site into the fields and press Submit.
    • Go back to Spambot countermeasures, select reCaptcha again and press Submit. This will change the countermeasure.

Some sites get hammered and even these steps are not enough. If this happens to you try these options:

  • phpBB 3.2 has an Akismet Anti-spam extension. Using it is not necessary free, but usually is for personal or club forums. The extension doesn’t cost anything but the Akismet service requests money depending on your usage. The extension also checks posts for spam.
  • If you don’t mind paying $8USD a year, my customers report 100% success with Cleantalk. You will have to download the modification or extension, create an account on Cleantalk and enter your registration key in phpBB. It’s worth the time and expense.
  • phpBB 3.1 and 3.2: Install the Stop Forum Spam extension. This checks the IP and some other information of the user registering or the guest poster against popular blacklists, and if there is a match they cannot post. It’s not 100% perfect so some legitimate people may not get through and it’s also possible some spam will get through.
  • phpBB 3.1: Install and use the Sortable Captcha extension
  • phpBB 3.0: Install the Advanced Block Mod. Warning: this is a very complex mod to install and configure correctly. You will need to point it to more updated blacklists. You may want to have me install it professionally.

I will cover how to remove spam posts in a future blog post.