Tag: spam posts

Cleantalk extension for phpBB can remove spam posts, plus its spam firewall feature is very useful

On: By: Mark

This is an update on an earlier post on removing spam posts.

Removing spam posts is hard because it requires actually reading the post and deciding if the post is spam or not and then using moderator tools to remove these posts. If your forum is overwhelmed with spam posts, this is a Herculean endeavor. Ideally though posts could be “read” by software and it would make the judgment on whether it is spam or not.

The Cleantalk extension for phpBB 3.1.x and 3.2.x can do just this as well as lots of other really cool tricks. My customers love Cleantalk, but the service is not free. However, it is so inexpensive that it easily justifies spending $8/year for the service. You can subscribe on the Cleantalk website. As of this writing, you can try it for free for 7 days. After 7 days, it won’t bring down your forum but it will stop working.

What is Cleantalk?

Cleantalk is essentially a huge database of addresses of known spammer sites. While it’s not perfect, based on the experience of my clients it is about 99% perfect. I originally recommended it as a spam registration solution for my clients. It still does that but is less necessary since phpBB 3.2. This is because since phpBB 3.2, version 2 of Google’s reCaptcha is supported. Unless it gets hacked, as long as you have it properly configured as a spambot countermeasure it should prevent virtually all spam registrations.

However, it has two powerful features that still keep it relevant for phpBB forums.

Cleantalk ACP Interface
Cleantalk ACP Interface

Installing and enabling Cleantalk

Cleantalk is installed like any other extension. While it can be downloaded from phpbb.com, you should download it from its GitHub page. This is because as of this writing the version on phpbb.com does not include the spam firewall feature, and you will probably want to enable this feature. You can access it through the Administration Control Panel: ACP > Extensions > Antispam by Cleantalk. Before you can do much with it you have to enter your Cleantalk key which you can get from their website or by pressing the button in the extension that should retrieve it for you.

Removing spam users and spam posts

As you can see from the image, once the extension is enabled and the key is properly configured there is a prominent Check users for spam button on its page within the Administration Control Panel. If you have lots of users, it may hang. Based on my experience though the next time you go into its interface you will see a list of potential spammers.

As I said, it is not perfect. So I recommend that for users with posts to check these out these users topics to make sure their posts are spam before deleting them. For those you want to delete, check the boxes next to their usernames and then press Delete marked. You can also press Delete all to remove all users and their posts. You may have to go through many pages to delete all spam users and their posts, but this is obviously much faster than doing a visual inspection of all your posts.

Spam firewall

This is a new feature which as of this writing is not available if you download the extension from phpbb.com. It keeps almost all spammers from hitting your site at all. Instead, Cleantalk’s servers grab it first. In the event the user is legitimate, there is a link that will take them to your website.

Why is this useful? Because it reduces the stress on your server by limiting it to legitimate traffic only. It speeds up the performance of your forum and makes it less likely that you will have to pay for the cost of a higher class of hosting to handle your traffic. Isn’t that worth $8 a year?

Stopping contact form spam

Cleantalk has one other useful feature: the ability to stop contact form spam. Of course you can disable the contact form (ACP > General > Contact page settings) and that will solve that issue. Or you can have Cleantalk essentially moderate it for you, passing on only valid contact forms to you. Simply check that option on the extension’s page and submit the form. Somewhat oddly, the phpBB group did not tie the contact form to the spambot countermeasure feature of phpBB. Perhaps that will come in a future release.

In any event for forums that get lots of spam and/or lots of traffic, using the Cleantalk service with the Cleantalk extension for phpBB is a no-brainer providing you know about it. Now you do!

Removing and preventing spam posts

On: By: Mark

Note: updated January 23, 2023

Note: my spam remover extension is now an approved extension. You may have to pay a fee to Akismet to use it. It can be used to find old spam in your board and remove it. 

Note: this post was updated on February 10, 2019 to bring it up to date.

Note: this post was edited on February 3, 2018 to keep it up to date, due to its popularity.

Note: This post was edited on July 22, 2018 to discuss tools for removing spam posts.

Back in 2015 I promised a subsequent post on removing spam posts from phpBB forums. Before talking about how likely spam posts can be removed let’s first talk about how to prevent them in the first place.

Preventing spam posts

You may want to adopt one or more of these strategies:

  • In 2018, the phpBB group approved the release of the Akismet anti-spam extension. This service uses the popular Akismet service, which is essentially a huge database of IPs and domains that have been reported to have sent out spam. Akismet is primarily used for comments on WordPress blogs, but was tailored by the extension developer to also work with phpBB. While the Akismet service can be free to use, it is not necessarily free. It is free for personal sites and blogs. If your forum is on your personal, noncommercial website, then presumably you can use it for free, although you are encouraged to donate anyhow. The extension though is free to install and use, as is true of all phpBB extensions. When properly enabled, the service will check new registrations and posts and will disallow them if they meet the spam threshold. Note that as of this writing it has no tool to go through existing registrations and posts to find and remove spam.
  • Similar to Akismet is the Cleantalk service. It’s arguably more affordable than Akismet, at least if you don’t qualify for Akismet’s free tier. You pay $8USD a year to subscribe to the service. You will have to download the Cleantalk extension for phpBB. (At this time an extension for 3.1 and 3.2 is available. However I recommend getting the latest version from GitHub, as it has features that may not appear on phpbb.com for months.) Install it, then configure it to check all posts for spam before allowing the post to be posted. As a bonus, it can check for spam in the contact form if that is enabled. There is no CAPTCHA built into the contact form. This is probably the most effective solution currently available. Note: the newer versions of this plugin also have a neat feature called Spam Firewall that can be enabled. It basically keep spambots from hitting your forum in the first place, saving you bandwidth and server resources.
  • Do not allow guests to post. Fortunately, phpBB comes configured this way by default. If you actually want guests to post:
    1. ACP > Users and groups > Group forum permissions
    2. Select the forums that you want guests to post in and submit the forum.
    3. Select the role for guests for each forum. Probably you want Limited access but may prefer to be more expansive with guests and give Standard access. Then click Apply all permissions.
  • Use the phpBB stop forum spam extension. This will check the IP of the poster against a popular known spammer’s database, but only this one list. It’s not foolproof, but it’s probably a 95% solution. Note that this extension works only for guest posts, so a registered user’s IP won’t be checked to see if their post contains spam. One advantage over Akismet and Cleantalk is it never costs any money to use this database. However, the process of checking the database can be slow.
  • Use moderators. Find active and trusted users to help moderate your forums. You can make them global moderators or give them permissions to moderate specific forums only. Moderators also need to learn phpBB’s moderation procedures. In most cases it takes a human to truly identify a spam post.
  • Encourage users to report spam posts. You might want to create an announcement to draw people’s attention to this feature of phpBB. It’s easy not to see it. For every post in the top right corner of the post there is a small button with an exclamation point (!) on it. The user can identify the reason for reporting the post, which can be it is spam. This will flag it for moderators or the administrator to review.
  • By default newly registered users to have their first three posts go through the moderation process before they can post. If you do not have moderators set up, then you as the administrator will have to review and approve these posts. Follow phpBB’s moderation procedures.

Use better registration procedures

If your board is clean of spam, upping your spambot countermeasures can help ensure that no spambots register. A spambot that succeeds in registering can create spam posts.

  • With the release of phpBB 3.2, phpBB can be integrated with the second generation version of reCaptcha. With phpBB 3.3, reCaptcha V3 is supported and should be used instead of reCaptcha V2 if possible. You need to go to the reCaptcha site, select the version of reCaptcha you want with the checkbox Captcha and generate a set of private and public keys for your domain (if you don’t have them already). Then configure the plugin: ACP > General > Board configuration > Spambot countermeasures. Look under Available plugins for reCaptcha and press Configure. Once the keys are entered you have to enable the plugin, which is done on the same page.
  • If you are using phpBB 3.1, the best out-of-box solution is to use the Q&A countermeasure. Make sure the question is not easily retrieved with an “I feel lucky” Google search.

Removing spam posts

The latest version of the Cleantalk extension has tools that can help identify and remove spam users by checking the IP they use with their database. If it matches, you have the option to remove their accounts and with it all their posts. It is possible but unlikely that it will give some false positives, in which case using this approach may delete a lot of legitimate posts. It requires subscribing to their service, which costs $8/year as this is written. For more information, see this blog post. There are some caveats:

  • If you have lots of users, it is likely you will get a timeout. 
  • The IP address database of spammers changes over time. So if you are trying to remove old spam accounts, it may miss them because the IP will no longer be in their database.

Consequently, my spam remover extension is a better option as it uses Akismet’s database, which appears to be a somewhat better database because spam is judged on factors other than the poster’s IP address.

Here are some other much more laborious means of identifying and removing spam posts:

  • An administrator or a forum moderator can manually remove any post he or she judges to be spam. Click on the little X icon in the top right corner of the post. If there is not much spam on your forum, this is generally the quickest approach.
  • If you allow guest to posts, a list of forums, topics and posts that have guest posts is useful. Administrators or moderators could then review these posts and delete them as needed. If you have phpMyAdmin, you can use it to run the following SQL to identify guest posts. (Select the forum’s database and then select the SQL tab.) Make sure you change phpbb_ as the table prefix if your config.php shows you have a different table prefix. The post text may look a little weird, as it is typically stored as HTML (phpBB 3.2) or in BBCode (previous versions), but it can be read.
SELECT f.forum_name, t.topic_title, p.post_subject, p.post_text
 FROM phpbb_forums f, phpbb_topics t, phpbb_posts p, phpbb_users u
 WHERE t.topic_id = p.topic_id and f.forum_id = t.forum_id AND p.poster_id = u.user_id and user_id = 1
 ORDER BY left_id ASC, t.topic_id DESC, post_id ASC
  • If older guest posts were valid but you notice a rash of spam guest posts after a certain time, you can see a list of posts on or after this time. In this example, January 1, 2016 is used.
SELECT f.forum_name, t.topic_title, p.post_subject, p.post_text
 FROM phpbb_forums f, phpbb_topics t, phpbb_posts p, phpbb_users u
 WHERE t.topic_id = p.topic_id and f.forum_id = t.forum_id AND p.poster_id = u.user_id and user_id = 1 AND p.post_time >= unix_timestamp('2016-01-01 00:00:00')
 ORDER BY left_id ASC, t.topic_id DESC, post_id ASC

The query will identify the forum, topic, post subject and post’s text. This query is ordered to present these posts in a way that is ordered the same way it usually is on the forum.

  • phpMyAdmin, which is generally available in your web host control panel, has an export capability. You could, for example, export this list as a comma separated values (CSV) value, import it into a spreadsheet like Excel and pass it out in a more human readable format to moderators for review. They will have to find these posts and delete them manually in phpBB.

Do not delete these using SQL, as you will mess up the topic post counts and possibly the number of topics in a forum count. Manually delete them on the view topic screen instead.