Ever since the extension system in 1.3 was announced, people have been asking questions about PunRes and future development of the site. I think it’s due time to announce some of the features and ideas I have for the next generation of PunBB’s mod community. Before PunRes was aimed at the sole mod developer writing the actual code. Now, when the extensions and styles have taken a more centralized role of the PunBB community, this is no longer the case.

PunRes will no longer be a forum where developer can get together and discuss how to hack up the PunBB code in different ways. This has always been a problem as the same discussions have been going on over at the official boards. PunRes will instead compensate for what the official board will not do, that is to host extensions and styles. Some degree of communication will be available, but it will be on the extension level, not on a global PunBB source level.

PunRes will make it easy for developers to organize their extensions and styles, and it will be just as easy for a board administrator to find the information they need. Here are some practical features that will most likely be in the next version of PunRes:

- Completely revamped interface with improved search and category implementation. This will also be accompanied with RSS listings for new releases, community news etc.

- Better recognition of well-written extensions. Every release will be moderated by trusted people in the community or otherwise labeled in some way that will tell the end-user that it is a safe extension to use. The moderation will take notice of security, compatibility (both database and other extensions), user-interface, coding standard and code effectiveness.

- The style previewer will be enhanced and styles will also be moderated before release to ensure end-user safety.

- Keep your board up-to-date by querying the PunRes database directly from your own board’s interface. Basically, you will have a list of your installed extensions and the interface will tell you whether or not there is a newer version released.

- You will be able to automatically post a release through the interface of PunXS (http://www.punxs.org/ for more info).

- Improved board statistics and removed public listings. An administrator will be able to use the graphs from PunRes on his/her own board, but PunRes won’t list most popular/active boards. This feature will simply exist as a tool for administrators to keep track of their board’s activity.

These are just some of the most significant changes, many good ideas posted in the PunRes feedback forum will also make it in. I’ve deliberately left anything related to the wiki out of here, as we’ve yet to see what road the official wiki will take.

I just exported, zipped and uploaded revision 1400 of the 1.3 tree to the download folder. That’s right, ladies and gentlemen, it’s time for bug crunching! Download PunBB 1.3 Beta.

Here are a few notes of interest to all potential beta testers:

1. This is a beta (and not in the Google sense of the word), so don’t expect everything to work perfectly. There are bugs. Don’t replace your production server install with this just yet.

2. The documentation is a bit lacking at the moment (which in turn is a bit of an understatement :D). An overhauled version of the 1.2 documentation is in the pipeline. Included in this, among other things, is a short manual aimed at extension developers.

3. Speaking of extensions, please dig into and tinker with the extension system. We appreciate any comments, thoughts or ideas you have on the subject. In particular, we are interested in requests for new hooks. We are aware that, even though the code currently contains a great number of hooks, there is need for more.

4. Another area of interest is the greatly improved update script. In PunBB 1.3, we are aiming for full UTF-8 compatibility. In order to accommodate this, the update script now does character set conversion to make sure as much as possible of existing content can be transferred over into valid UTF-8. This process is quite complex and the number of variations of characters sets out there is more or less infinite. Please put the update script through its paces. In order to realize proper UTF-8 support, we had to bump the MySQL requirement to 4.1.2. Hopefully, this shouldn’t affect too many of you.

5. As you might have realized, virtually every line of code and markup has been affected by the move to 1.3. What this means is that existing language packs are useless in 1.3. When the time comes for 1.3 final, it would be great to have a few additional languages available for download on the website. We would however appreciate if you could give us a few more days to tidy up some of the language files before you start. The new admin language file in particular, is a mess at the moment.

I think that about covers it for now. Please try to keep all discussions relating to the beta in the new 1.3 Beta talk.

Enjoy!

• Speed. Distributing a hotfix for a straight-up XSS or SQL injection vulnerability is a lot easier than packaging up a brand new release.
• Security. The faster we can fix something and the faster this fix can be applied to as many installs as possible, the better.
• Convenience. Installing a hotfix is ridiculously easy. PunBB tells you when there’s a new one and then you go to admin/extensions and click install.

Every once in a while, we will release a new version that incorporates the hotfixes that have been released for previous versions. The database update script will then automatically uninstall any hotfixes it supercedes.

Another cool thing is that since hotfixes are tied to a specific PunBB version, that means we can support an older version through hotfixes even when there are newer versions available. Someone might download PunBB 1.3.1, hack it to pieces and then be hesitant to upgrade to 1.3.3.

There’s still some work left on the extension system as a whole (more hooks as well as some minor additions to the manifest standard), but things are shaping up nicely.

An administrator of a PunBB forum clicks a link to an external page posted by a malicious user. On this external page, the malicious user has constructed a hidden form that automatically submits when anyone visits the page. The form submits to the profile.php script of the administrators PunBB install with the parameter update_group_membership, the user ID of the malicious user and the destination group - administrators. The administrator doesn’t even noticed that the form submits in the background and boom, the hacker has managed to escalate himself to administrator. He then logs in, deletes everything and posts his usual “h4cked by whatever” everywhere. Not good.

The above is one variation of what is commonly called a cross-site request forgery, or just CSRF (pronounced “sea surf”). The referrer check prevents this kind of attack because when the administrator visits the external page and the form gets submitted to profile.php, PunBB detects that it was submitted from an external page and spits out a warning/error message.

I have, over the years, received quite a few questions regarding the referrer check. Lots of people are uncertain about its efficiency considering the ease at which HTTP_REFERER can be spoofed. Spoofing of HTTP_REFERER is a non-issue though. A hacker can spoof his own HTTP_REFERER but we don’t care about that, we’re interested in the HTTP_REFERER of the forum administrator when he submits a form. This can’t, as far as I know, be modified externally.

Spoofing aside, the most common issue that users have with the referrer check is that for some users, it just won’t work properly. The reason for this usually is that they have some kind of “Internet security” software suite (I put that in quotes on purpose) installed that strips out HTTP_REFERER from all browser requests. Other users have encountered problems with proxies that strip out or alter HTTP_REFERER. The bottom line is that there are lots of ways in which the referrer check can fail, and we can’t have that.

In PunBB 1.3, the referrer check is no more. To replace it, we’ve implemented an anti-CSRF token system. Fancy words for something relatively simple. Here’s how it works.

All users browsing a PunBB 1.3 forum are assigned a temporary random SHA1 hash that we refer to as the CSRF token. The token is only valid for one session. As soon as the user times out or logs out, the token is destroyed. Now, armed with the token, whenever we present a form to an administrator or moderator, we include the token in a hidden field in that form. Here’s an example:

<form method="post" action="somescript.php">
    <input type="hidden" name="csrf_token" value="7fa7097b4dc1f3bc22895cd95e2183cbc165ece0" />
    the regular form fields...
</form>

When the form is submitted and somescript.php receives the form data, we compare the value of the hidden form field to the token we have stored for the user in question. If there’s a mismatch or if the token is missing, we display a message stating that there was a token mismatch.

This system prevents CSRF attacks as effectively as the referrer check, but it is much less prone to issues out of our control. The only real issue with the token system is that the token has a certain time to live. If an administrator or moderator navigates to a form in which the csrf_token field is included and then waits for an hour or so before he submits the form, he will be assigned a new token when the script that receives the form data loads and there will be a mismatch. It’s not the end of the world though. Hitting the back button, refreshing the page and then submitting again will solve the problem.

At this moment, the CSRF token system is only used for administrators and moderators, but in the future, we might enable its use for other user groups as well.

A number of basic schemes will be included with PunBB, these will be firstly simple file and folder stuctures such as
http://forums.punbb.org/forum-23.html or http://forums.punbb.org/forum/23/
for the 23rd forum, it will also include two similar schemes which will also incorporate text from the title of the forum or post into the URL (this is the SEF part), such as
http://forums.punbb.org/forum23-Feature-requests.html or http://forums.punbb.org/forum23/Feature-requests/

1.3 will also come with the functionality to add more schemes simply by dropping files into a folder in a similar way styles or languages are added now.

When creating the URL schemes a number of important considerations had to be taken, first of all as this is PunBB it had to be as fast and as simple as possible for the elements of a link to be converted to a URL, so this is done simply with a list of the different URL formats and how each one should be output. Another important consideration was how permenant a URL would be when admins have the option to switch schemes. This has lead to a “hybrid” .htaccess file being created which processes URLs from all the provided schemes, meaning even if a URL scheme is not currently in use, the URLs it generated will still work (assuming .htaccess still exists as it did).

To round this off, I think we’ve managed to produce a simple but effective way to scheme URLs, and have implemented it in such a way it won’t lead to dead ends (and inevitably poor search engine rankings) in the future, however since this isn’t simply PHP there are a lot more variations from setup to setup than we normally have to deal with, so here’s where you come in. If you are testing 1.3 it would be extremely useful if you could try out the schemes it currently has available, and report back if you have any issues. If you do have issues please give as much information as you can, apache version is critical as well as exactly what kind of error you are having (500 or 404 server errors etc.).

I talked about this in my previous post about the UTF-8 migration. However, in that post I discussed taking a database dump, running iconv on it and then re-importing it. With the new update script, that won’t be necessary unless the current encoding is non-ISO-8859-1 and the PHP environment lacks iconv and mbstring support. I believe this scenario will be relatively uncommon, particularly since any PHP5 environment should have iconv right out of the box. For ISO-8859-1, there’s no need for iconv or mbstring since we can use PHP’s built-in utf8_encode() function.

It would be great if you guys could test the new script. I’ve run it on a couple of semi-large database dumps and it appears to work just fine, but there’s only so many test cases I can come up with on my own. Do remember to make a backup of anything you attempt to convert.

My first entry on the blog will be a rather short one, but at least you see that I’m alive. ;)

I’ve comitted changeset 908 which adds the possibility to auto subscribe to topics you post in.

While I was adding this functionality I also modified how the checkbox for subscriptions works. In PunBB 1.2 the checkbox is not doing anything if you’re already subscribed to the topic, no matter if you check/uncheck the box, you stay subscribed. With this changeset for PunBB 1.3, the checkbox will become checked if you’re already subscribed, and if you decide that you no longer want to be subscribed while you’re making a post, just uncheck the checkbox and you will be unsubscribed when you post your reply.

Not that much, but at least something ;)

~Frank H

In keeping with the current stream of entries, this one is also about a new feature in PunBB 1.3: the improved password hashing implementation.

First though, it’s worth taking a look at how PunBB’s password hashing system currently works. PunBB 1.2 uses a function called pun_hash to decide which hashing algorithm to use. The function chooses SHA-1 if it is available: otherwise, it chooses MD5. The passwords stored in the database are simply hashes of the plaintext passwords: there is no salt added to the hashes. The values stored in cookies are MD5 hashes of password hashes from the database combined with the sitewide $cookie_seed. In case my explanation confused you, I’ve written a couple code examples below.

A password stored in the database is created by

pun_hash($plaintext_password)

and a cookie password hash is created by

md5($cookie_seed.pun_hash($plaintext_password))

1.2’s implementation is good, but it has several places it can be improved:

1. There is no salting done in the database, which means that any passwords grabbed directly from the database (via SQL inject, etc) are more vulnerable to attacks using rainbow tables.
2. Everyone’s cookie hash uses the same salt. Thus, a malicious user could take his/her cookie hash and his/her password (which he/she knows, obviously) and use brute force to find the value of $cookie_seed. Once the malicious user has that value, his/her brute force attacks just became simpler to do (although the malicious user would still need to steal another user’s cookie value via XSS or some other method).

So, how does 1.3 improve upon 1.2’s password security?

1. PunBB 1.3 requires SHA-1 hashing. The pun_hash function has been eliminated as a result. But have no fear, any passwords stored as MD5 are “upgraded” when the user logs in the first time.
2. The sitewide $cookie_seed has been abandoned in favor of a new feature I will talk about in just a few seconds. $cookie_seed added relatively little security overall and there was no real need to keep it hanging around.
3. All rows in the users table now have a new column, salt. That column stores a 12 character long salt which is randomly generated for each user. That’s right, PunBB now has per-user salts!

To go a little more in depth with #3, the salt is made up of 12 characters which have ASCII values between 33 and 126 (inclusive). That salt is used when storing the password in the database and when generating the cookie password hash (they are now exactly the same value). So, the code to generate a password hash now looks something like:

sha1($user_salt.sha1($plaintext_password))

where $user_salt is the randomly generated salt that each user has.

Some of the benefits of this new implementation:

• 2 users with the same plaintext password should not have the same hashed password (unless they randomly are given the same salt). In addition, a user who uses the same password on 2 user accounts will have 2 different password hashes (again, assuming the user does not randomly get identical salts).
• Because every password is salted and hashed in the database, it’s impossible for a malicious user to launch an attack against all passwords in the database at the same time. Each user’s password would have to be brute forced individually.
• There are only per-user salts, so a malicious user who brute forces his/her own hashed password will just get his/her per-user salt, which isn’t useful for breaking the passwords of others.

Of course, even with this wonderful salting, there is no substitute for a good, strong password. However, that’s a bit beyond the scope of this entry: if people are interested in learning some guidelines for secure passwords, they can read some of this topic.

~ Smartys

I hope my last post shone some light upon the upcoming extension system. I told you there where two major problems with today’s mods, and I addressed the first one. Now, it’s time for the second one.

Ever since mods where introduced by the community, and we agreed on the standard of the “ReadMe-file”, we’ve heard several complaints about what a pain they are to make. Not to mention when you where required to fix something, or God forbid, release a new version. That often led to rewriting the whole ReadMe-file. Being one of the first mod developers in the community myself, I was quick to realise that something needed to be done, and I created the “ModGenerator”. This was a web-based application that you could use to easily modify and create new ReadMe-files from your browser. It worked fine for a while, but due to the not so well written code and lack of time, the ModGenerator development was cancelled.

Some time passed and several other attempts to create a “ModGenerator” was made by various people. They mostly provided little help in the actual making of the mod, but made a nice tool to put your mod together in a ReadMe-file once you where done. Today, when we’re about the roll out the new extension system, the problem still remain, and it has probably grown even larger. You see, the extension, in it’s core, is made up from a somewhat more complicated XML file.

Rickard said to me once that we should think about creating something like the ModGenerator (which he was very fond of, *creds to me, yay*) for the extension system. I said I’d think about it, and I did, for quite some time. At first, I was focused on creating something web-based that would be located in the admin pages, allowing you to put your extension together, but I always got caught up in the swamp that is user-friendliness, and the balance between it and features. You don’t want it to be useless, but you also want it to be simple enough to be fast and easy to start working with. When this will aOf course, there will be an extension that lets you work with your extensions through PunBB’s admin panel. rrive, whether it’ll be here at 1.3 launch or not, we’ll see. This isn’t the main point of this post though. What I’m going to tell you is what happened to my ideas about how you would make life easier for the extension developers.

So, several months later, I began working with C# and I started thinking, what if you created a full blown application that would not only put your extensions together, it will help you write them as well. An environment that will help you organise your files and hooks, and provide powerful tools to speed up the development in general.

I present to you, PunXS!

What is it, you ask? Well, it’s all of the above. It’s written in C# so it’s based on .NET technology. It’ll be released under GPL, and there will be a working release of it before 1.3 comes out for everybody to start working on upgrading their mods to 1.3. It comes with a powerful hook-editor that let you code your hooks right in there with the rest of the PunBB code, and it finds the hooks automatically. This helps decreasing the need for new versions if hooks should be added or changed. Also, you can at any time, preview your extension in your local PunBB installation by a single click on your mouse.

- Ok, sounds great, is there a screenshot or something?

There’s a video:

Click here to view PunXS preview video

If you’re a mod developer, or a to-be extension developer, please take some time and try to come up with some features that you think will be useful. Not all features have been announced yet, so chances are they are already in there or on the todo-list, but that matters not, the more ideas the better!

Official PunXS Site

Kristoffer here, or Jansson as I’m also called. If you haven’t heard of me, I’m the guy behind PunRes. I’ve been a user of PunBB since sometime in beta, and a developer since November. What’s always been fascinating to me about PunBB is that the code is so simple and structured that anyone with even the smallest PHP knowledge are able to modify it to their own needs. This is very important to PunBB as it ships with a rather slim set of features.

It is almost impossible to make software that appeals to everybody. The reason for this is that everybody’s needs are different. However, there is a solution, and it’s quite simple: Let the users decide their own set of features! As I write this, there are 269 projects/modifications at PunRes and new ones are created at a regular basis. All the common features are there, private messaging, polls, attachments, gallery, wiki, you name it. There are several reasons modding has become such a widespread thing in the PunBB community. Firstly, because the code is kept simple enough for most people to understand it, we almost get this for free by not implementing huge and complex systems in the main package. Secondly, modding has been around since the release of PunBB 1.0, and even before. Thirdly, the availability. PunRes has always been there to keep all mods in one place for the users to browse and discuss modding.

There is only one downside with mods, and it’s huge. Actually, there’s two, but let’s concentrate on one of them. It relates to the way in which mods are installed. Installing a mod requires the user to modify the source manually. This can be a bit complicated for the inexperienced user, and there are often mistakes. Another problem is when updates to PunBB are released, you either have to install all the mods again manually, or apply the PunBB update manually, either way it can be very annoying. This also indirectly leads to less security because some people don’t bother updating at all. Wouldn’t it be awesome if you could just install a mod by a single click on your mouse?

Enter PunBB 1.3.

With 1.3, there will be Extensions. Extensions are not mods, though sometimes referred to as “the mods of 1.3″. Mods and extensions are completely different things created to solve the same problem; let the user decide. The extension, instead of editing the source directly, hooks into the code at specific locations causing PunBB to behave different. The source of the extension is stored separately in the database and can therefore be put in and be taken out at any given time without anyone touching a single line of PHP code. I will not go into any technical details at the time but focus on the advantage of this new system. Apart from the obvious ease of installation, it also makes updating PunBB as easy as cake.

Mods have never been officially supported, and Extensions in themselves won’t be either. However, now when the developer team consists of more than one guy we will be able to create “official extensions” that are sure to always be compatible with the latest version, plus they will come with the same exceptional quality as the rest of PunBB. It will also allow us to update the extensions without having to release a new version of PunBB each time, making the users that doesn’t use the feature less annoyed. Mods will probably continue to exist as they do today, though they just won’t be as many. PunRes will be focusing on extensions and mods will be shared primarily through the wiki.

What about the Plug-ins?

Yes, the plug-ins will also be replaced by the extension system as it will be possible to acquire the same functionality through extensions.

As I said above, there are two problems with today’s mods, the first one is the user’s problem, and the second one is the developer’s. Feel free to speculate about what it is, it shouldn’t be too hard to guess. There is a solution to the problem in the making, but I won’t reveal that just yet, it has been hinted about on the forums though :-)

Take care,
Kristoffer