Hi,
I hope my last post shone some light upon the upcoming extension system. I told you there where two major problems with today’s mods, and I addressed the first one. Now, it’s time for the second one.
Ever since mods where introduced by the community, and we agreed on the standard of the “ReadMe-file”, we’ve heard several complaints about what a pain they are to make. Not to mention when you where required to fix something, or God forbid, release a new version. That often led to rewriting the whole ReadMe-file. Being one of the first mod developers in the community myself, I was quick to realise that something needed to be done, and I created the “ModGenerator”. This was a web-based application that you could use to easily modify and create new ReadMe-files from your browser. It worked fine for a while, but due to the not so well written code and lack of time, the ModGenerator development was cancelled.
Some time passed and several other attempts to create a “ModGenerator” was made by various people. They mostly provided little help in the actual making of the mod, but made a nice tool to put your mod together in a ReadMe-file once you where done. Today, when we’re about the roll out the new extension system, the problem still remain, and it has probably grown even larger. You see, the extension, in it’s core, is made up from a somewhat more complicated XML file.
Rickard said to me once that we should think about creating something like the ModGenerator (which he was very fond of, *creds to me, yay*) for the extension system. I said I’d think about it, and I did, for quite some time. At first, I was focused on creating something web-based that would be located in the admin pages, allowing you to put your extension together, but I always got caught up in the swamp that is user-friendliness, and the balance between it and features. You don’t want it to be useless, but you also want it to be simple enough to be fast and easy to start working with. When this will aOf course, there will be an extension that lets you work with your extensions through PunBB’s admin panel. rrive, whether it’ll be here at 1.3 launch or not, we’ll see. This isn’t the main point of this post though. What I’m going to tell you is what happened to my ideas about how you would make life easier for the extension developers.
So, several months later, I began working with C# and I started thinking, what if you created a full blown application that would not only put your extensions together, it will help you write them as well. An environment that will help you organise your files and hooks, and provide powerful tools to speed up the development in general.
I present to you, PunXS!
What is it, you ask? Well, it’s all of the above. It’s written in C# so it’s based on .NET technology. It’ll be released under GPL, and there will be a working release of it before 1.3 comes out for everybody to start working on upgrading their mods to 1.3. It comes with a powerful hook-editor that let you code your hooks right in there with the rest of the PunBB code, and it finds the hooks automatically. This helps decreasing the need for new versions if hooks should be added or changed. Also, you can at any time, preview your extension in your local PunBB installation by a single click on your mouse.
- Ok, sounds great, is there a screenshot or something?
There’s a video:
Click here to view PunXS preview video
If you’re a mod developer, or a to-be extension developer, please take some time and try to come up with some features that you think will be useful. Not all features have been announced yet, so chances are they are already in there or on the todo-list, but that matters not, the more ideas the better!
WTF??? WINDOWS. I WAS EXPECTING MAC PROGRAMS NOT CRUDDY WINDOWS.
February 21st, 2007, at 8:03 pm #I was expecting something else. A mac program, perhaps.
February 21st, 2007, at 8:04 pm #That looks pretty interesting, will certainly make it much easyer to build extentions!
February 21st, 2007, at 8:05 pm #Sounds awesome for all the mod developers out there.
Since it’s based on .net technology do you have any plans to do a native linux version?
February 21st, 2007, at 8:13 pm #That looks very interesting…
February 21st, 2007, at 9:03 pm #I’ll be even more interested if this becomes available for Mac OS X. ;)
February 21st, 2007, at 9:41 pm #looks good m8. are you using sharp develop’s text editor in there?
C# is how I make my living and have been using it since it came out, so if you need a hand feel free to hit me up.
February 21st, 2007, at 10:23 pm #Ok… And what about linux users? :/
February 21st, 2007, at 10:30 pm #huh ! Scintilla like always !
February 21st, 2007, at 10:53 pm #Wow, looks great, but a mac app would be even better!
February 21st, 2007, at 11:04 pm #Jansson: what did you use to cap that vid? i’m always on the lookout for cool apps to do such things.
February 22nd, 2007, at 7:45 am #I hope Linux users will be able to run it with Mono. But I would prefere less heavyweight solution.
February 22nd, 2007, at 8:16 am #have none of you linux users ever heard of wine? I’m sure CrossOver will run it just fine if you’re unable to configure the standard wine to do by yourselves.
February 22nd, 2007, at 9:03 am #Uhm, while the Windows application looks great, don’t you think this is a bit overly complicated? I don’t want to purport WordPress as having the best solution for everything, because it’s basically a mess source-code wise, but its extension mechanism is so much cleaner than this. Instead of having an horrific mess of XML file configuration, all configuration needed is done within the extension’s PHP file(s) itself.
Hooks are basically events you can add an event handler to, with a priority added to adjust when the event is supposed to occur. A much better way to do this is of course to have a base PHP class that you can inherit and then override functions to override event handlers and in the constructor of the class add event handlers if you want to. That way, all the code is kept inside of PHP and it’s very easy to administer and develop on. Having configuration files as XML is all in all a terrible idea, since the PHP syntax itself works fine for configuration. Just use self-explanatory variable names or hash table keys and it’s easier to use than XML and much easier to parse and manage for PunBB.
I don’t want this to come out as a total “your idea sucks” post, because that’s not what I mean, I just think an extension system needs to be simpler, without the requirement of a complete IDE installed to be able to program it. Once an extension system requires more than a text editor, I’d say it’s too complex.
Oh and to the “Windows-only” discussion going on here, I’m sure the PunBB developers are able to get this to run under Mono in Linux and on OS X as well. But if they listen to my advice, they’ll create an extension system that doesn’t require anything like it at all.
I’d be more than happy to prototype something like this if you want to. Just drop me a line on my registered e-mail address.
February 22nd, 2007, at 12:43 pm #Ok, some clarification as some people seems to overlook things in the post.
PunXS will not be a requirement to write an extension. You can do it manually, through a normal text-editor, and there will be an extension that let’s you develop other extensions (works on all platforms). PunXS is merely intended to speed up the process.
I really want to go cross-platform, but right now, I’m focusing on getting the extension system finished, and developing PunXS simultaneously.
Let’s not make this into some win, mac and linux war. Cross-platform will come, though perhaps not in 1.0.
Now, let’s get back to coding :)
February 22nd, 2007, at 3:12 pm #What, no Amiga port ;)
February 22nd, 2007, at 3:55 pm #Asbjørn: I’m not sure where you get the idea that the XML based manifest is all that complicated. Here’s an example manifest:
http://punbb.org/stuff/manifest.xml.txt
I can’t see how describing that data and that code would be any simpler if it was done in PHP code. Regardless of whether it’s PHP or XML, if you’re going to do it by hand, you’re gonna need a “template” manifest. Also, the XML file is just a means of distribution. It’s parsed once.
I’d love to see a prototype of an alternative to this though. I’m not saying what we have now is the best solution period.
Doing a class based system is an alternative, but it would require a massive rewrite and thus, just isn’t an option.
February 22nd, 2007, at 3:57 pm #It looks very nice, and will definately be a great gift to the developers using Windows.
February 22nd, 2007, at 9:29 pm #Yes, Kristoffer, you should definitely finish the extension system first. :P
Also, I’m actually not too concerned about a cross-platform app, now that I think about it. I’ll probably just make some shell scripts (or AppleScripts, or whatever) to help me, while still using my current text editor.
Rickard, thanks for the brief example; It’s looks just how I expected it to.
February 22nd, 2007, at 11:19 pm #It’s not that the XML format in question is so complicated or that XML itself is so complicated, it’s just that it’s completely unecessary and causes an uneeded extra level of indirection between PunBB and the extension itself. If the extension can be coded in PHP and only PHP, it should be coded in PHP and only PHP.
Can you please just check out how WordPress’ extension mechanism works before you start paving the way with this overly complicated process? As I’ve mentioned, WordPress doesn’t do everything right and its code base is nowhere near perfect, but it has gotten some principles and architectural stuff right, and that’s worth noting and taking as an example. It’s after all an extremely well-known blogging tool that could, if PunBB had the same kind of extension system, make it a lot easier to start extension-programming for PunBB and level the existing WordPress competency.
February 23rd, 2007, at 7:53 am #If i’ve understood well PunXS is an editor to modify punBB source code and produce their extensions direclty in the PunBB source code.
But Is PunXS able to produce (by an export function or something similar) a complet package of the extension we’ve just wrote ?
- if so its great !
Or is it just a dedicated punBB editor ?
- if so, i dont see the advantage of this editor except for punbb1.2.x :(
regards.
February 23rd, 2007, at 3:35 pm #The extension system seems really simple to me. Maybe a class based system could be more efficient (I wouldn’t know), but I don’t particularly care at the moment; I’d like to see PunBB 1.3 sometime this year. ;)
February 23rd, 2007, at 5:37 pm #A mac program would be highly appreciated. thank you
February 24th, 2007, at 9:24 am #Bravo! Will make it a lot easier for me to upgrade my mods, and hopefully encourage others to build great extensions.
February 24th, 2007, at 9:43 am #Holy cow! I’m not even a developer and I can readily appreciate how cool that is.
Almost makes me want to learn to program just so I can use it! ;)
BTW, great screencast / screencap!
February 24th, 2007, at 9:29 pm #I just made finished a simple AppleScript Studio app (Mac OS X thing) that pulls out extension code (after creating some with my text editor of choice) and dumps it into a manifest file, along with the other properties (author, etc.). :D
Next step is making it put the code back into the PunBB files (so I can edit/preview it there). I’ll try to share it with other Mac users once it (and PunBB) are further along. :)
February 25th, 2007, at 6:05 am #Darn late night typos. s/are/is
February 25th, 2007, at 6:42 am #I’ve sketched up a very simple example of how a class-based Extension mechanism can work. See the “My Extension” implementation here: http://ulsberg.no/asbjorn/stuff/myextension.php.txt
And the “Extension” class definition here: http://ulsberg.no/asbjorn/stuff/extension.php.txt
You can also see the result of MyExtension’s execution here: http://ulsberg.no/asbjorn/stuff/myextension.php
The point I want to make is that you can easilly drop the XML file and still have the same loose coupling between PunBB and the extension, but with a lot less overhead. All PunBB would need to do is to create an instance of the MyExtension class (based on its knowledge of the Extension class which is defined) and then call the hook methods whenever they are meant to be called.
Implementing an extension can’t be much simpler than this, imo. You don’t need to know much about object orientation or anything, you just need to extend the Extension class and then define some properties (which ones that should be required is up to you, I made all of them required), an install() method, an uninstall() method and then all of the hooks they want.
February 26th, 2007, at 10:31 am #All I see is this…: function fn_navlinks()
…instead of this:
How would your extension be called from within PunBB?
February 26th, 2007, at 6:36 pm #I think WordPress ate my XML… I said: <hook id=”fn_navlinks”>
February 26th, 2007, at 6:37 pm #What do you mean? What don’t you understand? Can you please be a bit more elaborate? :-) The extension can be called from PunBB by putting it into an “extensions” folder. Then, in PunBB’s “Extensions Administration” interface, one can enable the extension and by doing that, PunBB will add the filename of the extension and its class name to a list of extensions to initialize on startup. On startup, it then goes through this list and initializes every extension in it and with the method_exists() function, inspects the extension classes to see if they contain methods that should be executed at given events (or so-called “hooks”).
February 27th, 2007, at 9:11 am #That looks horribly messy: you’d have to create a function for every hook. And every hook can only be used once too.
Unless I’m misinterpreting it ofcourse :)
February 28th, 2007, at 3:21 pm #Would you have to declare global variables in those functions?
February 28th, 2007, at 7:25 pm #guardian: yes.
February 28th, 2007, at 7:54 pm #What’s different about writing ‘function hook_name()’ and the XML equivalent? I mean, the function body (the code) would be the same, except you don’t have to write code embedded in an XML file, you can instead just write plain old PHP.
I don’t follow you when you think it’s messy. I also don’t follow you when you say that each hook can only be defined once. Sure it can only be defined once in each class (each extension), but what’s different in that from what we have now? If you want to execute several functions on a given hook, you can easilly do so within your PHP code by calling functions from the hook (or “event handler” as is common to call such things in event-driven OO stuff).
To access variables outside the class scope, those variables needs to be declared as global, yes.
March 1st, 2007, at 2:49 pm #Who said you *have* to type it in an XML file? You can write code in the PunBB files themselves, and then cut/paste into an XML file.
March 1st, 2007, at 8:09 pm #I must’ve misunderstood your code then.
I’m talking about this part:
/*
* Executes the instance of the class’ fn_navlinks() method if it exists.
*/
private function ex_fn_navlinks() {
if ($this->validate(’method’, ‘fn_navlinks’, false)) {
$this->fn_navlinks();
}
} // function ex_fn_navlinks
From what I understand of it, you’d need one of those functions for each hook in PunBB. Which would be painful.
March 1st, 2007, at 9:25 pm #Unless I’m wrong, feel free to point that out then :)
I was reading it that way, too.
March 2nd, 2007, at 1:21 am #Would there be any significant speed difference between this and the current hook cache?
March 2nd, 2007, at 1:56 am #Maybe a little, due to the current system using eval().
March 2nd, 2007, at 7:29 am #Uhm, why are you hung up in what’s going on inside the extension.php file? That’s not the class the extension author is supposed to write, that’s what PunBB developers write (or me, if they’d like help). If you want to loop through an array of hook names and check to see if they exist in the extended class, that’s possible too. You don’t have to hard-code each and every hook like I’ve done; it’s only one example of many ways to do it.
My point is that the XML file is completely unnecessary and that it will be faster and easier to develop this as a pure PHP script. It will also be a lot faster to execute since you don’t have to parse the XML files in PunBB and then eval() the code. You can also easier protect yourself against security risks that are often imposed by the eval() function.
Just to show how such a loop can be implemented, I’ve uploaded a new version of the extension (the file that would ship with PunBB):
http://ulsberg.no/asbjorn/stuff/extension.php.txt
And just to be clear, the extension.php file is not something extension developers should touch. It’s a file they include/require in their PHP file so that they can extend it with PHP’s object oriented stuff. The file an extension developer would create is this:
http://ulsberg.no/asbjorn/stuff/myextension.php.txt
Now, having those hooks inside an array doesn’t make much sense, since they would be called at different places in the code and not in a loop like this. The whole point is that a hook for e.g. “navlinks” is supposed to be called when the navlinks are to be rendered. But I’ve created code here that shows how it can be done in a less hard-coded way just to show that it’s possible.
March 2nd, 2007, at 9:14 am #Sorry for not replying to any of your posts until now Asbjørn. I’ve got the flu and my head isn’t working exactly the way I want it just yet. I’ll blame the flu if what I’m about to write doesn’t make any sense :)
Nice to see you’ve posted some example code. I have a few comments and questions.
First off, I get the impression you’ve misunderstood the current implementation somewhat. The XML manifest is only parsed once and that’s when you install or uninstall the extension. The code snippets are stored in the database (and cached to a PHP script). Therefore, claiming your suggestion would be faster because the current implementation has to parse XML doesn’t make sense. It should also be noted that eval() is called only once per hook, regardless of how many extensions implement that hook. In reality, I don’t believe there is any real performance advantage to either solution considering that your solution would require parsing one additional PHP script per extension and creating one new object per extension (every pageview).
Now, about your suggestion. I can see the benefit of being able to edit the extension code directly in a PHP script. It definitely has its advantages. It also has some disadvantages though. What I like about the current implementation is that the code you write in your extension will be executed where the hook is called. Essentially, it’s like copy and pasting code into the PunBB source. Extension authors don’t have to worry about variable scope or anything like that. I think it’s a more novice friendly approach.
I am not entirely sure what you think would be the best way to implement a system like the one you’ve sketched out. Obviously, we would need to keep track of which extensions are installed (in the database) in order to know which scripts to include at the beginning of each pageview and which objects to create. This is similar to what we do now. For this purpose we could have a global array of extension objects. I’m not sure how you think we should “call the hooks” though. For example, say we’re in viewtopic.php before we output a post and we want to allow extensions to fiddle with the arrays of post options before we output them. The only thing I can come up with is to iterate through the array of extensions, check if an object implements the hook method and if it does, call it. Was this what you had in mind?
One problem with the example code you posted, albeit probably a small one, is that your code relies on the object model in PHP5. We need PHP4 comparability. Unfortunately, the object model from PHP4 is extremely limited. For example, there’s no real encapsulation (all methods and attributes are public). PHP4 has constructors, but they are methods that have the same name as the class itself. There are no destructors. I don’t believe PHP4 has the property_exists() function. There’s a hack for getting the same functionality though.
I want to make it clear that we are open to these kinds of suggestions. We are in no way set on using the current implementation regardless of what pops up. If we can come up with a system that we as a community think is better, we’ll switch. I just don’t want to see anyone going into defensive mode on this.
March 2nd, 2007, at 5:46 pm #Just a small note, there is significantly more overhead using your “php only system” Asbjørn, php must maintain memory allocations for the complete class structure of every extension loaded. eg 30 classes > an array of hooks… So there is certainly not going to be a performance advantage over using included classes. Also keeping an array of 30 classes and looping through every class and verifing a method exists for every hook is also more taxing on php :)
March 3rd, 2007, at 4:55 am #Erm…
* So there is certainly not going to be a performance advantage over using included classes.
should be
* So there is certainly not going to be a performance advantage when using included classes.
March 3rd, 2007, at 4:57 am #There’s no one saying that the class-based solution would have to be included, parsed and loaded on each and every request. Today’s caching method could be used on a class-based solution as well. It would require PHP5, so I definately see a caveat there, but you could use Reflection_Method::export() to get the method body (the source code inside the function) and then cache that in the exact same way as you do now with the XML-based approach. There’s nothing in a class-based implementation that would require you to drop all the caching and optimizations you have in place now, the only thing that would change is how the practical bits are executed internally and how extension authors write their extension.
Having PHP4 compatability is a PITA, though, so I see your point there. Having to back-port all of the shiny PHP5 code to also work with PHP4 would not be fun. Let’s perhaps pick this discussion up again in one year or so when PunBB can (perhaps) require PHP5? ;-)
March 4th, 2007, at 12:35 pm #A little more food for thought is the following article based on XML configuration files in Java:
http://jroller.com/page/scolebourne?entry=configuration_in_java_it_sure
The same applies to XML configuration in PHP, only in PHP, the configuration syntax can be even simpler because PHP is dynamically typed and not class-dependant like Java.
March 14th, 2007, at 1:39 pm #One could also look at vBulletin. It’s in PHP, and has the same kind of plugin system distributed through an XML file.
March 15th, 2007, at 6:18 am #Mmmmmhhh…. .NET?
Unluckily I have Windows…. but I ever refuse to install .NET on my machine.
I suggest you to rewrite all the code into C, C++ or Python.
February 3rd, 2008, at 9:34 pm #Lucas: I’m curious what you find so appalling about .NET. Please tell us.
I work in .NET everyday and even though it has its quirks, I very much doubt I will ever sit down and write a Windows GUI app in C/C++ again. Not having to deal with Windows API and/or MFC is a godsend.
February 4th, 2008, at 1:02 am #Just the typical eletist, MS-hating, “my **** is huge cause I code in C++” idiot.
February 8th, 2008, at 4:19 am #where is the download link of that punxs software .. i hear alot about it but dont know where it is … can some one help
February 13th, 2008, at 7:31 pm #It’s not released yet SuperMAG.
February 14th, 2008, at 5:01 pm #oh ok .. but how can people make extensions now in punres when its not even released … are they beta users or something …
any way i hope its is release soon
February 16th, 2008, at 1:15 pm #Because we dugg through the extension file parser and looked at the sample extensions.
February 17th, 2008, at 10:50 am #If it’s interesting, here’s my solution to the problem:
May 5th, 2008, at 9:05 am #http://pastebin.ca/1007899