After a fair bit of work (feels like longer than 2 months) I’ve finally managed to get bug 382752 landed. What this gives us in simple terms is a set of functions that we can use in order to do unit testing on the extension manager. Alongside I have checked in the first unit test. Now if anything regresses bug 257155 we should know about it immediately.
Ignoring the regression detection, I’ve always found unit tests to be fantastically useful when developing new code or fixing bugs. Zipwriter is a prime example, with a large number of tests that I can run by typing a single command I can test whether the changes I have made have solved the problem and not introduced any other errors.
The next step of course is to start adding unit tests for the extension manager. I have some already in progress and hopefully soon some of the key parts of the EM should be getting checked on a daily basis.
Since the disclosure of potential vulnerabilities in the way Firefox (and other Mozilla applications) automatically update your add-ons we have been discussing how to tighten up the system in a way that is hopefully unnoticeable to users and not much extra work for add-on authors.
After a process of listening to authors on the newsgroups, forums and by email we now have a rough proposal of what changes we are looking to make. There’s still a few minor details to be ironed out of course. This is mainly of interest to add-on authors since there is an impact depending on how you host your updates. I’ve started threads on the newsgroup and forums so if you want to discuss the proposal there then that’d be good. I’d prefer it if you didn’t edit the main page of the wiki but feel free to stick small comments onto the discussion page.
Last night (or I should say early this morning) I got a chance to watch some of the webcasts from the Firefox Developer Conference in Tokyo. I had been wanting to attend one of the previous developer days but had to pull out due to accomodation problems so it was really great that webcasts were available, I believe this is the first developer day to do so.
I managed to catch most of the sessions on FUEL and XULRunner and some of the presentations on the development environments that extension authors use. The first two were good to watch, simple overviews of topics that can often get too bogged down in details. I won’t say I learned a great deal new from them, but then given my background I wouldn’t expect to. I really hope they make the recordings available so when the next person comes onto IRC asking what XULRunner is we can point them to it.
The presentations on development environments were somewhat more challenging to watch remotely, the webcast didn’t really have the resolution to bring out the detail of what the authors were doing and there was some time lagging on the video so what was being described didn’t correlate with what you could vaguely see on screen. Plus it was nearly 9am here so I had to call it a day at that point and get some sleep.
All in all what I saw of the event looked excellent. Top work by the translators, I don’t know if they were volunteers or hired in but they did a great job of handling all the technical terms that were flying about. I hope to make a dev day soon but till then lets keep up this theme of webcasting them so those who can’t be there still get a chance to be involved. One thought from the future, how about taking questions from the remote viewers over IRC?
We’re looking at the situation of automatic updates for add-ons and whether or not tightening up the security of such updates is a good idea or not (this is one good reason why it could be). After some initial talking I would like to get a little feedback from the add-on authors out there who host their add-ons on their own websites and not on addons.mozilla.org. Are you such a person? If so then please take a few moments to check out the thread in the forums or on the newsgroup and take a few moments to answer my questions.
Of course nothing is a foregone conclusion at this point and you might have noticed that I host my own extensions, not on addons.mozilla.org so I totally have the self-hoster’s needs at heart 🙂
Wow, it’s been a month to the day since my last post here, and quite a lot’s happened in that time. Those of you that keep up on Mozilla things might realise that I have changed jobs and I’m now working for Mozilla on the Firefox team under Mike Connor. I’m going to be putting work into the addons side of Firefox 3, in particular taking some of the main requirements as well as tackling some of the really irritating issues that have lain dormant for a little too long for my liking. Most exciting stuff for me right now (yes I know, I’m sad!) is that I’ve been working on doing unit tests for the extension manager component which makes testing new patches far easier to my mind as well as of course allow us to start catching regressions.
Getting this new position has been quite a fantastic achievement in my eyes and it’s allowed me to do other things that I’ve been needing to do for some time, like move house and various other personal goals that I won’t bore you with here.
In case you were wondering how this affects my extensions, well not much in all honesty. They are still all my personal work and all done in my personal time and the amount of time I have spare to work on them is (unfortunately) still about the same. I am however thinking about a fundamental change about how my extensions are available to the general public and in particular one that I think will encourage more outside contribution to my extensions, meaning that the burden is taken off me as a lone developer to add features and fix the bugs. I’m still mulling this over at the moment so watch this space for further news.
Update: Mozilla now produce intel gecko SDKs so there is no need to use the version I have put here, I’ll leave it for posterity though.
It’s currently a bit of a pain building xpcom components in intel macs. The only officially available sdk is ppc only. Until Mozilla come up with an official version, here is an intel build of it for those that want it: gecko-sdk-mac-intel-188.8.131.52.zip
As the name suggests it’s built against Gecko 184.108.40.206. To the best of my knowledge it’s right but please don’t bug me if you can’t get your component to work with it unless you’re pretty positive that it’s the sdk that’s wrong.
Right now I have no clue how you’d go about making a universal sdk, maybe if you know of a simple way then you could get in touch.
I can’t believe it’s been over a month since I wrote something here. Well I kinda can, lots of hectic stuff going on at my work right now which has been making finding time for Mozilla stuff tricky. Hopefully not for much longer.
I’m glad to say that I have managed to make great progress on the zip writer component. I have decided that dealing with multiple platform compiles for Nightly Tester Tools is just a bad idea, so instead I have pushed on with submitting the zip writer to Mozilla for review. Hopefully that will make it into tree where I (and of course anyone else) can just use it. There’s been a bunch of changes between the version I posted earlier and that that’s gone up for review, not least of which is a set of testcases that have made sure I didn’t break the old by making some cleanups.
Lots of people have been bugging me about when new versions of my extensions are going to be done. Sadly I can’t really do this. The old adage of “It’ll be done when it’s done” certainly applies. For most I don’t have a good handle on how much work is left and I certainly don’t know how much time will have to spend on them in the near future which sort of messes up any planning.
Oh and I’d just like to say hello to all you people reading planet out there. Assuming I haven’t broken my atom feed you should be able to see everything I write here from now on, you lucky lucky people!
I spent some more time this weekend hacking on my Zip Writer component. It’s now pretty capable. It can open existing zip files and remove entries and append to them, happily rewriting the headers and everything exactly as it encountered them. And the other major win is that I have deflate code up and running which makes compression possible.
All this has allowed me to reintroduce making extensions compatible during the install process in a far safer way than was the case before. NTT can now watch the EM datasource, spot that an extension has finished being downloaded, then open that extension’s xpi file and if necessary rewrite the metedata to claim compatibility with the current version of whatever app you happen to be running in.
Depending on time I may have the next release of NTT out the door by the end of next weekend, but things are hectic as usual and I have to be various places over the coming week.
So there’s been this bug in Firefox for … well quite a while where it would suddenly stop remembering your toolbar customisations, window positions and even make your bookmarks appear to not be there, and in Firefox 1.5, make the search bar non-functional.
Well I’m quite proud to say that after quite a lot of research, and help from those guys doing Firefox support, particularly stevee, I think I have a fix for at least part of the problem. This bug (or at least the part I’m interested in) is all caused by one corrupt file. For quite some time I was unable to reproduce, and it turns out that that’s because the issue actually resolves itself in Minefield which is what I use day to day. Once I started testing on BonEcho it became pretty obvious what was going on. So the patch I’ve just posted for review basically spots a corrupt file on startup, and deletes it. Short and sweet.
Of course it’s not really the solution to the problem, just a nice way to make Firefox recover without too much hassle. The real issue is how the file gets corrupt in the first place.
One of the annoying omissions from the Mozilla platform has always been the inability to create a zip file. It’s been bugging me for some time since it’s the only way for Nightly Tester Tools to properly manage overriding compatibility for an individual extension install, without doing dangerous things like it used to. There’s been a Google Summer of Code project on creating such functionality but it didn’t get all that far.
Well Saturday night I finally bit the bullet, after talking about it for a long time, and took a crack at it myself. I didn’t like the API’s presented in the SoC project so I decided to just start from scratch andknock up something really really simple. And when I say simple, I mean that initially all this thing was going to do was create a zip file (no editing existing ones), and add files and folders to it, with no compression. That makes it simple see, just add a few file headers here and there and all you’re doing is copying data around.
So onto the second implementation in C++ this time. I guess it was probably inevitable that it would come to it, but JS at least is real quick to prototype things like this in. The C++ implementation now works as well as I’d initially hoped, so since then I’ve of course got bored and started looking into having real compression in there and maybe editing zip files in the future. Like most of my code all this is open source, you can view it in my subversion repository. What’s more, since I moved on, Mook has taken the JS implementation, plumbed in some cleverness and made it work for binary files too!
Obviously the code is to be used at your own risk and right now it’s not all that thoroughly tested. Hopefully though if I can convince bsmedberg of my API’s, this code might end up making it onto trunk so everyone can use it.