Well it has been a mad few weeks. Between landing the new Get Add-ons pane, sheriffing the Firefox 3 beta 3 freeze, decimating old parts of the code and working through follow-up issues from the Get Add-ons pane and not to mention two 10 hour flights it really has been busy. Thankfully I’ve not just been totally buried under code, I’ve had some small spare time to tinker with a few other things (I need to to keep my sanity).

We’re getting better and better at adding quality control to our code. The QA team do their awesome work, we have some 50,000 automated unit tests now (depending how you count) and our tinderbox now has more machines running performance tests than it does machines building the Fox. One problem that all this brings though is a bit of information overload. The tinderbox is now so large that you can no longer glance to see if you can check in or not. Goodness knows how many pages you need to view to truly monitor all of the performance graphs. So, spurred on by Jonathan’s performance dashboard, I’ve been tinkering with some ways to present some of this data.

First off there is the old-style performance machines, pre-talos. These often find regressions first, because they cycle fast. Apparently that is going to improve but until then these old machines have useful data. Unfortunately the graphs they produce are … well lets just say pig ugly! It didn’t take too long to knock up something very basic, but nicer looking and allowing multiple results on the same graph. You can see it graphing the live data (yes it will appear blank for the age it takes to draw). Unfortunately the graph kit I’m using seems a little slow and not all that functional, but that could be changed. Then again I have just found out that new work on the graphs server pretty much blows this out of the water!

So that’s all pretty and good but doesn’t bring a much new to the table. Graphs are good but what we want to be able to do is relate the performance changes to the code that caused it. At this point Johnathan was kind enough to point me at Timeplot. This is a rather neat tool that lets you visualise numerical data on the same graph as defined events. That’s kinda perfect for this. It took me a little longer to hook this one up given the multiple sources and the kinda weird API that can’t normally load from anything other than a remote text file, but here is the result. You can see the performance data of course (Ts from the linux perf box), then each vertical band is a checkin which you can click on for more details.

In the end the timeplot is not massively useful. When you get down to it we tend to check in far more frequently than we gather performance information. I’ll be interested to see if the fast run talos boxes will improve matters. If we had many of them then we can start correlating the regressions across them and thus far reduce the range in which to look for potential causes. I’m interested in other ideas people might have for visualising these kinds of data, I think finding the right way to visualise our information is the key to managing things in the future.

Another thing that needs help is the tinderbox of course. While I was in Mountain View last week Johnathan and I did a quick brainstorm of what the tinderbox is used for. We think it is being used for too many different things and if we split the display out into just those tasks that individuals needed then it would be far more manageable. Maybe you have your own comments on uses we didn’t come up with.

Just finally, after playing with timeplot I was still in a tinkering mood. One thing I periodically have to do is look back over what work I did on specific days. Bugmail is a good indicator for this. Now though I am working in my own Mercurial repository which means I have a full checkin log of my own to look over. Given that it is interleaved with the full Mozilla CVS checkin log it still a little arduous to scan through. But know I have the tools and after not long I can know skim over my previous checkins and even see in depth what each involved (yes, sorry boss it doesn’t look like I did much coding today!).

I know all you extension authors out there have been understandably miffed at the add-on update security bits landing before you could do anything about it, so I’ve pushed hard and we can now make an early version of McCoy available.

McCoy is the tool to use if you are hosting your own add-ons and for whatever reason cannot use SSL to secure the updates. If you haven’t yet heard about the new security restrictions that will be in Firefox 3 or you don’t quite understand them yet then why not skip on over to the vastly improved add-on updates documentation and take particular note of the security section.

So what are you waiting for? Grab your copy of McCoy and get to work securing your add-ons.

I should point out that this is an early version, there are a bunch of improvements that we’d like to make so keep an eye on the homepage and here for updated releases. Also massive thanks to Mark Finkle for getting through the code reviews so quickly.

If you happen to hit any issues using McCoy or just have some ideas for it’s future then please file a bug (searching for dupes first of course!). Given that this is an early release there’s a number of issues already on file.

Yes, finally after months of twiddling my thumbs waiting for approval to land the zipwriter component has gone into trunk, and is enabled for all applications (some apps might need to add the interface to their packages list). If you want to use check out the interface, it’s fairly well documented I think. That magic contract ID you need is "@mozilla.org/zipwriter;1".

It’s been quite a long process both writing the code, getting reviewed and getting agreement for it to appear in Firefox so I hope all you extension authors and application developers are going to make good use out of it in your projects. Let me know what you come up with.

I have just checked in Bug 378216, and wanted to give a quick heads up on it.

What this means is that we are now enforcing a security restriction on all add-ons. To be specific, if an add-on does not provide a secure method of auto-updating then by default Firefox will refuse to install the add-on. If you have add-ons already installed that are insecure in this way then they will be automatically disabled.

The good news is that addons.mozilla.org already uses SSL for it’s updates, so any add-ons you have installed from there will be unaffected by this change. Equally any add-on authors who use SSL on their site, their add-ons will be unaffected. Personally I found 2 of my add-ons were disabled by it, that’s 2 out of nearly 20, so hopefully you won’t see a major impact.

For add-on authors there is an alternate way to provide secure updates without investing in an SSL key involving digital signatures, unfortunately we’ve had to hold off on providing the software to make that possible until the backend changes were complete and reviewed. I hope to have something usable available not too long after M8 is released. Keep an eye on this blog for an update on that.

If you want to see more of the specifics the best place to look is probably at the wiki page. This is all based around the discussions I started on various forums and newsgroups. Hopefully it’s not too much of a surprise to the add-on authors out there, if it is then I apologise, I tried to get the word out as best I could.

After a fair bit of work (feels like longer than 2 months) I’ve finally managed to get bug 382752 landed. What this gives us in simple terms is a set of functions that we can use in order to do unit testing on the extension manager. Alongside I have checked in the first unit test. Now if anything regresses bug 257155 we should know about it immediately.

Ignoring the regression detection, I’ve always found unit tests to be fantastically useful when developing new code or fixing bugs. Zipwriter is a prime example, with a large number of tests that I can run by typing a single command I can test whether the changes I have made have solved the problem and not introduced any other errors.

The next step of course is to start adding unit tests for the extension manager. I have some already in progress and hopefully soon some of the key parts of the EM should be getting checked on a daily basis.

Since the disclosure of potential vulnerabilities in the way Firefox (and other Mozilla applications) automatically update your add-ons we have been discussing how to tighten up the system in a way that is hopefully unnoticeable to users and not much extra work for add-on authors.

After a process of listening to authors on the newsgroups, forums and by email we now have a rough proposal of what changes we are looking to make. There's still a few minor details to be ironed out of course. This is mainly of interest to add-on authors since there is an impact depending on how you host your updates. I've started threads on the newsgroup and forums so if you want to discuss the proposal there then that'd be good. I'd prefer it if you didn't edit the main page of the wiki but feel free to stick small comments onto the discussion page.

Last night (or I should say early this morning) I got a chance to watch some of the webcasts from the Firefox Developer Conference in Tokyo. I had been wanting to attend one of the previous developer days but had to pull out due to accomodation problems so it was really great that webcasts were available, I believe this is the first developer day to do so.

I managed to catch most of the sessions on FUEL and XULRunner and some of the presentations on the development environments that extension authors use. The first two were good to watch, simple overviews of topics that can often get too bogged down in details. I won't say I learned a great deal new from them, but then given my background I wouldn't expect to. I really hope they make the recordings available so when the next person comes onto IRC asking what XULRunner is we can point them to it.

The presentations on development environments were somewhat more challenging to watch remotely, the webcast didn't really have the resolution to bring out the detail of what the authors were doing and there was some time lagging on the video so what was being described didn't correlate with what you could vaguely see on screen. Plus it was nearly 9am here so I had to call it a day at that point and get some sleep.

All in all what I saw of the event looked excellent. Top work by the translators, I don't know if they were volunteers or hired in but they did a great job of handling all the technical terms that were flying about. I hope to make a dev day soon but till then lets keep up this theme of webcasting them so those who can't be there still get a chance to be involved. One thought from the future, how about taking questions from the remote viewers over IRC?

We're looking at the situation of automatic updates for add-ons and whether or not tightening up the security of such updates is a good idea or not (this is one good reason why it could be). After some initial talking I would like to get a little feedback from the add-on authors out there who host their add-ons on their own websites and not on addons.mozilla.org. Are you such a person? If so then please take a few moments to check out the thread in the forums or on the newsgroup and take a few moments to answer my questions.

Of course nothing is a foregone conclusion at this point and you might have noticed that I host my own extensions, not on addons.mozilla.org so I totally have the self-hoster's needs at heart

Wow, it's been a month to the day since my last post here, and quite a lot's happened in that time. Those of you that keep up on Mozilla things might realise that I have changed jobs and I'm now working for Mozilla on the Firefox team under Mike Connor. I'm going to be putting work into the addons side of Firefox 3, in particular taking some of the main requirements as well as tackling some of the really irritating issues that have lain dormant for a little too long for my liking. Most exciting stuff for me right now (yes I know, I'm sad!) is that I've been working on doing unit tests for the extension manager component which makes testing new patches far easier to my mind as well as of course allow us to start catching regressions.

Getting this new position has been quite a fantastic achievement in my eyes and it's allowed me to do other things that I've been needing to do for some time, like move house and various other personal goals that I won't bore you with here.

In case you were wondering how this affects my extensions, well not much in all honesty. They are still all my personal work and all done in my personal time and the amount of time I have spare to work on them is (unfortunately) still about the same. I am however thinking about a fundamental change about how my extensions are available to the general public and in particular one that I think will encourage more outside contribution to my extensions, meaning that the burden is taken off me as a lone developer to add features and fix the bugs. I'm still mulling this over at the moment so watch this space for further news.

It’s currently a bit of a pain building xpcom components in intel macs. The only officially available sdk is ppc only. Until Mozilla come up with an official version, here is an intel build of it for those that want it: gecko-sdk-mac-intel-1.8.1.3.zip

As the name suggests it’s built against Gecko 1.8.1.3. To the best of my knowledge it’s right but please don’t bug me if you can’t get your component to work with it unless you’re pretty positive that it’s the sdk that’s wrong.

Right now I have no clue how you’d go about making a universal sdk, maybe if you know of a simple way then you could get in touch.