What’s the Future for Add-ons Management?

With Firefox 3 getting ever closer to release it is time (well, past time) to start thinking about the future. The Extension Manager has seen quite a number of improvements since Firefox 2. Many were designed to be invisible, generally improving stability and fixing oddities. Some are extremely visible such as the new security requirements and the addons.mozilla.org integration. The question is, what’s the next step for add-ons management?

Here are a few ideas that are floating around my mind to get you started:

  • Installing add-ons without restarts
  • Presenting more detail in the install dialog
  • Simplifying the UI
  • Deprecating install.rdf and replacing with a simpler xml or json format
  • Add-on dependencies with automatic resolution

So what have I missed? Please keep any comments restricted to add-ons management, either as a user installing and using add-ons or as a developer writing add-ons. While I am interested in the future of Firefox as a whole I don’t want this to be a mass feature request for the product.

Why 2 SDKs are better than 1

In the past the Gecko SDK was somewhat limited. You could compile against it, but only if you were using frozen components, of which there are exceptionally few. You can build an application with only them, but I’d be startled if any moderately complicated app or extension gets by with only them. Thankfully this has changed for 1.9 and the new style SDK contains all interfaces and headers, frozen and unfrozen. This gives you access to lots more, though has the minor disadvantage that you have to keep an eye on what you are using as it could break in the future.

Just one snag with this new SDK though. The Mac version was missing something. Like a bunch of the tools and the XUL framework snapshot. It turns out that the method of packaging the SDK really didn’t work too well when you’re building a universal XULRunner, which is of course what we do.

After a bunch of work by plasticmillion and myself and a bunch of reviewing and generally dealing with our pesky questions by bsmedberg, the Mac XULRunner tinderbox is now producing working SDKs for every build. And I do mean the plural there. Building just one unified SDK is pretty tricky, even manually, let alone trying to make the Mozilla build system do it. Instead two SDKs are build, one for the powerpc architecture and one for intel. This actually works out well if you are producing an unified application using the Mozilla build system. All you really need are these magic lines in the mozconfig for your application:

ac_add_app_options ppc  --with-libxul-sdk=<path to powerpc sdk>
ac_add_app_options i386 --with-libxul-sdk=<path to intel sdk>

I’m just about to update the McCoy application to use the SDKs rather than having to build a full XULRunner alongside. This will make building McCoy simpler, particularly a unified Mac version which for the initial release I pretty much had to stitch together by hand. It will also I hope resolve some stability issues on Linux. I hope to have a new version out soon with some popularly requested new features.

I should note that the powerpc SDK is not a pure powerpc SDK. It is designed to be used to compile the powerpc portion of an applicaton using a build machine that is intel based. Unfortunately some of the important build tools cannot be cross-compiled for the powerpc architecture since the dependencies are only one architecture. Hopefully this will suit the majority of people.

On Timezones, Testing and Deals with the Devil

Timezones make my head hurt. Not the concept, that is easy, but writing code that works correctly in different timezones. Throw daylight savings correction into the mix and you’ll often find me in a corner rocking gently. The problem of course is that I frequently have to deal with data that isn’t just the standard unix timestamp (or various orders of magnitude away from). A number of times a load of information has been thrown away, never to be recovered which makes life interesting. You end up having to decide which is the best path to take when all of them are wrong in various ways.

So why bring this up? Well America has just jumped to daylight savings time. And wouldn’t you know it, a few of Mozilla’s automated tests failed. Which is awesome of course. Always good to know something is up. The tricky thing with test failures is figuring out why they are failing. The automated tinderbox are remote machines you generally don’t have access to to debug on. This can be pretty unhelpful if the test doesn’t provide enough logged information. I’ve been pretty guilty of this myself, while a test that spots problems is fantastic, what is even better is one that provides enough information in the log to figure out why it failed. Think of it this way. If you write a test that (months, years) later fails then you’ll likely be asked why. What information do you need to find out? A few suggestions I’ve learned the hard way:

  • Always log the found and expected values when comparing (do_check_eq and similar does it for you but there are cases where you don’t use that).
  • If you have a loop in the test be sure to log what iteration the test failed in. A message that “false wasn’t true” isn’t a lot of use when that check is performed a hundred times.
  • If it is a long test why not just log a few checkpoints along the way, if the test crashes somewhere at least you’ll have a rough idea of where.

I will probably come back with more suggestions by the end of the week. I needed approval for a late fix I needed to land on Saturday and shaver was the only driver around at the time. He is often getting on at people (quite rightly) for not writing tests for the code they submit, so I made a deal with him that in return for the approval that I’d spend 2 days this week writing tests. I expect the majority of them to be testing for the UI side of the add-ons manager which is essentially untested right now and right now I have no clue how to go about testing. Should be fun!

Meet the New Website, Same as the Old Website, Roughly

I’ve finally taken the plunge and switched my website to a more modern blogging software (WordPress) and a dedicated media gallery (Gallery 2). Hopefully through the magic of redirects most shouldn’t notice much difference. I just hope planet hasn’t decided to dump all my posts onto the front page, if it has then I apologise.

I’ve also taken this opportunity to move all my extensions to addons.mozilla.org. Most are currently still in the sandbox, hopefully they will come public soon. I’m also using the nice new AMO API service to populate details on the add-on homepages, means there is only one place I need to make most changes to.

Trimming the Fat

As I mentioned in my last post and as I know everyone is well aware, tinderbox is starting to feel the strain. You already can’t just glance at it to see what is going on. The Firefox tree is currently showing build and test results from a total of 23 machines. The interesting thing is that not all of these machines are actually doing any compiling. The rest are merely running tests on builds produced by the other machines. That isn’t to say that those test results are less important but I wonder whether it is worth treating these differently.

Thinking about this difference I’ve spent a short time knocking together an experimental view of tinderbox. The idea is simple, the test boxes that aren’t actually compiling don’t get a dedicated column. We still want those results though, and luckily there is somewhere to put them, on the build that they are testing. So every actual compile ends up as a box within which can be multiple results from testers. This leaves us a very manageable 9 columns, 3 per platform.

This is quite early stages, the physical perf and test numbers aren’t visible yet, there are a couple of bugs I know about, it doesn’t auto update and I wouldn’t recommend you really use this, but I am interested to know what people think before I proceed any further with it. There are a few questions in my mind like how to colour the main box of the build (currently the worse result from the compile and all the test results), how to colour the header (I suspect the worst of the most recent results from all of the machines reporting in that column), where to even put the full perf/test numbers (there isn’t really room in some of the fast nightly boxes). More importantly is this even readable? I think it is good for some things, often tinderbox gets confusing when you see a talos result fail long after the nightly box went green, here that talos red would be back on the nightly box that failed. I can imagine problems trying to figure out why the header is red when it’s actually the talos results from build a couple of times ago that is causing it.

Anyway without further ado, take a look at Tindermerge and let me know your thoughts.

Visualising our Labours

Well it has been a mad few weeks. Between landing the new Get Add-ons pane, sheriffing the Firefox 3 beta 3 freeze, decimating old parts of the code and working through follow-up issues from the Get Add-ons pane and not to mention two 10 hour flights it really has been busy. Thankfully I’ve not just been totally buried under code, I’ve had some small spare time to tinker with a few other things (I need to to keep my sanity).

We’re getting better and better at adding quality control to our code. The QA team do their awesome work, we have some 50,000 automated unit tests now (depending how you count) and our tinderbox now has more machines running performance tests than it does machines building the Fox. One problem that all this brings though is a bit of information overload. The tinderbox is now so large that you can no longer glance to see if you can check in or not. Goodness knows how many pages you need to view to truly monitor all of the performance graphs. So, spurred on by Jonathan’s performance dashboard, I’ve been tinkering with some ways to present some of this data.

MultiperfFirst off there is the old-style performance machines, pre-talos. These often find regressions first, because they cycle fast. Apparently that is going to improve but until then these old machines have useful data. Unfortunately the graphs they produce are … well lets just say pig ugly! It didn’t take too long to knock up something very basic, but nicer looking and allowing multiple results on the same graph. You can see it graphing the live data (yes it will appear blank for the age it takes to draw). Unfortunately the graph kit I’m using seems a little slow and not all that functional, but that could be changed. Then again I have just found out that new work on the graphs server pretty much blows this out of the water!

So that’s all pretty and good but doesn’t bring a much new to the table. Graphs are good but what we want to be able to do is relate the performance changes to the code that caused it. At this point Johnathan was kind enough to point me at Timeplot. This is a rather neat tool that lets you visualise numerical data on the same graph as defined events. That’s kinda perfect for this. It took me a little longer to hook this one up given the multiple sources and the kinda weird API that can’t normally load from anything other than a remote text file, but here is the result. You can see the performance data of course (Ts from the linux perf box), then each vertical band is a checkin which you can click on for more details.

CheckinplotIn the end the timeplot is not massively useful. When you get down to it we tend to check in far more frequently than we gather performance information. I’ll be interested to see if the fast run talos boxes will improve matters. If we had many of them then we can start correlating the regressions across them and thus far reduce the range in which to look for potential causes. I’m interested in other ideas people might have for visualising these kinds of data, I think finding the right way to visualise our information is the key to managing things in the future.

Another thing that needs help is the tinderbox of course. While I was in Mountain View last week Johnathan and I did a quick brainstorm of what the tinderbox is used for. We think it is being used for too many different things and if we split the display out into just those tasks that individuals needed then it would be far more manageable. Maybe you have your own comments on uses we didn’t come up with.

Just finally, after playing with timeplot I was still in a tinkering mood. One thing I periodically have to do is look back over what work I did on specific days. Bugmail is a good indicator for this. Now though I am working in my own Mercurial repository which means I have a full checkin log of my own to look over. Given that it is interleaved with the full Mozilla CVS checkin log it still a little arduous to scan through. But know I have the tools and after not long I can know skim over my previous checkins and even see in depth what each involved (yes, sorry boss it doesn’t look like I did much coding today!).

Extension Authors, Say Hi to McCoy

McCoyI know all you extension authors out there have been understandably miffed at the add-on update security bits landing before you could do anything about it, so I’ve pushed hard and we can now make an early version of McCoy available.

McCoy is the tool to use if you are hosting your own add-ons and for whatever reason cannot use SSL to secure the updates. If you haven’t yet heard about the new security restrictions that will be in Firefox 3 or you don’t quite understand them yet then why not skip on over to the vastly improved add-on updates documentation and take particular note of the security section.

So what are you waiting for? Grab your copy of McCoy and get to work securing your add-ons.

I should point out that this is an early version, there are a bunch of improvements that we’d like to make so keep an eye on the homepage and here for updated releases. Also massive thanks to Mark Finkle for getting through the code reviews so quickly.

If you happen to hit any issues using McCoy or just have some ideas for it’s future then please file a bug (searching for dupes first of course!). Given that this is an early release there’s a number of issues already on file.

ZipWriter is Here!

Yes, finally after months of twiddling my thumbs waiting for approval to land the zipwriter component has gone into trunk, and is enabled for all applications (some apps might need to add the interface to their packages list). If you want to use check out the interface, it’s fairly well documented I think. That magic contract ID you need is “@mozilla.org/zipwriter;1”.

It’s been quite a long process both writing the code, getting reviewed and getting agreement for it to appear in Firefox so I hope all you extension authors and application developers are going to make good use out of it in your projects. Let me know what you come up with.

Add-on Security Restrictions Landed

I have just checked in Bug 378216, and wanted to give a quick heads up on it.

What this means is that we are now enforcing a security restriction on all add-ons. To be specific, if an add-on does not provide a secure method of auto-updating then by default Firefox will refuse to install the add-on. If you have add-ons already installed that are insecure in this way then they will be automatically disabled.

The good news is that addons.mozilla.org already uses SSL for it’s updates, so any add-ons you have installed from there will be unaffected by this change. Equally any add-on authors who use SSL on their site, their add-ons will be unaffected. Personally I found 2 of my add-ons were disabled by it, that’s 2 out of nearly 20, so hopefully you won’t see a major impact.

For add-on authors there is an alternate way to provide secure updates without investing in an SSL key involving digital signatures, unfortunately we’ve had to hold off on providing the software to make that possible until the backend changes were complete and reviewed. I hope to have something usable available not too long after M8 is released. Keep an eye on this blog for an update on that.

If you want to see more of the specifics the best place to look is probably at the wiki page. This is all based around the discussions I started on various forums and newsgroups. Hopefully it’s not too much of a surprise to the add-on authors out there, if it is then I apologise, I tried to get the word out as best I could.

Practice what you Preach

One of the main parts of my work for Mozilla at the moment is about securing add-on updates. The spec is now pretty near complete and the implementation is also pretty much complete so hopefully we can start pushing out the necessary tools to add-on authors real soon then land the work shortly after.

Of course it wouldn’t be right for me to push this out without first making my own extensions comply with the new requirements. So today I am rolling out updates to all of them, mostly just changing the update url to an SSL one, though a couple of the extensions (Nightly Tester Tools and /Find Bar/) have some additional updates.

Using SSL really will be the easiest way of hosting secure updates for your extensions and I urge you to use it. Assuming you have a sensible hosting package, adding SSL is really not as expensive as many expect. Godaddy offer SSL certificates for $18 per year (minimum of 2 years) and if you are like me and hosting open source extensions then you can get the first year for free (though that seems to take a few weeks longer than if you pay). It’s also pretty simple to set up assuming you have a decent webhost, Dreamhost just has one form to fill in.

It turns out that the hardest part of getting SSL was fixing the bugs in my CMS since it’s current version had never been used in an SSL environment before 😉