Developer Tools meet-up in Portland

Two weeks ago the developer tools teams and a few others met in the Portland office for a very successful week of discussions and hacking. The first day was about setting the stage for the week and working out what everyone was going to work on. Dave Camp kicked us off with a review of the last six months in developer tools and talked about what is going to be important for us to focus on in 2014. We then had a little more in-depth information from each of the teams. After lunch a set of lightning talks went over some projects and ideas that people had been working on recently.

After that everyone got started prototyping new ideas, hacking on features and fixing bugs. The amount of work that happens at these meet-ups is always mind-blowing and this week was no exception, even one of our contributors got in on the action. Here is a list of the things that the team demoed on Friday:

This only covers the work demoed on Friday, a whole lot more went on during the week as a big reason for doing these meet-ups is so that groups can split off to have important discussions. We had Darrin Henein on hand to help out with UX designs for some of the tools and Kyle Huey joined us for a couple of days to help work out the final kinks in the plan for debugging workers. Lot’s of work went on to iron out some of the kinks in the new add-on SDK widgets for Australis, there were discussions about memory and performance tools as well as some talk about how to simplify child processes for Firefox OS and electrolysis.

Of course there was also ample time in the evenings for the teams to socialise. One of the downsides of being a globally distributed team is that getting to know one another and building close working relationships can be difficult over electronic forms of communication so we find that it’s very important to all come together in one place to meet face to face. We’re all looking forward to doing it again in about six months time.

Firefox now ships with the add-on SDK

It’s been a long ride but we can finally say it. This week Firefox 21 shipped and it includes the add-on SDK modules.

We took all the Jetpack APIs and we shipped them in Firefox!What does this mean? Well for users it means two important things:

  1. Smaller add-ons. Since they no longer need to ship the APIs themselves add-ons only have to include the unique code that makes them special. That’s something like a 65% file-size saving for the most popular SDK based add-ons, probably more for simpler add-ons.
  2. Add-ons will stay compatible with Firefox for longer. We can evolve the modules in Firefox that add-ons use so that most of the time when changes happen to Firefox the modules seamlessly shift to keep working. There are still some cases where that might be impossible (when a core feature is dropped from Firefox for example) but hopefully those should be rare.

To take advantage of these benefits add-ons have to be repacked with a recent version of the SDK. We’re working on a plan to do that automatically for existing add-ons where possible but developers who want to get the benefits right now can just repack their add-ons themselves using SDK 1.14 and using cfx xpi --strip-sdk, or using the next release of the SDK, 1.15 which will do that by default.

The Add-on SDK is now in Firefox

We’re now a big step closer to shipping the SDK APIs with Firefox and other apps, we’ve uplifted the SDK code from our git repository to mozilla-inbound and assuming it sticks we will be on the trains for releasing. We’ll be doing weekly uplifts to keep the code in mozilla-central current.

What’s changed?

Not a lot yet. Existing add-ons and add-ons built with the current version of the SDK still use their own versions of the APIs from their XPIs. Add-ons built with the next version of the SDK may start to try to use the APIs in Firefox in preference to those shipped with the XPI and then a future version will only use those in Firefox. We’re also talking about the possibility of making Firefox override the APIs in any SDK based add-on and use the shipped ones automatically so the add-on author wouldn’t need to do anything.

We’re working on getting the Jetpack tests running on tinderbox switched over to use the in-tree code, once we do we will be unhiding them so other developers can immediately see when their changes break the SDK code. You can now run the Jetpack tests with a mach command or just with make jetpack-tests from the object directory. Those commands are a bit rudimentary right now, they don’t give you a way to choose individual tests to run. We’ll get to that.

Can I use it now?

If you’re brave, sure. Once a build including the SDKs is out (might be a day or so for nightlies) fire up a chrome context scratch pad and put this code in it:

var { Loader } = Components.utils.import("resource://gre/modules/commonjs/toolkit/loader.js", {});
var loader = Loader.Loader({
  paths: {
    "sdk/": "resource://gre/modules/commonjs/sdk/",
    "": "globals:///"
  },
  resolve: function(id, base) {
    if (id == "chrome" || id.startsWith("@"))
      return id;
    return Loader.resolve(id, base);
  }
});
var module = Loader.Module("main", "scratchpad://");
var require = Loader.Require(loader, module);

var { notify } = require("sdk/notifications");
notify({
  text: "Hello from the SDK!"
});

Everything but the last 4 lines sets up the SDK loader so it knows where to get the APIs from and creates you a require function to call. The rest can just be code as you’d include in an SDK add-on. You probably shouldn’t use this for anything serious yet, in fact I haven’t included the code to tell the module loader to unload so this code example may leak things for the rest of the life of the application.

This is too long of course (longer than it should be right now because of a bug too) so one thing we’ll probably do is create a simple JSM that can give you a require function in one line as well as take care of unloading when the app goes away.

Throwing xpcom docs out into the wild

It’s been quite some time since I last worked on my prototype XPCOM component/interface viewer and I have to face facts, it’s likely to be quite some time till I do again. I haven’t even had time to update it with the data from the latest 1.9.2 and trunk interfaces. Since I’m not likely to go anywhere else with this I’d love for someone else who thinks it is worthwhile to pick it up and run with it. I’ve just done a final commit adding a README on how to use the code (though I’m sure it is lacking in key ways so be prepared to have to experiment a bit) and the full source is available in my hg repository. The code is MPL tri-licensed so you should just be able to fork and go do your own thing. I should be able to answer any questions and would love to see it get finished and really usable somewhere.

Bumper XULRunner release day!

Thanks to the hard work of Nick Thomas and Lukas Blakk, today I can announce the availability of two new official releases of the XULRunner runtime and SDKs. XULRunner 1.9.0.12 is a maintenance release for the 1.9.0 branch (the code that matches Firefox 3.0.x). XULRunner 1.9.1 is the first official release of XULRunner on the 1.9.1 branch (which matches the code in Firefox 3.5.x). Unfortunately we’re not quite at the point of shipping XULRunner releases at the same time as Firefox 3.5.x releases, but we should have a 1.9.1.1 release soon.

Where is XULRunner 1.9.1?

A question I have been hearing quite a bit on IRC since Firefox 3.5 was released. We don’t quite have the build automation set up to do XULRunner 1.9.1 releases at the same time as Firefox yet, but Nick has been awesome and manually spun some release candidates. These should be pretty much good to go but it would be useful if anyone interested could try them out and report on any serious problems they see. Either just comment here or file a bug and mark it blocking bug 502915.

Without further rambling go get the release candidates.

Moar xpcom docs!

While many of you were out blowing things up over the weekend I spent a few days varying between getting quite a bit done and banging my head against my desk in frustration.

First I updated the XPCOM interface lists that I have been working on with the final release version of Gecko 1.9.0. A few people had asked me for it but I had hit upon the problem that I had lost some of the arcane scripts that I was using to gather the data and so I had to rebuild them first. I have dropped information about the beta versions of 1.9.1 now, just because it was less work that way, if people desperately want them back then I can probably do it.

Secondly I finally sat down and put some hard graft into my rewrite of the project. Some time ago I decided on what extra information I could easily find out automatically and wrote the new code to shove it all into a database. It took till this weekend for me to actually write the web front-end to it. Despite appearances it is actually a complete rewrite. Previously I was using hand-rolled database access code in php. The new version is based on Django which is apparently what all the cool kids are using these days. I was for the most part pleasantly surprised by Django, it makes quite a lot of things significantly easier, but it does suffer from the problem that most object mappers do, it never quite does exactly what you want (or expect). Still I’ve no doubt despite the teething problems I found I have got further like this than I would have by hand. Pro-tip to those thinking of using it, don’t trust it to be right when it tells you what line of a template is causing an error.

Not everything is implemented yet, and I’m sure there are a few bugs laying around to find, but you can take a sneak peek at it. This is what is new:

  • Lists components as well as interfaces
  • Shows what operating systems interfaces are available on (watch out, this can and does vary with the version of Firefox)
  • Shows what components implement a particular interface (again, can and does change with the Firefox version)
  • Shows what interfaces a component implements and for which platform (ditto, our platform is complicated ok?)

There is actually a lot more potential information that could go into the database, including listing for multiple applications, but I think the UI will probably not develop that much further, the additional information would just make things far to complicated to understand.

XULRunner 1.9.1b3 builds, get them while they’re hot

In the process of switching from CVS to Mercurial (and behind the scenes Tinderbox to Buildbot), quite a lot of Mozilla’s build and release systems have had to be updated. Sadly one of the casualties that has yet to be resuscitated are nightly builds of XULRunner on the 1.9.1 branch and trunk. Catlee is now doing some excellent work to get these up and running, but until that gets going here are some semi-official builds using the code matching Firefox 3.1b3. I say semi-official, they were built on regular build slaves using the standard configuration, but just done by hand. I also haven’t tested all of them yet so let me know if you hit any issues, they might be good indicators of issues we’ll see with the nightlies.

Updated Interface lists

I’ve generated a new database for my interface listing webapp so you can now see the current state of the 1.9.1b3pre and 1.9.2a1pre platforms. So for all you extension developers getting ready for the Firefox 3.1b3 release maybe you want to see what has changed since 3.1b2? Or since 3.0?

There are still more things I want to do with this web-app, but right now my time is being spent elsewhere so for now I’ll just keep it up to date with the beta releases. Once 3.1 final is released I’ll likely remove all the beta versions since they probably won’t be necessary then.

I have for the time being had to disable caching in the application, it was eating up literally gigabytes of disk space on my web server and not giving all that much of a speed-up.