Making it easier to check that your plugins are up to date

Keeping the software you use up to date is a crucial part of keeping yourself safe while browsing online. At Mozilla we work hard to help you get the most up to date version of Firefox and all the add-ons you have installed. For some time now security updates for Firefox have been installed without you needing to do anything. In Firefox 4 we made extension and theme updates behave similarly.

Plugins, like Flash, Quicktime and Java for example, are a little more difficult to update in this way though. They tend to require explicit permission to install new versions and so we haven’t quite gotten to the point of doing this completely automatically. Instead we developed the plugin check page which can quickly and easily tell you which of the plugins you have installed are old and need updating. It will also tell you where to go to update them.

The latest version of Firefox currently in beta makes it easier to get to the plugin check page. Simply go to the Add-ons Manager, click on the Plugins section and there is a link at the top of the page to check if your plugins are up to date.

Why do Firefox updates break add-ons?

Our success in switching to the new rapid release cycle for Firefox has stirred up lots of excitement in the community and I wouldn’t be surprised if that intensifies when we ship the next update to Firefox in 8 weeks time. People keep pointing out that everytime we update Firefox we break add-ons so surely faster releases means add-ons will get broken faster. Many people don’t really understand why Firefox updates should break add-ons anyway so here is my attempt at an explanation and how maybe rapid releases aren’t such a bad thing after all.

The crux of the matter is in how deeply we allow add-ons integrate with Firefox. Most browsers separate add-ons from their code using what can be called an add-on API. All the add-on can see and use is what the browser makes available through that API and this is normally a restricted set of functionality. Firefox has no such separation. Add-on code runs in exactly the same setting as the browser code, they can call any of the internal functionality that we write to make the browser work in the first place.

There are big advantages to the Firefox way:

  • Add-ons can do basically anything, allowing really complicated add-ons like Firebug and NoScript to exist.
  • Authors don’t have to wait for features to be exposed through some API to be able to use it, as soon as the feature is in the application add-ons can use it.

There are also some disadvantages:

  • Add-ons can do basically anything so you should be wary about installing add-ons from people you don’t trust!
  • Because we aren’t forced to provide an API specifically for add-ons sometimes what is there can be cumbersome for add-ons to use.
  • Add-ons rely on internal functions, if we make a change to one then it breaks the add-on.

It is that last point which explains why Firefox updates break add-ons. Any time we add, remove or change some code we run a risk that some add-on depends on it and so will break. Some bits of code are more important than others but we have such a vast library of add-ons now that it’s probably getting to the point where almost everything is in use. When people say that Firefox updates shouldn’t break add-ons what they don’t realise is that they’re asking us to stop making changes to Firefox: no new features, no bug fixes.

How does the Add-ons SDK fit in?

With the most recent update to Firefox we also made the first official release of the new add-ons SDK available. These are tools designed to make developing add-ons for Firefox easier and faster and they largely serve as an add-ons API sitting between the add-on and Firefox with one difference, the API is actually a part of the add-on so you can actually write your own additional APIs to supplement those in the core SDK.

Add-ons written with the SDK (and only using the core APIs) gain a lot of the benefits of add-ons for other browsers. The APIs in the SDK can be far more stable than those in the browser itself, as the browser code changes the internal SDK code can adapt to match allowing add-ons to work just by rebuilding with the newer SDK. This makes the problem of Firefox updates much smaller for those that can move their add-ons to the SDK.

I don’t foresee a future where the only add-ons are SDK based though. We’d lose too many really important add-ons that way so the SDK doesn’t completely solve the problem.

How does rapid release affect the problem?

As I’ve said, when Firefox updates it breaks add-ons. Ask any add-on author how much fun it was fixing their add-ons to work from Firefox 2 to Firefox 3 or 3.6 to 4 and you’ll probably get some pained expressions. Now think about how in the past we would do a Firefox update about once a year, now we are talking about it being more than eight times a year. Having given up maintaining my own add-ons I can hardly blame authors who are concerned that they won’t be able to keep up with the pace.

It actually isn’t all bad though. It’s not like we are making more changes than we were before, we’re just releasing them sooner in bite-sized chunks. Instead of one update a year that probably changes every piece of functionality an add-on might use we do 8 smaller updates, each one touching smaller parts of the code. This means it is more likely that an add-on can survive an update than before.

The stability of updates is a key change too. It used to be that add-on authors might choose to wait for the RC stage of an update before updating their add-ons. We’d make so many changes even during the beta cycle that sometimes it wasn’t worth trying to keep up. In our new cycle though there is a full 6 weeks in the beta cycle before release during which time almost nothing changes, no new features, extremely limited bugfixes, just things to solve any stability issues that have been found. Before that there is 6 weeks in the aurora cycle which again should be largely change free. Some new features might get removed at this point and there is the possibility of larger bugfixes, but based on the two aurora cycles we’ve had so far there are far fewer changes going in than there were in the final beta stages of Firefox 4.

I actually think that rapid releases can make it easier for add-on authors to keep up. Sure you have to check your add-on more often but most of the time it will probably just work, if not the scope of change is likely to be smaller and you have 12 weeks of pretty stable code to build against.

Why not just mark add-ons as working when they are?

A pretty common question asked is why we don’t just mark add-ons as working in the new update of Firefox when we know they are. The good news is that we actually are starting to do this. As we make changes to Firefox we plan to keep track of which changes have the potential to affect add-on compatibility and automatically scan add-ons on AMO to find ones that might break.

In the run up to the most recent update of Firefox we found nearly 4,000 add-ons that were unlikely to be affected by the update and just marked them as compatible with no need for the add-on author to do anything. For the others we emailed the author to explain what problems we had found. You can read more about the compatibility checking process we’ll be using.

As well as automated tools the add-on compatibility reporter provides add-on authors valuable feedback on how their add-ons are performing in Firefox. Users can install this tool and then use normally incompatible add-ons and file reports on whether they are working or not.

Faster releases are good for add-ons

Rapid release is all about getting new features into users’ hands faster. It has a side effect though, it gets new features and APIs into add-on authors’ hands faster too. Back when I was still writing add-ons I would frequently do so in response to some fantastic new thing that had been added to the Firefox nightlies. It could then be up to a year before I’d be able to release an add-on that normal users could actually make use of, by which point I’d often have lost interest in it.

I think that on the whole faster releases will be a good thing for add-ons. Add-on developers can get exciting new features into users’ hands faster and should only need to make smaller updates with each new Firefox. The new automated tools we are building to help authors understand when and why their add-ons are no longer compatible are going to be a massive improvement, especially as we get better at identifying the changes we make in Firefox that will affect add-ons.

Time will tell of course, but I do believe that the overall work an author has to do to keep an add-on up to date with Firefox is going to drop with the new rapid release cycle even though they may have to do something more often.

Creating custom add-on types just got easier

One of the nice features that we added to the add-ons manager in Firefox 4 was supportĀ  for custom add-on types that could be treated the same way as the built-in types, even showing up in the same UI if you did a little work. I blogged a basic example of how to do this and I know since then Greasemonkey and Stylish have been using the support.

A few days ago (and now in Aurora builds) I landed some improvements that simplify the hardest part of this, making the UI appear.

Registering a custom type

Previously in order to add a new type of add-on and have it appear in the add-ons manager you would have to do two things, first tell the add-ons manager module that you supported add-ons and secondly manually create some elements in the UI to allow the user to view them. These two things have now been combined:

AddonManagerPrivate.registerProvider(myProvider, [{
  id: "scripts",
  name: "User Scripts",
  uiPriority: 3500,
  viewType: AddonManager.VIEW_TYPE_LIST
}]);

This registers the provider as before but also tells the module that your provider supports an AddonType with the ID “scripts” and some other information. The documentation explains a little more about the properties. You’ll note it passes an array to allow for providers registering multiple types.

Currently you only need to do this if you want your add-on types to show up in the UI, however we may start requiring providers to register their types to help speed up some of the API calls. If you want to register your type and not have the UI automatically show it just don’t give it a viewType.

Breaking changes

Some of the changes we had to make may have broken some add-ons that have been doing this the hard way in the past. Here are the once I’ve identified and some ways to work around the problems:

Types must be registered

The UI now requires that if you’re displaying your add-ons in the regular list view then it must have been registered. If you only care about supporting Firefox 6 and higher then you can just remove your code for adding your type to the category list, if you want to support both then you should either only do that on versions before Firefox 6, or you could always manually add your type to the UI but also register your type with no viewType.

Some IDs changed in the UI

We made some changes to the IDs of elements in the add-ons manager UI, we dropped the “s” on the end of the type selectors. category-extensions became category-extension as an example.

Category items aren’t in the UI immediately

Because we now build the list of types to display in the UI (even the built-in types use the same type registration I’ve talked about here), the elements for those types aren’t in the UI until after the main initialize function for the window has completed. If you try to use overlays that refer to those elements or if you run script that tries to find these elements before the initialize function completes (it is a load event listener) then they won’t exist and you’ll probably find things broken.

 

How to disable extension compatibility checking on Nightly builds of Firefox

A long long time ago (I can still remember…) we changed the preference you use to disable compatibility checking for extensions. We still aim for users to instead use tools like the Add-on Compatibility Reporter to handle this (especially since we are going to start crowdsourcing data from it), but for developers who don’t want to install that but still want to use extensions on their nightly builds the new rapid release model would mean setting a new preference every 6 weeks.

We’ve now changed this and any nightly, try, or self-made builds (anything not on the aurora, beta or release channels) will instead use the single preference extensions.checkCompatibility.nightly from now on, regardless of version. This will be available from tonight’s nightly onwards.

What’s next for the Add-ons Manager?

Firefox 4 is just around the corner and it’s great to look back over just how far the Add-ons Manager has come since Firefox 3.6. In fact if you want to see the full history look at my earlier post that shows its evolution since Phoenix 0.2. We set out with some pretty lofty goals for Firefox 4 and I’m pretty excited at just how many of them we achieved. I hope everyone appreciates the hard work that Blair, Boriss, Justin, Henrik, Ben, myself and all the others put in to get us to where we are today.

Of course, we aren’t finished. There were a lot of things that we wanted to get done and didn’t get time for and I’m sure we’ll be getting lots of feedback from users of the final release to work on. Since the pressure has been off I’ve been skimming back over bug reports and notes to try to come up with an idea of what we want to work on over the next few releases of Firefox. As you may know the plan is to move to quarterly releases and not really block a release on any particular projects but we will still be getting an idea of what product drivers will prioritise work on and guess at what other things we can fit in.

UI refinements

In the closing stages of Firefox 4 there were a lot of small polish issues identified. We knocked down a lot of them but there are still plenty of tweaks that we’re going to be bringing to the UI to make sure it is as usable as we can make it. It isn’t really worth applying a fixed schedule to these as they are small one-off fixes that can just come as and when ready. We already have some that are basically ready to land.

Incremental API fixes

There are lots of things that I’d like to see done for the platform and APIs that developers use. Again there is no fixed schedule for these but the main things on my mind are changes that make developing restartless add-ons or extending the add-ons manager easier. I’ve already started work on a couple of these and expect to get them landed after the tree reopens.

Larger projects

There are of course also larger projects that will require more resources. I’ve started writing up short project pages for each of these. Each project should start with the people working on it coming to agreement on what the actual goals should be and an implementation plan so until the projects are actually picked up the project pages are mostly made up. I’ve put down a rough idea of when a project might make it into a Firefox release. These are just guesses, if the project turns out to be larger than expected they’ll come later, if people volunteer to work on projects or if the product drivers decide they are a priority then they could come sooner.

You can see that assuming a quarterly release schedule we have enough projects here to last till the end of 2011 at least, quite likely longer. Over that sort of time I guess it’s almost guaranteed that priorities and plans will change in the meantime but hopefully this gives an idea of what our thinking is at the moment. Expect to see progress on some of the things at the top of the list very soon!

Playing with windows in restartless (bootstrapped) extensions

Lots of people seem to be playing with the new support for restartless extensions (also known as bootstrapped extensions) coming in Firefox 4. Nothing could make me happier really. I’m not sure I can remember ever helping implement something which is will hopefully turn out to be a game changer for extension development in the future. The technical details of restartless extensions are talked about in a few places but one thing that is missing is what I think is probably going to be the most common code pattern in these extensions.

When you’re developing an extension it’s almost certain that you’re going to want to interact with the main browser window in some way. With the XUL based extensions this was a pretty simple process, normally your extension would be overlaying browser.xul and so any script you add there would automatically run for every new browser window and would have direct access to it (in fact this sometimes confused new developers who assumed their script would run just once and be globally available, not for every browser window opened).

In restartless extensions things are different. You have one script that runs when your extension is started and runs in its own sandbox so you have to take steps to get access to windows. Complicating matters your extension may be started when Firefox starts up and there are no browser windows or after when some already exist. The best way to go is when your extension is started to look for any existing browser windows and make your necessary modifications and then wait for new windows to open and modify those too.

Since this is such a common case (and vlad was trying to figure it out on IRC) I put together a very simple bootstrap script that just does this. It calls the setupBrowserUI function for every window that needs to be initialised by your extension and tearDownBrowserUI for every window that must be cleaned up when your extension is being disabled. This code isn’t the only way to do all this but it is fairly straightforward and works, feel free to take it and modify it for use in your own extensions. I’ll probably get it put onto MDC too.

const Cc = Components.classes;
const Ci = Components.interfaces;

var WindowListener = {
  setupBrowserUI: function(window) {
    let document = window.document;

    // Take any steps to add UI or anything to the browser window
    // document.getElementById() etc. will work here
  },

  tearDownBrowserUI: function(window) {
    let document = window.document;

    // Take any steps to remove UI or anything from the browser window
    // document.getElementById() etc. will work here
  },

  // nsIWindowMediatorListener functions
  onOpenWindow: function(xulWindow) {
    // A new window has opened
    let domWindow = xulWindow.QueryInterface(Ci.nsIInterfaceRequestor)
                             .getInterface(Ci.nsIDOMWindow);

    // Wait for it to finish loading
    domWindow.addEventListener("load", function listener() {
      domWindow.removeEventListener("load", listener, false);

      // If this is a browser window then setup its UI
      if (domWindow.document.documentElement.getAttribute("windowtype") == "navigator:browser")
        WindowListener.setupBrowserUI(domWindow);
    }, false);
  },

  onCloseWindow: function(xulWindow) {
  },

  onWindowTitleChange: function(xulWindow, newTitle) {
  }
};

function startup(data, reason) {
  let wm = Cc["@mozilla.org/appshell/window-mediator;1"].
           getService(Ci.nsIWindowMediator);

  // Get the list of browser windows already open
  let windows = wm.getEnumerator("navigator:browser");
  while (windows.hasMoreElements()) {
    let domWindow = windows.getNext().QueryInterface(Ci.nsIDOMWindow);

    WindowListener.setupBrowserUI(domWindow);
  }

  // Wait for any new browser windows to open
  wm.addListener(WindowListener);
}

function shutdown(data, reason) {
  // When the application is shutting down we normally don't have to clean
  // up any UI changes made
  if (reason == APP_SHUTDOWN)
    return;

  let wm = Cc["@mozilla.org/appshell/window-mediator;1"].
           getService(Ci.nsIWindowMediator);

  // Get the list of browser windows already open
  let windows = wm.getEnumerator("navigator:browser");
  while (windows.hasMoreElements()) {
    let domWindow = windows.getNext().QueryInterface(Ci.nsIDOMWindow);

    WindowListener.tearDownBrowserUI(domWindow);
  }

  // Stop listening for any new browser windows to open
  wm.removeListener(WindowListener);
}

Firefox 4 and the Add-ons Manager at Add-on-Con

As I mentioned before I was part of a presentation at Add-on-Con this year. Myself, Boriss and Justin talked about the new UI changes in Firefox 4 and about the main changes to the add-ons manager. If you’re particularly interested the slides are available here though I guess slides are often just tiny snippets of info from the actual session so if anything catches your eye you’ll need to get in touch and ask us about it.

Add-on-Con is here!

Next week is Add-on-Con 2010 and if you do any work in the add-ons space then you’re probably going to want to take a look at what is going on and hopefully sign up to attend. There are two days this year, one for some training and then the traditional day for keynotes and business/development tracks. I’ll be there for all of the main day and while I don’t think I am going to make the training day I should be there in the evening for the Mozilla party, be sure to sign up.

Mozilla are going to be there in full force of course:

  • Myk Melez will be talking about developing with our new Add-ons SDK
  • Jorge Villalobos will be talking about how to update for Firefox 4
  • Mark Finkle will talk about developing for Mobile
  • Pascal Finette will talk about business development
  • Justin Scott will be in the main keynote as well as talking about making add-ons that people will love
  • Justin, Jennifer Boriss and myself will be giving an overview of Firefox 4 and some of the new features you should be excited about
  • Jay Sullivan will be on the panel talking about the future of the browser

I’m quite looking forward to this. It’s been a long hard slog getting to the end of Firefox 4 (and we aren’t even there yet!) and one of the problems of being focused on all the blockers all the time is that all you are seeing is the bugs that users are hitting and features that really should be there. It’s quite a pessimistic way to see the project. I remember last year at Add-on-Con being really inspired to see all the great things that add-on authors were doing with the tools we give them (or sometimes in spite of the tools we give them!). It really was quite an uplifting experience and I’m hoping it happens again this year, don’t let me down!

Minor change coming to the interface amIWebInstallListener in Firefox 4.0 beta 8

We’re past API freeze so any API changes should be getting announced and communicated. In this case the change is tiny and unlikely to affect anyone. Have you heard of the interface amIWebInstallListener? If not then you can probably ignore this.

If you’re interested it is effectively what the add-ons manager backend code uses to communicate messages about webpage initiated add-on installations. An add-on or application might provide its own implementation if it wanted to provide its own UI for installs. Or an add-on might call it to get the normal UI to show up for some other cases but this is pretty rare.

The change is a method addition, onWebInstallDisabled is called when a webpage attempts to install an add-on but the preference xpinstall.enabled is false so installation is denied.

You can read a little more in the bug report or in the source IDL.

Don’t miss an exciting opportunity to shape the future of Firefox 4!

You might have heard of this web-browser. It’s called Firefox. You may have also heard that a new version is due out soon. As my part in its development I have helped completely reshape the way the add-ons manager looks. The good news is that the large bits of the changes are pretty much done, pretty much all that is left is a bunch of UI tweaks and some small behaviour changes.

This is where you come in. Yes you, the one reading this post. If you have an urge to help out you can look through this list of things we want to get done on the add-ons manager before release, pick one out and go nuts. These are all things that shouldn’t require very much experience working on the add-ons manager or even Firefox in general, though of course some are easier than others. We have some great information about how to get started and also a great community who can help out if you get stuck. So if you’ve ever had a hankering to get involved this would be a great time to dive in.

Once again that list is here: http://bit.ly/duYWdP, give me or Blair a shout if you have questions or don’t know where to get started.