Hacking on Tilt

Tilt, or 3D view as it is known in Firefox, is an awesome visual tool that really lets you see the structure of a webpage. It shows you just how deep your tag hierarchy goes which might give signs of your page being too complex or even help you spot errors in your markup that you wouldn’t otherwise notice. But what if it could do more? What if there were different ways to visualise the same page? What if even web developers could create their own visualisations?

I’ve had this idea knocking around in my head for a while now and the devtools work week was the perfect time to hack on it. Here are some results, click through for larger images:

Normal 3D view

Normal 3D view

Only give depth to links

Only give depth to links

Only give depth to links going off-site

Only give depth to links going off-site

Only give depth to elements that have a different style on hover

Only give depth to elements that have a different style on hover

This is all achieved with some changes to Firefox itself to make Tilt handle more generic visualisations along with an extension that then overrides Tilt’s default visualisation. Because the extension has access to everything Firefox knows about the webpage it can use some interested sources of data about the page, including those not found in the DOM. This one I particularly like, it makes the element’s depth proportional to the number of attached DOM event listeners:

Give depth to elements based on the number of attached event listeners

Give depth to elements based on the number of attached event listeners

Just look at that search box, and what’s up with the two buttons having different height?

The code just calls a JS function to get the height for each element displayed in 3D view. It’s really easy to use DOM functions to highlight different things about the elements and while I think some of the examples I made are interesting I think it will be more interesting to just let web-devs come up with and share their own visualisations. To that end I also demoed using scratchpad to write whatever function you like to control the visualisation. You can see a screencast of it in action.

Something that struck me towards the end of the week is that it could be awesome to pair this up with external sources of data like analytics. What about being able to view your page with links given depth proportional to how often users click them? Seems like an awesome way to really understand where your users are going and maybe why.

I’m hoping to get the changes to Firefox landed soon maybe with an additional patch to properly support extensibility of Tilt, right now the extension works by replacing a function in a JSM which is pretty hacky, but it wouldn’t be difficult to make it nicer than that. After that I’ll be interested to see what visualisation ideas others come up with.

The Add-on SDK is now in Firefox

We’re now a big step closer to shipping the SDK APIs with Firefox and other apps, we’ve uplifted the SDK code from our git repository to mozilla-inbound and assuming it sticks we will be on the trains for releasing. We’ll be doing weekly uplifts to keep the code in mozilla-central current.

What’s changed?

Not a lot yet. Existing add-ons and add-ons built with the current version of the SDK still use their own versions of the APIs from their XPIs. Add-ons built with the next version of the SDK may start to try to use the APIs in Firefox in preference to those shipped with the XPI and then a future version will only use those in Firefox. We’re also talking about the possibility of making Firefox override the APIs in any SDK based add-on and use the shipped ones automatically so the add-on author wouldn’t need to do anything.

We’re working on getting the Jetpack tests running on tinderbox switched over to use the in-tree code, once we do we will be unhiding them so other developers can immediately see when their changes break the SDK code. You can now run the Jetpack tests with a mach command or just with make jetpack-tests from the object directory. Those commands are a bit rudimentary right now, they don’t give you a way to choose individual tests to run. We’ll get to that.

Can I use it now?

If you’re brave, sure. Once a build including the SDKs is out (might be a day or so for nightlies) fire up a chrome context scratch pad and put this code in it:

var { Loader } = Components.utils.import("resource://gre/modules/commonjs/toolkit/loader.js", {});
var loader = Loader.Loader({
  paths: {
    "sdk/": "resource://gre/modules/commonjs/sdk/",
    "": "globals:///"
  },
  resolve: function(id, base) {
    if (id == "chrome" || id.startsWith("@"))
      return id;
    return Loader.resolve(id, base);
  }
});
var module = Loader.Module("main", "scratchpad://");
var require = Loader.Require(loader, module);

var { notify } = require("sdk/notifications");
notify({
  text: "Hello from the SDK!"
});

Everything but the last 4 lines sets up the SDK loader so it knows where to get the APIs from and creates you a require function to call. The rest can just be code as you’d include in an SDK add-on. You probably shouldn’t use this for anything serious yet, in fact I haven’t included the code to tell the module loader to unload so this code example may leak things for the rest of the life of the application.

This is too long of course (longer than it should be right now because of a bug too) so one thing we’ll probably do is create a simple JSM that can give you a require function in one line as well as take care of unloading when the app goes away.

What is Jetpack here for?

Who are the Jetpack team? What are they here for? A lot of people in Mozilla don’t know that Jetpack still exists (or never knew it existed). Others still think of it as the original prototype which we dropped in early 2010. Our goals have changed a lot since then. Even people who think they know what we’re about might be surprised at what our current work involves.

Let’s start with this basic statement:

Jetpack makes it easier to develop features for Firefox

There are a couple of points wrapped up in this statement:

  1. The Jetpack team doesn’t develop features. We enable others to develop features. We should be an API and tools team. We’ve done a lot of API development, but we should work with the developer tools team more to make sure that Firefox developer tools work for Firefox development, too.
  2. I didn’t say “add-ons.” I said “features”. The APIs we develop shouldn’t be just for add-on developers, Firefox developers should be able to use them too. If the APIs we create are the best they can be, then everyone should want to use them. They should demand to use them, in fact. It follows that this should make it easier for add-ons to transition into full Firefox features, or vice versa.

We happen to think that a modular approach to developing features makes more sense than the old style of using overlays and xpcom. All of the APIs we develop are common-js style modules (as used in Node and elsewhere) and we’ve built the add-on SDK to simplify this style of add-on development. That work includes a loader for common-js modules which has now landed in Firefox and is available to all feature developers. It also includes tools to take directories of common-js modules and convert them into a standalone XPI that can be installed into Firefox.

The next step in supporting Firefox and add-on developers is to land our APIs into Firefox itself so they are available to everyone. We hope to complete that by the end of this year and it will bring some great benefits, including smaller add-ons and the ability to fix certain problems in SDK based add-ons with Firefox updates.

Mythical goals

Jetpack has been around for a while and in that time our focus has changed. There are a few cases where I think people are mistaken about the goals we have. Here are the common things that were talked about as hard-goals in the past but I don’t are any longer.

Displace classic add-ons

Most people think Jetpack’s main goal is to displace classic add-ons. It’s obvious that we’ve failed to do that. Were we ever in a position to do so? Expecting the developers of large add-ons to switch to a different style of coding (even a clearly better one) without some forcing factor doesn’t work. The electrolysis project might have done it, but even supporting e10s was easier than converting a large codebase to the add-on SDK. The extension ecosystem of today still includes a lot of classic addons, and the APIs we build should be usable by their developers, too.

Forwards compatibility

Users hate it when Firefox updates break their add-ons. Perfect forwards compatibility was another intended benefit of the SDK. Shipping our APIs with Firefox will help a lot, as the add-ons that use them will work even if the specific implementation of the APIs needs to change under the hood over time. It won’t be perfect, though. We’re going to maintain the APIs vigorously, but we aren’t fortune tellers. Sometimes parts of Firefox will change in ways that force us to break APIs that didn’t anticipate those changes.

What we can do though is get better at figuring out which add-ons will be broken by API changes and reach out to those developers to help them update their code. All add-on SDK based add-ons include a file that lists exactly which APIs they use making it straightforward to identify those that might be affected by a change.

Cross-device compatibility

There’s a theory that says that as long as you’re only using SDK APIs to build your add-on, it will magically work on mobile devices as well as it does on desktop. Clearly we aren’t there yet either. We are making great strides, but the goal isn’t entirely realistic either. Developers want to be able to use as many features as they can in Firefox to make their new feature great. But many features in one device don’t exist or make no sense on other devices. Pinned tabs exist on desktop and the SDK includes API support for pinning and un-pinning tabs. But on mobile there is currently no support for pinned tabs. Honestly, I don’t think it’s something we even should have for phone devices. Adding APIs for add-ons to create their own toolbars makes perfect sense on desktop, but again for phones makes no sense at all.

So, do we make the APIs only support the lowest common denominator across all devices Firefox works on? Can we even know what that is? No. We will support APIs on the devices where it makes sense, but in some cases we will have to say that the API simply isn’t supported on phones, or tablets, or maybe desktops. What we can do, again by having a better handle on which APIs developers are using, is make device compatibility more obvious to developers, allowing them to make the final call on which APIs to employ.

Hopefully that has told you something you didn’t know before. Did it surprise you?

After an awesome Jetpack work week

It’s the first day back at work after spending a great work week in London with the Jetpack team last week. I was going to write a summary of everything that went on but it turns out that Jeff beat me to it. That’s probably a good thing as he’s a better writer than I so go there and read up on all the fun stuff that we got done.

All I’ll add is that it was fantastic getting the whole team into a room to talk about what we’re working on and get a load of stuff done. I ran a discussion on what the goals of the Jetpack project are (I’ll follow up on this in another blog post to come later this week) and was delighted that everyone on the team is on the same page. Employing people from all around the world is one of Mozilla’s great strengths but also a potential risk. It’s vital for us to keep doing work weeks and all hands like this to make sure everyone gets to know everyone and is working together towards the same goals.

Look Ma, no restarts!

An extension installed without restarting Firefox

This is a screenshot of some of my latest (and most exciting) work on my project to rewrite the extension manager. I’ve just implemented support for a special kind of extension that can install (and uninstall, and enable, disable, upgrade and anything else you can think of) without the user needing to restart Firefox. This is of course to allow add-ons developed on the Jetpack platform to install without restarts but the feature is going to be available to any extension author, there are just some restrictions to how these extensions work.

The precise spec for this isn’t final yet so I’m not going to show you precise examples but the basics are that unlike normal extensions the platform won’t load anything from the restart-less extensions except for one file. This simple JavaScript file will be called when the extension is loaded or unloaded allowing the extension to either set itself up or tear every trace of it out of the app. This may seem limited but there is actually very few things that an extension can’t do from this (and hopefully we’ll get those fixed), it just involves more manual work. Maybe we’ll simplify some of that in the future, but for now it’ll be pretty raw and ready for use. Of course the Jetpack platform basically does all the loading and unloading work for you…