Planning for the future

For some time now I’ve been throwing around ideas for new features that we may want for the add-ons manager. After lots of thought and trying to take on board comments from anyone that would pipe up I decided it was high time to put together an actual plan for what features I have decided should be pursued and what order to start tackling them.

The roadmap itself should be pretty self explanatory, just remember that nothing is ever set in stone. I’ll try to keep it updated as things change, features seem less important or new items are added. But for now this is how I see the add-ons manager evolving over the course of about the next year or so, which may be the next two versions after Firefox 3.1.

Obviously the ordering of the goals may not be quite to everyone’s taste, but I’ve tried to strike a balance between getting new features done and getting groundwork laid to make things better in the future. I’m happy to take any comments people might have into consideration for altering the plan, but chances are I won’t be making any big changes unless you have some pretty damning evidence that a change is necessary :)

Dividing the labour

Faaborg has started a discussion on the newsgroups on the relative importance of polish and blocker bugs (I’d provide a link bug Google Groups seems to be refusing to acknowledge existence of the post). I have been for quite some time now very focused on the in-depth blocker issues, partly because that is where all the interesting work is for me, but also because I think it makes more sense for me or one of the other guys with good understandings of how the extension manager works to be making these changes rather than throwing others in at the deep end.

This has of course left little time for me to fix the trivial issues that Faaborg talks about. The things that users probably notice more and make the product look complete or half-done. This is why I’m really pleased that when I posted a list of simple bugs that anyone should be able to tackle there was a fairly good response. Of the 12 bugs on the original list, 9 have been fixed. I wanted to extend my thanks to those guys that took the time to work on those, some of them probably ended up being a little harder work than anticipated.

Of course the polish list is almost never ending. I’ve been adding more to the list of good first bugs, if anyone is interested in helping out then don’t be afraid to just email me or find me on IRC as Mossop and ask how much work something really involves.

Finding API Changes

I started a small discussion on the newsgroups the other day wondering about the numbers of API changes that had landed on trunk since 1.9.0. It was a short discussion, maybe partly because all I had to go on were the vague metrics that mercurial could give me. It also evolved a little into a discussion of how add-on developers we’re meant to find out about API changes. It’s true that as developers we are getting better at trying to announce our changes as much as possible, but inevitably we miss some, and if you don’t track the announcements what then?

Yesterday on IRC ted jokingly made the comment that I could use the idl parser than bsmedberg wrote in python a short time ago to find out more about the API changes. I’ve been hard pressed on other work lately so it wasn’t long before the prospect of knocking together a little side project had me too interested to do anything else. In a short space of time I had something that could parse all the in-tree idl files into a sqlite database that I could then query. It wasn’t long after that that I got a small web app written to display the content of the database, but I spent some more time today getting it a bit more useful before I showed it to the rest of the world.

So here it is, the API viewer. You can look at any interface in any of the platform versions that I’ve bothered to scan. And here is the more interesting bit, you can compare two platforms to see what interfaces changed. Not happy there? Well you can also compare an interface in different platform versions in a sort of diff like display.

I should note that it is only indexing the core and Firefox interfaces for now, and it’s close to indexing all of the idl files from the sources, many of which don’t make it into the final build (which makes life interesting as it turns out some of them are invalid xpidl!). So you can’t trust that interfaces you find are necessarily useful etc.

The web app is really a prototype right now. Navigation is probably a bit clunky and it’ll need more work to be really useful but I wondered what other people thought of it. So what do you think? Is it useful? What other interesting information might we be able to grab automatically like this?

Update: I’ve added details of the current trunk, 1.9.1b2pre as of today so you can see what changes are coming since the last beta. Not sure how often I’ll keep that up to date though.

Frustration

For a while now I’ve been in a bit of a rut. I knew I was there I just hadn’t quite got the motivation to pull myself out of it. It wasn’t such a bad place to be, but the difference of being just ok with the way things are going and being exactly where you want to be and with the people you want to be with is a pretty large one. I’ve been exceptionally lucky that over the last few months a bunch of things have happened that will help take me to where I want to go and start some new and exciting changes in my life.

The problem is that almost none of them were instant changes, they are all gradual and still in the process of happening. Some of them may not even work out the way I’d like. It’s disconcerting to see where I want to be on the horizon yet now I don’t have control of getting myself there, I have to wait to for the cogs to turn. Now don’t get me wrong, I know that there are perfectly good reasons for having to wait, I don’t expect to be able to just move out to the States at a moments notice for example. But the wait and not knowing exactly what is going on is leaving me extremely frustrated. Do I assume everything is going to work out and look forward to my new life only to be disappointed later? Do I assume the worst and then not be prepared when it comes?

Frustration is something I’m normally fairly familiar with. It is after all pretty much the standard state of affairs for anyone working on Mozilla code, you have to wait for responses to design questions, you have to wait for patch reviews, you have to wait for the tree to be clear enough to land. All of this you slowly get used to, but it is still annoying. The problem is that now this is going on in pretty much all aspects of my life and is starting to drive me into the ground. I’ve been getting even less sleep than normal which is also leaving me kind of cranky.

Thankfully I have been able to arrange myself a temporary escape. Next month I’m going to go spend a week and a half with an friend in Austin, and while I’ll still be working for much of that time I can’t wait to hang out with him again. Also, just because it was fairly cheap, I’ll hop over to San Francisco for a long weekend with some other friends. Hopefully that will de-stress me enough to last the following months. And who knows some things may even have resolved themselves by then, or at least become a little clearer.

Updating wordpress posts without updating

Dear web (I’m not going to insult you and call you lazy like some people I’d mention),

Why is it that even if I’m just adding a tag to a wordpress post it deems it necessary to update the date of the post causing places like planet to republish the post? Surely there must be some way around this that I’m missing?

The Long and winding road

So as I’ve said before I’m planning on moving somewhere new. I’ve finally made up my mind and I’ll be trying to relocate to Mozilla’s Mountain View office. I say trying because of course just deciding is only the first bit, there are still visas to be arranged, contracts to be agreed, accommodation to find. It’s going to be a long road and I’m barely started.

Hopefully the guys in the Toronto office don’t hate me too much for snubbing them. While they are awesome and I would really love to work out of that office, I think I’m going going to get a better social life outside of work around San Francisco. I have friends inside and outside the city who I know will drag me out so I’m not at home alone every weekend like I am now.

I’ve been trying to work out what it’s actually going to cost me to live around the office. I’ve found it kind of strange that I can’t find any cost of living comparisons between here and there. Maybe I’ve missed something entirely but it all seems to be comparing US cities to each other, or UK cities. Never comparing US to UK. Seems like a bit of a gap in the market to me. Anyone have any good ideas for comparisons? I guess if there is a city in the US that works out somewhat similar to a place in the UK then it’d at least give me a rough guess. As it is I’m left trying to estimate all my bills and work it out that way with pretty much no clue how various things work out there.

Random music

I’ve known for a while that I’m not terribly good at varying the music I listen to. Even within my own music collection I tend to have a few favourite albums of the moment, and while those favourites vary I rarely listen to anything else. Except when I have the radio on, and then it is your usual popular stuff, some good, some pap. I still pick up new things from friends occasionally but I never really go out of my way to vary what I’m listening to. I started to change that a little over the last few weeks.

First I tried sticking most of my music collection into a playlist, putting it on shuffle and just listening through it. I didn’t get very far, maybe a couple of hundred tracks in. While it was pretty good to hear lots of stuff that I hadnt listened to in absolutely ages it wasn’t so great that it just kept jumping from genre to genre. I quite like listening to an entire set of songs in one style. Maybe I could figure out how to make it just randomly play entire albums or something.

Second I experimented with iTunes’ new “genius” feature. I actually used this because I needed a good playlist for going to the gym. It takes a certain kind of music and the theory was that by choosing one good track, iTunes would make me a playlist with similar stuff. It was pretty bad. I’m not sure how it matches up tracks but it certainly isn’t by music style. Who would put together Foo Fighters and Newton Faulkner?

Finally I have been trying Pandora. This is a pretty cool service all in all. I have no idea how they fund it just seemingly with ads, but you give it an artist and it just streams you music to listen to that is similar. It does a good job of matching up the music styles. The only place where I felt it fell down was the lack of variety. Maybe I was just unlucky but it seemed to just flip flop between about 4 artists for each channel I created, all of them fairly mainstream artists too, nothing particularly new, I owned most of the songs it was choosing to play for me.

Pandora is certainly pretty good for making me listen to new stuff but I wonder what else there is out there that I might be missing. Unfortunately I’m too lazy to go hunting so I hope someone can tell me…

Status quo ante

Looks like I can finally say that my body clock is fully switched back to UK time. I can be sure of this because I spent 2 hours trying to get to sleep last night. That hasn’t happened since I flew out to Toronto over 3 weeks ago. I had rather hoped that after my trip had resolved all the issues I was stressed about that I might be able to get myself back into some sort of sensible sleep pattern, but apparently not.

What is also back to normal is the wonderful Welsh weather. As I speak the drizzle is hammering against my window, as it has been for most of the last week. There are many things about this city I wll miss, but many I really won’t. I guess I still have to survive the winter though.

Sky Anytime, programmes you can’t watch whenever you don’t want to watch them

At my old house I used to have cable TV. Provided first by NTL who then got bought out by Virgin Media. In many ways they weren’t so good, however where Virgin excelled was in providing an excellent on-demand service. With the right TV package you could get instant access to the last 7 days highlights (or for some definition of) and entire series of popular programmes.

Sky can’t quite match the kind of choice Virgin can give, they just don’t have the ability to stream data on demand to an individual home. Instead they have come up with a kind of a halfway house solution. So long as you have their PVR (Sky+) you can switch on the Anytime service. You sacrifice a bunch of space on the PVR but overnight it downloads and stores programming, leaving you with a choice of maybe 10 programmes or movies to watch whenever you like.

The first downside to this is pretty obvious. All the thing is doing is just recording programmes you might want to watch while you sleep. Well I can set the PVR to record programmes I definately do want to watch while I sleep manually. But I guess maybe this will record things I wouldn’t have normally and convince me to watch something I wouldn’t normally, who knows.

Now for the next problem. Each downloaded programme comes from a tv channel, obviously. The programmes it downloads seem to just be a set list for the period, from all of the channels Sky offers. Now some of the channels Sky offers I do not subscribe to. Yet it still downloads programming from these channels. So does that mean I can get a sneak look at something from the films channel I dont normally have? No, what it means is that and entire movie that I’m not allowed to watch is taking up space on my PVR.

So basically, for the small sacrifice of losing space on my PVR I get to choose from programmes that 75% of the time I probably don’t want to watch, 20% of the time can’t watch and probably 5% of the time I have set to record myself anyway.

*all probabilities totally made up.

How extensions can slow down Firefox (my dirty little secret)

Tab Sidebar is probably my favourite extension that I’ve created. It is certainly the most polished, thanks mostly to other people pushing me to make it so. For those that haven’t used it it creates a thumbnail preview of all of your tabs in the sidebar. The thumbnails automatically update whenever the page changes, even things like popup menus generally show up. This automatic updating comes at a cost though.

In order to detect changes to the page content the extension mostly uses DOM mutation events. These are in theory sent out whenever a change to the page’s structure happens, which covers adding/removing content, style changes, text entry and all manner of things. What I wasn’t aware of when I first used this technique was that Gecko performs certain optimisations when there are no DOM mutation listeners registered for a document (the normal case). Simply the presence of a mutation listener impacts the speed of any DOM manipulation by the page. How much? Well nothing like a graph to illustrate things.

Performance impact of mutation listeners

Performance impact of mutation listeners

These are numbers gathered from the Dromaeo test suite, averaging 5 runs with and without Tab Sidebar installed. A little perf loss is inevitable as the extension must perform some checks on the mutation and occasionally repaint the thumbnail, but nowhere near the sort of regression that actually occurs. I’ve combined some of the test results for simplicity but it is pretty clear that while the regular JS tests are essentially unaffected, the DOM tests can have wildly different results. Don’t ask me why DOM Queries get faster with the extension installed, but DOM modifications are a nice round 4 times slower.

That isn’t to say that using mutation listeners is a complete sin to be avoided at all costs. In some cases they are the only safe option. I discovered this problem some time ago and never before found a better way to detect the changes for Tab Sidebar. Last I checked Firebug uses them in similar ways. The point I think is that there are many features in the platform that can have unexpected side effects.

Of course I guess I wouldn’t be making this post if I hadn’t finally come up with a solution. While I’m no longer actively working on my extensions, I’m afraid I couldn’t resist the opportunity of the new MozAfterPaint event that landed on trunk recently. Quite simply whenever an area of the page is repainted on the screen this event is dispatched with details of what the area painted was. This is absolutely perfect for Tab Sidebar, which isn’t surprising since the use case it was written for is essentially what Tab Sidebar is doing anyway. After a few hours of ripping out most of the event handling code I now have a working prototype that redraws solely based on this event, only painting the areas that might have changed.

Performance impact of the MozAfterPaint listener

Performance impact of the MozAfterPaint listener

Boy does it work well. As you can see from the graph essentially all of the performance loss is gone. For some reason the faster DOM queries are still there (who am I to complain if my extension makes Firefox run faster!). And what’s more the previews are now updated not just for DOM changes, on OSX at least it even sees repaints of plugins. It is quite bizarre to be watching a movie on youtube and see the little thumbnail at the side showing the movie as well.

I guess now I have whetted your appetite it would be unfair not to make the test version available. Use at your own risk of course, the sheer amount of code change means there are likely some problems lurking, but you can install Tab Sidebar 2.5a1 and see how it goes. If you set the preference extensions.tabsidebar.selecteddelay to -1 then it won’t even delay before repainting and just repaint the thumbnail as soon as the main page is painted, this actually seems to work out best for me, particularly on sites with fast animations.