technical – Page 20 – Oxymoronical

Less technical

I’m going to be starting to post some more personal posts to my blog. Chances are these are not going to be very interesting to the average reader of planet so I’ve also tinkered with the categories so they won’t get syndicated there.

This means a couple of things. Firstly if you are actually interested in the posts then you’ll have to subscribe to the personal category on my blog. Secondly (and I’m guessing this affects no-one) the urls of the categories on my blog have changed a little so if you had direct feeds to them you’ll have to update.

Ruminations on a fortnight

It’s been a hectic couple of weeks for me and I wanted to touch on a few of the highs and lows before I forget them all.

It started with flights to get me to Toronto (for those of you that don’t know I live and work out of the UK). I was speaking at the Toronto Developer Day as well as attending a Firefox team work week (two things that conflicted more than I would have liked unfortunately). I normally manage to find direct flights but this time I had to connect through Amsterdam which wasn’t too bad, even if they seem even more mad for security than Heathrow. Incidentally terminal 4 at Heathrow is miles nicer than 3 where I normally come from, even if the 6 police officers armed with automatic weapons was a little disconcerting.

The Developer Day seemed to be a success and I hope all the attendees got something out of it. I’d encourage those that didn’t to let us know where we went wrong of course, not point in us running these things if they don’t give you anything. I think the people at my session found it useful, though I suspect I really need to work on my presentation style for such things. I feel kinda bad though that I’m not more confident about speaking and so could have taken some of the massive workload that Mark Finkle took on off his shoulders.

The rest of the week was spent with the rest of the Firefox team either figuring out concrete plans for the immediate future or working on those plans, interspersed with pizza, barbecue and poutine (Who’d have thought that Canadians would mash together some of the quick takeaway foods I used to get here and serve it in a restaurant!).

Next stop was Mountain View, via Detroit this time. I have to say that Detroit is a really nice airport to change at, even if it does mean that you’re probably flying Northwest if you do so.

I had a weekend to kill so on the Saturday a friend took me to a Renaissance Faire. I thought it might be a bit too geeky even for my tastes at first but it turned out to be a lot of fun, not sure you’d ever get me dressing up to attend like many do though. Fairly fun to watch Americans celebrating a history they never had.

Sunday I headed to the Winchester mystery house. I probably wasn’t in a good mood for it to be honest, jet lag had hit me and then the tour guide was a stereotypical over the top American tour guide trying to make every little oddity out to be spooky. Still it was an interesting place to walk around and the gardens were quite spectacular.

Tuesday I headed into San Francisco proper and went to meet up with the guys at Songbird. One of the things I really want to do is avoid writing the add-ons manager for just Firefox so it was good to chat with them about what they would like to see and how important they find the future plans that we’ve already toyed with.

That evening I had a great meal with some new friends at a German restaurant (yes I had to go to California in order to eat at a German place for the first time). I also got to experience the fun of driving out of the city back to MV at eight in the morning. I think I’ll be avoiding that in the future.

Thursday it was fun to attend one of the monthly labs meetups. Pretty cool to see plans for what is coming both in experimental stuff and in Firefox itself. I think it’d be really useful to stream these events so more people can get an insight in what is coming and why. Too often we can land things in the nightlies seemingly without any discussion when in actual fact a lot of thought does go into pretty much everything.

Finally Friday morning was painful. I maybe took people too seriously when they advised me on how long it would take to get to the airport and to check in, and then I added some for safety leaving me getting up at 5am and then ending up waiting at the gate for an hour and a half.

Seriously, why are so many people carrying on multple bags rather than checking them. I suppose I understand the risk of lost luggage but it is getting really annoying when I have to stick my single carry-on bag under my feet, decreasing the already sparse space there, because the overhead lockers are filled with trolley cases.

Northwest has to be one of the worst airlines I have flown. Poor food, very little movie selection, no tv episodes available at all. Thank god I have taken to putting movies onto my iPod so I always have something decent to watch.

Now I’m sat in a hotel room trying to stay awake for a bit longer. After I get back to Heathrow, after some 20 odd hours travelling (and I can never sleep on planes) it seems to be a good idea for me to sleep before taking the final 3 hour drive back home. But I also need to start the shift back to UK time. I shall probably be waking myself at around 3am for a leisurely drive back to Wales and then slowly shifting myself a couple of hours a day for the rest of this week.

Toronto Developer Day

In case you hadn’t heard Mozilla are running a developer day in a week or so’s time in Toronto. The first day is aimed at giving a good overview of the Mozilla platform and what technologies it offers. The second day will be a more hands on workshop where you can try your hand at extensions, applications and testing with friendly experienced people around to lend a hand when you hit problems. It isn’t far off now so get yourself signed up if you’re coming.

I’m in the midst of trying to put together what I’ll be talking about so if you have anything you particularly want to hear then make a suggestion.

The great bug triage

I was very excited to learn at the Firefox Summit that Rob Strong was handing over ownership of the Extension Manager module to me. He did great work making the extension manager what it is today but has lately had to be more focused on Installer and windows integration issues.

Last week I spent a large part of my time trying to get myself up to speed on all the old filed bugs, clearing out things that clearly aren’t going to happen and trying to consolidate others. In particular I made the effort to go over every single unconfirmed bug and either resolve as appropriate or request any additional information from the reporter. It is rather sad that many of these were issues that noone commented in after the initial report or even worse the reporter responded with additional information but people (including me) dropped the ball and nothing further happened.

This big triage has been quite useful. Firstly the number of unconfirmed bugs for the add-ons manager has dropped from the mid 70s to under 30 (I would give you exact figures and pretty graphs but bugzilla’s charting is broken). I expect it to drop even further next week when I resolve the issues where the reporter is no longer responding to bugmail. Secondly it has allowed me to spot a pattern amongst 6 or so filed bugs which strongly suggests a new issue that can break add-on installations. While I have yet to be able to reproduce it in a test environment I certainly have more information to go on than previously.

Finally I have been marking all the trivial little issues that I think we could fix, but are unlikely to be priorities. These are the kinds of things that if you were interested in helping out and getting started in developing for Mozilla you could start out on. Many will end up being a few lines of patch, just need to figure out where to put it and test it.

So if you are interested in tackling any of these please give it a go. You can find me on IRC (for reasons best left alone I appear as Mossop) or by email if you need any guidance on getting started. I’m going to try to keep this list updated as I find new bugs that fit there, I still have yet to go through all of the 263 confirmed bugs, maybe a job for next week.

Add-ons Manager session notes

My head is slightly less fuzzy now and I’m almost back on UK time so I thought it would be a good opportunity to give a quick review of how the Add-ons Manager session I ran at the summit went. I won’t go into too much detail, just a look at the main points of discussion and how I felt the room reacted to the ideas. You can also see the slides I prepared or the notes that Basil kindly took for me without me needing to remember to ask. Most of the session was about things we might want to implement and what problems/benefits we might encounter.

Add-on locale packs

The idea of developing the UI for locale packs specific to extensions received a less enthusiastic response than I had expected. This may have been because there were more add-on developers and less localisers in the room, but the general feeling seemed to be that there was not a large benefit and the potential complications, particularly around updating, could cause some large problems.

Dependency resolution

The current add-on dependency implementation is a very rough start. Being able to automatically find and install dependant add-ons for the user, with clear UI to show what is happening, seemed to be something that didn’t have many downsides. There were also suggestions that AMO could handle dependencies and deliver sets of add-ons as multi-xpi packages, though this could cause issues in upgrades.

Add-on conflicts

The moment I mentioned the suggestion of a system to mark two add-ons as incompatible with each other half the room said they wanted it. It was pointed out that some add-ons were already disabling things they don’t like on install (if you hear of that happening without appropriate warning please let us know). It was also quickly realised that under the author’s control it could just become a means to avoid having to work to fix incompatibilities and to disable the competition. The suggestion of Mozilla maintaining a central list of incompatibilities seemed like a good way to go.

Install without restart

No surprise that this was seen as useful, particularly in non-Firefox applications that don’t have the wonder of session restore. We also started talking about the potential for lower impact extensions that just had a small sandbox to work in with a little piece of UI which could make loading and unloading dynamically far easier.

Install/uninstall hooks

We discussed how the current mechanisms for running setup and cleanup code for extensions were poor, there is no way to deal properly with uninstall in all situations. It was felt that add-ons should be able to provide a full uninstall script that would even run if uninstalled when disabled or in safe mode, but would not run if the add-on had been blocklisted. We also talked a little about sandboxing what this script might be able to access.

Replace RDF

I was surprised that there seemed little to no resistance to the prospect of introducing a new plainer XML form for install.rdf and update.rdf given some of the comments I had when I blogged about it last. We would of course still support the old forms for a time and make converters between the forms available. There didn’t appear to be any benefit in using JSON instead of XML.

Much of the rest of the slides were skimmed over as we had unfortunately run out of time by this point, but thankfully that was most of the more important stuff completed anyway. For the immediate future I think introducing replacements for RDF into Firefox 3.1 is potentially on the cards. None of the other items were ruled out, far from it, though the locale packs seemed less interesting to everyone, and I think there needs to be some very careful thought about the prospect of running an uninstall hook for an add-on in safe mode or when disabled.

Thanks Canada (no, really)

It’s 6:30am Sunday UK time (though in my weird method of getting over the jet lag that works out as just after lunchtime) and I’ve just got home to Swansea. As many others are I’ve been thinking back over the great times I’ve had over the past two weeks. Needless to say I got out alive, as I believe everyone did (though apparently some are still stranded in Vancouver), despite Canada’s attempts to break us.

After a short visit with my sister trying to avoid getting licked to death by her new puppy I spent a week in Toronto at the Mozilla offices there. Thanks guys for having me, it was a fun week and I’m sorry I brought the Welsh rain over with me. Although I do have a shared office that I work from for most of the week I forget how good it is to actually be in a place where others are working on the same stuff. Maybe I’ll give in to Beltzner and move out there.

The summit was nothing short of amazing. So much larger than the last one and so much more enjoyable for it. As with all of these events I never got chance to do the things I had planned on and missed meeting many of those I wanted to, but met others I had never known and did more interesting things anyway. Dan and the rest of the organising team should be seriously proud with how the event went and how they coped with the upsets.

I think the Add-ons Management session I ran on Tuesday went pretty well all things considered. I have never really done interactive sessions before and combined with the hangover I was sporting I was a little nervous about it. Thankfully you can always rely on the Mozilla community to be talkative so in the end we actually ran out of time to talk over some of the ideas for the future of the Add-ons manager. Hopefully those there found it useful, I know I will take a lot from being able to get feedback and suggestions on my plans.

I guess the only minor disappointment (which is always the case since it would be impossible to avoid) is that there were many sessions I wanted to attend but conflicted with others. I hope there are some notes around for some of them so I can skim over what I missed.

Somehow I managed to find the journey home easier than normal. I suspect it helps that there were so many of us at the airport and I did the journey in a couple of stages to avoid the drive home immediately after getting off a plane which always turns into a dangerous race between traffic and my body trying to fall asleep.

I forgot how cool it is to drive down the motorways at 4am in the pitch black with nothing but good music and your thoughts to enjoy. Now, if only there were some shops open so I could actually get some lunch and a decent cup of English tea…

Keeping Track

In the past year or so that I’ve been working for Mozilla I’ve found myself slowly working my way through a bunch of different ways to keep track of all of the work on my plate. I still don’t think I’ve found the best way so I wondered what other people do to manage such things?

For a rough idea of my work, most of my work is bug oriented. Either bugs that I want to work on when I get some time, bugs I am actively working on, bugs I am waiting for review on, bugs I am reviewing, bugs I am waiting to check in, bugs I want to backport to a branch… and so on. Then there is other work like planning future work and working with extension authors to help resolve their problems.

A few different approaches I’ve tried:

Bug ASSIGNED status

Simplest of all, gives me a long list of stuff I am looking at. Totally fails for prospective work though and the list gets somewhat daunting.

Status whiteboard flags

I tried for a while sticking special flags into the status whiteboard that could then be spotted in special bug queries so I could see what state each bug was in. This kind of worked, though I imagine irritated some with bugspam as I maintained the flags and a couple of times other people changed the flags.

Bugzilla tagging

Bugzilla has this tagging feature that basically lets you create named buglists. Basically like using the whiteboard stuff without having to use the whiteboard. Sadly Bugzilla’s tagging UI sucks really hard.

Bookmark tagging

With Firefox 3 I can bookmark my bugs and tag them, and use smart bookmark folders as my buglists right? Well sort of, the folder list isn’t really enough detail and there are issues with manually created smart folders that kept losing my bookmarks.

Task management software

Finally bit the bullet and stuck all of this stuff into some task managent software (Omnifocus as it happens). This is so far going ok but seems to require more dedication to keep the list up to date, I think because of having to switch to a different application so much.

So what are other people doing?

Filing good Extension Manager bugs

The Extension Manager is pretty complex and so it can be difficult to gather the sort of information needed in a bug report to really diagnose what is going on. When the problem is related to extension installation, upgrade, uninstall or enable/disable, these suggestions should help get as much information as possible into a bug report.

Be specific in your description

While it may appear that your problem happens for “every add-on you try to install” or for “every website” the reality is that add-ons are complex things. A feature of the 20 add-ons you tried might not be present in the one add-on that the developer tests with. If you say precisely what add-ons you tried then the developers will test with those add-on which gives us a better chance of reproducing your problem.

The other part of this is to be specific in the steps to reproduce. Try to include what windows appear, what buttons you click on, when you restarted Firefox. Too much information is not a bad thing here, the more closely a developer can follow your exact steps the better.

Attach the cache files

In your profile directory are a set of files that the extension manager uses. Attaching copies of them after the problem has occured gives us an idea of what state the extension manager got itself into, ideally shut down Firefox straight after the problem and copy them out of the profile before starting Firefox again, then attach them to the bug report you file.

You want the files extensions.log (only in Firefox 3), extensions.ini, extensions.cache and extensions.rdf. If any don’t exist then say that in the report.

Turn on logging

If the problem is reproducible then you can turn on additional logging to get some more information on what the extension manager is doing. Type about:config into the address bar and look for javascript.options.showInConsole and extensions.logging.enabled. Make sure they both have the value true (double click to change).

Now open the Error Console from the Tools menu, clear it to start fresh and then perform whatever action causes the problem. Hopefully some messages will appear in the error console. Include these in the bug report.

Should AMO allow adverts for pay-for add-ons?

Assuming you agree that paying for some add-ons is ok then you have to ask what we do about people using AMO as a marketing platform. This is a tough question since we risk devaluing AMO as a website if it just gets filled up with adverts. I don’t believe that there is an official policy on this. It is such a rare issue right now that maybe one isn’t necessary, but here are are my thoughts on what such a policy might say.

In order to even advertise a pay for add-on the developer must have uploaded a real add-on. If this add-on is just junk, i.e. doesn’t really do anything and is just a means for getting a page onto AMO, then it should be removed. I have seen add-ons like that (that weren’t advertising anything) getting removed as a matter of course anyway so I think this is already working.

The next question is whether the add-on is a basic version of the premium add-on. If so then I think it is totally fair that the developer can include text in the page to note that a pay for version is available elsewhere. I’m less enamoured with the idea of including screenshots of the pay-for version on AMO. I think regardless of how well labelled they are there is a strong risk of confusing the user there.

Trial versions of add-ons pose a question. Should AMO be allowing add-ons on the site that will intentionally stop working after a period of time, or that list features in menus that only pop up a message about needing to pay to activate? I don’t think so. I believe a user should be safe to download add-ons from AMO that are whole products and they can count on to continue to work.

If an add-on on AMO is unrelated to a premium version by the same author then more care has to be taken with advertising different add-ons on the same page. I don’t see a problem with a short note at the bottom mentioning that other add-ons from the author are also available elsewhere, but I don’t think mentioning what the other add-ons are or what they do is a good idea.

Overall I think short notes are the way forward, anything where the add-on is more of an advert than an add-on in its own right is something that needs to be carefully looked at.

Update: A couple of commenters have raised the question of whether AMO should just include pay for add-ons in its listing, something that I completely failed to consider. I’m not sure which side I am on on that yet. I think if handled correctly it could work, but maybe I’ll think on it and do a follow up post on the subject.

Should developers charge for add-ons?

I’m very surprised that many people are questioning whether developers should even be allowed to charge for add-ons. Traditionally it is true that add-ons have been freely available, but I’ve known of pay for add-ons for at least 2 years now (that add-on is still available so must be doing ok) and I imagine there have been some around for longer. Some companys’ entire business rests on their Firefox add-ons. That sort of situation wouldn’t exist if no-one ever paid for add-ons in some way.

I believe that developers should feel welcome to do whatever they please. If they believe their work is good enough to charge for and users think it is good enough to pay for then great. There are probably more funded add-ons out there than most people realise. Charging for download is just one mechanism for recouping some cash for development. I use adverts on my website to help fund some of my development. Some developers ask for donations, others are supported by virtue of their add-on driving traffic to their website. Do you think Google give you the Google Toolbar for free purely out of the kindness of their hearts?

I’m no expert in the matter, but I suspect there would be something seriously wrong with the add-ons ecosystem if it was only populated by free items. I’m pretty sure that people actually getting paid for their work will feel more driven to release higher quality add-ons as well as put more pressure on Mozilla to fix the issues that they find in the platform during their development. This helps all developers, not just those charging their users.

I suspect that people who don’t like the idea of add-on developers making some money out of their add-ons simply don’t fully understand just how much work developing add-ons involves. It takes lots of time and effort to get something to the level of quality you are happy with releasing. Those that aren’t developing for a company are literally taking weeks and months of time away from their friends and family to provide users with something useful. You can argue that if they enjoy the work then they shouldn’t be charging for it, I enjoy my day job yet I still expect to get paid. Believe me that working through all the little bugs your users find but often you can’t see is generally not much fun compared to the actual development of a new feature. Money provides that little extra incentive that might mean the difference between an add-on continuing to be maintained and it dying out.

If you don’t like that a developer is charging for their work, even if the same add-on was previously available for free then I don’t really think you have any right to complain. If you don’t think that what they are offering is worth the price then don’t buy it. I wonder, if you were to try disabling all the add-ons you are using for a day, how many of them would you so sorely miss that you could see yourself paying a small price for them.