mozilla – Page 19 – Oxymoronical

Filing good Extension Manager bugs

The Extension Manager is pretty complex and so it can be difficult to gather the sort of information needed in a bug report to really diagnose what is going on. When the problem is related to extension installation, upgrade, uninstall or enable/disable, these suggestions should help get as much information as possible into a bug report.

Be specific in your description

While it may appear that your problem happens for “every add-on you try to install” or for “every website” the reality is that add-ons are complex things. A feature of the 20 add-ons you tried might not be present in the one add-on that the developer tests with. If you say precisely what add-ons you tried then the developers will test with those add-on which gives us a better chance of reproducing your problem.

The other part of this is to be specific in the steps to reproduce. Try to include what windows appear, what buttons you click on, when you restarted Firefox. Too much information is not a bad thing here, the more closely a developer can follow your exact steps the better.

Attach the cache files

In your profile directory are a set of files that the extension manager uses. Attaching copies of them after the problem has occured gives us an idea of what state the extension manager got itself into, ideally shut down Firefox straight after the problem and copy them out of the profile before starting Firefox again, then attach them to the bug report you file.

You want the files extensions.log (only in Firefox 3), extensions.ini, extensions.cache and extensions.rdf. If any don’t exist then say that in the report.

Turn on logging

If the problem is reproducible then you can turn on additional logging to get some more information on what the extension manager is doing. Type about:config into the address bar and look for javascript.options.showInConsole and extensions.logging.enabled. Make sure they both have the value true (double click to change).

Now open the Error Console from the Tools menu, clear it to start fresh and then perform whatever action causes the problem. Hopefully some messages will appear in the error console. Include these in the bug report.

Should AMO allow adverts for pay-for add-ons?

Assuming you agree that paying for some add-ons is ok then you have to ask what we do about people using AMO as a marketing platform. This is a tough question since we risk devaluing AMO as a website if it just gets filled up with adverts. I don’t believe that there is an official policy on this. It is such a rare issue right now that maybe one isn’t necessary, but here are are my thoughts on what such a policy might say.

In order to even advertise a pay for add-on the developer must have uploaded a real add-on. If this add-on is just junk, i.e. doesn’t really do anything and is just a means for getting a page onto AMO, then it should be removed. I have seen add-ons like that (that weren’t advertising anything) getting removed as a matter of course anyway so I think this is already working.

The next question is whether the add-on is a basic version of the premium add-on. If so then I think it is totally fair that the developer can include text in the page to note that a pay for version is available elsewhere. I’m less enamoured with the idea of including screenshots of the pay-for version on AMO. I think regardless of how well labelled they are there is a strong risk of confusing the user there.

Trial versions of add-ons pose a question. Should AMO be allowing add-ons on the site that will intentionally stop working after a period of time, or that list features in menus that only pop up a message about needing to pay to activate? I don’t think so. I believe a user should be safe to download add-ons from AMO that are whole products and they can count on to continue to work.

If an add-on on AMO is unrelated to a premium version by the same author then more care has to be taken with advertising different add-ons on the same page. I don’t see a problem with a short note at the bottom mentioning that other add-ons from the author are also available elsewhere, but I don’t think mentioning what the other add-ons are or what they do is a good idea.

Overall I think short notes are the way forward, anything where the add-on is more of an advert than an add-on in its own right is something that needs to be carefully looked at.

Update: A couple of commenters have raised the question of whether AMO should just include pay for add-ons in its listing, something that I completely failed to consider. I’m not sure which side I am on on that yet. I think if handled correctly it could work, but maybe I’ll think on it and do a follow up post on the subject.

Should developers charge for add-ons?

I’m very surprised that many people are questioning whether developers should even be allowed to charge for add-ons. Traditionally it is true that add-ons have been freely available, but I’ve known of pay for add-ons for at least 2 years now (that add-on is still available so must be doing ok) and I imagine there have been some around for longer. Some companys’ entire business rests on their Firefox add-ons. That sort of situation wouldn’t exist if no-one ever paid for add-ons in some way.

I believe that developers should feel welcome to do whatever they please. If they believe their work is good enough to charge for and users think it is good enough to pay for then great. There are probably more funded add-ons out there than most people realise. Charging for download is just one mechanism for recouping some cash for development. I use adverts on my website to help fund some of my development. Some developers ask for donations, others are supported by virtue of their add-on driving traffic to their website. Do you think Google give you the Google Toolbar for free purely out of the kindness of their hearts?

I’m no expert in the matter, but I suspect there would be something seriously wrong with the add-ons ecosystem if it was only populated by free items. I’m pretty sure that people actually getting paid for their work will feel more driven to release higher quality add-ons as well as put more pressure on Mozilla to fix the issues that they find in the platform during their development. This helps all developers, not just those charging their users.

I suspect that people who don’t like the idea of add-on developers making some money out of their add-ons simply don’t fully understand just how much work developing add-ons involves. It takes lots of time and effort to get something to the level of quality you are happy with releasing. Those that aren’t developing for a company are literally taking weeks and months of time away from their friends and family to provide users with something useful. You can argue that if they enjoy the work then they shouldn’t be charging for it, I enjoy my day job yet I still expect to get paid. Believe me that working through all the little bugs your users find but often you can’t see is generally not much fun compared to the actual development of a new feature. Money provides that little extra incentive that might mean the difference between an add-on continuing to be maintained and it dying out.

If you don’t like that a developer is charging for their work, even if the same add-on was previously available for free then I don’t really think you have any right to complain. If you don’t think that what they are offering is worth the price then don’t buy it. I wonder, if you were to try disabling all the add-ons you are using for a day, how many of them would you so sorely miss that you could see yourself paying a small price for them.

Money money money

The question about developers charging money for the use of add-ons seems to be getting brought up again lately. One of the more active theme developers has started to charge for premium versions of their themes, leaving free basic versions still available on addons.mozilla.org. There are a couple of different issues with this, both of which many users are taking exception to. Apparently once I started writing about this I couldn’t stop so I’ve split this out into a couple of different posts to follow.

Add-on Developers, Get the News you Crave

Being an add-on developer can be hard work. So many different places to look for information about releases, new features and changes to the platform. How can you be expected to keep up with all this?

Well starting next week hopefully we’ll be taking a big step in the right direction. Mozilla are launching about:addons, a newsletter dedicated to getting add-on developers all the important information they need. There will be announcements about when to check your add-ons against new releases of applications, new features added to addons.mozilla.org, plans for the future that you can get involved with and early warning of API changes.

So what are you waiting for? Go and sign up to receive the newsletter direct to your inbox. We’ll also be posting each issue to a blog but the email is more convenient in my opinion. You can of course unsubscribe any time you like. The newsletter should be out about once a month or occasionally more often if there is something important to announce.

Why not let me know if there are important kinds of news that we should be including in the newsletter.

What’s the Future for Add-ons Management?

With Firefox 3 getting ever closer to release it is time (well, past time) to start thinking about the future. The Extension Manager has seen quite a number of improvements since Firefox 2. Many were designed to be invisible, generally improving stability and fixing oddities. Some are extremely visible such as the new security requirements and the addons.mozilla.org integration. The question is, what’s the next step for add-ons management?

Here are a few ideas that are floating around my mind to get you started:

Installing add-ons without restarts
Presenting more detail in the install dialog
Simplifying the UI
Deprecating install.rdf and replacing with a simpler xml or json format
Add-on dependencies with automatic resolution

So what have I missed? Please keep any comments restricted to add-ons management, either as a user installing and using add-ons or as a developer writing add-ons. While I am interested in the future of Firefox as a whole I don’t want this to be a mass feature request for the product.

Why 2 SDKs are better than 1

In the past the Gecko SDK was somewhat limited. You could compile against it, but only if you were using frozen components, of which there are exceptionally few. You can build an application with only them, but I’d be startled if any moderately complicated app or extension gets by with only them. Thankfully this has changed for 1.9 and the new style SDK contains all interfaces and headers, frozen and unfrozen. This gives you access to lots more, though has the minor disadvantage that you have to keep an eye on what you are using as it could break in the future.

Just one snag with this new SDK though. The Mac version was missing something. Like a bunch of the tools and the XUL framework snapshot. It turns out that the method of packaging the SDK really didn’t work too well when you’re building a universal XULRunner, which is of course what we do.

After a bunch of work by plasticmillion and myself and a bunch of reviewing and generally dealing with our pesky questions by bsmedberg, the Mac XULRunner tinderbox is now producing working SDKs for every build. And I do mean the plural there. Building just one unified SDK is pretty tricky, even manually, let alone trying to make the Mozilla build system do it. Instead two SDKs are build, one for the powerpc architecture and one for intel. This actually works out well if you are producing an unified application using the Mozilla build system. All you really need are these magic lines in the mozconfig for your application:

ac_add_app_options ppc  --with-libxul-sdk=<path to powerpc sdk>
ac_add_app_options i386 --with-libxul-sdk=<path to intel sdk>

I’m just about to update the McCoy application to use the SDKs rather than having to build a full XULRunner alongside. This will make building McCoy simpler, particularly a unified Mac version which for the initial release I pretty much had to stitch together by hand. It will also I hope resolve some stability issues on Linux. I hope to have a new version out soon with some popularly requested new features.

I should note that the powerpc SDK is not a pure powerpc SDK. It is designed to be used to compile the powerpc portion of an applicaton using a build machine that is intel based. Unfortunately some of the important build tools cannot be cross-compiled for the powerpc architecture since the dependencies are only one architecture. Hopefully this will suit the majority of people.

On Timezones, Testing and Deals with the Devil

Timezones make my head hurt. Not the concept, that is easy, but writing code that works correctly in different timezones. Throw daylight savings correction into the mix and you’ll often find me in a corner rocking gently. The problem of course is that I frequently have to deal with data that isn’t just the standard unix timestamp (or various orders of magnitude away from). A number of times a load of information has been thrown away, never to be recovered which makes life interesting. You end up having to decide which is the best path to take when all of them are wrong in various ways.

So why bring this up? Well America has just jumped to daylight savings time. And wouldn’t you know it, a few of Mozilla’s automated tests failed. Which is awesome of course. Always good to know something is up. The tricky thing with test failures is figuring out why they are failing. The automated tinderbox are remote machines you generally don’t have access to to debug on. This can be pretty unhelpful if the test doesn’t provide enough logged information. I’ve been pretty guilty of this myself, while a test that spots problems is fantastic, what is even better is one that provides enough information in the log to figure out why it failed. Think of it this way. If you write a test that (months, years) later fails then you’ll likely be asked why. What information do you need to find out? A few suggestions I’ve learned the hard way:

Always log the found and expected values when comparing (do_check_eq and similar does it for you but there are cases where you don’t use that).
If you have a loop in the test be sure to log what iteration the test failed in. A message that “false wasn’t true” isn’t a lot of use when that check is performed a hundred times.
If it is a long test why not just log a few checkpoints along the way, if the test crashes somewhere at least you’ll have a rough idea of where.

I will probably come back with more suggestions by the end of the week. I needed approval for a late fix I needed to land on Saturday and shaver was the only driver around at the time. He is often getting on at people (quite rightly) for not writing tests for the code they submit, so I made a deal with him that in return for the approval that I’d spend 2 days this week writing tests. I expect the majority of them to be testing for the UI side of the add-ons manager which is essentially untested right now and right now I have no clue how to go about testing. Should be fun!

Trimming the Fat

As I mentioned in my last post and as I know everyone is well aware, tinderbox is starting to feel the strain. You already can’t just glance at it to see what is going on. The Firefox tree is currently showing build and test results from a total of 23 machines. The interesting thing is that not all of these machines are actually doing any compiling. The rest are merely running tests on builds produced by the other machines. That isn’t to say that those test results are less important but I wonder whether it is worth treating these differently.

Thinking about this difference I’ve spent a short time knocking together an experimental view of tinderbox. The idea is simple, the test boxes that aren’t actually compiling don’t get a dedicated column. We still want those results though, and luckily there is somewhere to put them, on the build that they are testing. So every actual compile ends up as a box within which can be multiple results from testers. This leaves us a very manageable 9 columns, 3 per platform.

This is quite early stages, the physical perf and test numbers aren’t visible yet, there are a couple of bugs I know about, it doesn’t auto update and I wouldn’t recommend you really use this, but I am interested to know what people think before I proceed any further with it. There are a few questions in my mind like how to colour the main box of the build (currently the worse result from the compile and all the test results), how to colour the header (I suspect the worst of the most recent results from all of the machines reporting in that column), where to even put the full perf/test numbers (there isn’t really room in some of the fast nightly boxes). More importantly is this even readable? I think it is good for some things, often tinderbox gets confusing when you see a talos result fail long after the nightly box went green, here that talos red would be back on the nightly box that failed. I can imagine problems trying to figure out why the header is red when it’s actually the talos results from build a couple of times ago that is causing it.

Anyway without further ado, take a look at Tindermerge and let me know your thoughts.

Visualising our Labours

Well it has been a mad few weeks. Between landing the new Get Add-ons pane, sheriffing the Firefox 3 beta 3 freeze, decimating old parts of the code and working through follow-up issues from the Get Add-ons pane and not to mention two 10 hour flights it really has been busy. Thankfully I’ve not just been totally buried under code, I’ve had some small spare time to tinker with a few other things (I need to to keep my sanity).

We’re getting better and better at adding quality control to our code. The QA team do their awesome work, we have some 50,000 automated unit tests now (depending how you count) and our tinderbox now has more machines running performance tests than it does machines building the Fox. One problem that all this brings though is a bit of information overload. The tinderbox is now so large that you can no longer glance to see if you can check in or not. Goodness knows how many pages you need to view to truly monitor all of the performance graphs. So, spurred on by Jonathan’s performance dashboard, I’ve been tinkering with some ways to present some of this data.

First off there is the old-style performance machines, pre-talos. These often find regressions first, because they cycle fast. Apparently that is going to improve but until then these old machines have useful data. Unfortunately the graphs they produce are … well lets just say pig ugly! It didn’t take too long to knock up something very basic, but nicer looking and allowing multiple results on the same graph. You can see it graphing the live data (yes it will appear blank for the age it takes to draw). Unfortunately the graph kit I’m using seems a little slow and not all that functional, but that could be changed. Then again I have just found out that new work on the graphs server pretty much blows this out of the water!

So that’s all pretty and good but doesn’t bring a much new to the table. Graphs are good but what we want to be able to do is relate the performance changes to the code that caused it. At this point Johnathan was kind enough to point me at Timeplot. This is a rather neat tool that lets you visualise numerical data on the same graph as defined events. That’s kinda perfect for this. It took me a little longer to hook this one up given the multiple sources and the kinda weird API that can’t normally load from anything other than a remote text file, but here is the result. You can see the performance data of course (Ts from the linux perf box), then each vertical band is a checkin which you can click on for more details.

In the end the timeplot is not massively useful. When you get down to it we tend to check in far more frequently than we gather performance information. I’ll be interested to see if the fast run talos boxes will improve matters. If we had many of them then we can start correlating the regressions across them and thus far reduce the range in which to look for potential causes. I’m interested in other ideas people might have for visualising these kinds of data, I think finding the right way to visualise our information is the key to managing things in the future.

Another thing that needs help is the tinderbox of course. While I was in Mountain View last week Johnathan and I did a quick brainstorm of what the tinderbox is used for. We think it is being used for too many different things and if we split the display out into just those tasks that individuals needed then it would be far more manageable. Maybe you have your own comments on uses we didn’t come up with.

Just finally, after playing with timeplot I was still in a tinkering mood. One thing I periodically have to do is look back over what work I did on specific days. Bugmail is a good indicator for this. Now though I am working in my own Mercurial repository which means I have a full checkin log of my own to look over. Given that it is interleaved with the full Mozilla CVS checkin log it still a little arduous to scan through. But know I have the tools and after not long I can know skim over my previous checkins and even see in depth what each involved (yes, sorry boss it doesn’t look like I did much coding today!).