Broken executables in extensions in Firefox 3.6

If you are an extension developer and include executable files in your XPI package (binary or shell scripts) then you may be seeing problems in Firefox 3.6.

Back between Firefox 3.6 beta and Firefox 3.6 RC we took a small fix to the extension manager that changed how we extract the files from the XPI package. The fix involved adjusting how we accessed files to avoid hitting problems with certain anti-virus tools that would occasionally lock files in the middle of extraction making us fail to install the add-on. A side effect to this fix leaves us setting file permissions on the extracted files in a slightly different way to previously. This side effect means that the executable permission is getting stripped from all extracted files. If you try to execute these files with nsIProcess it will likely fail.

There is a bug on file and I have a patch almost complete so hopefully this should be fixed in Firefox 3.6.1 but until then you can workaround this in your extension by setting the executable permissions on the file yourself. Assuming you have a file variable that is an nsIFile pointed at the executable, just do:

file.permissions = 0755;

In case you were wondering why we enforce file permissions at all, it is because it turns out there are quite a lot of different zip tools that developers use to build add-ons. Some of them are unfortunately broken and embed bogus permissions into the generated XPI. After extraction it leaves us with files that are unreadable/unwritable which makes the add-on fail to work correctly. This most commonly affects developers on windows (where permissions are a little laxer in general) who get odd bug reports from users on Linux which is respectfully refusing to use the files. The easy fix is to enforce read/write permissions on the extracted file in the first place.

Zooming along, hopefully as fast as before

I’ve just landed a fix to a bug that has irritated me ever since page zoom started getting remembered for sites. It fixes a real problem you find if you both use zoom a fair bit, and load pages in background tabs. When you finally decide to look at that tab there is this little pause (or long pause if the page is large) and sometimes a visual jump as it re-zooms the content. It also changes where the page is scrolled to which is very irritating if you have just clicked a link to a specific line in some source code for example.

The fix was relatively simple, the problem is it causes a little extra time to be spent loading background tabs. About 3ms from the numbers I have. Normally this is small enough that you wouldn’t notice in comparison to how long a page takes to load. One concern though is how this impacts session restore where you have a lot of background tabs all trying to load at once. The tests I’ve run say pretty much what I’d expect, Firefox opens, all your previous tabs are displayed and start loading with no change due to this fix, the reason being that the additional time is spent later in the load cycle of the tabs.

Of course tests never quite mirror reality so I’d like all you nightly testers to keep an eye out to see if you notice a sudden change in the performance of the browser when you open it and session restore has a lot of tabs to restore. Obviously before filing a bug do the good thing and try it a few times both with nightlies with the fix (tomorrow’s should be the first) and nightlies without. If you do see definite issues then file a bug and CC me. Assuming there are no issues then I’ll be trying to get approval to land this for Firefox 3.1 in the near future.

Dividing the labour

Faaborg has started a discussion on the newsgroups on the relative importance of polish and blocker bugs (I’d provide a link bug Google Groups seems to be refusing to acknowledge existence of the post). I have been for quite some time now very focused on the in-depth blocker issues, partly because that is where all the interesting work is for me, but also because I think it makes more sense for me or one of the other guys with good understandings of how the extension manager works to be making these changes rather than throwing others in at the deep end.

This has of course left little time for me to fix the trivial issues that Faaborg talks about. The things that users probably notice more and make the product look complete or half-done. This is why I’m really pleased that when I posted a list of simple bugs that anyone should be able to tackle there was a fairly good response. Of the 12 bugs on the original list, 9 have been fixed. I wanted to extend my thanks to those guys that took the time to work on those, some of them probably ended up being a little harder work than anticipated.

Of course the polish list is almost never ending. I’ve been adding more to the list of good first bugs, if anyone is interested in helping out then don’t be afraid to just email me or find me on IRC as Mossop and ask how much work something really involves.

The great bug triage

I was very excited to learn at the Firefox Summit that Rob Strong was handing over ownership of the Extension Manager module to me. He did great work making the extension manager what it is today but has lately had to be more focused on Installer and windows integration issues.

Last week I spent a large part of my time trying to get myself up to speed on all the old filed bugs, clearing out things that clearly aren’t going to happen and trying to consolidate others. In particular I made the effort to go over every single unconfirmed bug and either resolve as appropriate or request any additional information from the reporter. It is rather sad that many of these were issues that noone commented in after the initial report or even worse the reporter responded with additional information but people (including me) dropped the ball and nothing further happened.

This big triage has been quite useful. Firstly the number of unconfirmed bugs for the add-ons manager has dropped from the mid 70s to under 30 (I would give you exact figures and pretty graphs but bugzilla’s charting is broken). I expect it to drop even further next week when I resolve the issues where the reporter is no longer responding to bugmail. Secondly it has allowed me to spot a pattern amongst 6 or so filed bugs which strongly suggests a new issue that can break add-on installations. While I have yet to be able to reproduce it in a test environment I certainly have more information to go on than previously.

Finally I have been marking all the trivial little issues that I think we could fix, but are unlikely to be priorities. These are the kinds of things that if you were interested in helping out and getting started in developing for Mozilla you could start out on. Many will end up being a few lines of patch, just need to figure out where to put it and test it.

So if you are interested in tackling any of these please give it a go. You can find me on IRC (for reasons best left alone I appear as Mossop) or by email if you need any guidance on getting started. I’m going to try to keep this list updated as I find new bugs that fit there, I still have yet to go through all of the 263 confirmed bugs, maybe a job for next week.

Filing good Extension Manager bugs

The Extension Manager is pretty complex and so it can be difficult to gather the sort of information needed in a bug report to really diagnose what is going on. When the problem is related to extension installation, upgrade, uninstall or enable/disable, these suggestions should help get as much information as possible into a bug report.

Be specific in your description

While it may appear that your problem happens for “every add-on you try to install” or for “every website” the reality is that add-ons are complex things. A feature of the 20 add-ons you tried might not be present in the one add-on that the developer tests with. If you say precisely what add-ons you tried then the developers will test with those add-on which gives us a better chance of reproducing your problem.

The other part of this is to be specific in the steps to reproduce. Try to include what windows appear, what buttons you click on, when you restarted Firefox. Too much information is not a bad thing here, the more closely a developer can follow your exact steps the better.

Attach the cache files

In your profile directory are a set of files that the extension manager uses. Attaching copies of them after the problem has occured gives us an idea of what state the extension manager got itself into, ideally shut down Firefox straight after the problem and copy them out of the profile before starting Firefox again, then attach them to the bug report you file.

You want the files extensions.log (only in Firefox 3), extensions.ini, extensions.cache and extensions.rdf. If any don’t exist then say that in the report.

Turn on logging

If the problem is reproducible then you can turn on additional logging to get some more information on what the extension manager is doing. Type about:config into the address bar and look for javascript.options.showInConsole and extensions.logging.enabled. Make sure they both have the value true (double click to change).

Now open the Error Console from the Tools menu, clear it to start fresh and then perform whatever action causes the problem. Hopefully some messages will appear in the error console. Include these in the bug report.

Long Standing Crappiness

So there’s been this bug in Firefox for … well quite a while where it would suddenly stop remembering your toolbar customisations, window positions and even make your bookmarks appear to not be there, and in Firefox 1.5, make the search bar non-functional.

Well I’m quite proud to say that after quite a lot of research, and help from those guys doing Firefox support, particularly stevee, I think I have a fix for at least part of the problem. This bug (or at least the part I’m interested in) is all caused by one corrupt file. For quite some time I was unable to reproduce, and it turns out that that’s because the issue actually resolves itself in Minefield which is what I use day to day. Once I started testing on BonEcho it became pretty obvious what was going on. So the patch I’ve just posted for review basically spots a corrupt file on startup, and deletes it. Short and sweet.

Of course it’s not really the solution to the problem, just a nice way to make Firefox recover without too much hassle. The real issue is how the file gets corrupt in the first place.