Zip Writer Goodness

I can’t believe it’s been over a month since I wrote something here. Well I kinda can, lots of hectic stuff going on at my work right now which has been making finding time for Mozilla stuff tricky. Hopefully not for much longer.

I’m glad to say that I have managed to make great progress on the zip writer component. I have decided that dealing with multiple platform compiles for Nightly Tester Tools is just a bad idea, so instead I have pushed on with submitting the zip writer to Mozilla for review. Hopefully that will make it into tree where I (and of course anyone else) can just use it. There’s been a bunch of changes between the version I posted earlier and that that’s gone up for review, not least of which is a set of testcases that have made sure I didn’t break the old by making some cleanups.

Lots of people have been bugging me about when new versions of my extensions are going to be done. Sadly I can’t really do this. The old adage of “It’ll be done when it’s done” certainly applies. For most I don’t have a good handle on how much work is left and I certainly don’t know how much time will have to spend on them in the near future which sort of messes up any planning.

Oh and I’d just like to say hello to all you people reading planet out there. Assuming I haven’t broken my atom feed you should be able to see everything I write here from now on, you lucky lucky people!

Zip Writer Update

I spent some more time this weekend hacking on my Zip Writer component. It’s now pretty capable. It can open existing zip files and remove entries and append to them, happily rewriting the headers and everything exactly as it encountered them. And the other major win is that I have deflate code up and running which makes compression possible.

All this has allowed me to reintroduce making extensions compatible during the install process in a far safer way than was the case before. NTT can now watch the EM datasource, spot that an extension has finished being downloaded, then open that extension’s xpi file and if necessary rewrite the metedata to claim compatibility with the current version of whatever app you happen to be running in.

Depending on time I may have the next release of NTT out the door by the end of next weekend, but things are hectic as usual and I have to be various places over the coming week.

Long Standing Crappiness

So there’s been this bug in Firefox for … well quite a while where it would suddenly stop remembering your toolbar customisations, window positions and even make your bookmarks appear to not be there, and in Firefox 1.5, make the search bar non-functional.

Well I’m quite proud to say that after quite a lot of research, and help from those guys doing Firefox support, particularly stevee, I think I have a fix for at least part of the problem. This bug (or at least the part I’m interested in) is all caused by one corrupt file. For quite some time I was unable to reproduce, and it turns out that that’s because the issue actually resolves itself in Minefield which is what I use day to day. Once I started testing on BonEcho it became pretty obvious what was going on. So the patch I’ve just posted for review basically spots a corrupt file on startup, and deletes it. Short and sweet.

Of course it’s not really the solution to the problem, just a nice way to make Firefox recover without too much hassle. The real issue is how the file gets corrupt in the first place.

Zip Writer

One of the annoying omissions from the Mozilla platform has always been the inability to create a zip file. It’s been bugging me for some time since it’s the only way for Nightly Tester Tools to properly manage overriding compatibility for an individual extension install, without doing dangerous things like it used to. There’s been a Google Summer of Code project on creating such functionality but it didn’t get all that far.

Well Saturday night I finally bit the bullet, after talking about it for a long time, and took a crack at it myself. I didn’t like the API’s presented in the SoC project so I decided to just start from scratch andknock up something really really simple. And when I say simple, I mean that initially all this thing was going to do was create a zip file (no editing existing ones), and add files and folders to it, with no compression. That makes it simple see, just add a few file headers here and there and all you’re doing is copying data around.

Rather surprisingly it almost worked. Lots of problems handling CRC calculations in JavaScript, but got there in the end. Well to a point. It managed to write out zip files when the files added were not too binary in nature. I made the mistake of attempting all this in JavaScript which it turns out doesn’t work with binary data too well. Particularly not when the Mozilla interfaces I used pass then around as char* data types which js then turns into a unicode string.

So onto the second implementation in C++ this time. I guess it was probably inevitable that it would come to it, but JS at least is real quick to prototype things like this in. The C++ implementation now works as well as I’d initially hoped, so since then I’ve of course got bored and started looking into having real compression in there and maybe editing zip files in the future. Like most of my code all this is open source, you can view it in my subversion repository. What’s more, since I moved on, Mook has taken the JS implementation, plumbed in some cleverness and made it work for binary files too!

Obviously the code is to be used at your own risk and right now it’s not all that thoroughly tested. Hopefully though if I can convince bsmedberg of my API’s, this code might end up making it onto trunk so everyone can use it.