As I mentioned in my last post and as I know everyone is well aware, tinderbox is starting to feel the strain. You already can’t just glance at it to see what is going on. The Firefox tree is currently showing build and test results from a total of 23 machines. The interesting thing is that not all of these machines are actually doing any compiling. The rest are merely running tests on builds produced by the other machines. That isn’t to say that those test results are less important but I wonder whether it is worth treating these differently.
Thinking about this difference I’ve spent a short time knocking together an experimental view of tinderbox. The idea is simple, the test boxes that aren’t actually compiling don’t get a dedicated column. We still want those results though, and luckily there is somewhere to put them, on the build that they are testing. So every actual compile ends up as a box within which can be multiple results from testers. This leaves us a very manageable 9 columns, 3 per platform.
This is quite early stages, the physical perf and test numbers aren’t visible yet, there are a couple of bugs I know about, it doesn’t auto update and I wouldn’t recommend you really use this, but I am interested to know what people think before I proceed any further with it. There are a few questions in my mind like how to colour the main box of the build (currently the worse result from the compile and all the test results), how to colour the header (I suspect the worst of the most recent results from all of the machines reporting in that column), where to even put the full perf/test numbers (there isn’t really room in some of the fast nightly boxes). More importantly is this even readable? I think it is good for some things, often tinderbox gets confusing when you see a talos result fail long after the nightly box went green, here that talos red would be back on the nightly box that failed. I can imagine problems trying to figure out why the header is red when it’s actually the talos results from build a couple of times ago that is causing it.
Anyway without further ado, take a look at Tindermerge and let me know your thoughts.
6 thoughts on “Trimming the Fat”
Very awesome. The little spinner is a bit unclear — is it waiting for data to load or the build to finish?
Click-to-expand on the Talos/Unit Test/Perf bubbles would be sweet (so you can drill down and see how many tests failed)
The spinners are just on builds or tests that are waiting to complete, ideally this would auto update every 5 minutes or so so they would eventually go away and go green/red/orange
As long as it’s easy to figure out (by clicking through stuff) which tests failed and which test machine is involved (so you can clobber it as needed, say), this looks awesome. Much much better than the current tinderbox setup!
At first glance: I like it a lot!
What I’m missing there is the more detailed data we see on tinderbox waterfall (i.e. exact times of build starts, relation to checkins, test/perf results, download links, etc.) – but that could all be added with overlayed on-click-displayed boxes like we have on the "L" links on regular tinderbox waterfall displays.
Else, this looks really awesome and makes it much easier to see what’s going on. The spin buttons are probably not needed though, we already know that yellow is "build running" 😉
This looks great! Why not just have three columns, one per platform?
I wouldn’t worry about the perf numbers, I’d just indicate whether it was up or down compared with last time.. if it’s within a certain range, consider it identical and don’t mention it.
Comments are closed.