Kiss the old testing infra (Revision 3 Mac minis) goodbye

It is with great joy that today we are turning the page on our old Release Engineering infrastructure. Since Q1 2010, we have run unit tests and talos jobs on rev3 Mac minis which have seen seven different end-user operating systems: xp, win7-32/64-bit,10.5, 10.6, Fed12 32/64-bit. We stopped support for OS X 10.5 and moved our 10.6 tests to rev4 Mac minis over the past two years, and back in April we moved most of our Fedora32/64-bit test/talos jobs to Ubuntu32/64 VMs in AWS and on iX hardware.

I'm so high! Have you seen this infra? It's magical! Rainbows for you and me.

Several weeks ago, we had another major milestone in the transition of the XP and w7 architectures when new iX machines (the same type of hardware we used for Ubuntu and Windows 8) started arriving at our Vantage data center in Santa Clara.  Thanks to DCOps and RelOps, we got them up and running this May, and we enabled identical test jobs on them.  Today, we are disabling unit tests and talos jobs on our old Rev3 Mac mini machines for Firefox 23 and Firefox 24 development trees for Windows XP and Windows 7.  This change moves the last two platforms off the rev3 minis and allows them to be retired once they finish riding the trains for ESR17 in December.

What is shiny about this infrastructure? 

Q Fortier and Mark Cornmesser (from RelOps) wrote steps for the initial installation using Microsoft's WDS/MDT framework, which allows for modular, shared task sequences between systems. Changes to the machines can be made after the fact using Microsoft's GPO (Group Policy Object), and, in the future, support will also be added for puppet.  This is the same setup as with our new Windows 8 infrastructure. This helps us deprecate our lovely (...not) OPSI system and it takes away the pain of deploying things manually through VNC or SSH (I know, I know :S ) on our Win7 machines.

Another neat thing is that we will be able to remotely power cycle these machines like the rest of our infrastructure since they have out-of-band management support via IPMI. We can also increase capacity of this infrastructure more easily than the Mac minis since the iX hardware has a longer and more predictable purchasing lifecycle than Apple machines.

We now have 130 nodes for each Windows flavor (Win8, Win7 and WinXP) taking care of all of the jobs on mozilla-central based trees plus mozilla-aurora.  For now, the rev3 machines will ride the trains until ESR17 is killed in December. We could try to fast track this process by uplifting some patches but we don't see a pressing reason to add more human hours into it.

It's been a pleasure to be part of this large testing infrastructure refresh and working together with arr, coop, dmoore, dustin, edmorley, Fredo, hwine, Hubear, markco, melissa, philor, Q, rail, RyanVM, tfranco, van, vinh, sal and devs. My apologies if I missed anyone.

If you're curious, all 5 different operating systems were set up under this huge tracking bug: https://bugzilla.mozilla.org/show_bug.cgi?id=820243

iX systems:

http://www.ixsystems.com

EDIT: 30-May: I added few more people to the thank you list.

---

This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

Running Windows 8 tests visibly on tbpl

A couple of weeks ago we announced that we were running Windows 8 test jobs on the Cedar tree but we're now also running it on mozilla-central, mozilla-inbound and try.

We have 3 types of jobs hidden for these reasons:

• Reftests will soon land on mozilla-inbound and be fixed
• Debug mochitest-2 and debug mochitest-other are intermittently failing (more often than Windows XP and Windows 7)

If you want to lend a hand with these last issues please visit bug 847447 and see how you can help.

We will soon enable it on every other branch besides mozilla-esr17 in bug 847471.

It's good to see these running officially :)

---

This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.

Running Firefox unit tests on Windows 8

Last week, we started running Windows 8 unit tests in the Cedar development tree:
https://tbpl.mozilla.org/?tree=Cedar&jobname=Rev3 WINNT 6.2

Unfortunately a bunch of jobs are not passing all tests :( We have filed "Bug 847447 - Figure out list of failing win8 jobs and chase them down" to figure how to solve those unit test failures.
If you can give jimm and others a hand with those bugs that would be greatly appreciated!

Once we add the remaining Windows 8 machines to the pool of machines, we we will enable the test jobs on every other development tree and hide them until green. This way, we can have a better idea of the load to expect.

To track the project as a whole, visit: "[Tracking bug] test 32bit windows build (which include metro functionality) on 64bit version of win8".

---

This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.