So what use are metrics from bad processes

Bill very kindly followed up on my earlier post and quite correctly pointed out:

“You cant collect useful metrics until you have a repeatable process because anything you collect wont relate to the process when its performed again – differently – cause its not repaeated”.

I agree 100%, however my let out is the phrase ‘Useful Software Metrics‘. If we have no process to measure then find what can be measured now. Some examples might be: What is the salary bill for our developers? How many support calls do we get? Can we estimate the amount of rework we do from the timesheet system? and so on…

Get some simple numbers — they do not need to be precise or low level. This is your marker that gives you something to aim at and some justifcation for a budget; in the worst case they may be no more than guesses — which is fine as long as everyone knows that. As you implement your processes then your metrics can become more sophisticated and the old metrics will become useless. Just keep asking the questions: (basically the Deming PDCA cycle)

1. How much is what I am doing costing me today?
2. How can I measure that better than I am now?
3. How can I make what I do better and by how much do I think I can improve it?
4. Did I improve it?

And further to Bill’s chastisement, sadly the word metrics does not seem to have increased the traffic to my blog .

powered by performancing firefox

So how can I pay for process improvement?

A frequent lament of process improvement champions is that management will not support the costs or show the support required. In my previous post Bill comented that the purpose of process is reduce variability and improve consistancy.

This is of course quite correct, but not the complete story that we present to management when asking for real money and time to implement process improvement. We need to show $ value for the our activity, so we need to dig behind the quality message and provide simple metrics that show the current cost of our process, and project the savings and additional value we can achieve by investing in improvment.

You’d better start gathering those metrics and instrumenting your process now!

powered by performancing firefox

The myth of the reproducible build

I have just finished reading Martin Fowler‘s excellent paper on Continuous Integration (CI). Well worth a read and if you are not using CI then I urge to spend time implementing it. However there is one idea that I did want to pick up on:

In general you should store in source control everything you need to build anything, but nothing that you actually build. Some people do keep the build products in source control, but I consider that to be a smell – an indication of a deeper problem, usually an inability to reliably recreate builds.

I think in the context of CI or when using most build tools this approach is quite correct. We have limited control on the build environment and the products it creates. Better to start afresh each time. But there is a problem with this: this is a not in fact a reproducible build; it is a reproducible build process – which is not at all the same thing.

In my view a reproducible build is an activity that, when given a specific set of input artifacts (all at a given revision), will create a bit for bit identical output to the previous build. In practice this is hard to achieve for two reasons

1. We usually have poor control or knowledge of the environment on which the build is performed. E.g. exactly which 3rd party libraries are used, are we certain that no old copies of artifacts are being used in the build.
2. Things such as date and time differences can modify the built files in trivial ways.

Now insisting on exact bit for bit reproduction may seem a little harsh, but it does allow the use of checksum hashes to verify the integrity of our different environments. For some people this is absolutely critical as production environments are audited. For the rest of us it’s also important, for example we can ensure that the correct runtime environment is being used in test to reproduce a production problem.

A way to achieve this is to preserve the outputs from our builds in our version control system. Then instead of ‘reproducing the build’ we release our build artifacts as often as required.

However their are two caveats to this: we need to tag the created output files showing

1. Which source files (and versions) were used as input. This includes files such as build scripts
2. Shows under which configuration the files were built (should relate back to the software tools, libraries etc. used). Note that this mean all system build environments should be subject to rigorous change control.

Now that we have this information we can also clearly document how our software was build and what file revisions were used to build it. This information is vital, even if the built outputs are not saved in our version control repository.

A minor side affect is that we can now re-use built files in later builds, providing the tags are correct. There are still some systems where this potential saving is a significant benefit as complete builds can take days.

powered by performancing firefox

What is Process?

zx12bob asks what is process and who needs it. The answer to this question is easy — the details and implementation are of course complex, but at the highest level it comes down to three things.

• We must do things cheaper
• We must do things faster
• We must do things better

What does that mean?

Cheaper

• Use Fewer people
• Peform less rework (rework costs money)

Faster

Offer useful business services to users as quickly as possible

Better

Improve the user experience with better design and fewer bugs

The way to achieve this is to have everyone consistantly use the best process for the project.

So now the complex questions:

• What is the best process
• How do we use it consistantly?

powered by performancing firefox

The dangers of scientific ignorance

New Scientist has a opinion piece about the rise of home schooling in the US, it’s approach to scientific truth and it’s long term goals. It makes chilling reading to realise that the world’s biggest military and economic power could end up in the dark ages, unable to understand how to survive in the second millennium.

Let’s hope they don’t drag the rest of us with them…

powered by performancing firefox

No support for Building from Subversion

I mentioned a few days ago that I was looking to write a paper about building from Subversion using strict configuration management control.

There seems to be no previous art on how to do this, even in a similiar way to the current way that GNU Make suppors tools like RCS (admintally RCS is a file based version control system as apposed to directory based like CVS and Subversion).

So I a appear to be breaking new ground .

I had an epithany today relating to the use of listing a Build Of Lost as a Subversion repository. Make should be able to update the build revion with the details of the files used as inputs — the slightly tricky part is to conconcate the information from source files to final built targets via the intermediate built files.

Update 14/Nov/06 — I did find this about using Ant and Subversion, this list of VC Ant tasks and this list of optional Ant tasks

powered by performancing firefox

Xen and the art of world domination — yet more about Novell and Microsoft

As I considered writing yet another post on that deal I got to wondering about what Novell has that Microsoft want. The answer came in a Xen like moment — “Server Consolidation and Virtualisation”. Novell have invested heavily in the Xen OSS hypervisor product and can do some impressive things with it. Of course all of this is Open Source so the technology can be customised for any solution required.

On the appropriate Intel or AMD hardware Xen should be able to run Windows and Linux unmodified. This gives Microsoft a powerful weapon when selling into the server market in large accounts “Look Ms Customer, we can support all your server needs on commodity hardware — you can now deal with a single vendor, Microsoft”.

Remember that the Novell deal also covers System Management Services — I suspect that will be extended to cover not just Linux, but process management on Xen. For Novell that will be simple.

The Novell deal again “A five year deal with Microsoft to dump Novell/SUSE | Linux Journal”

This article touches on the patent issues of the deal and raises some issues of concern and interest.

• Is Novell violating the GPL?
• What the hell happens in five years time when the deal is over
• Is Microsoft still trying to spread FUD about Open Source, this time using patents
• Can we expect the FSF to take Novell to court over the possible GPL issues

I am also intrested in the role of Xen in Mircosoft’s plans — more on that later

powered by performancing firefox

Process Documentation, a simple approach

I have been talking about Lightweight Processes and suggesting that all process must be adequtelty documented.

This got me thinking about easy, lightweight, method to document lightweight process — in the project Wiki of course. To that end I have started up a small project to create Lightweight Process Documenation support.

It will have to wait until I complete some other things, but I am hoping it will easy to create some CSS and a few XHTML tables that can be used to guide the documentation process.

[Update July 2008: I haven't got much done with the CSS etc. However I did post a simple example of what it might look like]

powered by performancing firefox

In need of a decent calendar

Arrh!! For the second time in a week I have doubled booked myself .

I need an alternative calendar system. Currently I use the daily pages in my journal to make a diary note. However this does not seem to be working well anymore. I think a combination of not being able to read my own writing and not copying family appointments into my journal are the main reason.

I have also have access to Google Calendar and a Zimbra Calendar. However they are on servers and I want something I can carry and use off-line. Since my Windows PDA does not sync with my Linux software I feel very stuck.

When I was using MS Outlook then the PDA thing worked very well…

Anyway — sincere apologies to those who have been inconvenienced by my lack of control and effective tools

A great Web Application

I came across a really intretsing web site the other day called Lulu, a commercial site that allows you to publish and sell books (and other media). It is intresting for two reasons:

• What it allows you do by creating small print runs. e.g. create documentation for your product, print internal process manuals, etc. Prices seem resonable — I would be intrested to know at what point it becomes cheaper to use a traditional printer.
• How the website is designed. It’s easy to use, has a clear product message and is well layed out

It was a very ‘obvious’ web site, even though the service being offered is quite complex.

powered by performancing firefox

Some seeds flourish

I was browsing c.o.l.a this evening when I discovered that GLLUG is still going strong.

This brings back memories as I helped start the group back in 1997, the first meeting was at my then employers premises. However we had no regular place to meet for some time until GND very kindly stepped in and gave us a home until well after I moved to Australia.

BTW, I spent days coming up with the ‘cute’ name, it was a reference to the famous Linux virtual beer.

The group now has it’s own web domain and meets in central London. Well done to Glluger’s past and present for keeping the torch burning. It can be hard work but it’s an important job.

powered by performancing firefox

Did Novell made a deal with the devil?

Well a few days ago I wondered about the newly announced alliance between Novell and Microsoft. Now that some of the dust is settling down I am a little disappointed in Novell. One item that particularly caught my attention was the promised improvement for Open Office interoperability. It appears that the plan is to write filters so that Open Office can read and write Microsoft’s new ECMA approved Office Open XML format. What is wrong with the ISO approved Open Document Format is a mystery to me.

Given our experience with Microsoft and their sponsorship of C++/CLI at ECMA, in my opinion an attempt to foist .NET into the C++ community, the prospect of supporting MS Office in it’s continuing dominance fills with me disgust.

I suspect I will be migrating from OpenSUSE to Debian or Ubunto soon.

powered by performancing firefox

A normal Sunday

In order to make readers in the northern hemisphere jealous about the
great weather, and make it appear that I do really have a normal life,
I thought I would post news of our Sunday here in Melbourne.

First it’s a lovely sunny day today but only 21C so wonderfully cool.
This evening we are having a barbecue after visiting our local butcher
– lamb steaks, chicken and sausages. See the BBQ in the attached
photograph.

All washed down with local beer and wine! Australia — bloody nice one
day, perfect the next.

We also ordered our Christmas turkey which will also go on the BBQ,
which usually means we eat about 4:30 on Christmas day as it takes so long to cook

Holy Cow! Novell and Microsoft

So Novell has into jumped into bed with Microsoft. This opens up a number of interesting questions:

1. How will this affect RedHat? Consider also the announcement from Oracle earlier in the week
2. What is the medium to long term cost for Novell? Often Microsoft’s partners come off worse
3. Is the safe harbour from MS patent litigation important for customers? Here in Australian it is not something that I see people talking about that much as an issue
4. Will this improve Novell business? I am guessing that the answer is yes, they should be able to improve the service offering to cover more Windows under Xen
5. Will Microsoft offer Linux implementation services from their consultants? I would guess the will just hand ball that straight to Novell — stick to your knitting
6. Will the interoperability between MS Office and Open Office mean better filters for Open Office or new filters for MS Office? I assume that MS Office must have new filters so that it can remain in government offices where ISO document formats are mandated
7. What will the services management interoperability offer Microsoft? It should help Microsoft protect their turf with customers who must introduce Linux into an existing MS Infrastructure, and of course it will mean that customers will have to upgrade their current SMS installations.

powered by performancing firefox

Subversion based software building

I am in the process of designing a mechanism to build software from a Subversion branch/tag under strict CM control and hopefully track the build output and preserve them in Subversion as well.

The draft paper is up at http://alecclews.pbwiki.com/SubversionBuilding. Please feel free to come along and add comments

Nasty Obsession

I have become obsessed with the statistics page on my blog: I visit it several times a day; I become pleased when the graph goes; disappointed when the graph goes down; and keep wondering if I should blog differently to show up in the search engines.

It’s all rather pathetic really.