What tools does Alec use?

After Perl Mongers last night there was a short discussion over tools. Here is my preferred list

• Vim editor
• Git version control (and GitHub to host my remote repos)
• Linux operating system
• Perl and CPAN for programming. Not a prefect language, but has wide applicability to the the types of problems I solve
• Google search engine
• Email (gmail)
• Bash and the UNIX command line tools (sort, find, grep etc etc)
• OpenOffice
• Paper journal/notebook (currently using a Moleskine, but I prefer a Miquelrius)

Top Tip: Upgrade Git on Cygwin

The current version of git installed with Cygwin (1.5.x) has at least one annoying bug so it’s worth building  the latest version (NB I have almost all of Cygwin installed – you may need to install additional development tools and libs):

1. Download and unpack the source code. Pointer here
2. Run ‘./configure –prefix=/usr/local’
3. Run ‘make test’
4. Ignore errors relating to not detecting unwritable repos
5. run ‘make install’
6. run ‘hash -r’
7. Make sure it’s installed with ‘type git’ and ‘git –version’

Update March/09 — I found more detailed instructions

Handy Hack: Comparing two directory trees on Solaris 8

After some information from jim mcnamara I developed a script to compare two directory trees. I needed something that worked on Solaris 8 so it uses basic UNIX tools (and dircmp was not very useful). If you are using a more modern system then try this

#!/bin/ksh

#  Compare 2 directories for differences (e.g. current release and new release)

CURRENT=$1 ; shift
NEW=$1; shift

CURRENTDIFFFILE=/tmp/$(echo $CURRENT | sed -e 's/[/.][/.]*/_/g')
NEWDIFFFILE=/tmp/$(echo $NEW | sed -e 's/[/.][/.]*/_/g')

# File pattens not to be included in compare
cat << 'EOF' > /tmp/patternFile
^.git/
^.*\.log
^logs/.*
^.*\.tmp
^.*\.err
EOF

for i in $CURRENT $NEW ; do
	cd $i
	# cut removed ./ that find puts at start of path name
	find . -type f -print |cut -b3- |egrep -v -f /tmp/patternFile | sort > /tmp/$(echo $i | sed -e 's/[/.][/.]*/_/g')
	cd -
done

echo New files
comm -13  $CURRENTDIFFFILE $NEWDIFFFILE

echo
echo Deleted files
comm -23  $CURRENTDIFFFILE $NEWDIFFFILE

echo
echo files that are different
# xargs has limitations so must pipe into a second shell
comm -12  $CURRENTDIFFFILE $NEWDIFFFILE | xargs -i echo cmp -s \"$CURRENT/{}\" \"$NEW/{}\" \|\|  echo {} changed | sh

It’s all about the process stupid!

BBC NEWS | Health | Surgical checklist ’saves lives’

Using a simple surgical checklist during major operations can cut deaths by more than 40% and complications by more than a third, research has shown.

Does ANYONE need persuading on the value of a process when performing technically complex work? Including software development and service management…

Application deployment and change auditing with git

Updated 6/Feb/09

I’m currently working on on a project to deploy some complex banking systems and we have some somewhere in the region of 40 environments (various testing, production and development instances)

I’ve written some Perl to automatically configure environments using Text::Template. Now I’d like to automate deployment and have some form of change auditing (c.f.AIDE or Tripwire).

So a new project (which I hope will also  have some useful ideas for others):

• Project Outline and User stories
• General Approach using git (other tools could probably be used)
  • Define a staging repo
  • Create a remote branch of each deployment environment
  • Create a repo on each environment
  • Define a custom merge driver that overwrites (old files with new releases)
  • For each release
    • Create a new branch (overkill?)
    • Add new release to new branch
    • tag
  • To deploy
    • Merge (using new driver) onto environment branch
    • tag
    • Push to remote
    • On remote perform git reset --hard HEAD
    • configure
    • add and commit configuration changes
    • on staging pull config changes from remote repo
  • Now use git status etc. on environments to track any changes
  • Staging should have historical record of changes

Top Tips: Using the EEEPC 1000H

I’ve has my EEEPC 1000H for over a week now and I’m so totally enamoured of the small device that it’s become my sole platform for all my computing. Obviously this has involved some adjustments.

1. The keyboard requires considerable patience — at least for someone with my large fingers
2. My fingers are having to learn new positions to try and stop keep brushing the trackpad and moving the cursor to somewhere completely different. This is currently the most frustrating aspect of using the machine.

Some things to try.

• Get a bluetooth mouse
• Try the LXDE desktop and the Xfce4 desktop. They are both tuned for low resource use
• Consider using DevilsPie to reduce the amount of screen space used by removing the title bar and forcing all screens to be full maxiumum, partucurly of your screen is smaller than 10″
• Configure Skype to use the hda sound devices
• Carry a short USB cable for thumb drives etc. Thy can’t always fit in space you have at the side of the netbook
• Get used to using keyboard shortcuts — it mush  easier than using the trackpad
• Try using Vim and it’s plugins rather than  a large IDE
• Try using an alternative email  and PIM program to Evolution. Thunderbird for instance works better on the smaller screen and is faster
• Use multiple desktops and flip between them using <ctrl><alt><arrow> (Gnome default), for instance when referring to other material in the middle of witing an email. It’s an easier way of switching back and forth between two applications that using something like <alt><tab>
• Use the <F11> in Firefox to move into full screen mode (many other applications have a similar feature)
• Do as much as possible by typing at the terminal

Thanks to git I’m releasing IP into the wild!

This week I at last did something I’d wanted to do for a long time and  release some of Voga’s intellectual property to the wider world.

I have already made material available either through this blog or via Slideshare. However for a geek it’s obviously better to publish complete projects in an Open Source manner so  they can be freely copied, modified and shared.

For historical reasons this had been inconvenient to do as I used to hold everything in a single Subversion repository. Not only did this contain my potentially open material, but also my customers private data and my business records (yes, I keep my accounts database in a version control repository!). A few months I migrated my Subversion  repo to git, but I still had a single large repo and no time or skill to fix it.

However today I bit the bullet

1. Removed all my possibly public data to another directory structure
2. Tidy up my current repo by removing all the old ‘trunk’ directories — as a solo consultant I was lucky and had never had to branch my projects. Commit my new streamlined repo

Now to work on the the public projects

1. Identify a complete project and make sure it had no proprietary data. This is potentially time consuming
2. Tidy up some of the names
3. Create a new git repo; add the project files; and commit
4. Create a new repo on Github for my project
5. Follow the simple instructions to upload my project to github

To Do

• Update existing references to my projects I can find and update to point to the repo
• Add the correct license files
• Start again for as many projects as possible, please be patient

I’ll still keep presentation copies on Slideshare because that is much more convient for most people.

A big thank you to the git developers and github for making this so easy

Handy Hack: Make failure optional

When programming it’s always a good idea to check return status to make that no error has occured. In Perl we often write

somethingWeWantToDo or die "we failed because .....";

However when running in a test environment we may not have all our infrastructure in place to make our operation pass when needed (ideally we should use mock objects or some other testing framework). A  hack is to pass a flag into the program telling it that we are running in a test jig and we don’t care if our operation fails. The solution is simple, but a pain to work out after lunch…

somethingWeWantToDo or $testJig or die "we failed because .....";

If we fail and we are not using our test jig then we will exit with the appropriate diagnostics. i.e. failure is optional in a test jig

This is much easier than cluttering the code with conditonal if statements.

Setting up Serena Dimensions Design Parts and Allocating Roles

Once thing that often confuses Dimensions CM administrators and users is how to set up and use an effective Design Part Structure (DPS). There is no ‘once size fits all’ answer but I hope the following comments will help guide people to a more comfortable structure.

Handy Hack: Installing Ubuntu 8.10 on EEE PC 1000H

The 10″ EEE PC 1000H makes a good choice for people looking for a notebook with a little more storage and bigger keyboard than Asus’ previous model. Unfortunately, whilst the local Harvey Norman had a good price, I was still forced to pay the Microsoft tax. So the third job after I got home (after upgrading the BIOS and checking the hardware worked ) was to install Ubuntu 8.10. The process was pretty straight forward because I had a current Ubuntu system to create the USB boot disk and a wired network connection so that I could install the wireless tweaks. However it took a fair mount of Googling to hopefully this should save some folks a bit of time.

1. Get a copy of the 32-bit Ubuntu 8.10 ISO image. I used the Desktop version, alternative should work fine as well
2. Use the Ubuntu System->Administration -> Create Startup USB tool to copy Ubuntu to a USB stick
3. Shutdown Windows on the EEE. I had to do this via Windows shutdown to get the correct prompts on power up
4. Power up the EEE with the ESC key pressed. You should be prompted for a boot device
5. Boot from USB
6. Install Ubuntu in the normal fashion
7. Connect to wired Internet connection
8. Get latest updates
9. Follow these instructions
10. You should now have a working EEE PC with wireless

To configure Evolution email use ALT-f at the end of each configuration screen (the buttons can’t be seen)

Try setting the panels to autohide this will give you a little more screen room

The EEE PC trackpad buttons are pushed from the front by the way, not the top.

How to survive the emotional pressure of working at home

When you work from home there are various things you need to do to look after yourself:

1. Identify what you need to stay sane and make sure work out how to find or do it
2. Set up networks of friends on services such as twitter or facebook. Spend a limited of time each catching up with them — but be careful not to let it affect productivity too much.
3. Make sure you get out off the ‘office’ for coffee breakfast or lunch. I like to go to a local coffee shop
4. Join various groups relevant to your professional or personal interests and consider attending group meetings. IT people based in Melbourne can find a list at Perl Net
5. Set boundaries to help you switch off from work and do something else. Having an private office and defined office hours can help
6. Make sure that the people you live with understand when you are work and can’t be usually expected to join in other activities
7. Be flexible and break the schedule and work practices sometimes. After all flexibility is the main reason to telecommute

He also helps to create a decent working environment (space, light, furniture, storage, IT systems). I made some notes on this

What is Application Lifecycle Management?

The analysts and tools vendors have been talking about Application Lifecycle Management (ALM) for many years now and I think they have (of course) been somewhat self serving in their definitions. ALM seems to  include whatever product category they have in the current price list and then promising their integrations’ would be managements answer to cost and governance issues. I hope to say more about integrations in the future, but I’d like to look at the scope of ALM for now.

If you ask vendors and consultants (including me) about ALM scope you’ll get answers such:

• Requirements engineering
• Software Configuration management
• Dependency Tracking
• Change control and work tracking
• Test management
• Software development tools and process
• Software building and processes
• Deployment
• Process Governance

Depending on the vendor you might even get more traditional: Project and Program Management;  Project and Application Portfolio Management/Analysis; and Service Management activities thrown in as well.

The usual sales pitch is that everything is managed via a common tooling and provides a common repository. The major, and often single, selling point of this approach is improved governance and business oversight of the SDLC.

However I think it’s useful and important to look at the definition of ALM from a customers perspective rather than as analysts or vendors.

It seems that a more pragmatic approach is to describe ALM as the methods, processes and tools that support the management of change for software systems — with particular, but not exclusive, emphasise on the SDLC. In this context it does not matter if we are using Agile or Waterfall approaches, we chose the best tools and processes for our particular situation. This of course implies that in eighteen months time we may need to re-tool.

Furthermore the methods tools and processes of project management (PM) are much more mature and there is no real need to include them in this evolving segment — they have their own. Obviously PM will have an influence and impact on ALM and we need to design our ALM approach to dovetail with it. I realise that most vendors will not agree with this.

I have come to the conclusion in order to ’sell’ ALM into the teams that will use it we need to ensure, front and centre, that ALM provides a ‘better/faster/more’ improvement in daily productivity. That should be the primary focus of what is delivered.

Governance, audit etc should be secondary attributes and natural site affects of using ALM (important though they are of course and usually the primary sales message).

So at a practical level we need to be able to provide teams and their specialists with the best tools they need to do their job. Additionally we need to provide a supporting layer of data flows, events and data repositories to

1. Assist teams with traceability
2. Provide the tracking and management visibility to effectively steer the ship

So the point of this post is to plead with vendors to stop selling a one size fits all tool suite with deep integrations and start enabling shallow API and effective integrations to allow a truly best of breed approach for customers software purchases. It’s a shame that ALF project is shuting down.

Yes! At last someone explains the difference between LinkedIn and Facebook

The Ultimate Social Media Etiquette Handbook: The Most Egregious Sins on Social Media Sites, Exposed » techipedia | tamar weinberg

Forgetting that some individuals won’t network with you on a “personal” space like Facebook without knowing who you are, even with the proper introduction. If you’re looking to establish a professional relationship with someone, consider LinkedIn. Otherwise, consider building up a rapport with an individual before randomly adding them as your friend. Some people require face-to-face meetings before they invite you into their private lives. After all, Facebook was a tool that college students were using before it was open to the public, and some still use it as a purely personal and not a professional tool. LinkedIn is still seen as the more professional of the two.

I am one of these strange people, so please don’t try and initiate a business relationship with me on facebook. However I’d be delighted to consider a professional connection on LinkedIn

Handy Hack: Using FriendFeed to follow more than status updates from ANYONE

Now that twitter has become mainstream (Even the Australian prime minister has a microblog)  it can be useful to consolidate all of a contact’s information into one place (instead having to monitor their blog, microblog, flickr account etc etc) which is where a service life FriendFeed (ff) comes in.

On ff you create create an account that sucks in all your other feeds (blog, twitter, identi.ca, flickr, digg, slidshare and so on) into a single place. All people have to do is to subscribe to  that single account (probably via RSS or IM) and they get everything you are doing — not just your twitters. At the time of writing ff supports:

• 12seconds
• Amazon.com
• Backtype
• Blog
• brightkite.com
• delicious
• Digg
• Diigo
• Disqus
• Facebook
• Flickr
• Furl
• Gmail/Google Talk
• Goodreads
• Google Reader
• Google Shared Stuff
• identi.ca
• iLike
• Intense Debate
• Jaiku
• Joost
• Last.fm
• LibraryThing
• LinkedIn
• Ma.gnolia
• Mister Wong
• Mixx
• Netflix
• Netvibes
• Pandora
• Picasa Web Albums
• Plurk
• Polyvore
• Pownce
• Reddit
• Seesmic
• SlideShare
• SmugMug
• StumbleUpon
• tipjoy
• Tumblr
• Twine
• Twitter
• Upcoming
• Vimeo
• Wakoopa
• Yelp
• YouTube
• Zooomr

That’s all very well and good, but what if your network buddy does not have an account on FriendFeed? That’s were ff Imaginary friends come it. You can create an proxy account and then add the external accounts your contact posts on. All the account types listed above can be added and anything not in the list can created using the Blog type which expects an RSS or Atom feed. If you are adding a WordPress blog then make sure you follow this suggestion to get post titles to display correctly.

So currently I have 21 ‘real’ contacts on ff, but another 17 proxy contacts setup, including @stephenfry. Not only do I get his twitter posts but also notification of his wonderful blog posts, all in one place.

What a young developer needs to know

(Updated 5/Oct/08 and 11/Nov/08)

I occasionally ponder on what skills and knowledge a novice who wants to become a skilled developer should acquire. In no particular order here is a suggested list of things to consider:

• Basic accounting: This is useful from both an IT perspective — a lot of IT systems are used for financial management; and from a general work perspective because it helps with your future in business
• Version control concepts and tools
• Waterfall Software Development Lifecycle (SDLC): Requirements, design, building, testing, release
• Agile SDLC such as XP and/or Scrum
• How to use a command line
• Basic compiler architecture — the 7 phases of compiling
• Data representation: Number bases (binary, Hex, BCD); character sets; XML markup
• Basic algorithms: Sorting, Linked lists etc, recursion,…
• Important design patterns
• Testing and quality as a substantial stand alone topic
• Programming in at least three languages: C (or Java, C++, Ada etc), Perl (or Python, Ruby etc) plus one other (e.g. Lisp, Haskell). At least one needs to be an OO language and OO programming should be a familiar habit. Perl does not count as OO for this discussion.
• How to use a word processor and spreadsheet
• Basic SQL, RDBMS architectures, 1st, 2nd & 3rd Normal forms and basic DB schema design
• TCP/IP networking and basics of http, email, ftp, etc. Needs to include conceptual understanding of DNS and related security issues
• Basic IT security concepts
• Basic IT System Management
• Personal task and work management (e.g. GTD or GSD, suggested by Sinewalker)
• Maintaining personal work records in a  log book

Further topics

• More advanced RDBMS use and schema design
• Embedded systems
• How to use LaTeX or Docbook
• Application engineering for quality (maintenance, extensibility, performance etc)
• OS design
• Requirements engineering
• UML

Further suggestions anyone?

Packing your laptop for travel and working on the road

Updated 6/Oct/08

When you travel any distance with a laptop it helps to have some good habits for packing, carrying and using your computer

What to pack:

1. If you travel abroad then a universal mains socket adaptor is required. I quite like this one, make sure it’s as robust as possible because sometimes they can be a little fragile.
2. USB Charging cables for phones, iPods etc. Saves taking the mains based charger and having to share the mains adaptor when abroad. Some laptop BIOS settings allow you to configure the USB ports to supply power even when the laptop is off or on battery, which is useful for overnight charging but you can flatten your laptop battery so be careful
3. Security cable for laptop. I prefer a combination lock to avoid having a key to lose. Get one with as long a cable as possible as anchor points sometimes need a stretch
4. If you use a mouse then get a smaller laptop model to pack — it could save you get getting RSI using the built in trackpad or trackpoint all the time. The really small laptop mice may be too small for prolonged use or larger hands so check before buying.
5. A small mouse matt. Hotel room desks etc. are often covered with glass (or worse dirt) and will not work with laser mice so get a small mouse mat. Pack it so it stays flat
6. I have always carried a spare LAN cable, however I am using it less and less as wireless becomes more common.  Depending on the places you go it can be useful, especially in hotel rooms, but beware buying retractable cables (sight unseen) as some are bulky.
7. Have small bags to place cables and mice in. It makes finding things a lot easier. Ziplocks plastic bags will do when you can’t get nylon or cloth
8. Headphones can be useful for music and VoIP calls (needs a mike as well). I don’t recommend USB headphones as they may be bulky and take up a sometimes precious USB connection. You need to experiment to see if a bluetooth headset works for you. If you can spring for headphones that work on the plane as well that’s a bonus but I have not bothered for a number of years as airlines now provide reasonable quality headsets (ignoring issues of noise cancelling headphones). It seems to be hard to protect expensive headsets without bulky cases so I get cheaper headsets and save the space.
9. Consider the use and packing of a 3G modem. In Australian hotels broadband access can be very expensive (A$20-30/night) and with the added convenience of 3G it can make a lot of sense. Shop around for the best deals. In the US free wireless access seems to be included as part of the hotel package — but check before leaving. And check how much 3G access costs when using global roaming, it can be VERY expensive.

Avoid wrapping the power leads tightly around your power brick — it stresses the cables and causes failures (I had a colleague who did it all the time)

Have a specified order and placement for all your items in your luggage. Do it enough and you won’t have to think about it and you’ll know were everything is.

Invest in a good quality laptop backpack. It will save your spine from being pulled to the side  and leave the hands free for other things. Business travelers can get smart, black, bags now. It should have a padded laptop compartment, or use a laptop sleeve. In addition make sure that the bag has enough additional packing for your other travel needs. e.g.

• Paper notebook/journal/Filofax
• Pens
• Paperwork (use file folders to protect loose paper sheets), magazines and books
• On longer trips passports, toiletries and spare plastic bags. I wrote some additional notes on trans-Pacific travel earlier
• Anything else that supports your travel flow and work flow.

N.B. When travelling, even with carry on luggage only, assume you will loose access to your suitcase for 24 hours and need to work/live as best you can from your backpack

Lastly make sure that you have the correct work processes and tools on hand to make the best use of your time away

How software developers can make customers life hard

For the benefit of all you young developers out there who are being steered into a life of developing useful and valuable software I’d though offer a few hints and tips on slowing thing down a bit for your customers to increase support revenue:

• Make error messages cryptic or generic e.g. “Error writing file”
• Do not document the underlying architecture, diagnostic features or data formats
• Make sure that products require significant setup after installation or, better still, each time they used in a new project (excellent way to increase consulting revenue)
• Make the product appear flexible and powerful by doing the same thing different ways in various places

Surely it can’t be so hard to be a better (Emacs and Docbook) geek?

I recently decided, despite using nothing but Linux on my laptop, that I had become a suite using purely graphical Office tools (Evolution for calendar, email etc, OpenOffice to write documents and presentations and so on). I was becoming a very rusty geek!

Time for Plan A. Migrate email, task management etc to emacs, start using Docbook for presentations and document creation (in Emacs of course) with git to protect my precious work.

However so far it all feels a bit hard…  There is no consensus on the best way to configure Emacs for laptop email using IMAP services (mix and match any of Dovecot, fetchmail, Gnus, RM, procmail, etc etc); and getting a decent fop processor working is harder than it should be (I got going instead with dblatex in the end) — now writing Docbook documents seems hard as well — all that XML  *sigh*. I’m not even feeling at home with git yet .

Update 29/Aug/08: Sacha Chua has some great advice to share on using Gnus for email

Anyone have some tips or encouragement to share?

Pre Sale demonstration techniques for Open Source Geeks

Open Source Industry Australia were kind enough to invite me to present last week on pre-sales demonstrations techniques. My main message was about delivering a Value based message — which is explained rather well by the following

Farting in Church « The Sales Wars

Here’s what decision makers care about:

* Improved Revenues
* Lowered Expenses
* Efficiency of Operations
* Managed Risk

As well as talking about a Value based approach I also added in some general tips. Enjoy, and of course feedback is always welcome.

Handy Hack: Run vim for Windows under Cygwin

UPDATED 30/Jan/09. With apologies for any confusion, previous example was completely wrong

For reasons I cant quite remember running the Windows gvim.exe binary directly from bash can lead to problems parsing path names. So I created an function as follows:

function gvim() { (
unset a
for i in "$@"; do
    case $i in
        -*)a=" $i $a";;
         *) if [[ -f $i ]] ; then
  a=" $a $(cygpath --absolute --dos "$i" )"
     else
  touch "$i" &&
  a=" $a $(cygpath --absolute "$i" )" &&
  rm -f "$i"|| echo "failed"
     fi
    esac
done
/progra~1/vim/vim72/gvim.exe $a&)}