The Ship Metaphor

When managing a Software Engineering project you have a problem - you need nothing less than to be able to see the future.

Whoever is paying for the project, not only wants you to see and predict the future, but they expect it of you.  Its no use telling them how unreasonable they're being.

As the technical expert responsible for delivering the project you are expected not only to master the technical challenges but also to master time and space itself. Software project sponsors act surprised and start talking about hiring someone else for the job if you even suggest that you can't see or predict the future.

Your choices are to change careers, or fail to fulfil these expectations.  I'm offering a third option - maybe you can learn how to predict the future.

This is for you, if you are responsible for team leading, or software project management. It's also for you if you are a developer, programmer or software engineer - because if you are any good at doing those things, soon enough you'll be asked to lead a team or manage a project. I'm a software engineer and now my job has me doing both software engineering and project management. I'm writing code and deciding what code to write.

Do you believe I can show you how to predict the future? Well, I do not have all the answers, but I have seen how deep the rabbit-hole goes. I'm prepared to lend you my crystal ball. The first step is to understand that you need it.

Maybe you're a lucky young Software Engineer, and no-one has ever airily asked you that question, as though they couldn't care less about the answer, as though as soon as the words are out of your mouth they're not going to write them down and hold them in evidence against you: "How long would something like that take to write...?   If you believe you don't need the crystal ball I have one thing to say to you - muahahahaha!

Lets assume you're taking the Red Pill.  First we have to start with some basic philosophies.

Getting Meta

First, we must understand the power of metaphor in Software Engineering. There are two things to understand about metaphor in this work of ours - metaphors are real, and metaphors are helpful.

Do you think there actually are threads and loops inside computers? When you ask "who broke the build?" do you realise how many metaphors you're mixing there? Because what we as Software Engineers do everyday involves moving electrons around inside silicon, we have to use layers of metaphor to even begin to talk about what we do.

We encode these metaphors into the artefacts (source control, compilers, IDE's) that we use everyday, until we forget that the metaphors are not tangible things - they are constructs that we use to talk about aspects of a complex system, instantiated inside digital computing hardware.

So when it comes to Software Engineering, the closest thing we have to reality, is a big pile of metaphors, stitched together by lines of code, development tools and our daily practise. For us, metaphors are real.

I'm not going to argue this point any further here. If you don't believe me, read Steve McConnell's Code Complete, in which he starts out (Chapter 2) by talking about various project level metaphors and how they can be helpful on the one hand and misleading on the other. McConnell's metaphor of choice is construction of software, but he looks at others including writing software - as an author does. Both of these terms which we nowadays think of as very real activities, are in fact metaphors for Software Engineering. Also the Pragmatic Programmer is a slim volume full of pithy tips that you can apply straight away in your personal software engineering work, and in it the authors Hunt & Thomas use a metaphor of the software craftsman. They also use a number of anecdotes and illustrative yarns to explain their points.

The second point is that we adopt metaphors when they are helpful. Now when I say helpful, I choose that word carefully. As McConnell points out (pp 11-12) they are not complete answers. They can be helpful in the same way that painting a picture can be worth a thousand words. But a picture can be subject to interpretation, and since it hides detail that we don't need right now, that detail can come back to bite us later. We need to be careful about pushing the metaphor too far.

The takeaway from this is that a metaphor in software engineering can be a powerful tool.  The metaphor I'm about to present can help you wrangle projects as well as any web-based tool, spreadsheet or cheat sheet because it will help show where the project, the team and the other elements in the problem of software project management fit.  This tool is called "The Ship"- welcome aboard.

The Software Project as Ship Voyage

As I said at the outset, the problem we are trying to solve first with our software project management is that the people who are paying for what we do, and the people who care about the outcome, want us to predict the future.

They keep asking "are we there yet?" and "at least give me an estimate - it doesn't have to be accurate - are we maybe halfway there?".

Trouble is we don't have a map, and worse, there aren't really any roads for this voyage. We could go in any direction, and even though "that way" seems right we could easily be off piste by a considerable amount without realising it. So its less like a road-trip and more like a voyage at sea, and a voyage of exploration at that.

When the project is a voyage, it seems if we keep putting miles behind us then surely we'll arrive where we need to be. The trouble is we are absolutely awful at estimation. Putting a finger in the wind, with no other information apart from a bit of technical guesswork is always wrong. So we don't know how many miles it is to go, and we are not sure how fast we can go either.

But alongside the estimation problem of how long, there is another problem at least as bad - direction. Now that you are on board, are you feeling all at sea? Let me chart the course for you: what we have is a navigation problem.

This is where the Ship Metaphor comes in:

The Software Project is a Ship sailing away from port, with a talented crew, bound on a voyage across the Feature Sea. The owner of the ship waves us off, the captain steers a course, while the engineer stokes the boilers. But its not clear where we are going exactly.

Recently I read an interesting blog post comparing a Software Project to a hike along the coastline. I like the argument, for its story about how we repeatedly have to disappoint the stakeholders who are expecting us to arrive at the destination. But it does not really capture the problem of knowing exactly what route to take. It is assumed that the coastline route is already a given, but how do you know that?

Ed Yourdon in his book Death March: The Complete Software Developer's Guide to Surviving "Mission Impossible" Projects discusses doomed projects. He likens such projects to an army marching along, hungry and tired towards a destination that they will never be able to reach. Yourdon identifies the "Marine Corp" mentality as being one of the things that allows a Death March project to even start, because we as Software Engineers (I use that term interchangeably with Developer) think we're tough, and we won't give up when the going is hard. Yourdon's book is great because he talks a lot about the people involved in these projects and what their roles are. But even though he talks about the march, and the army, he never discusses how they pick their battles, or what grounds they fight on.

So the first one of these metaphors is good at explaining about the people who get disappointed when you don't arrive - those people are called the customers. The second one is good (pg 50) at explaining about the people who are involved in the actual project itself - the owner, the customer and so on; and also the team (pg 99) but it doesn't explain about the navigation issue either. But both of these metaphors fall down in that important way - they don't explain direction.

The key idea to capture when you use the Ship to understand your project, is that even when you don't think you are making choices about what features to implement, you actually are.  When you say "I don't have time to do requirements", or "We already know what a web-based issue tracking system looks like - lets just get started", what you are really saying is we'll implement what we like, instead of the features that are needed.  If you're super lucky there'll be some overlap.  But just coding - just putting miles behind the ship - is only half of the story, the other half is what direction you're pointing in.

Now the Ship metaphor is not perfect either, for example its lacking when it comes to modelling the role of the customer. But that turns out to be handled, and I'll explain below. But next we're going to look deeper into the bowels of the ship.

The Anatomy of the Good Ship Project

The key roles in the day-to-day operation of the Ship as she steams across the Feature Sea are:

Engineer - burns cash, makes the ship go

Captain - steers ship

OK, so what does that map to in a real-world project?  Obviously the Engineer is, well, the Software Engineers. As the days and weeks of the project go by, the Software Engineers are burning cash.

As the Engineers burn the cash, they provide the motive force that makes the ship move along its course, putting more and more implemented features behind it as it goes. Cutting code as the screw cuts through the water, the ship moves forward.

As a Software Engineer myself, this term burning cash make me uncomfortable - but its exactly how the owner of the project who is paying my salary sees it. If you're in a start-up this is exactly what is happening. There's a reason its called the "cash burn rate". But the ships owner knows the project is doing something important, and so they supply the cash to burn. However it sure seems to them like its going up in smoke sometimes. And that is especially the case, when a wrong turning has been taken and we have to go back on our course we had taken. And if the cash runs out, that ship isn't going anywhere.

So while the Engineers job seems simple, the job description for the Captain is bit less clear.

Does my project even have a captain? You can easily find out who the captain is by asking, "Who decided that you'd implement this feature?"

Sometimes the answer to that is that one of the engineers did. Sometimes a project owner has given over a spreadsheet with a list of requirements like "Must have bluetooth" or "Social Media integration"; and someone on the engineering team decides what that actually means. In this case your ship is more or less running without anyone at the wheel. Every so often someone from the engine-room comes up to the bridge, and adjusts the ships wheel a bit before lashing it in place and going back down below again.

When the Engineer is the only one steering, not a lot of steering actually gets done.  Usually it happens without even looking out the window to see where the Ship is headed.  If the real-life project has someone in the role of project manager, even if that is the same person as the team lead, then there might be someone on the wheel a bit more frequently and the ship has a chance of staying off the rocks before it leaves the harbour.

What about a one-person project?  There is always an Engineer and a Captain, even if they are the same person. Someone makes those decisions about what to write, and someone cuts the code. Trouble is since these are fundamentally different jobs, and even a natural tension between the roles, failure to find two different people to do them can mean your project is dead in the water.

Another failure mode is that you have a Captain, but the Engineers mutiny.  They have no faith in the Captain, maybe because of bad decision making or maybe because the owner of the ship didn't lay any ground rules for the ships charter.  Often its simply because the software engineers do not believe the job is important.  They don't hate the PM, they don't mind having them around, but hey - they'd be better off without a PM, so they could just get on with the job.

Engineers tend to believe that since the ship won't go anywhere without them, that they are therefore all that is needed. We're down here making things happen, and them up there - they have no idea the technical prowess required to churn out code, they say. That might be true, especially if there is a non-technical project manager; or one of the engineers is making the decisions on what course to take and thus "them up there" is management or ownership.

But the problem with this belief is made completely obvious when you look at the ship: exactly which code should be churned out?  Even if you know which code, what order do you do it in?  Why would anyone care about it? What use is the code you've written anyway?

There can be bad Captains too. These are the people who talk about Engineers as "resources" and act as though understanding the technical aspects of the software is not their job. For Captains who come from the ranks of the Engineers the pathology can be reversed - they are too close to the Engineers and spend too much time below deck.

But in my experience the biggest problem is that Engineers don't think a Captain is needed at all.  So let me digress a bit.

Use and Code

Let us take a quick detour to discover why the role of Captain is so important. If you're a project manager, or like me a team lead who has to write code and do project management, then you are probably already convinced. But let me illustrate the problem, just quickly, that occurs when you do not set the course of your voyage adequately.

Here is a picture of two different types of Software Projects, taken as a snapshot at a point in time.

In the first one we have a bunch of code in a software repository, gleaming in perfection, unsullied by actual users. This project type is called Useless Software, because no-one is using it. But you might be surprised to discover many engineers think such projects are wonderful - the code is brilliantly written, the algorithms are cool, and the technologies employed are the latest and greatest. The sorts of euphemisms we employ to describe projects in this state are Research Project, Development Sandbox, Proof of Concept and many others.

In the second type we have code as before, but we also have something magical: we have users. The users actually use the code, and what's more they complain about it. The complaints are called bugs and feature requests. This is Software in Use. You might be surprised to find that there are engineers who don't like projects like this. They don't like users who seem in their eyes to be unappreciative of the engineer's brilliance, and instead ask for such things as actually having the software do what they need, or supplying business value. Read Alan Cooper's The Inmates Have Taken Over the Asylum for a description of this if you don't believe that this happens.

Now, sometimes a project starts out like the first type, and at some point becomes like the second type; or at least that is the plan. But let me tell you, if a project starts out like the first type it is very likely doomed to remain like that. And the reason for that is that us engineers don't like it when dumb users tell us what what direction to go. Thus we will put off for as long as possible having any interaction with these users.  As long as the project is still the same project it will be stuck.  A Useless project cannot break out of its mould any more than you can lift yourself up by your shoelaces.  The only way is to throw away the first one, and start a new one planned from the outset to be Useful.

In terms of the ship, if there is no-one on the bridge at the start of the voyage, the engineers will quickly decide they know exactly what is required. The steering wheel will be lashed so the ship is pointing out to sea, they will burn the cash, make the ship go at full speed, and when it runs aground somewhere, or runs out of cash and stops, only then will they go topside to see what is going on. At this point, the project will be many nautical miles from where it needs to be, the code is left there gleaming in its repository, the voyage is over; and the crew wander off whistling nonchalantly hoping that no-one notices.

So steering the ship is important.  Are you on board with that?  OK, we agree.  So how do you do it?

Steering the Ship

The voyage of the Ship is about the daily life of the Software Project team. Its about the work being done, to move the project forward; and its about the minor corrections made so often to keep the ship on course.

Now I want to look a bit more at that issue of customers - which I mentioned about was something that is not immediately well explained by the Ship Metaphor.

Well there's a reason for that - those working on most projects do not get to see customers as part of their daily work. Even so called agile projects don't often have customers turn up to their sprint review meetings, held as they are every week (or two) on the dot.

Its the job of the Captain to keep his spy-glass trained on the sea ahead looking for the slightest sign of that that El Dorado, those plentiful fishing grounds or brave new lands. Every spot on the charts which the Captain has spread on his map table represents a different combination of features. The Feature Sea has an infinity of possible destinations, and when the ship arrives at the right one, the voyage will be complete.

If you like to extend metaphors too far, maybe the ship casts out its nets at this point, and catches a haul of fish which represents the successful delivery of the software. Maybe the customers will appear like mythical mermaids to welcome them. Or maybe its like discovering Atlantis, and gazing down into the water at this otherwise undistinguished spot on the ocean will reveal a city of users below.

But before we get too carried away lets think about what is important - its what the Captain has to be doing to reach the "X marks the spot" spot successfully. Imagine there really is a giant X floating on the surface of the Feature Sea, and that is how you know the voyage is done. It really is going to be pretty hard to see, even from the bridge of the ship, isn't it?

And what is worse if the voyage is any distance at all you have a big problem - the curvature of the earth means that you almost certainly won't have any sight of it until you get a lot closer.

The Captain must search for every clue, every warning about where that X is; and pay extra attention to every sign that rocks are ahead, storms are brewing or hazards to navigation are present.

So where do those clues come from? Customers.

While the customers don't appear as figures in their own right in the Software Project Ship, they are our stars to steer by.

So the thing that we reach is not a school of tasty customers, but a product that is represented by a latitude and longitude on the Feature Sea.

The X Marks the Product

In his essential book The Mythical Man Month, Fred Brooks debunks the idea that putting lots more Engineers onto a late project can save it. This is just putting more engineers onto shovelling cash into the boiler of the ship - that boiler can only get so hot before just blowing up - it certainly won't get you to the end of the voyage more quickly.

But more importantly Brooks talks about the Product (pg 4) and debunks the idea that two engineers working in their garage can produce an amazing piece of software that a large organisation could not produce in the same time. What Brooks describes - and he knew this in the 1970's so we should know it now - is that "works for me on my machine" does not equate to delivered software. To deliver software you need to turn it into a product - that is it needs to respond to the demands on quality, documentation, and maintenance that go with it being a product.

Refer back to the diagram in the previous section: the Software in Use is a product; and the Useless Software is not.

The difference is the customers. The difference is Software in Use is the one that supplies value. Business value is the fish-finder of the ship steering caper. Value is the magnetic north on the Captains compass, and the customers are the stars to steer by.

The best possible course to steer is the one that is the shortest distance to the point that represents the optimal combination of features for our product. To reach this point, the Captain checks at regular intervals via the chronometer, compass and instruments, making corrections when its necessary.

The longer the ship steams in the wrong direction, the most cash must be burnt to make up the distance on getting back on course.

Proponents of agile will recognise this regular check and correction as the sprint, with its review and planning. And of course this whole sprint thing is about running a course too, another +1 for our navigation metaphor.

No Customers

But what if there are no customers?

There are plenty of software projects where you'll here those running the show say, "But I don't really have any customers" or "There's no way for me to know what customers think - I'm too far away from all that". 

If that is true, then those projects are doomed. These are the Useless Projects. But the thing is more often than not, its just that the Captain on those projects is derelict in his duty.

If you are the one project managing then you absolutely must find the business value, and start plotting the course to see how you are going. Ask yourself: "Is this software is of any use at all to anyone?" Then find out how to know what that someone expects, what they need and what represents value to them.

If you have requirements - great. Don't just believe those though, go and find the customers as well. Maybe there is a front-person, acting as a customer liaison, or customer site "project manager". If you are not getting accurate readings from this person about what business value is, then either work with them to find that information, or work around them. If you don't you'll be part of a failing project.

I love the section in the Pragmatic Programmer entitled The Cat Ate My Source Code (pg 2). It's about taking responsibility for things that are "out of my control". The Software Engineers that can do this are the ones who become leaders. Don't just accept that there are "no customers".

Ideally you'll have someone to help you. This means having a new ships officer on the bridge - the Product Owner can be a key figure. When the Product Owner role and the Project Manager role are in separate heads, the dialogue that results exposes a veritable wealth of intelligence on customer value. If they are the same person neither role is done properly.

The Product Owner is the one who sees the product - the X marked on the Feature Sea - as a thing in it own right, and works toward it.

Alright I'm sold already.  Lets get working.  How do I work smarter, and how do I see ahead?

How Does the Ship Help Me See the Future?

So if it wasn't obvious before, you have to be the Captain here.  And you get to see the future, like I promised, but only in a very special limited way - hey, you weren't expecting miracles were you?

The good thing is that even though its limited, it is still very important. Here is how you do it.

Go back and look at the picture near the top of this article, with that ship. It's around 2/3 of the way along its journey, right?

That is the limitation. You can only see the future when you are part-way there. But that is not too bad, hey?

Now in reality that line is not straight, there'll have been a few kinks on the way. But you can draw a straight line through that approximates it well enough.

The most important thing is that after steaming all this distance, and working with your crew, learning the seas, you know how much ground you can cover before the project reaches its end.

The end here means different things depending on how your project is being run. If you have a decent product, then hopefully this won't be the end, but just a destination or the end of a leg, on a continuing voyage. It could be a software release, or another milestone where you have to check in a first delivery. Alternatively it could be when the first round of funding cuts off and a new case must be made for more cash to burn.

Whatever the case, your ship is steaming toward this point and your job is to make it the best possible collection of features covered for the distance travelled.

Given you're 2/3 of the way there, you've probably still got the ability to make one or two more minor course corrections, but given your readings of the stars (customer intelligence) has been accurate you'll be very close.

The key information you have is velocity. This is the standard speed that your ship can go. Accurately knowing both your velocity, your current position and heading you can predict the future.

If someone asks you where you will be on the feature sea in 6 weeks you can say "right here" and show the exact point on the map. You can give the answer backed by evidence, and the confidence of all the recordings you've made in the ships log to date.

If someone asks when will we reach the point X on the Feature Sea you can say "10 weeks time" with a degree of certainty that would not be possible if you were blind to all that information.

Now of course despite all that certainty and confidence you can still be wrong. But chances are if you are wrong it won't be by much. And certainly your answer will be far far better than any finger-in-the-air estimate or crystal ball gazing.

Also I said the limitation on the future-gazing is that you only get to do it 2/3 of the way there. Its actually better than that: you can give estimates before that, they're just not as accurate. So you can have something to say to your inquisitors that are asking you "when will we get there". You can say "Hey - let me do some sums, and I'll get back to you". And you do your first few miles at sea and give them a first estimate. Pretty soon after than you come back to them with a better estimate. And when you get to 2/3 of the way you can tell them with quite a lot of confidence.

Even better, if your project is a success, and you get to keep steaming on after your first release or first product, and the team is more or less the same, then you have all that knowledge to base your course predictions on.

Books are available - such as Agile Estimating and Planning by Mike Cohn - tell you how to do the mechanics of this future gazing. I think the picture of the Captain with a sextant on the cover of Cohn's book is a vindication of the Ship metaphor, right there. The other vital ingredients when it comes to the mechanics of this daily work between the Captain and the Engineer are documented in various books on how to run teams in Scrum or Agile modes.

But don't think that you got all the way to then end of this article, expecting to hear how to predict the future only to hear me say "Use Agile". No I'm not saying that is the answer.

I am saying you have to use Agile, or something like it because otherwise you won't be able to know the velocity.  You'll also need sprints to make the course corrections I'm talking about or incorporate the intelligence you are getting from your readings of your customer value compass.

Future posts on this blog will cover tips, techniques and traps for you to consider in your work of measuring velocity and setting the course.  If you're concerned because there's no picture here covering that part of it, I will cover that in articles to come.

But that is not all I am saying. That's just the mechanics of it.  There is something else.  Remember the metaphor.

The key to all of it is that you as project manager must be the Captain. You must own this role and live up to it.  That is the very first thing: before worrying about what kind of spyglass or sextant to buy.

You must believe in the role of Captain, and its importance, at the same time managing humility in your work with the other members of the crew. If you are also an Engineer, and you do your work as Captain by switching hats, you must be extra careful to examine every decision you make. Is this guided by customer value?  Am I abandoning the helm to spend too much time below with my friends in the nice warm engine room?

Next you as Captain must be a master of communications, and a diplomat of the high-seas and a resourceful solver of all kinds of problems relating to people. You need to manage your people, even as you work as one of them. You need to look to the horizon and yet still spot when the bilge is overflowing. Your job is so wide-ranging that the best way to to describe it is that you have to do everything else that is not being done by the Engineers, including figuring out what needs to be done. Its a very diverse skill set. But if it seems over-whelming don't forget that it includes delegation.

There are many Captain-less ships out there foundering on reefs, or lost at sea. There are many abandoned repositories filled with code that will always remain useless.

No matter what your project is, if you want to deliver it as a product to someone who will care about using it, then the challenge for you is to go up to the bridge of that ship, put on that Captains hat, grab the wheel and set course.

Epilogue

This article has set the manifesto and tone for this blog. The goal is to hear your comments, and to respond to feedback, questions and explosions of outrage. I plan to keep writing and exploring all of the aspects of this model of Software Project management, and to incorporate my experiences in the hope that its useful.

If you find any value here, please leave comments, subscribe to my twitter feed, connect with me via Google plus or the rss to keep up to date, as I keep posting more here about Coding on Course.

Thanks for reading!