Is building software unpredictable?

Had a really interesting discussion at work yesterday.

We had our usual development meeting and I presented a lightning talk titled "The Software Craftsman and Tools". In a nutshell, I started off the talk by stating though the strong emphasis on Software Craftsmanship here at work is fantastic, we must also be conversant with the tools and platform that we are working on.

I then went one step further in saying that those two elements are not necessarily that important, without an overall vision (ie. goal). This overall vision is basically where the business wants to be in the future (ie. within a time period)

I then finished my talk by stating that I believed that "Only by having an architectural vision, can a software craftsman; who is conversant with his/her tools set, be truly effective in delivering value to the business"

Which then lead into our fishbowl conversation, "Do we need an overall architecture vision here at work?"

What really intrigued me was the statement made by one of the developers here, which goes along the lines "... building software is inherently unpredictable, thus its unnecessary to have an overall architectural vision." [not in these exact words, something similar]

This statement really bothered me and got me thinking late into the night. What is this unpredictability that he is referring to? To me, when building software there are basically two types of requirement:

1. Functional Requirement (aka What the Business wants)
2. Non-functional Requirements. (What we build to support the software)
   

The only thing that is unpredictable is the functional requirements (aka "What the Business wants"), because processes can change, scenarios that were not considered before, etc.

So that leaves us with the Non-functional Requirements. What are non functional requirements? These are things like

1. How do we scale the application?
2. How do we monitor the health of the application?
3. and so on.
   

NOW, do we pull this non-functional requirements out of thin air? No we don't. We asked the business (product owners, stake holders, etc)

1. Why is this software important to you?
2. How do you see this software fitting into the overall strategic vision of the business?
3. What is the expected uptake and potential growth of user base for this piece of software?

Its only by understanding what this piece of software means to the business, can we clearly articulate the non-functional requirements. And leading on from there, we can define an architectural vision... aka a roadmap -> Where the software needs to be, and How we are going to get there.

So when an important application which suddenly becomes popular with end users and is one of the major revenue earners for the company starts performing sluggishly, we SHOULD NOT shrug our shoulders and give the excuse that this was never in the requirements. It probably was in the requirements (ie. the non-functional requirement), but no one actually thought about having a conversation with the relevant people and thinking about the bigger picture and the future.

So I putting it out there, "We need to determine what parts are unpredictable and what are not." Building software is NOT unpredictable. We are just not asking the right questions.

In summary, I still believe we need to have an overall architecture vision (which is derived from the business strategic vision). Its only within the confines of this vision, can a software craftsman who is conversant with his/her tools be truly effective in delivering value to the business.

Just my 2 cents :)