The Power of “Defensive Pessimism” in Building Great Software

March 15, 2018

Can software be beautiful? Certainly a great looking and intuitive interface which enables people using the app to accomplish their tasks with little effort and minimal friction could be called beautiful. Software developers are privy to another kind of beauty: The inherent beauty in well-constructed software that makes it easy for a software team to effortlessly integrate disparate pieces into a compound whole. Well-constructed software can be appreciated much the same way that a beautiful painting, a sculpture, a building or a piece of music can be appreciated.

But beautiful software is not necessarily great software. Ideally, great software is great because it empowers people. It can give them what could be described as superhero-like capabilities. We definitely want people to feel like superheroes when using our software, but we want them to identify more with Superman or Wonder Woman than with The Greatest American Hero. In other words, they should be able to achieve great things, but, unlike The Greatest American Hero’s Ralph Hinkley, they should not be rendered powerless without a sufficiently detailed instruction manual.

For software to empower people in this way, it must be designed from the ground up to be anticipatory. Great software often feels omniscient. It makes the difficult look easy, even though, ironically, making the difficult look easy is really quite hard. As Steve Jobs is said to have put it, “Simple can be harder than complex. You have to work hard to get your thinking clean to make it simple. But it’s worth it in the end because once you get there, you can move mountains.”

As software developers, we must also strive to anticipate the events and conditions our software may be forced to deal with if we wish to create great software. The wireframes or mockups we receive from designers tend to focus on the so-called “happy path”. These are the things that the people using our software will hopefully be doing most of the time, and they include such things as writing great novels, sending money to friends, depositing checks, or collaborating and communicating with colleagues. These concepts are the things people would mention when describing our software to others.

Other events require error handling and recovery. These are the things which result from software being used in the real world. They are expected, but, hopefully, infrequent. Network requests may fail. The device may run out of memory or storage capacity. Great software accounts for these scenarios and provides a fluid though perhaps degraded experience in spite of their presence.

Finally, there are exceptional events. These are failures from which we cannot recover programmatically, and include hardware failures or assumptions about external dependencies which have held true in the past but which have since changed and upon which we can no longer rely.

Software is best constructed by taking the existence of these types of scenarios into account from the beginning instead of “bolting them on” later. To me, great software must be robust by design.

The benefits of robust-by-design software

Software that is built from the ground up anticipating the various ways things can go wrong is more likely to be of the necessary quality and to deliver a rock-solid user experience. Robust-by-design software will also be less likely to crash or behave in an unexpected manner in the course of operation.

In addition, the resulting software will provide a better user experience. Things will be more fluid, and errors, when they undoubtedly occur, will be handled smoothly. The app won’t unexpectedly jump between screens or overlay elements from the anticipated “good state” with elements from the “error state”. They won’t show blank screens or display a spinner ad infinitum. Furthermore, it will be easier to avoid these unwanted situations.

The app will also be more secure as it will not crash as often or continue to operate in an unexpected state. The software will also safely clean up after itself in these situations (i.e., close open files or overwrite memory to remove sensitive information) thereby also increasing security.

The app will also be easier to maintain as the code will be better constructed. It will be less likely that other developers will cause software to regress as these scenarios will be more explicit in the code. The app will also be easier to test as it will be structured as a collection of components, thereby helping with separation of concerns.

Creating robust-by-design software requires us to think about as many of the various scenarios that we can to make sure we cover all the things which can go wrong. This is a skill that software developers must hone. Modern software is quite complicated, and there are often many things that can go wrong at any time. But, like any skill, one can get better at anticipating these scenarios. The more we practice this approach, the more scenarios will be known to us and the better we will become at thinking about new ones.

Why being a ‘defensive pessimist’ matters

I often tell my colleagues that I spend more than 90 percent of my time working to make sure the software I create handles those scenarios which occur less than 10 percent of the time. A lot of that time is spent trying to find those scenarios which are not on the happy path. Thinking of those scenarios can be hard, making sure the software is able to handle those scenarios well is easier (though not easy). Making sure the software operates well in the presence of those scenarios is made easier when the need to do so is taken into account from the beginning.

Obviously there’s no expectation or requirement that we think of every possible thing that can go wrong, but the more such scenarios we think of the better. It makes it more likely that the scenarios we had failed to think of may be covered by the scenarios of which we did think. Furthermore, any new scenarios will be easier to incorporate later as we already have support for alternate paths and do not need to bolt those on well into the development cycle.

Another way to say this is that, as software developers, we should think about our software through the lens of “defensive pessimism”. Defensive pessimism is a cognitive strategy whose practitioners work through all the things which could go wrong and plan accordingly. According to a New York Magazine article on defensive pessimism, while it might seem better to expect things to go well and not worry about negative outcomes, it most certainly is not better. According to research conducted by Dr. Julie Norem, a psychology professor at Wellesley College and a leading researcher of the defensive pessimism concept, defensive pessimists actually benefit from all the worrying they do as they approach situations more fully prepared.

That’s exactly what we want to do when developing software. We could be optimists, and assume that everything will work out okay, but we will quickly find that that’s unrealistic in practice. We could be pessimists and assume everything will always go wrong, but then we’d never write any useful software.

Instead, we should strive to be defensive pessimists who create beautiful and useful software that is robust enough to remain beautiful and useful when the inevitable happens and things go wrong.

Software which anticipates our next action can feel magical. As software developers, we can create a more solid foundation for our software and make it more likely to achieve that vaunted status if we work on anticipating all of the various scenarios it may have to deal with and incorporate that support from the beginning.

As the saying goes, the devil is in the detail. In other words, the details are from where that beauty comes. Getting these details right will make it more likely that the software we create is in fact great software.

SEE ALSO: The power of permanence in a “Command Z” world>

Please leave this field empty.


Thank You

Welcome to the YML family!