The Skeptical Software Development Manifesto
By Ed Johnson
Argumentation cannot suffice for the discovery of new work, since the subtlety of Nature is greater many times than the subtlety of argument. – Fracis Bacon
The over-enthusiastic and often uncritical adoption of XP and Agile tenets by many in the software development community is worrying.
It is worrying because it attests to the willingness of many developers to accept claims made on the basis of argument and rhetoric alone. It is worrying because an over-eagerness to accept technical and methodological claims opens the door to hype, advertising and wishful thinking becoming the guiding forces in our occupation. It is worrying because it highlights the professional gulf existing between software engineering and other branches of engineering and science, where claims to discovery or invention must be accompanied by empirical and independently verifiable experiment in order to gain acceptance.
Without skepticism and genuine challenge, we may forfeit the ability to increase our domain’s body of knowledge in a rational and verifiable way; instead becoming a group of fashion followers, darting from one popular trend to another.
What is needed is a renewed sense of skepticism towards the claims our colleagues make to improved practice or technology. To that end, and to lend a little balance to the war of assertion initiated by the Agile Manifesto1, I would like to posit the following alternative.
The Skeptical Software Development Manifesto
We are always interested in claims to the invention of better ways of developing software. However we consider that claimants carry the burden of proving the validity of their claims. We value: • Predictabilityovernovelty • Empiricalevidenceoveranecdotalevidence • Factsanddataoverrhetoricandphilosophy
That is, while there is value in the items on the right, we value the items on the left more.
Our skepticism is piqued by claims and rhetoric exhibiting any of the following characteristics: • Animprecisionthatdoesnotpermitfurtherscrutinyorenquiry • Themischaracterizationofdoubtasfearorcynicism • Logicalandrhetoricalfallaciessuchasthoselistedbelow:2 Argumentum Ad Hominem Reference to the parties to an argument rather than the arguments themselves. Appeal To Ignorance The claim that whatever has not been proved false must be true, and vice versa. Special Pleading A claim to privileged knowledge such as “you don’t understand”, “I just know it to be true” and “if you tried it, you’d know it was true.” 3 Observational Selection Drawing attention to those observations which support an argument and ignoring those that counter it. Begging The Question Supporting an argument with reasons whose validity requires the argument to be true. Doubtful Evidence The use of false, unreasonable or unverifiable evidence. False Generalization The unwarranted generalization from an individual case to a general case; often resulting from their being no attempt to isolate causative factors in the individual case. Straw-Man Argument The deliberate distortion of an argument to facilitate its rebuttal. Argument From Popularity Reasoning that the popularity of a view is indicative of its truth. e.g. “everybody’s doing it, so there must be something to it.” Post Hoc Argument Reasoning of the form “B happened after A, so A caused B”. i.e. confusing correlation and causation. False Dilemma Imposing an unnecessary restriction on the number of choices available. e.g. “either you’re with us or you’re against us.” Arguments From Authority Arguments of the form “Socrates said it is true, and Socrates is a great man, therefore it must be true”. We are especially cautious when evaluating claims made by parties who sell goods or services associated with the technology or method that is the subject of the claim.
Principles Behind The Skeptical Software Development Manifesto
We follow these principles: • Propositions that are not testable and not falsifiable are not worth much. • Our highest priority is to satisfy the customer by adopting those working practices which give us the highest chance of successful software delivery. • We recognize that changing requirements incur a cost in their accommodation, and that claims to the contrary are unproven. We are obliged to apprise both ourselves and the customer of the realistic size of that cost. • It is our responsibility to identify the degree/frequency of customer involvement required to achieve success, and to inform our customer of this. Our customer has things to do other than help us write their software, so we will make as efficient use of their time as we are able. • We recognize that controlled experimentation in the software development domain is difficult, as is achieving isolation of variables, but that is no excuse for not pursuing the most rigorous examination of claims that we can, or for excusing claimants from the burden of supporting their claims. • Quantification is good. What is vague and qualitative is open to many interpretations.
1 http://agilemanifesto.org/ 2 The Demon-Haunted World, C. Sagan and A. Druyan, Ballantine Books, 1996 3 How To Win An Argument, 2nd Edition, M. Gilbert, Wiley, 1996