The Objective of Software Development is Software Development

I don’t like what is happening to the software industry. I’ve been a developer since 1988 and at Microsoft wrote earliest versions of software that many of you use. In the late 80s and early 90s Microsoft had a management policy whose foundation was enabling concentration and focus, minimizing interruptions because they knew that doing so enabled our best work.

We called this “flow.”

That recognition has vanished entirely, from Microsoft and with few exceptions from the entire industry. It has been replaced with bizarre and ritualistic “methodologies” that from all I’ve seen add little or no value; on the contrary, they emphasize interruptions. We now have “masters,” managers, and executives who have never written code and who view any desire to reduce interruptions as effete. Meetings are seen as legitimately productive uses of time. They almost never are.

When people are constantly interrupted they cannot concentrate. When they can’t concentrate they write code with more bugs. The response to this is not to return what worked so well before but to further adulterate the experience with having developers write tests. WRITE tests. Not perform manual testing before tossing it over the wall to QA, who could do their testing however they wanted, unprejudiced by knowledge of the code and not having the developer’s blind spots. These blind spots are an important point that I rarely see recognized.

It’s vital to have other people do testing; some will agree with me on that but I’ve read a lot of verbiage arguing otherwise, that testing is solely the responsibility of the developer, and in my experience this verges on madness. I will achieve far better results by writing design documents because in doing so I will run across those blind spots before I start to write. Were I to write unit tests I would forget the same cases in the tests as I would later overlook in the code. And unit tests are a lot of work and often of little value.

When I read others’ writing on software practices I read more and more about testing, less and less about good coding practices. A third of a century in the industry and I see code is even less legible than it was in 1988, and even more poorly structured.

There is even an approach called test-driven development that regards development as a secondary practice; if you must cut corners to meet a deadline, cut them in the code, not the tests.

People have forgotten that the objective of software development is software development. Testing is a markedly different skill that not all developers do as well as they do designs.

Before this began I usually worked closely with testers, writing tracelogging instead of unit tests and spending substantial amounts of time in the tester’s office, without bug-tracking or triage. He finds a bug, we read the tracelogging, I fix and he retests, total elapsed time ten minutes. The time to do a bug-tracking entry was more than twice this, and that was just the beginning. We bypassed all that and worked very well together.

Re testing, I think it’s important to look at what ecosystem it grew out of. So the original major TDD proponents almost universally were Smalltalk (and thence Ruby) programmers. And the nature of development using those languages lends itself to an approach where the developer tests they code as they write it. They aren’t statically typed languages and they allow very fast iteration; they’re extremely freeform. So slowing down the speed of development by carefully testing at each stage, if combined with solid requirements and sensible data structures, can work well (although it needs a specific kind of focus, which is why I think it’s so ritualistic). And they were leaning on previous academic literature regarding improving software quality.

I don’t think that it’s an approach that is always useful, but it can help enormously in certain circumstances. On the one hand it can slow development, cause a false impression of correctness in a programmers head, and cause a narrowing of focus that can cause a programmer to lose sight of the overall picture. But at the point a programmer is writing the code, they are best placed to verify that the code they are writing is correct and matches the requirements they are given. I don’t think it’s always as simple as having it always externally tested – it’s extremely context sensitive, and unit testing by the developer of the code is very often a very good thing. One of the aims of TDD is to attempt to enable that without breaking out of a flow state (how achievable that is is another matter, and regardless of how salty you are, I find it very hard to disagree with most of your points)

2 Likes

Nothing breaks flow like shit not working and not having any idea why. That’s why one writes tests as they go. I don’t follow the TDD workflow where one absolutely must write tests firsts, but I certainly don’t consider a unit finished until it has decent test coverage (Dog knows I haven’t been following that lately, but my hypocrisy isn’t at issue here).

Professional software houses still have their own QA departments that also write their own tests, but they’re usually of the black-box variety to validate the design requirements, not the technical specifications made by the source (written down or not, an interface is a spec). Smaller outfits that are not developing MS Office don’t need a separate QA department any more than a middle manager needs their own secretary. Powerful and comprehensive test harnesses are in the hands of developers nowadays, and they’re not going back to their own fiefdom.

2 Likes

First of that, my sincere thanks for a well thought-out and honest reply. You exhibit none of the absolutism or fanaticism that I have seen in so many discussions about TDD.

You mention writing tests as part of the development process, which I often do albeit as temporary code to be thrown away rather than checked in, since appropriately simple components tend to be immutable and complete and absolutely should not be intertwined with others that are unrelated. I will make a temporary doggerel call to the methods of the new component, write several normal-usage, edge-case, and error-condition calls and when satisfied, delete them.

I worked with one fanatic dev lead and in talking about this I asked him that if he had a function which couldn’t be broken before the Throne of God, e.g. it simple returned a constant or it added two integers, would he write a unit test for it? I was expecting him to laugh and say “of course not!” but he said that yes, he would, because anything short of 100% code coverage is dereliction of duty, or something military-sounding like that.

Generally when things mysteriously don’t work there is some hidden factor I don’t know about yet; in a recent job I had to learn JavaScript and I never did quite completely figure out why Promises and async/await worked perfectly in most circumstances yet in other apparently identical cases failed. And yes this is frustrating.

When it comes to talking TDD there is an orthodoxy that some follow and some don’t; I don’t believe the original texts require that developers are solely responsible for their own testing, yet I have read a lot of people saying this with table-pounding conviction. I have also heard many times that the tests must entirely precede development and, cue the marching zombie army,

When the tests all pass you’re done.

I am very adamant about design documents (I double as a technical writer) and I do far more planning than most everyone I have worked with, yet I have never seen a case more complex than Hello World when I did not learn things during development that I hadn’t thought of in design. Which means that tests written before code will need to be revisited again and again. I will not accept a client who wants me to work without a spec, and a list of known bugs is not a spec. Requirements spec is unconditional, functional spec is desirable; implementation specs are too volatile and I don’t write them anymore.

I hope I haven’t left the impression that I am opposed to time spent testing. In one job a few years ago I was in my usual role as server developer and was horrified to learn that the company used no staging server; we tested on our own machines, then zipped up some directories and unpacked them on the live server. I am not making this up. I insisted on creating a staging server for testing and customer complaints all but ceased.

2 Likes

In any methodology, there’s hardly anything more annoying than a True Believer. I remember when XP (eXtreme Programming, or as I call it, EXTREEEEEEEEEME RADICAL BRAWNDO INFUSED PROGRAMMING!!!1one ) hit the scene. That was the one that introduced things like “pair programming”, which I don’t think we need to revisit here. But holy crap, were the newfound adherents to this new approach ever annoying to deal with. Nothing like the zeal of the newly-converted.

Thing is, the industry has matured, taken the bits from XP that worked, scrapped the parts that didn’t. But development methodologies do continue to at least incrementally change and adapt, and there’s always going to be some doofus that picks up a new methodology and decides it’s their Golden Hammer. Goes without saying it’s even worse when said doofus is a manager. That’s human nature, and we won’t solve it until the robots take over.

I will make a temporary doggerel call to the methods of the new component, write several normal-usage, edge-case, and error-condition calls and when satisfied, delete them.

That’s basically what unit tests are, or at least what they start like. You just don’t delete them, and you have a test harness that runs them automatically. After you collect a few of these ad-hoc calls , you start filling in with tests for the rest of the interface spec (every public method, more or less). Then you can look at a test coverage graph, or even get the coverage info in your IDE (hit a key and lines turn green or red depending on if they’re covered). The speed and capacity of modern platforms makes all that pretty much effortless.

The industry is a lot more accessible to newcomers and hobbyists than it used to be, and that brings in a diverse range of experience and personality types. During tech booms, there tends to be a rash of “Bro-Driven Development”, where the most irritating of these types rises to the top. But there’s still islands of sanity even in a sea of chaos, if one knows where to look.

1 Like

Anecdotes and personal impressions are fine, but they should not be used exclusively to make decisions.

Let’s look at the research:
TDD reduced defect rate by 50% at IBM
TDD passed 18% more black box tests while taking 16% more time compared to waterfall
TDD at Microsoft resulted with a greater than 2x increase in code qualtiy with 15% extra time
Continuous Integration Testing improves productivity and code quality

Dogmatic adherence to any development methodology, to include Flow, is dangerous. But, Test Driven Development produces code with fewer defects, which indicates that there is some validity to it. Continuous Integration testing produces fewer defects, so there must be some validity to it.

Strict adherence to any development methodology orthodoxy is dangerous and blinds you to the benefits of incorporating other approaches into your workflow. Every development methodology is flawed, but the research indicates that methodologies that emphasize testing reduce defects, so those methodologies must have some valuable lessons that should be incorporated into our workflows.

2 Likes

It should at least have test coverage, which is to say some other code should be using it, and that code should be tested. If you’ve covered all the code except for said bulletproof method, then that method is just dead code that can be deleted.

For a real-life example of this, I had a method adminTypes() that was just return [SCHOOL_ADMIN, SUPER_ADMIN]. No need to test that directly, but test failure in another method showed I forgot to add DISTRICT_ADMIN to that list. My code was just returning a constant, but it was the wrong constant.

Assuming something slightly less trivial, but still Cannot Ever Break, there’s still the notion that your code is a logical proposition, but only verified by execution. If you have static types, then this actually is a formalism (called the Curry-Howard Isomorphism). One of the nice things about static types is that they allow so-called Property Tests, which effectively generate millions of cases of test input for you, edge cases and all. To say nothing of never having to test the type of a return value again.

What one thinks is absolute, immutable, and bulletproof is not necessarily always the case. For example, order of matrix multiplication might be associative in theory, but on actual hardware, you can end up with numerical instability that will cause pixels to clip through a wall. It’s nice to be able to test assumptions about perfection in an imperfect world (and better, to have tools that do it for you).

It’s nothing I’d imagine testing ahead of time, but it’s nice to have test points where one can just drop in the two rendering paths in isolation and see where they differ, rather than just trying out random fixes and hope QA stops reporting “Annoying shimmer effect on walls”. Testable software tends to be much more modular, and that’s a benefit that’s independent of the tests themselves. That’s something you only get when the developer also writes tests.

1 Like

I think the ability to formally verify code is one of the most exciting features of Rust. (I dislike the amount of Rust fanaticism, but I’ve found the language great to work with. And it’s the language that sold me on their style of documentation with automated doctests.)

Specifically, the RustBelt team has put out a lot of good work in verifying the foundations of Rust: https://plv.mpi-sws.org/rustbelt/

This sort of this is my favorite type of refactoring/optimizations to work on.

Funny, that kind of thing is my absolute least favorite. Actually I just took that example from a game development anecdote I read way back in olden times of yore: my boss could tell you about the fun physics bugs he had to work out when writing Myth: TFL. I bet he wishes he had the testing capability of today’s machines.

Speaking of bosses, I’d best get back to doing actual work :wink:

1 Like

Flow is a state of mind, not a methodology.

Whatever you want to call it, you appear to be describing a software development technique, approach, or methodology. Whatever you wish to call it, you seem very adamant that it is the One True Way.

Amen to that.

Happy to hear you specify PUBLIC methods because I’ve read a lot of advocacy of testing every method, which means making private methods into public ones just to be testable, and that’s where I get off the elevator.

Where I differ is the idea of writing tests before coding. That’s lunacy.

I still delete them. If I end up altering a method that was purportedly immutable I will write new tests. Problem is, that pretty much never happens.

> Whatever you want to call it, you appear to be describing a software development technique, approach, or methodology. Whatever you wish to call it, you seem very adamant that it is the One True Way .

Now you’re just being combative. Later.

I’m not trying to be combative at all. I’m just disagreeing with what you think.

You seem to be describing a code development [state of mind, technique, methodology, or whatever word feels accurate] and advocating for its superiority to other code development methodologies/practices/frameworks/ect. To me, it appears that you are advocating for your code development state of mind/methodology/technique/practice/framework/etc as a One True Way, or a single correct way to develop software. I disagree that Flow is a One True Way because there are no One True Ways.

If the alternative is little or no testing then the reduction of defects by doing some testing is hardly revelatory. This in no way argues for reversing the order of the work.

If the methodology means more meetings, count me out. Because the best I can do in the great majority of meetings is stay awake.

If you look at the academic studies looking into the potential for benefits in TDD (the handful I linked and the multitude of others), the control groups used regression testing written after the software was developed. The studies specifically were looking at the key feature of TDD - when testing is incorporated into the development process.

Nothing about TDD requires increasing your meetings or firing the QA team.

And to reiterate - I do not see TDD as a One True Way because there are no One True Ways. But research shows that it reduces defects, so there must be some good practices in the TDD methodology that can be broadly applied to reduce deficits compared to a traditional post testing methodology.

Please pass this on to the many, many obsessives who proclaim that if it isn’t TDD then it isn’t software development.

If I had a nickel every time I’ve read that I could buy a pretty nice CD box set.

THIS

I’d be very careful of falling into the trap of “what I see, is what is” at this point, simply because one is biased to ones self and their own experiences.

If you want evidence of getting near, or even into this trap. Examine how many times you use the word “I” or “me” when explaining your position. If you rely primarily on your own experiences to justify a general point, you end up in what is essentially a logical fallacy.

edit Also, watch out for Straw Man fallacies as well

1 Like

I have never met anyone who says that any software development methodology that is not TDD is not software development, but if I ever do I will tell them. There are lots of ways to write code. Some of them make ‘better’ code. Some of them do not.

Could be a nomenclature quibble but when I say “methodology” I am referring to that agile/scrum horseshit. I never heard of waterfall except from people deprecating it.

And if you want to tell me that agile and scrum don’t add shit-tons of meetings then I’ll just file that in the same round basket that includes “Microsoft says.” Go read about Microsoft in Glassdoor reviews and tell me with a straight face that you believe a word. “There’s nothing they could possibly improve!!!” All with wordings as suspiciously identical as RW trolls on political forums. I worked there for ten years between 1989 and 2009 and saw their disintegration with my own eyes.