Anti-Patterns: TED (Test-Eventually Development)

Software developers are good at writing applications. Testers are good at testing applications. In the software development world where separation of concerns is a never-ending quest, it seems logical to apply the rule to the software development cycle. “Let the developers code; QA can be responsible for making sure it works.” It provides an object-oriented management approach where each team is responsible only for what they are best at, and the two teams interface through one simple property, each: the result of their logic and expertise. The system is of logical design. It appears to be a logical pattern. Or so it seems.

When we practice this pattern, when our developers focus purely on writing code and not on writing quality code, when quality assurance is the last stop before the bus leaves the station, the development cycle suffers and projects run long and over budget. Modern software development is about immediate feedback. We must develop our applications on short timelines and with fewer bugs than ever before. Because of this, the role of the tester and the role of the developer have changed. Quality is everyone’s job.

From Tester to Quality Analyst

The staffing model surrounding quality has gone through a mind shift. Quality Control is now Quality Assurance. I have worked for companies in the past that felt the name change meant simply a new run of business cards, and each one of these companies is dead or dying. Successful business endeavors understand that there is more to the title change than find-and-replace in Microsoft Word.

Gone are the days where testers are isolated from the development process, the days where testers are corporate grunts that follow a decreed, rock-chiseled click-path like a group of monkeys. Quality Control would run a predefined script against random sampling of product batch, and fire green or red flares into the sky. Red flares cause the batch to be redone and shipped. Problem solved. Lather, rinse, repeat. But this model of quality is expensive. The bad product batch is already completed, and the money on production, materials, resources, and staffing has already been spent on building botched bits; now the company is also spending money on time, since the batch must be rebuilt, delaying delivery. Though output testing is still an integral component of quality assurance, the primary responsibility has shifted to answering “How could this have been prevented?”

Don’t test bugs out of your application, design them out.

Quality Assurance, the “just a new business card” kind, is the final gate before shipment. Quality Assurance, the “successful business endeavor” kind, is the final gate before shipment, the first gate before development, and every gate along the way. Quality Assurance is involved in requirements capturing, ensuring that the feature request is no longer a single-line description of “Add a memo field to the database to store Item Version.” Instead, Quality Assurance ensures that the requirements include the business problem, the intended solution, and use cases, all of which lead to discovery that the field should be an integer, that the field value should be unique for each item, and that only the most recent version should be returned on item search. Without this effort, that discovery occurs later in the development process, often when we present the implemented solution to the customer to a response of “That’s not what I meant.” We add scope to change the feature, which changes budgets, changes timelines, and all for an issue that we could have prevented up front during the design phase. Eliminate the ambiguity, the confusion, the rework, and the scope slip. Eliminate the potential of “subject to interpretation,” where what the client described, what the business analyst recorded, and what the developer implemented all differ.

Quality Is for Developers, Too

The result of the shift to Quality Assurance is that ensuring a quality product becomes everyone’s job. If a bug made it through to final testing, the system failed. Something went wrong. The defective code was already written, the resources used, the money spent, and now we’re losing time. Budget and timeline overruns ensue. Final testing should be nothing more than an exercise to validate success, and to do that developers need to write correct functionality and quality code the first time.

For developers, quality code is about unit tests. It’s about Steve Ballmeresque “Unit tests, Unit tests, Unit tests.” Code should be self-testing, with feature code being developed concurrently with, and driven by, test code. Using unit testing frameworks like MSTest, MBUnit, or NUnit, combined with test-driven development patterns, developers can write test code that instantly and constantly analyzes the correctness of their feature code. Positive use cases can be tested automatically. Boundary conditions can be tested automatically. Negative conditions can be tested automatically. Traditional test cases can be converted to automated functional tests. Each line item in the requirements document can be converted into code and an executable, reusable test. And we can execute these tests many, many times in an automated fashion, eliminating much of the burden from old-school Quality Control’s post-development testing. The traditional test model is replaced with instant feedback and instant verification on the status of a function, a feature, or an entire build, shortening the development cycle, and helping developers create correct functionality and quality code the first time.

Walking the Path

I understand that this appears to be a lot of extra work. “My development and quality teams are already stretched to capacity. This all sounds nice, but we just don’t have time for it.” And to be honest, in the short term it is more work. If you are in the final stretch of a development phase, and a major release is only a month away, you won’t see any timeline gains before your deadline. Remodeling your pursuit of quality cannot be executed overnight, and it won’t save you from an imminent disaster. Projects that benefit most will implement these changes throughout their development cycle, and walk the path from the start, so consider implementing changes with the kickoff of your next development cycle.

Include QA in all aspects of your project lifecycle. Again, it sounds like a lot of work, so start small; involve QA in design meetings before any coding has started. In my experience, Business Analysts tend to have a corporate or marketing approach to solving a client’s problems. Architects use a technical approach. Neither seems to reliably keep the application’s primary audience in mind, and the result is a technically sound application that solves the business problem, but is very painful to use. Quality Assurance has a lot of experience with this pain from repetitive use of the system during testing; by applying this experience, they can provide valuable feedback at the design phase to create more intuitive stories and click-paths. QA is also behaviorally trained to approach any task with a pessimistic eye. “I know that this application isn’t perfect. There are bugs in here somewhere and I will find them.” This same eye that will identify bugs in a completed application will also identify bugs in the problem statements, requirements, specifications, and story boards. If not identified and corrected, these bugs in the design phase will become bugs in the application, and will be more expensive to fix.

Developers should start with writing unit tests for all new code. Do not worry about back-coding tests for all existing/legacy code; test coverage for the full application will come with time. Even if this initiative generates just a small handful of tests, it is better than no tests at all, and again, coverage will come with time. As a clarification, new code should include not only development implementing new features, but also development correcting open bugs and issues. I consider issue resolution (or bug fixing) the best way for a developer that is new to unit testing to become familiar with authoring tests. We know that the code is broken, flawed, or incorrect in some way, and issue resolution presents developers with an opportunity to write a surrounding test that exploits the problem as well as succeeds upon correction. A developer’s natural first step is to reproduce the bug as a method of understanding the issue-we are hard-coded to do so-and to the same effect, developers test the code after making the correction. Writing an appropriate test, fixing the code, and verifying that the test succeeds are steps that follow this same habitual pattern: prove the existence of a problem, fix the problem, and verify. Because of our diligence during issue resolution, unit tests provide a method for automation, as we can let the computer do the testing for us. However, we still seem to test the code ourselves, despite the available automation, which helps us also validate our tests and refactor our concepts and understanding of unit testing when we find flaws that the test did not identify. Each test we write is more effective than the last, until the ah-ha moment occurs where we understand the power of the unit test. Because of this opportunity, it is easier to learn test authoring while correcting code than while implementing new features. But, be aware of the pitfalls; many of these opportunities can be short-circuited by simply executing the code. It is vital to identify why the code is flawed, as this flaw is what the unit test should exploit, and what leads to unit test enlightenment.

All new features should have a full set of unit tests; we do not want to dig a deeper hole by creating more functionality without the self-validation of unit tests. Also, writing tests while correcting issues will, over time, provide test coverage for the old legacy code. Implementing test-driven practices for any code modification will temper your application for the quality assurance battlefront. The basic, simple, fundamental, why-didn’t-you-catch-this-earlier bug will be caught by the developer, removing much QA involvement on code-based defects, and making resolution for these issues faster and cheaper than a model without unit tests. QA can focus on application polish and identifying more robust and complex issues, like identifying performance problems, memory leaks, or SQL injection vulnerabilities.

Success Is a Team Sport

Everyone shares the responsibility of producing a quality application. Involve QA and development in the business analysis and requirements capturing processes, and design the bugs out of the system, where correcting them is at its most inexpensive point. Replace the test-eventually development model with a test-driven development model, and write tests religiously, where tests can ensure that the right code was written, and defects can be caught early where, again, they are cheaper to fix. Abandon silos and tear down walls; make quality a collaborative effort so that a project’s success is the responsibility of the entire team.

Jay Harris

By: Jay Harris

Jay is a .NET developer, a software consultant, and is president of Arana Software (www.aranasoft.com). He has been developing on the web for 15 years, since he abandoned VB3 for JavaScript because he didn’t have to wait for a compile. With a career focus on end-user experience, he is a strong advocate of practices and processes that improve quality through code, ranging from automated testing, continuous integration, and performance analysis, to designing applications from the user’s perspective. Jay is also an active speaker and leader in the developer community, serving as President of Ann Arbor .NET Developers and is co-founder for Lansing Give Camp.

Originally from Rochester, New York, he and his wife, Amy, have lived in Michigan since 2003. They like Michigan, but still consider themselves tourists, and probably always will.

jay@aranasoft.com

Test-Eventually Development

When writing business logic is more important than writing the right business logic.

My friend Jeff and I were developing a charity application at the 2008 Ann Arbor Give Camp, and we were running behind due to a number of bugs in the system. I was poking fun at Jeff because he hadn’t written any unit tests for his code.

Jeff: “Well, I was going to write them eventually. I just haven’t gotten around to it, yet.”

Jay: “So we’ve abandoned Test-Driven Development for Test-Eventually Development?”

We laughed, and TED was born.