This is my test plan. There are many like it, but this one is mine.
My test plan is my best friend. It is my life. I must master it as I must master my life.
Without me, my test plan is useless. Without my test plan, I am useless. I must fire my test plan true. I must shoot straighter than my enemy [defects] who is trying to kill me. I must shoot him before he shoots me. I will...

Revised Rifleman's creed, US MOD, revised to be more useful.

This is a grouping article, holding an introduction, and links to more narrow subject content. It is an epic, in that this joins many story points together (maybe I am overusing this terminology, but I think it fits).

Why developers use test doctrines?

  • It enables you to charge more per hour, as your work is better quality;
  • Using TDD enables your timescales to be meaningful (so can charge more);
  • Using BDD enables more confidence that your solution matches the requirements (so can charge more);

Project managers, you like tests because

  • Tests reduce risk, so you have more control;
  • TDD means all data needed for the solution is known at the start, so you have more confidence in being able to deliver;
  • BDD and TDD reduce the amount of rework needed, so timescales are more secure;
  • I think TDD reduces costs (mostly the last point), so the TCO is much smaller;
  • Testing allows revenue generation faster, as the solution is in a reliable state faster;
  • To be equal with motivation, the above points mean you can charge more.

Business owners

In analogy, compare software purchase to shoes:

  • If you live in a world where you don't need shoes, I have nothing to offer you (i.e. bar staff won't hire developers);
  • You can buy a pair of shoes frequently (i.e. less good code, less reliable deadlines), or a decent pair that lasts longer (software engineering process). If I run in plimsoles, I need to replace them after about 5mi, which is often 1days distance. Managing shoe supply on that frequency is very demanding. I don't like being constrained to walking, to save the shoes;
  • For alot of things, it is more common sense to use a highstreet shoe than a custom handmade pair (good developers will use frameworks and common libraries, then your custom bits);
  • Due to the special nature of software; clever well-made shoes ~ for example DocMartin ~ don't go out of fashion (e.g. CMS features, so your software says your marketing current story);
  • When buying, economies of scale make a large impact on price (i.e. buying a pair of Brand X from a big city outlet will cost less than the same Brand X shoe in a 'boutique shop' in a small town). Against software engineering, hiring the right person from the start is cheaper;
  • When applying these techniques, you can get revenue earlier, more reliably/consistently and at lower risk.

What are unit tests?

  • A unit test is the process of checking results for a given input; on every method in the project you are building. This helps isolate failures, and the systematic approach to the tests provides a more systematic code quality.
  • The basic Wiki article, the artoftesting or techtarget. My unit-tests article is a longer version on this paragraph.

What is TDD?

  • TDD is the process of verifying your intentions at the cheapest point (first), by writing lots of unit tests. By focussing on deliverables first and making an executable specification; then doing a second leg to tidyup and improve code quality; one is often faster.
  • Is is a contraction of test-driven development.
  • For the formal definition, please read wiki.
  • For some other notes or more, and so on

What is BDD?

  • BDD is extending the ideas of TDD; to reduce the risk of miscommunication between the stakeholders and the developers, add an executable specification in business English. It is necessary to use BDD and TDD, as BDD focusses on how users solve their problems, not details on whether a particular class is correct.
  • Is is a contraction of behaviour driven development.
  • A full definition is on wiki, and it is discussed in 1 and 2

What is Regression testing?

  • Regression testing is a process of applying your existing unit population after later changes to ensure that you haven't broken existing and older features.
  • Regression testing is discused in more detail 3 4
  • In practice this should be done in every project, as it reduces costs. It is very simple to setup. I would suggest adding this step to your build/deploy tool.

What is Mutation testing?

  • This is currently fashionable in high quality PHP build processes. Oddly this has no acronym yet. Mutation testing tools report weak areas of the test coverage by appling a mutation to the target code.
  • This is discussed in more detail in 5 6 7.
  • I have no further details to log in this section at present; there are mutation test frameworks in PHP, Python, JS. Lastly a vast number in Perl, presumably as it is easy to make errors in Perl.

What is boundary testing?

  • Boundary testing is unit-testing where the extreme values are checked; to confim that possible error states are met correctly.
  • This is discused in more detail at 8 9.

My projects & tools

  • As I started using unit tests in PHP a long time ago, I have been using phpunit a long time. If I am writing code with a good level of understanding and confidence in the spec I apply TDD properly, and use a skel-gen, I think this saves lots of time.
  • I gave a talk on “Test Strategies && Process” to PHPhants
  • I am accumulating some small support scripts (like everyone does), to reduce duplicated work. These are mostly base classes with utility functions. An ongoing project is available from 10. The most significant one is a JS script that extracts all the human text from a rendered page, so it can be spell checked
  • I am unable to find a skel-gen tool for Perl, so I created punit, source code is available from a VCS. The article link holds my research looking for existing solutions before I wrote this. I have yet to do the paperwork process to get this in CPAN, but I intend to.
  • NB: Strict TDD practise is harder in Perl, as it doesn't have Interfaces.
  • My article on unit-tests, on logging.
  • JS test frameworks, I wrote some notes on JS memory management, Node test tools [XXX] (mostly the same as JS, so article got delayed).
  • I haven't written much volume of Python, however I use unittest, which ships as a standard library.
  • When I make API that emit complex documents, and so use the appropriate tool: a schema validator. Around 70% of my requirements are met by feeding output into a syntax checker, or doing the same in the browser.
  • I use BDD tools to verify complex rules on REST API are correctly followed. When the business asks for alot of behaviour, for example on price discounting, this delivers quality quickly. I am told this is wrong by BDD purists, but the process delivers results. To list a set of conditions, a validation schema is the wrong tool.
  • I sometimes use the techniques listed towards the end of this article, as the sitaution demands.

Why do I test like this?

I don't have any deep and needy feeling about tests. The various flavours of tests are a risk assessment, or rather the avoidance of risk. Things normally go wrong, not on first version, but on the later edition that just make a simple change. These above technologies are the only known solution to being able to avoid these risks.
A software engineer is someone who creates tools and manage the risks on creating them. A developer is just doing the first clause. I think in an Agile environment, software engineers are more useful.

Testing Epic

RSS. Share: Share this resource on your twitter account. Share this resource on your linked-in account. G+

Testing Epic

RSS. Share: Share this resource on your linked-in account. Share this resource on your twitter account. G+ ­ Follow edited