The term is being used to describe the context under which activities such as automated unit, integration, performance, and integration tests are executed, with that context being the release of frequent small batches of code to production, with minimal interaction from humans. It refers to the process of assessing risk on a continual basis in high frequency delivery environments.
It describes the inclusion of automated checks (or tests) into your development workflow, and a 'shift left' in terms of where testing happens in the development workflow. The human interaction aspect of continuous testing refers to the increasing validation of business requirements before they hit development teams through prototyping and business value analysis, and an increase in, or rather, focus on the amount of in team testing and crowd testing that occurs.
Most will recognise these activities to be an integral part of any lean, agile, XP, continuous deployment or continuous delivery workflow or pipeline.
Wikipedia describes continuous testing as
..the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate.I'm not a big fan of this term to describe these activities as it conjures up the notion that continuous testing is a separate set of activities and not integral to a successful development workflow or pipeline. It drives a wedge once again between the concept of testing and development which need to go together hand in hand. A wedge that has troubled us so much in the waterfall years, and a wedge that has still existed throughout the general adoption of agile practices.
Do we really need the term continuous testing to group together activities, processes, mechanisms, etc. that are already happily described in detail by those championing devops, continuous delivery, continuous deployment, xp, etc.? What value is this terminology bringing to the software delivery table? I just find it confusing.
I would hazard a guess and state that the term is being used in organisations where testing is sequential and costly, and not yet integral to the development workflow, and where little or no test automation exists to gain a traction with difficult to implement activities. Conceptually I can see how the term can help those in waterfall environments, or sudo agile environments to begin embracing a shift left in their testing process.
Whilst more mature developers and teams won't be impacted too much by the term, it's the consultancies, the large archaic organisations, and misinformed software professionals that worry me. Already terms such as devops, test automation, etc. have been hijacked by many to construct new functional silos within the development ecosystem, rather than embracing it as a cultural change. The very same could happen with continuous testing. Look out soon for job ads with the title "wanted - head of continuous testing"!
When you read some of the writing and blog posts on continuous testing, you will notice that some have attached roles to continuous testing, as if it was now someone's responsibility. The holistic value that continuous testing attempts to describe gets blown away once it becomes someone's responsibility.
Whilst I complete agree with and promote most of the practices I see being written about, I can't help but think that grouping things like automated unit or performance testing, or monitoring under the banner of continuous testing could encourage the removal of the responsibility of quality from the entire team, handing it over to a separate group, taking many organisations right back in time. Please don't let there be a Head of Continuous Testing appearing in your organisation!
Some Continuous Testing sources
https://www.stickyminds.com/interview/putting-quality-first-through-continuous-testing-starwest-2015-interview-adam-auerbach
https://blog.parasoft.com/continuous-testing-devops-infographic
https://www.soasta.com/webinars/continuous-testing-in-devops/
