Testing Thoughts

I am not the original author of this post’s contents.

I recently found this email reply from John Mitchell while discussing the testing aspects of micro services in our project. It is so informative that I didn't want this to get lost in emails. Adding this here.

(Names have been changed to keep the focus on the topic under discussion)

Hi PersonA,

I think our discussions about the Quality Criteria were pretty clear…

• And to be even more precise, they must be able to be automatically run as part of the appropriate phases in the pipeline.
• APIs
• Message / file formats, etc.
• UI’s which they provide
• Telemetry which they generate for business/application/operation/etc. monitoring/alerting/tracking/etc.

Martin’s slide that you pointed is no excuse to get out of any of the above. He’s pointing out, as I did, that false metrics such as “code coverage” percentages are misleading and inefficient ways to measure actual quality of the software (as opposed to trying to force the team to spend time creating “tests”).

Again, as we discussed in our last meeting, if you achieve anything even remotely close to my list above (and on the quality criteria page) you will have created software that is quite high quality.

As I noted, people must stop over-indexing on this so-called end-to-end testing to the exclusion of the rest of the criteria. That’s what’s lead to the current mess that we have with the AppA bundle today. It’s precisely the fact that each of those various angles/levels of testing are done (automatedly) that provide what we have talked about in terms of being able to bracket problems for much faster triage and debugging, achieve the sorts of testability for multiple uses, provide reality based assurances to InfoSec, the app teams, the management, etc. that are stuff really does work.

Thanks,

John

Read More