I recently submitted a patch for Test::More that improves subtest readability, but might break fragile test modules. As part of the pull-request discussion, I promised to blog about the change to warn people, so here it is.
When Test::More added subtests (usable as of 0.96), the name of the subtest only appeared at the end of the output, like this:
ok 1 - regular test
ok 1 - first test
ok 2 - second test
ok 3 - third test
1..3
ok 2 - my first subtest
That’s a bit confusing, because it looks like the indented tests belong to the “ok - regular test”, not the “ok 2 - my first subtest”.
In Test::More 0.99, a leading comment was added to clarify:
ok 1 - regular test
# Subtest: my first subtest
ok 1 - first test
ok 2 - second test
ok 3 - third test
1..3
ok 2 - my first subtest
That’s better, but still hard to pick out in a large amount of test output.
My pull request outdents the comment like this:
ok 1 - regular test
# Subtest: my first subtest
ok 1 - first test
ok 2 - second test
ok 3 - third test
1..3
ok 2 - my first subtest
I think this is much easier to read because it brackets the subtest information at the same indentation level. The subtest details are clearly between the opening comment and the matching “ok”.
This is likely to be merged and released soon.
Since comments don’t have any meaning in TAP, it shouldn’t change how tests are parsed. But CPAN test modules might — in their own tests — be looking for specific output and start breaking. Such tests are inherently fragile, and should be changed to use a TAP parser instead of parsing literal text, but in the meantime, if your favorite test class fails its tests, this Test::More change might be the reason.