Known issue. It was discussed over 4 years ago on the quilt-dev list:

https://lists.nongnu.org/archive/html/quilt-dev/2018-01/msg00000.html

Unfortunately we could not find a clean fix. Since then I'm using the workaround I posted in the discussion thread in the SUSE package so that it passes the test suite reliably. I'll attach it to this bug. It's a bit more costly than yours, as it requires running the problematic diff command twice, but I think it's more readable and actually tests the expected behavior more accurately.

I'll commit the workaround to the upstream quilt repository so that we no longer have to carry it as a separate patch in distribution packages.

We had a run of the quilt test suite fail:

[faildiff.test]
[1] $ mkdir patches -- ok
[3] $ quilt new test.diff -- ok
[6] $ cat > test.txt -- ok
[8] $ quilt add test.txt -- ok
[13] $ chmod -r test.txt -- ok
[15] $ quilt refresh -- ok
[18] $ echo 1 -- ok
[21] $ chmod +r test.txt -- ok
[25] $ printf "\\002\\000\\001" > test.bin -- ok
[26] $ quilt add test.bin -- ok
[29] $ printf "\\003\\000\\001" > test.bin -- ok
[30] $ quilt diff -pab --no-index -- failed
Diff failed on file 'test.bin', aborting !~ (Files|Binary files) a/test\.bin and b/test\.bin differ
Binary files a/test.bin and b/test.bin differ != Diff failed on file 'test.bin', aborting

Note how the test read the right lines, but in the wrong order.

This is because there's an inherent race condition in the test logic. run redirects stderr to stdout and expects the lines to be read in the order that they were written, but thanks to buffering that isn't always going to happen.

A race-free solution would probably involve not merging stdout and stderr, and matching on them explicity.