When the numbers and the interviews disagree
Most journals treat a contradiction between methods as a problem to fix before submission. We think it is often the finding, and we will publish it.
A service platform cuts average handling time by a fifth. Interviews with the people it serves find them markedly less satisfied, and able to say precisely why. Both measurements are competently made. They do not agree.
What usually happens next is that one of them is cut. The author anticipates a reviewer asking them to reconcile it, finds they cannot, and solves the problem by submitting the half that holds together. Two papers, two venues, two audiences that do not read each other.
The journal exists in large part to stop that happening.
Why the contradiction is usually real
Technical systems are optimised against a measure. The measure is a proxy, chosen because it can be counted. The gap between the proxy and the thing it stands for is not noise to be designed away; it is a substantive property of the system, and the people inside it experience it directly.
When handling time falls and satisfaction falls with it, the most likely explanation is not that one instrument is broken. It is that the system got better at the thing being counted by getting worse at something adjacent that nobody counts.
What we ask instead
We will not ask you to reconcile findings that do not reconcile. We will ask you to:
- Show that both measurements are sound, in enough detail that a reader can judge.
- Rule out the boring explanations, sampling, timing, instrument change, a confound that would produce the divergence artefactually.
- Say what you think is going on, and be clear about which parts of that are evidenced and which are your reading.
- Resist the urge to resolve it in the discussion section. An honest "we do not know why" beats a mechanism you cannot support.
Who reviews it
Papers of this kind get reviewers from both sides, someone who can assess the quantitative work and someone who can assess the qualitative, and we tell them explicitly that disagreement between the two strands is not a defect to be corrected. That instruction matters: the usual default is for each reviewer to treat the other method as the weak link.
Methods papers on how to design, analyse and report this kind of study are welcome in their own right, and are reviewed and published on the same terms as empirical work.