This test case is way too big. Can you please give me something where:

• I don't need Docker
• There is a simple test, without all of phpunit, where two code coverage runs show this problem?

You need to show what your expected result is (not just "two functions in file", but the real contents, and what you expected. Small test cases are best, and ones that don't require PHPUnit are highly recommended.

I will do my best to reproduce a more minimal, self-contained example. Apparent scope + size were one of the reasons I went with the Docker + PHPUnit route, but I understand the debugging difficulty.

Hi derick - I updated the linked repository to include a more self-contained example that doesn't run on Docker and doesn't use PHPUnit. It looks like any run through a file, regardless of if the functions are static or not, will produce a different count of the functions within the file.

Direct link to the 3 files needed for this example:
https://github.com/jmrieger/xdebug-issue/blob/master/driver.php
https://github.com/jmrieger/xdebug-issue/blob/master/SampleClass.php
https://github.com/jmrieger/xdebug-issue/blob/master/SampleStaticClass.php

It's not as "small" as it could be, but I did want to demonstrate that the unexpected behavior isn't limited to whether a function is static or not.

Additional clarity on my statement above: The first pass through when processing code coverage returns a result I expect - it identifies the actual number of functions present in the specified file, as well as coverage information for each of the functions, regardless of if the function was called.

The second pass through each file, the number of functions present in the specified file only includes functions which were called during the code coverage run. I expect the behavior to be the actual number of functions in the file, with only the branch and path information varying.

Hi,

I've had a moderately good look, and with your smaller example I can reproduce it. I think I know that the problem is that the xdebug_code_coverage_stop() call also removes all discovered information regarding branches, paths, and possible executable/non-executable lines. This can be fixed, but it is hard to do that without refactoring much of this code. I therefore don't think I want to do the refactoring for Xdebug 2.8, but rather leave it to 3.0. It will also likely drastically improve performance.

With as caveat being me not knowing how PHP_CodeCoverage exactly works, I do believe Sebastian has some workarounds for the merging of runs. In theory, you should be able to find out for each "run" which lines, paths, and branches have been hit; just not which ones that haven't been. You should be able to extrapolate that from earlier runs (and you'll know which lines won't have been hit too). If it's code that hasn't been seen before, these new functions will have the full list of branches, paths, and lines in the output. I know, it's not ideal, but I think it is possible to construct a full result for each non-first "run" with some logic.

Happy to chat about this on IRC or something like that about it — please email if you'd like to do that.

For now, I'll have to re-target this for Xdebug 3.0.

cheers,
Derick

Re-targeting to 3.0 is more than acceptable, given the scale of changes you mentioned.

I did note that much of Sebastian's code revolves around merging runs, though how I ran into this issue was related to that implementation - if you unintentionally forget the coverage data (or even intentionally discard it, say for indirectly covered code), you lose the visibility into earlier coverage data.

I'll check my calendar and see if there's a time we can meet on IRC. Any additional support that I can provide to help with the improvements I'm more than glad to provide.

Thanks,
Joe

Hi!

I have now finally sat down to have a good look at this, as well as some
other issues. In order to fix this, I had to split the analysing part
(scanning opcodes) and collecting data during execution, into two separate
data structures. This work is part of the improve-code-coverage <https://github.com/derickr/xdebug/tree/improve-code-coverage>_ branch.

The good news is that this fixes the problem, and several related parts.

The bad news is that this makes code coverage twice as slow.

This is likely because there are now more data structures, and hence memory
allocations and frees are involved, but I've not been able to pinpoint the
real cause, nor have I come up with some plan to address this new slow down.

I therefore don't want to merge this branch yet, and I'm moving the target
for this Xdebug 3.2. But, please try out the new branch, as it's possible
that there are still some pending issues.

cheers,
Derick

The fix for 0001799 is the same, so I'm closing this issue out.