View Issue Details

IDProjectCategoryView StatusLast Update
0001717XdebugCode Coveragepublic2019-12-09 08:04
Reporterjhassler Assigned To 
PriorityhighSeveritymajorReproducibilityalways
Status closedResolutionfixed 
Product Version2.8.0 
Target Version2.8.1Fixed in Version2.8.1 
Summary0001717: Code coverage turned slow after update from 2.7.2 to 2.8.0
Description

The code coverage performance in our CI pipeline suddenly decreased after Oct 31 due to the update from xdebug 2.7.2 to 2.8.0.

Before (xdebug 2.7.2): 8 minutes
After (xdebug 2.8.0): > 3.5 hours

PHPUnit 7.5.12
PHP 7.1

I switched the pipeline back to 2.7.2 and the coverage performance was back to normal - that's why I can exclude side effects due to other versions.

Steps To Reproduce

Can someone confirm this?

Tagsperformance, php7.1, php7.2, php7.3
Operating SystemDebian
PHP Version7.1.30-7.1.34

Activities

Anthony

2019-11-19 16:12

reporter   ~0005180

We've also had similar performance issues. We originally noticed it in CI, but didn't realise what the problem was. Our PHPUnit tests with coverage reports went from taking a little over 2 minutes, to over 25 minutes. When I looked back over the logs I realised that the last fast run was with Xdebug 2.7.2, and the first slow one was using 2.8.0. Rolling back to 2.7.2 has restored performance.

We're running this in Docker. I've attached a copy of the Dockerfile, along with the PHP config.

I've also run Strace against both 2.7.2 and 2.8.0 to see what's going on. The specific command that I ran INSIDE the container was:

 strace -r -c vendor/bin/phpunit --testdox --colors=always --coverage-html report

I've attached the output from Strace for both 2.7.2 and 2.8.0 as well.

I hope this helps. In the meantime, we're locked to the older version and our tests are running quickly enough for us

Anthony

2019-11-19 17:31

reporter   ~0005181

Just tried again with this command: strace -r -Tfe trace=open vendor/bin/phpunit --testdox --colors=always --coverage-html report

Here's a sample of the output:

 O running tests     0.971306 open("/var/www/html/tests/../spool/phpunit", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 9 <0.001046>
 O running tests     0.557823 open("/var/www/html/vendor/phpunit/phpunit/src/Framework/MockObject/Matcher/ConsecutiveParameters.php", O_RDONLY) = 9 <0.000551>
     0.020636 open("/var/www/html/vendor/phpunit/phpunit/src/Framework/MockObject/Stub/ConsecutiveCalls.php", O_RDONLY) = 9 <0.000791>
     0.053992 open("/var/www/html/src/Foo.php", O_RDONLY) = 9 <0.000989>
     1.311633 open("/var/www/html/vendor/phpunit/phpunit/src/Framework/MockObject/Stub/ConsecutiveCalls.php", O_RDONLY) = 9 <0.001083>
     0.006750 open("/var/www/html/src/Foo.php", O_RDONLY) = 9 <0.001176>
     0.005525 open("/var/www/html/vendor/phpunit/phpunit/src/Framework/MockObject/Matcher/ConsecutiveParameters.php", O_RDONLY) = 9 <0.000333>
     0.009173 open("/var/www/html/src/Foo.php", O_RDONLY) = 9 <0.000794>
     0.003583 open("/var/www/html/src/Foo.php", O_RDONLY) = 9 <0.000782>
 O running tests     0.085541 open("/var/www/html/src/Bar.php", O_RDONLY) = 9 <0.000818>

I've tried running it with the volume in default, delegated and cached modes, but the performance is the same

savar

2019-11-19 23:51

reporter   ~0005182

same issue for us. using phpdbg helps a little bit but it is still slower than the 2.7.2 version of xdebug but extremely faster than 2.8.0 xdebug

challgren

2019-11-22 22:10

reporter   ~0005183

Same issue for me. 2.7.2 resulting in my CI run time of 8 minutes. With 2.8.0 it times over after 1 hour.

challgren

2019-11-22 22:31

reporter   ~0005184

FYI for my CI I use php:7.3-alpine, php:7.2-alpine and php:7.1-alpine. Happens on shared Gitlab CI runners and local runner on MacOS 10.14.6. Rolling back to xdebug-2.7.2 improved the run times.

derick

2019-11-25 07:42

administrator   ~0005185

FWIW, I can reproduce this, and I know where the problem lies — but not quite yet what the problem is, and how to solve it.

derick

2019-11-29 15:47

administrator   ~0005191

Fixed in GIT for 2.8.1

Anthony

2019-12-03 12:24

reporter   ~0005192

We're still finding performance very slow in CI with 2.8.1

Attached is the strace that I ran locally. It took over an hour to run coverage locally with strace. CI has gone from 1m 52s to 9m 58s for PHPUnit with coverage alone. It's better then 25m, but it's still too slow to be practical for us.

We're only on PHP 7.2, looking to upgrade to 7.3 some time soon. Because we use ElasticBeanstalk and they always take almost a year to catch up to the latest PHP release we've got almost a year before we'll be able to upgrade to PHP 7.4, and since 2.7 won't work with 7.4 we'll see how the situation looks then.

tersmitten

2019-12-06 14:25

reporter   ~0005194

We're still finding performance very slow in CI with 2.8.1

We too.

Our builds used to take 15-20m, now > 2h

tersmitten

2019-12-06 14:28

reporter   ~0005195

We're using PHP 7.1.33 btw (on Ubuntu 16.04)

PHPUnit version 5.7.27

alex.m

2019-12-09 08:04

reporter   ~0005204

Confirm. Ubuntu 16.04 + PHP 7.1 + Xdebug 2.8.1 extremely slow