View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0001123XdebugUsage problems (Wrong Results)public2015-03-04 13:102018-07-17 07:22
ReporterravageAssigned Toderick 
PrioritynormalSeveritymajorReproducibilityalways
Status closedResolutionfixed 
PlatformLinux 3.13.0-46-generic x86_64OSUbuntu ServerOS Version14.04
Product Version2.3.1 
Target VersionFixed in Version2.4.0beta1 
Summary0001123: With Xdebug 2.3.1, PHPUnit with coverage is exponentially slower than without
DescriptionAfter upgrading to Xdebug 2.3.1, running the unit test suite of an internal project takes exponentially more time with code coverage than without.
Before with Xdebug 2.2.5 (and similiar), it took FAR less time with code coverage.

It's a CakePHP 2.6.x application using PHPUnit 3.7.38 (use of PHPUnit 4.x+ not possible).

The suite contains:
Tests: 821, Assertions: 2815, Incomplete: 15.

PHP 5.5.9-1ubuntu4.6 (cli) (built: Feb 13 2015 19:17:11)

Expecially integration tests of controllers are way slower than before.

Nothing else has been changed except the Xdebug version.

Interestingly I have yet not found any reports of 2.3.1 being so slow.
Currently I do not need to upgrade to the Xdebug 2.3.x but I wanted to document the bad performance I experienced, even if applies only to a fraction of other users.
Steps To Reproduce--------------------------------
Without Code Coverage
user@my_server:/var/www/cakephp_app/app$ sudo php Console/cake.php test app AllAppTests --stderr --working .
--------------------------------
Xdebug 2.2.5
Time: 2.24 minutes, Memory: 314.25Mb
--------------------------------
Xdebug 3.2.1
Time: 2.3 minutes, Memory: 314.50Mb
--------------------------------
-> About the same time to process
================================

--------------------------------
With HTML Code Coverage
user@my_server:/var/www/cakephp_app/app$ sudo php Console/cake.php test app AllAppTests --stderr --working . --configuration phpunit-html-coverage.xml
--------------------------------
Xdebug 2.2.5
Time: 7.69 minutes, Memory: 317.25Mb
--------------------------------
Xdebug 3.2.1
Time: 33.66 minutes, Memory: 318.00Mb
--------------------------------
-> Takes about FOUR TIMES the time to process
================================

--------------------------------
With HTML Code Coverage
user@my_server:/var/www/cakephp_app/app$ sudo php Console/cake.php test app AllAppTests --stderr --working . --coverage-clover ~/cakephp_app.clover.xml
--------------------------------
Xdebug 2.2.5
Time: 8.24 minutes, Memory: 329.00Mb
--------------------------------
Xdebug 3.2.1
Time: 33.92 minutes, Memory: 329.50Mb
--------------------------------
-> Takes about FOUR TIMES the time to process
================================
Additional Information
----------------------

xdebug
xdebug support enabled
Version 2.2.5
IDE Key root

Supported protocols Revision
DBGp - Common DeBuGger Protocol $Revision: 1.145 $

Directive Local Value Master Value
xdebug.auto_trace Off Off
xdebug.cli_color 0 0
xdebug.collect_assignments Off Off
xdebug.collect_includes On On
xdebug.collect_params 0 0
xdebug.collect_return Off Off
xdebug.collect_vars Off Off
xdebug.coverage_enable On On
xdebug.default_enable On On
xdebug.dump.COOKIE no value no value
xdebug.dump.ENV no value no value
xdebug.dump.FILES no value no value
xdebug.dump.GET no value no value
xdebug.dump.POST no value no value
xdebug.dump.REQUEST no value no value
xdebug.dump.SERVER no value no value
xdebug.dump.SESSION no value no value
xdebug.dump_globals On On
xdebug.dump_once On On
xdebug.dump_undefined Off Off
xdebug.extended_info On On
xdebug.file_link_format no value no value
xdebug.idekey no value no value
xdebug.max_nesting_level 100 100
xdebug.overload_var_dump On On
xdebug.profiler_aggregate Off Off
xdebug.profiler_append Off Off
xdebug.profiler_enable Off Off
xdebug.profiler_enable_trigger Off Off
xdebug.profiler_output_dir /tmp /tmp
xdebug.profiler_output_name cachegrind.out.%p cachegrind.out.%p
xdebug.remote_autostart Off Off
xdebug.remote_connect_back Off Off
xdebug.remote_cookie_expire_time 3600 3600
xdebug.remote_enable Off Off
xdebug.remote_handler dbgp dbgp
xdebug.remote_host localhost localhost
xdebug.remote_log no value no value
xdebug.remote_mode req req
xdebug.remote_port 9000 9000
xdebug.scream Off Off
xdebug.show_exception_trace Off Off
xdebug.show_local_vars Off Off
xdebug.show_mem_delta Off Off
xdebug.trace_enable_trigger Off Off
xdebug.trace_format 0 0
xdebug.trace_options 0 0
xdebug.trace_output_dir /tmp /tmp
xdebug.trace_output_name trace.%c trace.%c
xdebug.var_display_max_children 128 128
xdebug.var_display_max_data 512 512
xdebug.var_display_max_depth 3 3
TagsNo tags attached.
Operating SystemUbuntu Server 14.04
PHP Version5.5.5-5.5.9

Activities

Arjen

2015-03-23 12:57

reporter   ~0003052

I can confirm this.

2.2.7 with coverage: 3 min 45 sec
2.3.2 with coverage: 20 min
2.3.2 without coverage: 1 min 35 sec

gcth

2015-04-27 10:57

reporter  

xdebug-2.3.2-coverage-time.png (11,537 bytes)

gcth

2015-04-27 10:59

reporter   ~0003073

Same for me, for jenkins jobs, with phpunit 4.6 and php -v:

PHP 5.6.8-1~dotdeb+wheezy.1 (cli) (built: Apr 22 2015 04:03:53)
Copyright (c) 1997-2015 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2015 Zend Technologies
    with Xdebug v2.3.2, Copyright (c) 2002-2015, by Derick Rethans
    with Zend OPcache v7.0.4-dev, Copyright (c) 1999-2015, by Zend Technologies

I've attached build time trend with coverage enabled.

sebastian

2015-05-27 11:46

reporter   ~0003089

It is to be expected for the execution to be slower when XDEBUG_CC_BRANCH_CHECK is enabled. But without it the performance of the code coverage data collection with Xdebug 2.2 and 2.3 should be the same.

Arjen

2015-05-27 11:52

reporter   ~0003090

Just to be clear: this slowdown is with XDEBUG_CC_BRANCH_CHECK disabled (not enabled).

mindplay

2015-06-04 09:09

reporter   ~0003108

I can confirm the performance problem with 2.3.2, which came with the WPN-XM stack - downgrading to 2.2.7 is my temporary work-around too, it performs normally.

sebastian

2015-06-04 09:40

reporter  

xdebug_227_callgrind.out.12939 (829,645 bytes)

sebastian

2015-06-04 09:40

reporter  

xdebug_232_callgrind.out.13320 (835,496 bytes)

sebastian

2015-06-04 09:43

reporter   ~0003109

I attached two valgrind/callgrind profiles generated with PHP 5.6.9 and both Xdebug 2.2.7 and 2.3.2.

$ git clone https://github.com/sebastianbergmann/money.git
$ cd money
$ valgrind --tool=callgrind /usr/local/php-5.6.9/bin/php ./build/tools/phpunit.phar -c build

Arjen

2015-06-24 13:34

reporter   ~0003128

I'm not sure, but I think this line https://github.com/xdebug/xdebug/blob/1eb61105f811d09bc90109c611083ac0fad62dc4/xdebug_code_coverage.c#L108
should read "if (XG(code_coverage_branch_check)) {"

derick

2015-10-24 23:11

administrator   ~0003194

Fixed for 2.3.4 (if that gets released) and Xdebug 3.0.0

kaplun

2018-07-17 07:22

reporter   ~0004682

It looks like this problem still exists with PHP 7.2.6, XDebug v2.6.0, PHPUnit 7.2.3, and with processIsolation="false"

Issue History

Date Modified Username Field Change
2015-03-04 13:10 ravage New Issue
2015-03-23 12:57 Arjen Note Added: 0003052
2015-04-27 10:57 gcth File Added: xdebug-2.3.2-coverage-time.png
2015-04-27 10:59 gcth Note Added: 0003073
2015-05-27 11:46 sebastian Note Added: 0003089
2015-05-27 11:52 Arjen Note Added: 0003090
2015-06-04 09:09 mindplay Note Added: 0003108
2015-06-04 09:40 sebastian File Added: xdebug_227_callgrind.out.12939
2015-06-04 09:40 sebastian File Added: xdebug_232_callgrind.out.13320
2015-06-04 09:43 sebastian Note Added: 0003109
2015-06-24 13:34 Arjen Note Added: 0003128
2015-10-24 23:11 derick Note Added: 0003194
2015-10-24 23:11 derick Status new => closed
2015-10-24 23:11 derick Assigned To => derick
2015-10-24 23:11 derick Resolution open => fixed
2015-10-24 23:11 derick Fixed in Version => 2.3.4
2015-11-05 15:08 derick Fixed in Version 2.3.4 => 2.4.0beta1
2016-07-31 12:36 derick Category Usage problems => Usage problems (Crashes)
2016-07-31 12:38 derick Category Usage problems (Crashes) => Usage problems (Wrong Results)
2018-07-17 07:22 kaplun Note Added: 0004682