0001123: With Xdebug 2.3.1, PHPUnit with coverage is exponentially slower than without - MantisBT

ID	Project	Category	View Status	Date Submitted	Last Update
0001123	Xdebug	Uncategorized	public	2015-03-04 13:10	2018-07-17 07:22

Reporter	ravage	Assigned To
Priority	normal	Severity	major	Reproducibility	always
Status	closed	Resolution	fixed
Platform	Linux 3.13.0-46-generic x86_64	OS	Ubuntu Server	OS Version	14.04
Product Version	2.3.1
Fixed in Version	2.4.0beta1

Summary	0001123: With Xdebug 2.3.1, PHPUnit with coverage is exponentially slower than without
Description	After upgrading to Xdebug 2.3.1, running the unit test suite of an internal project takes exponentially more time with code coverage than without. Before with Xdebug 2.2.5 (and similiar), it took FAR less time with code coverage. It's a CakePHP 2.6.x application using PHPUnit 3.7.38 (use of PHPUnit 4.x+ not possible). The suite contains: Tests: 821, Assertions: 2815, Incomplete: 15. PHP 5.5.9-1ubuntu4.6 (cli) (built: Feb 13 2015 19:17:11) Expecially integration tests of controllers are way slower than before. Nothing else has been changed except the Xdebug version. Interestingly I have yet not found any reports of 2.3.1 being so slow. Currently I do not need to upgrade to the Xdebug 2.3.x but I wanted to document the bad performance I experienced, even if applies only to a fraction of other users.
Steps To Reproduce	Without Code Coverage user@my_server:/var/www/cakephp_app/app$ sudo php Console/cake.php test app AllAppTests --stderr --working . Xdebug 2.2.5 Time: 2.24 minutes, Memory: 314.25Mb Xdebug 3.2.1 Time: 2.3 minutes, Memory: 314.50Mb -> About the same time to process With HTML Code Coverage user@my_server:/var/www/cakephp_app/app$ sudo php Console/cake.php test app AllAppTests --stderr --working . --configuration phpunit-html-coverage.xml Xdebug 2.2.5 Time: 7.69 minutes, Memory: 317.25Mb Xdebug 3.2.1 Time: 33.66 minutes, Memory: 318.00Mb -> Takes about FOUR TIMES the time to process With HTML Code Coverage user@my_server:/var/www/cakephp_app/app$ sudo php Console/cake.php test app AllAppTests --stderr --working . --coverage-clover ~/cakephp_app.clover.xml Xdebug 2.2.5 Time: 8.24 minutes, Memory: 329.00Mb Xdebug 3.2.1 Time: 33.92 minutes, Memory: 329.50Mb -> Takes about FOUR TIMES the time to process
Additional Information	xdebug xdebug support enabled Version 2.2.5 IDE Key root Supported protocols Revision DBGp - Common DeBuGger Protocol $Revision: 1.145 $ Directive Local Value Master Value xdebug.auto_trace Off Off xdebug.cli_color 0 0 xdebug.collect_assignments Off Off xdebug.collect_includes On On xdebug.collect_params 0 0 xdebug.collect_return Off Off xdebug.collect_vars Off Off xdebug.coverage_enable On On xdebug.default_enable On On xdebug.dump.COOKIE no value no value xdebug.dump.ENV no value no value xdebug.dump.FILES no value no value xdebug.dump.GET no value no value xdebug.dump.POST no value no value xdebug.dump.REQUEST no value no value xdebug.dump.SERVER no value no value xdebug.dump.SESSION no value no value xdebug.dump_globals On On xdebug.dump_once On On xdebug.dump_undefined Off Off xdebug.extended_info On On xdebug.file_link_format no value no value xdebug.idekey no value no value xdebug.max_nesting_level 100 100 xdebug.overload_var_dump On On xdebug.profiler_aggregate Off Off xdebug.profiler_append Off Off xdebug.profiler_enable Off Off xdebug.profiler_enable_trigger Off Off xdebug.profiler_output_dir /tmp /tmp xdebug.profiler_output_name cachegrind.out.%p cachegrind.out.%p xdebug.remote_autostart Off Off xdebug.remote_connect_back Off Off xdebug.remote_cookie_expire_time 3600 3600 xdebug.remote_enable Off Off xdebug.remote_handler dbgp dbgp xdebug.remote_host localhost localhost xdebug.remote_log no value no value xdebug.remote_mode req req xdebug.remote_port 9000 9000 xdebug.scream Off Off xdebug.show_exception_trace Off Off xdebug.show_local_vars Off Off xdebug.show_mem_delta Off Off xdebug.trace_enable_trigger Off Off xdebug.trace_format 0 0 xdebug.trace_options 0 0 xdebug.trace_output_dir /tmp /tmp xdebug.trace_output_name trace.%c trace.%c xdebug.var_display_max_children 128 128 xdebug.var_display_max_data 512 512 xdebug.var_display_max_depth 3 3
Tags	No tags attached.

Operating System	Ubuntu Server 14.04
PHP Version	5.5.5-5.5.9

Arjen 2015-03-23 12:57 reporter ~0003052	I can confirm this. 2.2.7 with coverage: 3 min 45 sec 2.3.2 with coverage: 20 min 2.3.2 without coverage: 1 min 35 sec

gcth 2015-04-27 10:59 reporter ~0003073	Same for me, for jenkins jobs, with phpunit 4.6 and php -v: PHP 5.6.8-1~dotdeb+wheezy.1 (cli) (built: Apr 22 2015 04:03:53) Copyright (c) 1997-2015 The PHP Group Zend Engine v2.6.0, Copyright (c) 1998-2015 Zend Technologies with Xdebug v2.3.2, Copyright (c) 2002-2015, by Derick Rethans with Zend OPcache v7.0.4-dev, Copyright (c) 1999-2015, by Zend Technologies I've attached build time trend with coverage enabled.

sebastian 2015-05-27 11:46 reporter ~0003089	It is to be expected for the execution to be slower when XDEBUG_CC_BRANCH_CHECK is enabled. But without it the performance of the code coverage data collection with Xdebug 2.2 and 2.3 should be the same.

Arjen 2015-05-27 11:52 reporter ~0003090	Just to be clear: this slowdown is with XDEBUG_CC_BRANCH_CHECK disabled (not enabled).

mindplay 2015-06-04 09:09 reporter ~0003108	I can confirm the performance problem with 2.3.2, which came with the WPN-XM stack - downgrading to 2.2.7 is my temporary work-around too, it performs normally.

sebastian 2015-06-04 09:43 reporter ~0003109	I attached two valgrind/callgrind profiles generated with PHP 5.6.9 and both Xdebug 2.2.7 and 2.3.2. $ git clone https://github.com/sebastianbergmann/money.git $ cd money $ valgrind --tool=callgrind /usr/local/php-5.6.9/bin/php ./build/tools/phpunit.phar -c build

Arjen 2015-06-24 13:34 reporter ~0003128	I'm not sure, but I think this line https://github.com/xdebug/xdebug/blob/1eb61105f811d09bc90109c611083ac0fad62dc4/xdebug_code_coverage.c#L108 should read "if (XG(code_coverage_branch_check)) {"

derick 2015-10-24 23:11 administrator ~0003194	Fixed for 2.3.4 (if that gets released) and Xdebug 3.0.0

kaplun 2018-07-17 07:22 reporter ~0004682	It looks like this problem still exists with PHP 7.2.6, XDebug v2.6.0, PHPUnit 7.2.3, and with processIsolation="false"