0001168: Possible infinite loop when PHP's executor shuts down - MantisBT

ID	Project	Category	View Status	Date Submitted	Last Update
0001168	Xdebug	Uncategorized	public	2015-06-23 08:21	2016-12-04 18:33

Reporter	phil-lavin	Assigned To
Priority	normal	Severity	block	Reproducibility	random
Status	closed	Resolution	fixed
OS	Debian 8	OS Version	3.16.0-4-amd64
Product Version	2.2.6
Fixed in Version	2.5.0

Summary	0001168: Possible infinite loop when PHP's executor shuts down
Description	I've had random PHP-FPM processes consuming 100% of CPU whilst being unable to interrogate them via most means (e.g. strace). I spent a lot of time pulling info on the process to little avail however, finally, the below GDB backtraces show the process stuck in a while loop within xdebug shutdown code.
Steps To Reproduce	Hard to reproduce - will probably happen to a single php-fpm process once or twice a month.
Additional Information	GDB BT is at https://gist.githubusercontent.com/phil-lavin/2b8a1bb91f7e0bfe10c7/raw/9a2095bd60b11dca131a62d0f38065db181a1e2b/gistfile1.txt Other debug info gathered is as follows: /proc/$PID/fd: 2 sockets, and a deleted /tmp file /proc/$PID/stack: [<ffffffffffffffff>] 0xffffffffffffffff Process STAT reported by ps: 'R' /proc/$PID/syscall: 'running' /proc/$PID/wchan: 0 /proc/$PID/status: https://gist.github.com/phil-lavin/052b593848900ce1c532 /proc/${PID}/sched: https://gist.github.com/phil-lavin/82445465f2e5e7c5329d pstack was unable to connect to the process /proc/$PID/maps: https://gist.github.com/phil-lavin/fafca59e9c984975f5ce gdb `dump memory /tmp/wtf 0x7ffec5048000 0x7ffec5069000`: Let me know if you want this and I'll e-mail it... not 100% confident there's nothing sensitive in it
Tags	No tags attached.

Operating System
PHP Version	5.6.5-5.6.9

derick 2015-06-25 13:39 administrator ~0003135	Hmm, this issue has been reported before, but I could never find why it happens, as it seems so difficult to reproduce. Is there a way how you can (reliably) reproduce this? Do you know with which request/code path this happens? Is it in a single or multi-threaded environment?

phil-lavin 2015-06-25 13:47 reporter ~0003139	Hi Derick, I can guarantee that it'll happen some time in the next month but I cannot easily reproduce on a whim. It's a single threaded php-fpm implementation. I'll be at phpwarks tonight if you want to discuss or poke around the affected server.

tony2001 2015-10-22 16:10 reporter ~0003181	I can see that it's stuck here: void xdebug_llist_empty(xdebug_llist l, void user) { while (xdebug_llist_count(l) > 0) { xdebug_llist_remove(l, XDEBUG_LLIST_TAIL(l), user); } } with tail == NULL llist_remove() immediately returns, but the l->size is still 1 (in my case), so this loop has no end. I'll try to reproduce it, too, but no guarantees, sorry..

derick 2015-10-22 16:12 administrator ~0003182	Good luck Antony! I never managed :-/

derick 2015-11-20 17:55 administrator ~0003246	Changing this to "acknowledged", but I won't work on this unless somebody shows me a reproducible case.

derick 2016-12-03 13:37 administrator ~0003885	Is this something either of you have been able to conclusively reproduce yet? If not, I think I will close this issue.

derick 2016-12-04 14:51 administrator ~0003894	I have added the defensive check for 2.5.0RC2.