View Issue Details

IDProjectCategoryView StatusLast Update
0002179XdebugCode Coveragepublic2023-07-18 11:11
Reporterondrej.frei@lokalise.com Assigned Toderick  
PrioritynormalSeveritymajorReproducibilitysometimes
Status resolvedResolutionno change required 
Product Version3.1.3 
Summary0002179: Bad coverage
Description

In some situations (nested ifs) lines are reported as covered even though they are not. This can heavily skew the code coverage number.

Steps To Reproduce

Please refer to https://github.com/freiondrej-lokalise/bad-coverage-reproduction with live reproduction details.

TagsNo tags attached.
Operating System
PHP Version8.1.10-8.1.19

Activities

derick

2023-07-04 16:50

administrator   ~0006580

I have had a look at this, and although i can indeed reproduce it, I can't do anything about it. PHP seems to be not generating JMP instructions on the right line. With the attached file, the VLD tool produces:

$ php -dvld.active=1 -dextension=vld.so /tmp/2179.php 
Finding entry points
Branch analysis from position: 0
1 jumps found. (Code = 62) Position 1 = -2
filename:       /tmp/2179.php
function name:  (null)
number of ops:  1
compiled vars:  none
line      #* E I O op                           fetch          ext  return  operands
-------------------------------------------------------------------------------------
   29     0  E > > RETURN                                                   1

branch: #  0; line:    29-   29; sop:     0; eop:     0; out0:  -2
path #1: 0, 
Function getbyfilename:
Finding entry points
Branch analysis from position: 0
2 jumps found. (Code = 43) Position 1 = 2, Position 2 = 15
Branch analysis from position: 2
2 jumps found. (Code = 43) Position 1 = 4, Position 2 = 14
Branch analysis from position: 4
1 jumps found. (Code = 79) Position 1 = -2
Branch analysis from position: 14
1 jumps found. (Code = 42) Position 1 = 15
Branch analysis from position: 15
1 jumps found. (Code = 62) Position 1 = -2
Branch analysis from position: 15
filename:       /tmp/2179.php
function name:  getByFilename
number of ops:  17
compiled vars:  !0 = $this_file_format, !1 = $file_format
line      #* E I O op                           fetch          ext  return  operands
-------------------------------------------------------------------------------------
    5     0  E >   EXT_STMT                                                 
          1      > JMPZ                                                     <true>, ->15
    7     2    >   EXT_STMT                                                 
          3      > JMPZ                                                     <false>, ->14
    9     4    >   EXT_STMT                                                 
          5        ASSIGN                                                   !0, !1
   10     6        EXT_STMT                                                 
          7      > EXIT                                                     
   11     8*       EXT_STMT                                                 
          9*       EXIT                                                     
   12    10*       EXT_STMT                                                 
         11*       NEW                                              $3      'Exception'
         12*       DO_FCALL                                      0          
         13*       THROW                                         0          $3
         14    > > JMP                                                      ->15
   16    15    >   EXT_STMT                                                 
         16      > RETURN                                                   null

branch: #  0; line:     5-    5; sop:     0; eop:     1; out0:   2; out1:  15
branch: #  2; line:     7-    7; sop:     2; eop:     3; out0:   4; out1:  14
branch: #  4; line:     9-   10; sop:     4; eop:     7; out0:  -2
branch: # 14; line:    12-   12; sop:    14; eop:    14; out0:  15
branch: # 15; line:    16-   16; sop:    15; eop:    16; out0:  -2
path #1: 0, 2, 4, 
path 0000002: 0, 2, 14, 15, 
path 0000003: 0, 15, 
End of function getbyfilename

Function getbyfilename2:
Finding entry points
Branch analysis from position: 0
2 jumps found. (Code = 43) Position 1 = 2, Position 2 = 7
Branch analysis from position: 2
2 jumps found. (Code = 43) Position 1 = 4, Position 2 = 6
Branch analysis from position: 4
1 jumps found. (Code = 42) Position 1 = 7
Branch analysis from position: 7
1 jumps found. (Code = 62) Position 1 = -2
Branch analysis from position: 6
Branch analysis from position: 7
filename:       /tmp/2179.php
function name:  getByFilename2
number of ops:  9
compiled vars:  !0 = $this_file_format, !1 = $file_format
line      #* E I O op                           fetch          ext  return  operands
-------------------------------------------------------------------------------------
   20     0  E >   EXT_STMT                                                 
          1      > JMPZ                                                     <true>, ->7
   22     2    >   EXT_STMT                                                 
          3      > JMPZ                                                     <false>, ->6
   24     4    >   EXT_STMT                                                 
          5        ASSIGN                                                   !0, !1
          6    > > JMP                                                      ->7
   28     7    >   EXT_STMT                                                 
          8      > RETURN                                                   null

branch: #  0; line:    20-   20; sop:     0; eop:     1; out0:   2; out1:   7
branch: #  2; line:    22-   22; sop:     2; eop:     3; out0:   4; out1:   6
branch: #  4; line:    24-   24; sop:     4; eop:     5; out0:   6
branch: #  6; line:    24-   24; sop:     6; eop:     6; out0:   7
branch: #  7; line:    28-   28; sop:     7; eop:     8; out0:  -2
path #1: 0, 2, 4, 6, 7, 
path 0000002: 0, 2, 6, 7, 
path 0000003: 0, 7, 
End of function getbyfilename2

In the first function getByFilename the JMP on line 12/op 14, should really have been generated for line 14 where the else is, and similarly for the JMP on line 24/op 6 in the second function, which should be on line 26. I don't know whether it is possible to fix this in PHP, but I will have a talk with some of the other engine folks about it.

2179.php (912 bytes)   
<?php

function getByFilename()
{
    if (true) {
        if (
            false
        ) {
            $this_file_format = $file_format; // will be shown as not covered
            exit;   // will be shown as not covered
            die;    // will be shown as not covered
            throw new Exception(); // will be shown as covered because it's the last line in the block
        }
    } else {    // the else is crucial even when it's not actually called. and when it's empty. When removed, the coverage bug stops happening.
    }
}

function getByFilename2()
{
    if (true) {
        if (
            false
        ) {
            $this_file_format = $file_format; // will be shown as covered because it's the last (=only) line in the block
        }
    } else {    // the else is crucial even when it's not actually called. and when it's empty. When removed, the coverage bug stops happening.
    }
}
2179.php (912 bytes)   

derick

2023-07-05 14:53

administrator   ~0006585

This is going to be fixed in PHP 8.2.9: https://github.com/php/php-src/pull/11598#pullrequestreview-1514703137 — so I'll be closing this issue.

ondrej.frei@lokalise.com

2023-07-18 11:11

reporter   ~0006605

Thank you @derick! Much appreciated.

Issue History

Date Modified Username Field Change
2023-05-11 11:09 ondrej.frei@lokalise.com New Issue
2023-07-04 16:50 derick Note Added: 0006580
2023-07-04 16:50 derick File Added: 2179.php
2023-07-04 16:51 derick Assigned To => derick
2023-07-04 16:51 derick Status new => confirmed
2023-07-05 14:53 derick Status confirmed => resolved
2023-07-05 14:53 derick Resolution open => no change required
2023-07-05 14:53 derick Note Added: 0006585
2023-07-18 11:11 ondrej.frei@lokalise.com Note Added: 0006605