Add inputs matrix-php-versions-unit and matrix-php-versions-functional

[BrianHenryIE]

Allows setting the list of PHP versions in the matrix from the caller workflow.

The key is to use fromJson:

matrix:
  php: ${{ fromJson( inputs.matrix-php-versions-unit ) }}

The caller then looks like:

jobs:
  test:
    uses: wp-cli/.github/.github/workflows/reusable-testing.yml@main
    with:
      matrix-php-versions-unit: "['7.1', '7.2', '7.3', '7.4', '8.0', '8.1', '8.2', '8.3']"
      matrix-php-versions-functional: "['7.1', '7.2', '7.3', '7.4', '8.0', '8.1', '8.2', '8.3']"

And the matrix doesn't run PHP 5.6 etc at all:

The PR also removes the minimum-php input, since that will be addressed by the PHP matrices.

There will still be an issue with the extra "include" matrix declarations

.github/.github/workflows/reusable-testing.yml

Lines 88 to 124 in b0868c0

	include:
	- php: '7.0'
	wp: 'trunk'
	mysql: '8.0'
	- php: '7.0'
	wp: 'trunk'
	mysql: '5.7'
	- php: '7.0'
	wp: 'trunk'
	mysql: '5.6'
	- php: '7.4'
	wp: 'trunk'
	mysql: '8.0'
	- php: '8.0'
	wp: 'trunk'
	mysql: '8.0'
	- php: '8.0'
	wp: 'trunk'
	mysql: '5.7'
	- php: '8.0'
	wp: 'trunk'
	mysql: '5.6'
	- php: '8.1'
	wp: 'trunk'
	mysql: '8.0'
	- php: '8.2'
	wp: 'trunk'
	mysql: '8.0'
	- php: '5.6'
	wp: '3.7'
	mysql: '5.6'
	- php: '5.6'
	wp: '6.2'
	mysql: '8.0'
	- php: '8.3'
	wp: 'trunk'
	mysql: '8.0'

The documentation says:

Note: All include combinations are processed after exclude. This allows you to use include to add back combinations that were previously excluded.

I wasn't able to use conditional logic / expressions in the matrix exclude field.

Any ideas how to handle these combinations? It might be easiest to have all elements in the primary matrices, then exclude what is known to be incompatible combinations, then allow further exclusion to be provided via input.

[danielbachhuber]

Out of curiosity, what happens when this is invalid JSON?

[BrianHenryIE]

With value "'7.1', '7.2', '7.3', '7.4', '8.0', '8.1', '8.2', '8.3']" (I just deleted the leading bracket)

Error when evaluating 'strategy' for job 'unit'. brianhenryie/bh-wp-github-actions-tests/.github/workflows/reusable-testing.yml@master (Line: 26, Col: 14): Unexpected value '7.1'

[BrianHenryIE]

The latest commit rewrites the functional tests's matrix to only use exclude rules. I wrote a script to verify the calculated list of combinations from before and after is the same.

Script:

<?php
// https://github.com/wp-cli/.github/blob/b0868c02792af659157c03a38254c37e90b099ca/.github/workflows/reusable-testing.yml#L88-L124

// Existing matrix:
$php = ['7.0', '7.1', '7.2', '7.3', '7.4', '8.0', '8.1', '8.2'];
$wp = ['latest'];
$mysql = ['8.0'];
$include = [
	'7.0, trunk, 8.0',
	'7.0, trunk, 5.7',
	'7.0, trunk, 5.6',
	'7.4, trunk, 8.0',
	'8.0, trunk, 8.0',
	'8.0, trunk, 5.7',
	'8.0, trunk, 5.6',
	'8.1, trunk, 8.0',
	'8.2, trunk, 8.0',
	'5.6, 3.7, 5.6',
	'5.6, 6.2, 8.0',
	'8.3, trunk, 8.0',
];

// Expand the matrix to explicitly list all combinations:
foreach( $mysql as $m){
	foreach( $wp as $w){
		foreach($php as $p) {
			if( ! in_array( "$p, $w, $m", $include, true ) ) {
				$include[] = "$p, $w, $m";
			}
		}
	}
}

echo count($include) . ' combinations.\n\n';

// New matrix:

$new_php = ['5.6', '7.0', '7.1', '7.2', '7.3', '7.4', '8.0', '8.1', '8.2', '8.3'];
$new_wp = ['3.7','6.2','latest','trunk'];
$new_mysql = ['5.6','5.7','8.0'];

$exclude_rules = [
	['wp'=>'latest', 'mysql'=> '5.6'],
	['wp'=>'latest', 'mysql'=> '5.7'],

	['php'=> '5.6', 'wp'=>'latest'],
	['php'=> '5.6', 'wp'=>'trunk'],

	['php'=> '7.4', 'mysql'=>'5.6'],
	['php'=> '8.1', 'mysql'=>'5.6'],
	['php'=> '8.2', 'mysql'=>'5.6'],
	['php'=> '8.3', 'mysql'=>'5.6'],

	['php'=> '5.6', 'mysql'=>'5.7'],
	['php'=> '7.4', 'mysql'=>'5.7'],
	['php'=> '8.1', 'mysql'=>'5.7'],
	['php'=> '8.2', 'mysql'=>'5.7'],
	['php'=> '8.3', 'mysql'=>'5.7'],

	['wp'=> '3.7', 'mysql'=>'5.7'],
	['wp'=> '3.7', 'mysql'=>'8.0'],

	['php'=> '7.0', 'wp'=>'3.7'],
	['php'=> '7.1', 'wp'=>'3.7'],
	['php'=> '7.2', 'wp'=>'3.7'],
	['php'=> '7.3', 'wp'=>'3.7'],
	['php'=> '7.4', 'wp'=>'3.7'],
	['php'=> '8.0', 'wp'=>'3.7'],
	['php'=> '8.1', 'wp'=>'3.7'],
	['php'=> '8.2', 'wp'=>'3.7'],
	['php'=> '8.3', 'wp'=>'3.7'],

	['php'=> '7.0', 'wp'=>'6.2'],
	['php'=> '7.1', 'wp'=>'6.2'],
	['php'=> '7.2', 'wp'=>'6.2'],
	['php'=> '7.3', 'wp'=>'6.2'],
	['php'=> '7.4', 'wp'=>'6.2'],
	['php'=> '8.0', 'wp'=>'6.2'],
	['php'=> '8.1', 'wp'=>'6.2'],
	['php'=> '8.2', 'wp'=>'6.2'],
	['php'=> '8.3', 'wp'=>'6.2'],

	['php'=> '7.1', 'wp'=>'trunk' ],
	['php'=> '7.2', 'wp'=>'trunk'],
	['php'=> '7.3', 'wp'=>'trunk'],

	['php'=> '8.3', 'wp'=>'latest'],

	['php'=> '5.6', 'wp'=>'3.7', 'mysql'=>'8.0'],
	['php'=> '5.6', 'wp'=>'6.2', 'mysql'=>'5.6'],
];

// Expand the exclude rule for everything they match in the matrix
$exclude = [];
foreach( $exclude_rules as $exclude_rule ) {
	foreach( $new_mysql as $m){
		foreach( $new_wp as $w){
			foreach($new_php as $p) {

				if(isset($exclude_rule['php']) && ($exclude_rule['php'] !== $p)){
					continue;
				}
				if(isset($exclude_rule['wp']) && ($exclude_rule['wp'] !== $w)){
					continue;
				}
				if(isset($exclude_rule['mysql']) && ($exclude_rule['mysql'] !== $m)){
					continue;
				}
				echo "Excluding: $p, $w, $m – matches ". json_encode( $exclude_rule )."\n";

				if( in_array("$p, $w, $m", $include, true )) {
					throw new Exception( "Error: should NOT exclude: $p, $w, $m");
				}
				$exclude[] = "$p, $w, $m";
	}}}
}

$new_include = [];
$new_include_arr = [];

foreach( $new_mysql as $m){
	foreach( $new_wp as $w){
		foreach($new_php as $p) {
			if(
				! in_array( "$p, $w, $m", $include, true )
				&& ! in_array( "$p, $w, $m", $exclude, true )
			) {
				echo "Need to exclude: $p, $w, $m\n";
			}

			if(! in_array( "$p, $w, $m", $exclude, true ) ){
				$new_include[] = "$p, $w, $m";
				$new_include_arr[] = ['php'=>$p,'wp'=>$w, 'mysql'=>$m];
			}
		}
	}
}

foreach( $include as $item ) {
	if( ! in_array( $item, $new_include, true ) ) {
		throw new Exception("New include list is missing item: $item");
	}
}

$equal_count_message = count( $include ) === count( $new_include ) ? 'SUCCESS' : 'FAILURE';
echo 'count( $include ): ' . count( $include ) . ' === count( $new_include ): ' .count( $new_include ) . ' – ' . $equal_count_message . "\n";

echo "        exclude:\n";
foreach( $exclude_rules as $rule ) {
	$first = true;
	foreach( $rule as $name => $value ) {
		echo '          ';
		echo $first ? '- ' : '  ';
		echo "$name: '$value'\n";
		$first = false;
	}
}

[swissspidy]

Here's an alternative approach: #78

[swissspidy]

Closing in favor of #78