Latest Update 2021-01-25 14:25 EST

Notes:

1. Avoid unnecessary issue bumping. Please test this pull request and report issues or approve working state.
   See comment.
2. Stability of this pull request's changeset needs additional testing from the community, such as unintentional updates upon requiring this plugin.
   See comment.
3. Project is stuck in the process of transferring ownership to a new team within the Wikimedia Foundation.
   See comment.
4. Since v1.4 of the plugin does not support Composer v2, the plugin is ignored when updating to v1.5. As a result, Composer is unaware of the merged dependencies and removes them. Either update the plugin with Composer v1 first or run composer update again.
   See comment.
5. Unintentional lock file updates caused by isFirstInstall() on Composer v2. Listening only to post-update-cmd instead of post-install-cmd might be more sensible. See comment and followup.
6. Please be patient, we don't want to push broken changes.

How to test:

{
    "repositories": [
        {
            "type":"vcs",
            "url":"https://github.com/mcaskill/composer-merge-plugin"
        }
    ],
    "require": {
        "wikimedia/composer-merge-plugin": "dev-feature/composer-v2 as 1.5.0"
    }
}

Tasks:

• [x] Find alternative to install inherited dependencies without affecting locked dependencies.
  See comment #1, comment #2, comment #3.
• [x] Rebase to include #197 and #199
• [x] Update port of MultiConstraint to latest version of composer/semver.
  See comment.
• [x] Method Factory::getLockFile() does not exist in Composer 1.
  See comment.

Replaces #187, #185, as fix for #184 (as per composer/composer#8726)

I've tested using the example and the unit tests and it works in Composer v1 and v2.

This alternative proposal forgoes PRE_DEPENDENCIES_SOLVING entirely to avoid over-complicating the codebase and streamline the plugin's merge logic.

The issue with the previous attempts (as well as attempts on other plugins) stems from a miscommunication from the Composer team of the major difference in how v1 and v2 resolve and install dependencies.

[…] Roughly speaking:

• Composer v1: Composer resolves dependencies (dispatching *_DEPENDENCIES_SOLVING), iterates packages (while dispatching PRE_PACKAGE_* before PRE_FILE_DOWNLOAD), then finally writes the lock file.
• Composer v2: Composer resolves dependencies (dispatching PRE_POOL_CREATE), writes the lock file, dispatches PRE_OPERATIONS_EXEC, downloads the packages (while dispatching PRE_FILE_DOWNLOAD), then iterates packages (while dispatching PRE_PACKAGE_*).

In particular, people are assuming that PRE_OPERATIONS_EXEC is a replacement for PRE_DEPENDENCIES_SOLVING since they are dispatched in a similar-looking routine. The closest event to the latter would actually be PRE_POOL_CREATE. — composer/composer#8726

Back to my proposal.

Currently, the use of PRE_DEPENDENCIES_SOLVING in the plugin is used to inject duplicate requirements (usually with a different version constraints) into the solver (while distinguishing between require and require-dev).

What I propose replaces the duplicate links tracking in ExtraPackage by back porting Composer 2's new static MultiConstraint::create() method which can be used to resolve complex-constraints early on. In turn, this makes the need for PRE_DEPENDENCIES_SOLVING obsolete.

I hope this PR will, at the very least, help to figure out how to support Composer v2.

If you've landed here looking for a version of the wikimedia/composer-merge-plugin compatible with Composer 2.0, please see https://github.com/wikimedia/composer-merge-plugin/pull/189 and help test!

It seems like composer 2.0 is right around the corner. Downloading https://getcomposer.org/composer.phar already gives you a 2.0 build.

Are there any plans to make this plugin compatible? It currently explicitly requires a v1 composer api to be present.

Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Root composer.json requires wikimedia/composer-merge-plugin 1.4.1 -> satisfiable by wikimedia/composer-merge-plugin[v1.4.1].
    - wikimedia/composer-merge-plugin v1.4.1 requires composer-plugin-api ^1.0 -> found composer-plugin-api[2.0.0] but it does not match your constraint.

It would be great if this addon also supported merging of PSR-4 autoload classes:

"autoload": {
    "psr-4": {
        "Boardworld\\": "app/third_party/boardworld/src/"
    }
}

Additionally relative paths in included composer files should have their paths resolved correctly.

I needed to pull a centralized list of required packages into multiple composer configs. Since composer has the ability to read JSON contents from a remote source via HTTP it was a straight forward change to composer-merge-plugin to correctly validate that the url exists (but still use glob for local files) and pass a RemoteFileSystem object to JsonFile constructor if we're dealing with a URL.

Running composer with the merge plugin just exits successfully (return 0) with only half the work done. I need to run it a second time to finish the work.

Example: https://integration.wikimedia.org/ci/job/mwext-WikidataQuality-repo-tests-mysql-hhvm/164/consoleFull (search for composer update and see were on the second invocation it still has more work to do).

 $ vendor/bin/phpunit --verbose
 PHPUnit 5.0.5 by Sebastian Bergmann and contributors.

 Runtime:       PHP 5.6.14 with Xdebug 2.3.3
 Configuration: /work/GIT/composer-merge-plugin/phpunit.xml.dist

 ......................                                            22 / 22 (100%)

 Time: 232 ms, Memory: 17.75Mb

 OK (22 tests, 223 assertions)

Motivation

I am trying to use composer-merge-plugin with a composer-patches variation, which is late enough for composer-merge-plugin to have merged all sections.

My root composer.json has something like:

{
    "extra": {
        "merge-plugin": {
            "merge-extra": true,
            "merge-deep": true,
            "include": "composer.local.json"
        },
        "patches": {
            "wikimedia/composer-merge-plugin": {
                "Add tests for merge-deep option": "patches/add-tests-for-merge-deep-option.diff"
            },
            "anothervendor/some-project": {
                "some-patch": "patches/some-patch.diff"
            }
        }
    }
}

while my composer.local.json has:

{
    "extra": {
        "patches": {
            "wikimedia/composer-merge-plugin": {
                "Allow merging of sections in a deep way": "patches/add-merge-deep-option.diff"
            },
            "somevendor/some-project": {
                "some-patch": "patches/some-patch.diff"
            }
        }
    }
}

Approach

This adds a merge-deep option, which only really is meaningful for the extra sections using an utility function from Drupal.

This makes the merge plugin much more useful for plugins that have deeper structures.

This also allows both kinda workflows that the merge plugin is used with:

• Defining some base skeleton files and including that, then overriding some keys (replace = TRUE)
• Overriding the global composer.json with a local variant.

Hopefully no-one will need this together, because that use case is not yet 'done', however I feel this is complex enough as is, to be a follow-up discussion.

Notes

Unfortunately Drupal 8 at the moment does not allow to reference the NestedArray component directly via a subtree split, so I had to copy the code for now.

• Also see:
• https://github.com/composer/composer/issues/5232 - where I am suggesting to give composer-merge-plugin a chance to always be run after init.
• https://github.com/douggreen/drupal-composer-installer/pull/9 - where I am using this functionality.

First reported by @ghprod in https://github.com/wikimedia/composer-merge-plugin/issues/71#issuecomment-154435583

composer.json:

{
    "require": {
        "wikimedia/composer-merge-plugin": "dev-master"
    },
    "config": {
        "preferred-install": "dist"
    },
    "extra": {
        "merge-plugin": {
            "include": "composer.local.json"
        }
    }
}

composer.local.json:

{
    "repositories": [
        {
            "type": "vcs",
            "url": "https://github.com/ghprod/laravel-menu"
        }
    ],
    "require":{
        "lavary/laravel-menu": "dev-master#956e50f"
    }
}

Result:

$ composer -vvv update
[...snip...]
Loading plugin Wikimedia\Composer\MergePlugin
  [merge-plugin] Loading composer.local.json...
  [merge-plugin] Adding vcs repository github.com/ghprod/laravel-menu
  [merge-plugin] Merging lavary/laravel-menu
Loading composer repositories with package information
[...snip...]
Downloading https://api.github.com/repos/ghprod/laravel-menu
Downloading https://api.github.com/repos/ghprod/laravel-menu/contents/composer.json?ref=master
Downloading https://api.github.com/repos/ghprod/laravel-menu/commits/master
Downloading https://api.github.com/repos/ghprod/laravel-menu/tags?per_page=100
Downloading https://api.github.com/repos/ghprod/laravel-menu/git/refs/heads?per_page=100
[...snip...]
Reading composer.json of lavary/laravel-menu (master)
Downloading https://api.github.com/repos/ghprod/laravel-menu/contents/composer.json?ref=956e50fbf0d99b2804539069b012ad52b4851dd2
Downloading https://api.github.com/repos/ghprod/laravel-menu/commits/956e50fbf0d99b2804539069b012ad52b4851dd2
[...snip...]
Downloading http://packagist.org/p/lavary/laravel-menu%24cb8432bd74b707af0300352a1d49742389810f45e5646550e5dca3a823fe4c55.json
[...snip...]
Analyzed 1426 packages to resolve dependencies
Analyzed 1200 rules to resolve dependencies
[...snip...]
  - Installing lavary/laravel-menu (dev-master bebce99)
Downloading https://api.github.com/repos/lavary/laravel-menu/zipball/bebce996ef08cf911da0139c30a2da03a2768326

The installed revision (bebce99) is the head of the dev branch of the project as listed on Packagist. The alternate VCS repository was not used.

If the require for "lavary/laravel-menu" is moved to composer.json, results are slightly different but still not correct. In that case the logs report:

  - Installing lavary/laravel-menu (dev-master 956e50f)
Downloading https://api.github.com/repos/lavary/laravel-menu/zipball/956e50f

This download is still from the wrong repository in this case and the files fetched from GitHub are not not actually for the desired hash.

If the "repositories" are moved to composer.json instead, the new output is:

  - Installing lavary/laravel-menu (dev-master 956e50f)
Downloading https://api.github.com/repos/ghprod/laravel-menu/zipball/956e50fbf0d99b2804539069b012ad52b4851dd2

This is the desired result for all three tests.

I think the issue here is the order that the repositories are added to Composer\Repository\RepositoryManager. It may take an upstream patch to fix this properly.

I've got a project with a main composer.json and a merged composer.json which I'm trying to use to install a private git project.

With the repository and require config in the merged JSON file the project isn't installed with composer giving a 'package ... could not be found in any version' error.

Moving the repository and require to the main composer.json files results in the package being installed.

A package which is listed on packagist (doesn't require a repository entry in the composer config) installs ok from the merged file.

The installed composer-merge-plugin is at 361e098955069062c2d688a2c116d7dd837acb38

I am using composer-merge-plugin and trying to update from 8.2.7 to 8.3.0 using the following process

• Update composer.json with the new version of drupal
• Run the following command composer update drupal/core --with-dependencies

I am receiving the following error.

Your requirements could not be resolved to an installable set of packages. Problem 1 - Can only install one of: symfony/psr-http-message-bridge[1.0.x-dev, v0.2]. - Can only install one of: symfony/psr-http-message-bridge[v0.2, 1.0.x-dev]. - Can only install one of: symfony/psr-http-message-bridge[1.0.x-dev, v0.2]. - drupal/core 8.3.0 requires symfony/psr-http-message-bridge ^1.0 -> satisfiable by symfony/psr-http-message-bridge[1.0.x-dev]. - Installation request for drupal/core 8.3.0 -> satisfiable by drupal/core[8.3.0]. - Installation request for symfony/psr-http-message-bridge v0.2 -> satisfiable by symfony/psr-http-message-bridge[v0.2].

It seems like composer is looking at my current lock file (with version 0.2 for psr-http-message-bridge) instead of the merged version from the core/composer.json in drupal core.

I am not sure how this plugin works with lock files after looking through it a bit and was wondering if anyone could provide some insight if this is a higher composer issue or a merge-plugin issue

Is there any way to execute scripts from composer.local.json? I trying to allow custom sites to add additional themes and ensure that theme is activated after install or update. Perhaps with an option to append or override scripts from the parent composer.json file?

Currently, composer depends does not know about dev dependencies coming from a merge, which can be confusing. This PR exposes these dependencies to the depends command.

Refs #159

Little bit ugly with that qualifier \Composer\Filter\PlatformRequirementFilter\PlatformRequirementFilterFactory::fromBoolOrList() instead of an import but \Composer\Filter\PlatformRequirementFilter\PlatformRequirementFilterFactory does not exist in Composer 1.

Is it possible to add option to avoid composer update is executed after composer install is executed? Although I expected to deploy packages with exact version combinations recorded in composer.lock which was tested, it was unexpectedly overwritten by composer update executed by composer-merge-plugin, resulting deploying something not tested. My deployment places my source code to the server without any composer dependency, then runs composer install to install packages recorded in composer.lock including those merged by composer-merge-pluign. However, after this, composer-merge-plugin runs composer update because it is the first time composer-merge-plugin is installed on the sever. For my system, this behavior is unnecessary since the dependency is already merged in composer.lock, and even problematic since it make installation with composer.lock meaningless.

This is similar issue to #235 , yet I'm hoping to have an option to suppress this behavior of force executing composer update after composer install.

Like this: https://github.com/wikimedia/composer-merge-plugin/compare/master...pikanji:composer-merge-plugin:option-to-prevent-automatic-update?expand=1

Thanks.

This solves several issues, including but not limited to the following: Make a composer.json similar to this:

{
    "require": {
        "wikimedia/composer-merge-plugin": "dev-master"
    },
    "extra": {
        "merge-plugin": {
            "include": [
                "composer.local.json",
                "extensions/*/composer.json"
            ],
            "recurse": true
        }
    }
}

With this set up, the following composer.local.jsons give problems:

{
  "require": {
    "mediawiki/user-functions": "dev-REL1_35"
  },
  "extra": {
    "merge-plugin": {
      "include": [
        "extensions/*/composer.json"
      ]
    }
  }
}

Gives the error Fatal error: Cannot redeclare <some function/class> (previously declared in <directory>/extensions/<name>/<file>:<line>) in <directory>/extensions/<name>/<file> on line <line>, as described in this issue on this repo and this issue in the composer repo

{
    "require": {
        "mediawiki/semantic-media-wiki": "3.2.3"
    },
    "extra": {
        "merge-plugin": {
            "include": [
                "extensions/*/composer.json"
            ]
        }
    }
}

Gives the error

Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - my/root-package is present at version 1.0.0 and cannot be modified by Composer
    - mediawiki/semantic-media-wiki[3.2.3] cannot be installed as that would require removing my/root-package[1.0.0]. They both replace mediawiki/semantic-mediawiki and thus cannot coexist.
    - Root composer.json requires mediawiki/semantic-media-wiki 3.2.3 -> satisfiable by mediawiki/semantic-media-wiki[3.2.3].

which is also discussed in this issue on this repo (Which also says "[..] it would be good if there was some way to indicate not to merge the configs of any packages that are already included by the main composer.json file since their configs will already be evaluated by Composer when they're required in the main composer.json file.", which is what this pull request does). Solving the issue by using the merge-replace: false option gives the same Fatal error: Cannot redeclare problem described above.

This pull request solves all these issues.

Recreated an up-to-date PR. Credit goes to the authors of the initial pull request: https://github.com/wikimedia/composer-merge-plugin/pull/219

I have created this new pull request because the original one was not being updated to fix the indentation.

Solves the warning PHPUnit\Framework\TestCase::prophesize() is deprecated and will be removed in PHPUnit 10. Please use the trait provided by phpspec/prophecy-phpunit that appears when running tests.

Also let provideFireInit actually provide distinct data.