Allow composer packages to define compilation steps

Overview

Composer Compile Plugin Build Status

The "Compile" plugin enables developers of PHP libraries to define free-form "compilation" tasks, such as:

  • Converting SCSS to CSS
  • Generating PHP wrappers based on an XML schema

For site-builders who use these PHP libraries, compilation tasks run seamlessly during the regular download (composer install, etc).

For developers who publish PHP libraries, a task can be as simple as:

{
  "require": {"civicrm/composer-compile-plugin": "~0.14"},
  "extra": {
    "compile": [
      {"run": "@sh cd css; cat one.css two.css three.css > all.css"}
    ]
  }
}

Tasks may be defined in several ways, including:

  • Shell command (@sh cat file-{1,2,3} > big-file)
  • PHP method (@php-method MyBuilder::build)
  • PHP eval (@php-eval file_put_contents('big-file', make_big_file());)
  • PHP script file (@php-script my-script.php)
  • Composer subcommand (@composer dump-autoload)

Features:

  • Easy to enable. No manual configuration for downstream site-builders. Framework agnostic.
  • Plays well with other composer tooling, like forked repositories, composer-patches, composer-locator, composer-downloads, and the autoloader.
  • Allows library repos to remain "clean" without committing build artifacts.
  • Runs locally in PHP. Does not require external/hosted services or additional interpreters.
  • Supports file monitoring for automatic rebuilds (composer compile:watch)
  • Enforces permission model to address historical concerns about composer hooks and untrusted libraries.
  • Integration-tests pass on both composer v1.10 and v2.0.

Documentation

  • site-build.md: Managing the root package (for site-builders)
  • tasks.md: Working with tasks (for library developers)
  • evaluation.md: Evaluate and compare against similar options
  • develop.md: How to work with composer-compile-plugin.git (for plugin-development)

See also

  • composer-compile-lib: Small library of opinionated helpers/examples for specific compilation tasks -- meta-PHP, SCSS, etc
  • composer#1193: Old discussion thread about post-install hooks for dependencies
Comments
  • Adds circular reference test, dependency test and prefilters packages before sorting.

    Adds circular reference test, dependency test and prefilters packages before sorting.

    to address https://github.com/civicrm/composer-compile-plugin/issues/13

    This checks what happens if adding a package A that requires package B that requires package A.

    opened by stesi561 15
  • composer 2.3 not compatible

    composer 2.3 not compatible

    It seems to have made a change so that eventname can't be null for EventDispatcher. Admittedly 2.3 was just released today, but civi download fails at the compilation task.

    Error: Argument 1 passed to Composer\EventDispatcher\Event::__construct() must be of the type string, null given, called in .../vendor/civicrm/composer-compile-plugin/src/Event/CompileTaskEvent.php on line 55
    In Event.php line 49:
                                                                                   
      [TypeError]                                                                  
      Argument 1 passed to Composer\EventDispatcher\Event::__construct() must be of the type string, null given, called in .../vendor/civicrm/composer-compile-plugin/src/Event/CompileTaskEvent.php on line 55          
                                                                                   
    
    Exception trace:
      at phar:///usr/local/bin/composer/src/Composer/EventDispatcher/Event.php:49
     Composer\EventDispatcher\Event->__construct() at .../vendor/civicrm/composer-compile-plugin/src/Event/CompileTaskEvent.php:55
     Civi\CompilePlugin\Event\CompileTaskEvent->__construct() at .../vendor/civicrm/composer-compile-plugin/src/TaskRunner.php:205
     Civi\CompilePlugin\TaskRunner->runTask() at .../vendor/civicrm/composer-compile-plugin/src/TaskRunner.php:169
     Civi\CompilePlugin\TaskRunner->run() at .../vendor/civicrm/composer-compile-plugin/src/TaskRunner.php:74
     Civi\CompilePlugin\TaskRunner->runDefault() at .../vendor/civicrm/composer-compile-plugin/src/Command/CompileCommand.php:75
     Civi\CompilePlugin\Command\CompileCommand->execute() at phar:///usr/local/bin/composer/vendor/symfony/console/Command/Command.php:298
     Symfony\Component\Console\Command\Command->run() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:1015
     Symfony\Component\Console\Application->doRunCommand() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:299
     Symfony\Component\Console\Application->doRun() at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:334
     Composer\Console\Application->doRun() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:171
     Symfony\Component\Console\Application->run() at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:130
     Composer\Console\Application->run() at phar:///usr/local/bin/composer/bin/composer:83
     require() at /usr/local/bin/composer:29
    
    opened by demeritcowboy 3
  • putenv/getenv doesn't work in php when thread safety is disabled

    putenv/getenv doesn't work in php when thread safety is disabled

    Apparently some distributions of php have Thread Safety => disabled (e.g. php -i | grep Thread). This makes getenv always return false, so it can't get the task. I don't have an immediate workaround other than to do it on another system and transfer it over, or instead write to known fixed filename and then read that back. Mostly putting this here in case other people run into this.

    It's in src/TaskTransfer.php in export() and import().

    WARNING: Failed to read compilation-task from COMPOSER_COMPILE_TASK. Please use "composer compile".
    PHP Fatal error:  Uncaught InvalidArgumentException: Invalid file reference (tpl-file=NULL) in .../sites/all/modules/civicrm/vendor/civicrm/composer-compile-lib/src/Tasks/Template.php:62
    Stack trace:
    #0 .../sites/all/modules/civicrm/vendor/civicrm/composer-compile-lib/src/Tasks/Template.php(21): CCL\Tasks\Template::assertFileField(Array, 'tpl-file')
    #1 .../sites/all/modules/civicrm/vendor/civicrm/composer-compile-lib/src/Tasks.php(24): CCL\Tasks\Template::compile(Array)
    #2 Command line code(1): CCL\Tasks::template(Array)
    #3 {main}
      thrown in .../sites/all/modules/civicrm/vendor/civicrm/composer-compile-lib/src/Tasks/Template.php on line 62
    > @php -r 'require_once '\''.../sites/all/modules/civicrm/vendor/autoload.php'\''; Civi\CompilePlugin\TaskTransfer::import(); \CCL\Tasks::template($GLOBALS["COMPOSER_COMPILE_TASK"]);'
    Script @php -r 'require_once '\''.../sites/all/modules/civicrm/vendor/autoload.php'\''; Civi\CompilePlugin\TaskTransfer::import(); \CCL\Tasks::template($GLOBALS["COMPOSER_COMPILE_TASK"]);' handling the shell-runner event returned with error code 255
    
    opened by demeritcowboy 3
  • PHP warning when running with -v on windows for COMPOSER_COMPILE_TASK

    PHP warning when running with -v on windows for COMPOSER_COMPILE_TASK

    Warning: Use of undefined constant COMPOSER_COMPILE_TASK - assumed 'COMPOSER_COMPILE_TASK' (this will throw an Error in a future version of PHP) in Command line code on line 1

    This line puts double-quotes around it: https://github.com/civicrm/composer-compile-plugin/blob/af9686c4511ad4d2dde3c32aafa4637579e1085d/src/Handler/PhpMethodHandler.php#L21

    On windows the double-quotes disappear at some point along the way and so php thinks you're using a constant that hasn't been defined.

    Since shell escaping is hard enough as it is and on windows is extra-bizarre, just putting this as a placeholder for now. With a quickie test it works to just put single quotes in that code line, i.e. \'. I might test that out a bit more.

    opened by demeritcowboy 3
  • Error in composer prevents install of some packages

    Error in composer prevents install of some packages

    There seems to be an error occurring when installing some drupal modules via composer when civicrm is already installed.

    Back trace shown:

    Exception trace:
     () at /vendor/civicrm/composer-compile-plugin/src/Util/ComposerPassthru.php:72
     Civi\CompilePlugin\Util\ComposerPassthru->run() at phar:///usr/local/bin/composer/src/Composer/Plugin/PluginManager.php(196) : eval()'d code:117
     Civi\CompilePlugin\CompilePlugin_composer_tmp5->runTasks() at n/a:n/a
     call_user_func() at phar:///usr/local/bin/composer/src/Composer/EventDispatcher/EventDispatcher.php:164
     Composer\EventDispatcher\EventDispatcher->doDispatch() at phar:///usr/local/bin/composer/src/Composer/EventDispatcher/EventDispatcher.php:96
     Composer\EventDispatcher\EventDispatcher->dispatchScript() at phar:///usr/local/bin/composer/src/Composer/Installer.php:338
     Composer\Installer->run() at phar:///usr/local/bin/composer/src/Composer/Command/RequireCommand.php:248
     Composer\Command\RequireCommand->doUpdate() at phar:///usr/local/bin/composer/src/Composer/Command/RequireCommand.php:205
     Composer\Command\RequireCommand->execute() at phar:///usr/local/bin/composer/vendor/symfony/console/Command/Command.php:245
     Symfony\Component\Console\Command\Command->run() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:835
     Symfony\Component\Console\Application->doRunCommand() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:185
     Symfony\Component\Console\Application->doRun() at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:281
     Composer\Console\Application->doRun() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:117
     Symfony\Component\Console\Application->run() at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:113
     Composer\Console\Application->run() at phar:///usr/local/bin/composer/bin/composer:61
     require() at /usr/local/bin/composer:24
    

    I can reproduce this on a clean install by doing the following: composer create-project drupal/recommended-project civicrm cd civicrm/ composer config extra.enable-patching true composer config minimum-stability dev composer require civicrm/civicrm-{core,packages,drupal-8}:'~5.32' composer require drupal/drupalauth4ssp

    Failure happens at the last step. Have also tried --with-all-dependencies and wiping vendor and composer.install in case this helps.

    I suspect this might be related to: https://lab.civicrm.org/dev/drupal/-/issues/150

    opened by stesi561 3
  • PHP 8.1 / PHP 8.0 compatibility issue

    PHP 8.1 / PHP 8.0 compatibility issue

    this plugin is not compatible with PHP 8.1 / PHP 8.0 smoke billows out of this area:

      Composer\EventDispatcher\Event::__construct(): Argument #1 ($name) must be of type string, null given, called in vendor/civicrm/composer-compile-plugin/src/Event  
      /CompileTaskEvent.php on line 55```
    opened by olstjos 2
  • Post-install listener is overly sensitive to composer start-state

    Post-install listener is overly sensitive to composer start-state

    Consider this update:

    • Checkout civicrm-core at revision 6f776cf7a8. The composer.lock uses composer-compile-plugin@v0.8.
    • Run composer install
    • Checkout civicrm-core at revision 29ff32ead9. The composer.lock uses composer-compile-plugin@v0.10. Additionally, there's a new item in the task-list -- based on the newer @php-script type.
    • Run composer install

    This fails:

    Screenshot from 2020-09-30 21-50-32

    The problem is that composer initialized with plugin v0.8 (which lacks @php-script support), fetched all the new packages, and then tried to process the new tasks using the old code (still retained in memory).

    I think the solution is to change this detail about the hook:

    • Right now, there's a listener for POST_INSTALL_CMD which instantiates TaskList and TaskRunner.
    • Alternatively, the listener for POST_INSTALL_CMD should start a new subprocess with @composer compile. In the new subprocess, it would load the newly downloaded code. (The main process would still exist, but it wouldn't do anything substantive, so it doesn't matter if it has the old code in memory)

    Now, as long as the old+new versions both follow this convention, you'd expect that version-switches would be more robust...

    opened by totten 1
  • ComposerPassthru - Fix error when no options are passed through

    ComposerPassthru - Fix error when no options are passed through

    Example error below:

    Generating autoload files
    3 packages you are using are looking for funding.
    Use the `composer fund` command to find out more!
    Error: implode(): Invalid arguments passed
    In ComposerPassthru.php line 62:
      implode(): Invalid arguments passed
    install [--prefer-source] [--prefer-dist] [--prefer-install PREFER-INSTALL] [--dry-run] [--dev] [--no-suggest] [--no-dev] [--no-autoloader] [--no-progress] [--no-install] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--apcu-autoloader-prefix APCU-AUTOLOADER-PREFIX] [--ignore-platform-req IGNORE-PLATFORM-REQ] [--ignore-platform-reqs] [--] [<packages>...]
    Error: Process completed with exit code 1.
    
    opened by totten 0
  • Tests - Add php81 to matrix. Update examples to work on composer latest snapshot.

    Tests - Add php81 to matrix. Update examples to work on composer latest snapshot.

    Update the test matrix to include PHP 8.1. Add a few updates to address some issues with running tests. There should be no changes affecting a typical runtime.

    opened by totten 0
  • Test suite - Fix a little random bit-rot

    Test suite - Fix a little random bit-rot

    • LocatorTest - Upstream is failing to install on composer v2. Skip for now.
    • DownloadTest - Upstream is now working on composer v2. Enable it.
    • VeryLargeTaskTransferTest - Fix warning on PHP 7.4
    opened by totten 0
  • PackageSorter - Add+fix SelfReferenceTest (Drupal 8.7.11)

    PackageSorter - Add+fix SelfReferenceTest (Drupal 8.7.11)

    The CiviCRM CI threw up an error when installing on Drupal 8.7.11, which has a peculiar situation in the dependency-graph. The root package (drupal/drupal) both

    • Replaces drupal/core-file-cache, and
    • Requires drupal/core-file-cache

    This patch makes two changes (one long, one short):

    • (Long) Fully resolve aliases upfront, before sorting. To wit: drupal/core-file-cache should be treated as drupal/drupal.
    • (Short) Skip self-requirements. To wit: it's silly for drupal/drupal to require drupal/drupal
    opened by totten 0
  • Incremental compilation

    Incremental compilation

    If you run composer install or composer compile, and if it has enough information about the inputs (watch-files) and outputs (out-files), then it will do a staleness check before compiling.

    If we don't have enough information, then err on the side of recompiling too much.

    I'm probably going to leave this sitting in a PR for a while because it adds some systemic complexity and I'm a little ambivalent about cost/benefit (maintenance/risk/performance/need/utility). Gonna let this marinate...

    marinate 
    opened by totten 0
Owner
CiviCRM
CiviCRM
This composer plugin is a temporary implementation of using symbolic links to local packages as dependencies to allow a parallel work process

Composer symlinker A Composer plugin to install packages as local symbolic links. This plugin is a temporary implementation of using symbolic links to

Pierre Cassat 18 Nov 9, 2021
Composer Repository Manager for selling Magento 2 extension and offering composer installation for ordered packages.

Magento 2 Composer Repository Credits We got inspired by https://github.com/Genmato. Composer Repository for Magento 2 This extension works as a Magen

EAdesign 18 Dec 16, 2021
Composer plugin that wraps all composer vendor packages inside your own namespace. Intended for WordPress plugins.

Imposter Plugin Composer plugin that wraps all composer vendor packages inside your own namespace. Intended for WordPress plugins. Built with ♥ by Typ

Typist Tech 127 Dec 17, 2022
A Laravel package to help track user onboarding steps

A Laravel package to help track user onboarding steps This package lets you set up an onboarding flow for your application's users. Here's an example

Spatie 467 Dec 26, 2022
Enforce that your classes get only instantiated by the factories you define!

Enforce that your classes get only instantiated by the factories you define!

null 3 Nov 15, 2021
The package provides an expressive "fluent" way to define model attributes.

The package provides an expressive "fluent" way to define model attributes. It automatically builds casts at the runtime and adds a native autocompletion to the models' properties.

Boris Lepikhin 506 Dec 28, 2022
Attributes to define PHP language extensions (to be enforced by static analysis)

PHP Language Extensions (currently in BETA) This library provides attributes for extending the PHP language (e.g. adding package visibility). The inte

Dave Liddament 70 Dec 19, 2022
[READ-ONLY] Properties define model metadata.

Charcoal Property Properties define object's metadata. They provide the building blocks of the Model's definition. Properties are defined globally for

The Charcoal PHP Framework 0 Jun 21, 2022
Private, self-hosted Composer/Satis repository with unlimited private and open-source packages and support for Git, Mercurial, and Subversion.

Private, self-hosted Composer/Satis repository with unlimited private and open-source packages and support for Git, Mercurial, and Subversion. HTTP API, HTTPs support, webhook handler, scheduled builds, Slack and HipChat integration.

Łukasz Lach 112 Nov 24, 2022
Private Composer registry for private PHP packages on AWS Serverless

Tug Tug is a Composer private registry for private PHP packages installable with Composer (1 and 2). The main idea of this project is to have an inter

Fxp 33 Oct 5, 2022
A workbench for developing Composer packages.

studio Installation Usage Workflow Command Reference License Contributing For enterprise Develop your Composer libraries with style. This package make

Franz Liedke 1.1k Jan 3, 2023
Installed composer packages info

PackageInfo This package was highly inspired from ocramius/package-versions I needed some methods to read data from the composer.lock file fast...this

Martin Keckeis 7 Jun 2, 2022
Composer addon to efficiently get installed packages' version numbers

Package Versions composer/package-versions-deprecated is a fully-compatible fork of ocramius/package-versions which provides compatibility with Compos

Composer 1.4k Dec 27, 2022
This composer plugin allows you to share your selected packages between your projects by creating symlinks

Composer - Shared Package Plugin This composer plugin allows you to share your selected packages between your projects by creating symlinks. All share

L'Etudiant 169 Sep 20, 2022
Ied plugin composer - Inspired Plugin Composer: Create, publish and edit plugins from within Textpattern CMS.

ied_plugin_composer Create, publish and edit plugins from within Textpattern CMS. Creates a new page under the Extensions tab where you can edit and e

Stef Dawson 8 Oct 3, 2020
Magento-composer-installer - Composer installer for Magento modules

!!! support the maintainer of this project via Patreon: https://www.patreon.com/Flyingmana Magento Composer Installer The purpose of this project is t

null 213 Sep 24, 2022
Composer registry manager that help to easily switch to the composer repository you want

CRM - Composer Registry Manager Composer Registry Manager can help you easily and quickly switch between different composer repositories. 简体中文 Install

Tao 500 Dec 29, 2022
Dependency graph visualization for composer.json (PHP + Composer)

clue/graph-composer Graph visualization for your project's composer.json and its dependencies: Table of contents Usage graph-composer show graph-compo

Christian Lück 797 Jan 5, 2023
Composer Registrar Composer Plugin for Magento 2

This module add a global registration.php that replace the default glob search performed for each request to discover the components not installed from composer.

OpenGento 3 Mar 22, 2022