Giving the command: ./vendor/bin/drall exd cr --drall-workers=4 on a site where there are 4 installations, it should do the command under one command time but it's visibly and measurable that it runs serial still, instead of parallel.

Requirements

• Say, there are 200 sites.
• The user wants to run 5 instances of drush, thereby running commands on 5 sites at a time.
  • User defines number of parallel processes, e.g. --drall-processes=5.
• Ensure intelligible output.
  • If we run a command like foo &; bar & the lines of output of foo and bar get mixed up.
  • We need to ensure that the output of each drush command appears together and is readable.

Solution

• If --drall-processes, say n, is greater than 1, then continue, otherwise, there's nothing special to do.
• Create a unique ID for the command, e.g. a UUID.
• Get a list of all the targeted sites.
• Create a JSON file in the tmp directory to track command progress.
  • Let's call this the task file.
• Launch n drall commands (let's call these workers) with a reference to the task file.
  • Let's call these Drall workers.
  • E.g. drall --drall-task=/path/to/task.json ... &;.
• Each Drall worker picks up a site,
  • Takes a task from the list (if it's not empty).
  • Does the task.
  • Obtains a lock on the list.
  • Updates the list.
  • Prints output.
  • Releases lock on the list.

Tasks

• [x] Decide task file specifications.
  • [x] Decide task file name and location.
  • [x] Decide task file structure.
• [x] Create an internal command exec:task.
  • [x] Takes a task path as an argument.
  • [x] Implement task file locking mechanism, i.e. only one process can write to the task file.
  • [x] Implement task execution mechanism, i.e. execute a task/command from the task file and mark it as done.
• Update the exec:drush command.
  • Support a --drall-processes=n parameter.
  • If n > 1, then generate a task file and launch Drall workers to execute the tasks.
• [x] Update tests
• [x] Update documentation

Tasks

• [x] Use amphp/process libraries
  • [x] Limit max-workers to match --drall-workers value.
  • [x] Update tests
• [x] Remove all code from the old implementation of parallel execution
  • [x] Remove Runner/*
  • [x] Remove Queue/*
• [x] Update README, if required.

Tasks

• [x] PR: Issue #38
• [x] PR: Create and use RunnerAwareTrait.
• [x] PR: Create ExecRunner which should use exec() for capturing output and exit code.
• [x] PR: Create TestCase::testCreateTempFilePath().
• [x] PR: Create models: Queue\*.
• [x] PR: Create exec:queue command.
• [x] Implement --drall-workers=n in exec:shell and exec:drush commands.
• [x] Ensure proper test coverage.
• [x] Update README

Requirements

• Passing the --root option should make Drall do site/alias discovery in the correct directory instead of just using PWD.
• [x] Works with Drupal document root
• [x] Works with Composer project root

Requirements

• Currently, the dev workflow assumes that Drupal is always installed, i.e. drush site:install.
• In reality, we don't need the Drupal database to develop or test drall unless we really want to test commands that interact with Drupal's database.
• Thus, a good solution will be to make the database setup optional.
• This will speed up build times for the Docker setup and in turn, speed up GitHub actions.

Tasks

• [x] Make drush site:install optional.
• [ ] Do not require the database service in CI builds.

Requirements

All sites in a multi-site don't always have site aliases. Many projects rely on drush's --uri option to run the command on the correct site. Thus, we need to allow users to choose whether they want to use the URI or Drush aliases.

Todo

• [x] If the user runs a command with --uri=@@uri, we replace the @@uri with individual site URIs before sending the command to Drush.
• [x] If the user runs a command with @@site, we replace the @@site with a site alias's name part, i.e. excluding the env part.
• [x] If the user runs a command without any of the above, we use --uri=@@uri by default.

Tasks

• [x] Add commands to show a list of sites present in the multi-site installation
  • [x] Detect sites using sites/*/settings.php
  • [x] Detect sites using drush alias definitions
  • [x] Preferably, the list of sites should be iterable in shell scripts
• [x] Use a SiteDetector service to make the code re-usable.

Notes

• Only support D8+.
• Use PHP 8 or higher.

There used to be an extra space in front of certain Drush output but now that space is gone. This PR updates the expected output to match the new Drush output which fixes the tests.

What's done

• Set up locking mechanism using lock files.
• Create models to represent Queue data and Queue items.
• Create model for reading and writing to Queue files.

Motivation

• When running a drall command with -vvv, the -vvv gets passed to Drush.
• Drush output can get very verbose with -vvv and it makes it hard to read the Drall log messages.

Proposed solution

• Use something like --drall-verbose and --drall-debug to control Drall's verbosity.
• These options should not be forwarded to Drush commands.

Not only Bitbucket, it seems Drall gets stuck without any message when it is executed on aliases that refer to remote environments, e.g.

drall exd @@site.dev core:status

Maybe there's some kind of dialog that's present in there that we can't see? This seems like a dealbreaker.

Requirements

• Allow users to view the report in different formats.
• Some basic formats I to start with are shown below.

Tasks

• [ ] Use jigarius/phprogress.
• [ ] Implement the default format.
  • This is what we already have at the moment, just move the code into a formatter.
• [ ] Implement the dots format.
• [ ] Implement the bar format.

Formats

Names for these formats have not been decided. Feel free to make suggestions.

Default

Running on donnie.com
Cache rebuild successful.

Running on leo.com
Cache rebuild successful.

Running on ralph.com
Cache rebuild successful.

Running on mikey.com
Command failed.
Error: Could not connect to database.

Running tmnt.com
Cache rebuild successful.

Dots

...F.

Bar

60% [====================>----------] [3/5]

Requirements

• Drall should work on Windows
  • This will mainly involve using the Windows directory separator \.

If I get 25 👍🏽 on this issue, I'll start working on it.

When I simple give the command $ ./vendor/bin/drall exd uli I receive URLs which does not contain the domains:

Running: drush --uri=eaad uli
http://eaad/en/user/reset/1/1658916912/XtxmYUYDfYWCNDiF2Jf43DCXuQmpeNo6JbRW1NFIFhQ/login
Running: drush --uri=escaide uli
http://escaide/en/user/reset/1/1658916913/nFrSxO18O9BEHqC5E9cYO13pnJpp_2_28_qo5Ai0Rc4/login
Running: drush --uri=portal uli
http://portal/en/user/reset/1/1658916913/ugMuDeCVgV0Q5LgUJWTl7fPJ3N7lnDPebipyeVGtoeQ/login
Running: drush --uri=vaccine uli
http://vaccine/en/user/reset/1/1658916913/zzinbUGo-UrOm4izTtFtSByJqQQofUl6AHGIzIxgaMY/login

To have it correctly I need to run ./vendor/bin/drall exd @@@uri uli

Running: drush @eaad uli
http://ecdc-eaad.lh:8080/en/user/reset/1/1658916788/TpF3oxM2c5b3xuyisYbgfdzJqkBSRvNMgP27JjX4KlA/login
Running: drush @escaide uli
http://ecdc-escaide.lh:8080/en/user/reset/1/1658916789/xrZNk927t2Yalo1n3fpPrdRcMY3ZfEN9j_ylpAgjq9o/login
Running: drush @portal uli
http://ecdc-portal.lh:8080/en/user/reset/1/1658916789/gIvVSelKq1VyWkXExzQ7kjWFZ_5ZCrYcXI8FKeal6Gk/login
Running: drush @vaccine uli
http://ecdc-vaccine.lh:8080/en/user/reset/1/1658916789/NXtzQYeDinF04dF4nU5DyZR3zWMrKp-f4ZjHF4bAKDI/login

I would expect to have the correct url with the simple exd uli command too.

NOTE: here for us the @@@uri does work as the directory and the drush alias name is actually the same, otherwise it won't work.