A dev-friendly approach to handle background jobs in Magento 2
Overview
💭
Now and then we need to create processes that can take some time to execute, and that doesn't necessarily need to be done in real time. Like (but not limited to) third-party integrations.
For example, let's say you need to reflect product changes made by the storekeeper through the admin panel to their PIM/ERP. You can observe the catalog_product_save_after event and push the changes, but this would make the "Save" admin action become a hostage of the third-party system response time, potentially making the store admin reeealy slow.
But fear not citizens, because we are here!
Install
🔧
This module is compatible with both Magento 2.3 and Magento 2.4, from PHP 7.2 to 7.4.
composer require discorgento/module-queue
bin/magento setup:upgrade
Usage
⚙️
There's just two steps needed: 1) append a job to the queue, 2) create the job class itself (similar to Laravel).
Let's go back to the product sync example. You can now write the catalog_product_save_after event observer like this:
class ProductSaveAfter implements \Magento\Framework\Event\ObserverInterface
{
protected $queueHelper;
public function __construct(
\Discorgento\Queue\Helper\Data $queueHelper
) {
$this->queueHelper = $queueHelper;
}
/** @inheritDoc */
public function execute(
\Magento\Framework\Event\Observer $observer
) {
// append a job to the queue so it will run later in background
$this->queueHelper->append(
\YourCompany\YourModule\Jobs\SyncProduct::class, // job class, we'll create it below
$observer->getProduct()->getId(), // job "target", in that case the product id
['foo' => $observer->getFoo()] // additional data for later usage (optional)
);
}
}
Now, create the job itself, like app/code/YourCompany/YourModule/Jobs/SyncProduct.php:
// the job should implement the JobInterface
class SyncProduct implements \Discorgento\Queue\Api\JobInterface
{
protected $productRepository;
protected $productSynchronizer;
public function __construct(
\Magento\Catalog\Api\ProductRepositoryInterface $productRepository,
\YourCompany\YourModule\Helper\Sync\Product $productSynchronizer
) {
$this->productRepository = $productRepository;
$this->productSynchronizer = $productSynchronizer;
}
/**
* @param int|string|null $target The product id
* @param array $additionalData Optional extra data inserted on append
*/
public function execute($target, $additionalData)
{
// retrieve the product and sync it
$product = $this->productRepository->getById($target);
$this->productSynchronizer->sync($product);
}
}
And.. that's it! In the next cron iteration (which should be within the next minute) your job will be executed without compromising the performance of the store at all, assuring a smooth workflow for both your clients and their customers.
💡 Tip: any async process can benefit from this approach, your creativity is the limit.
Debugging
🪲
Developer Mode
You can force the queue execution whenever you want through bin/magento discorgento:queue:execute command: 
Production Mode
The queue should be running alongside with the store cron. You can check for errors in var/log/discorgento_queue.log log file.
Roadmap
🧭
- add a safety lock to prevent jobs from overflowing each other;
- add an option on admin allowing to choose between cron and rabbitmq backend;
- create console commands to execute (discorgento:queue:execute) and clear (discorgento:queue:clear) the queue;
Footer notes
🗒
- Magento can do this natively through Message Queues, but those are ridiculously verbose to use;
- issues and PRs are welcome in this repo;
- we want YOU for our community;
0 Aug 2, 2022
4 Dec 17, 2022
28 Dec 19, 2022
2 Dec 30, 2022
187 Nov 4, 2022
1.1k Jan 5, 2023
26 Apr 5, 2022
5 Jun 13, 2023
4 Feb 22, 2022
14 Sep 12, 2022
762 Dec 30, 2022
109 Oct 21, 2022
80 Aug 20, 2022
78 Dec 26, 2022
58 Jan 1, 2023
6 May 31, 2023
![[Proposal] - Add support to phpunit ^9](https://avatars.githubusercontent.com/u/5350377?v=4)
17 Sep 12, 2022
4.6k Dec 16, 2022
719 Jan 1, 2023
15 Jan 7, 2022
18 Dec 31, 2022
208 Nov 2, 2022
3 Mar 4, 2022
356 Dec 29, 2022
62 Apr 1, 2022
8 Sep 8, 2022