Clean up your Magento database by removing orphaned, unused and wrongly added attribute, attribute values and settings (for M2).

Overview

Magento 2 EAV Cleaner Console Command

Purpose of this project is to check for different flaws that can occur due to EAV and provide cleanup functions.

Usage

Run bin/magento in the Magento 2 root and look for the eav: commands.

Commands

  • eav:config:restore-use-default-value Check if config admin value and storeview value are the same, so "use default" doesn't work anymore. Delete the storeview values.
  • eav:attributes:restore-use-default-value Check if product attribute admin value and storeview value are the same, so "use default" doesn't work anymore. Delete the storeview values.
  • eav:attributes:remove-unused Remove attributes with no values set in products and attributes that are not present in any attribute sets.
  • eav:media:remove-unused Remove unused product images.

Dry run

Use --dry-run to check result without modifying data.

Installation

Installation with composer:

composer require magento-hackathon/module-eavcleaner-m2

Contributors

  • Nikita Zhavoronkova
  • Anastasiia Sukhorukova
  • Peter Jaap Blaakmeer

Special thanks to

  • Benno Lippert
  • Damian Luszczymak
  • Joke Puts
  • Ralf Siepker
Comments
  • There are no commands defined in the

    There are no commands defined in the "eav:media" namespace

    Hello, I'm running Magento 2.3.6 with PHP 7.3. I have tried making sure the module is enabled. Other commands show up and work just fine. Any advice for me?

    opened by cypher909 7
  • Remove data from 'wrong' table

    Remove data from 'wrong' table

    When the backend_type of an attribute changes (eg, from varchar to text), Magento will not clean up / migrate the data from the old table automatically. Extensions should do this as part of changing the backend_type, but not all extensions follow best practices. This pull request updates the SQL used to identify orphaned data to also detect data in the 'wrong' table.

    opened by fredden 3
  • Skip placeholder files

    Skip placeholder files

    When running bin/magento eav:media:remove-unused --dry-run, I see that the tool wants to remove placeholder images:

    ## WOULD REMOVE: /placeholder/default/photo-coming-soon3.jpg ##
    ## WOULD REMOVE: /placeholder/default/photo-coming-soon3_1.jpg ##
    ## WOULD REMOVE: /placeholder/default/photo-coming-soon3_2.jpg ##
    ## WOULD REMOVE: /placeholder/default/photo-coming-soon3_3.jpg ##
    

    I would think we'd want to skip these, correct? Since they are referenced in core_config_data? If you agree, we can modify the console command class to skip /placeholder like you do /cache, as seen in \Hackathon\EAVCleaner\Console\Command\RemoveUnusedMediaCommand::execute:

        private function isInCachePath(?string $file): bool
        {
            return strpos($file, '/cache') !== false;
        }
    

    If this makes sense to you, I can submit a PR, if needed.

    opened by perryholden 3
  • Error in tables

    Error in tables

    When I ran: bin/magento eav:attributes:remove-unused --dry-run

    I receive:

    In Mysql.php line 599:

    SQLSTATE[42S02]: Base table or view not found: 1146 Table 'ee.catalog_product_entity_static' doesn't exist, query was:
    SELECT COUNT(*) FROM catalog_product_entity_static WHERE attribute_id = ?

    In Mysql.php line 110:

    SQLSTATE[42S02]: Base table or view not found: 1146 Table 'ee.catalog_product_entity_static' doesn't exist, query was:
    SELECT COUNT(*) FROM catalog_product_entity_static WHERE attribute_id = ?

    In Mysql.php line 91:

    SQLSTATE[42S02]: Base table or view not found: 1146 Table 'ee.catalog_product_entity_static' doesn't exist

    bug 
    opened by teodorpav 3
  • Show all images for deletion.

    Show all images for deletion.

    Hi, I ran it with --dry-run mode but it showed that it will delete all photos, not just those that are not in catalog_product_entity_media_gallery.

    Magento 2.4.3-p1

    opened by teodorpav 3
  • Unused images are not deleted, because deleted products are still contained in catalog_product_entity_media_gallery

    Unused images are not deleted, because deleted products are still contained in catalog_product_entity_media_gallery

    The command "eav:attributes:remove-unused" does not show any images to be deleted. This is because Magento does not delete the deleted products from the catalog_product_entity_media_gallery table. The table now has more than 20000 entries, even though our store only has just under 4000 products. I found a similar error description here: https://magento.stackexchange.com/questions/238244/magento2-manage-and-understand-images-of-deleted-products-behavior

    I am using Magento 2.3.7p3.

    Does anyone have an idea what this is due to?

    opened by FlobeiGM 2
  • Feature Request: Add --force command to eav:media:remove-unused

    Feature Request: Add --force command to eav:media:remove-unused

    I added eav:media:remove-unused command to a cron due to how my Magento site adds products, it syncs up images nightly and wipes all product images with new ones. Magento leaves the old images on the hard disk so I want it to be cleaned up daily.

    I added eav:media:remove-unused to a cron but because of the prompt I don't think it runs - I amended the vendor file to do this which works for me but am scared of it being overwritten in the future - hence the feature request! Otherwise I'll make a plugin to properly extend it.

    protected function configure()
        {
            $this
                ->setName('eav:media:remove-unused')
                ->setDescription('Remove unused product images')
                ->addOption('dry-run')
    			->addOption('force');
        }
    
    
      public function execute(InputInterface $input, OutputInterface $output): void
       {
           $fileSize   = 0;
           $countFiles = 0;
           $isDryRun   = $input->getOption('dry-run');
           $isForce   = $input->getOption('force');
    
           if (!$isDryRun && $input->isInteractive() && !$isForce) {
    

    Adding a --force command would be useful for those who want this in a cron.

    opened by scotty0100 1
  • Move null value check outside loop

    Move null value check outside loop

    When reviewing this module for suitability and quality, I noticed a performance issue in the eav:attributes:restore-use-default-value command. When dealing with small databases, this inefficiency does not cost significantly; however, when working with large databases, this is causing significant delay within each loop iteration. This pull request resolves this inefficiency, allowing the command to run more quickly on all database sizes.

    I recommend reviewing this with 'ignore white-space' enabled. I have tested this on Magento v2.4.3-p1.

    opened by fredden 1
  • Respect db_prefix

    Respect db_prefix

    When using a database prefix, the commands in this module do not work. This change allows the same to work. I've tested this on Magento version 2.4.3 with db/table_prefix set in app/etc/env.php

    opened by fredden 1
  • Replace use of fetchAll() for database queries

    Replace use of fetchAll() for database queries

    When running bin/magento eav:attributes:restore-use-default-value on a website with a large catalogue, PHP ran out of memory. This is because the command attempted to load all values into memory, however as only one row is processed at a time, replacing this with the resource iterator makes sense. With the changes in this pull request I am now able to use all commands on this website.

    opened by fredden 0
  • Remove config values matching config.xml

    Remove config values matching config.xml

    This pull request enhances the eav:config:restore-use-default-value command to also remove values from the database which match those set in etc/config.xml.

    This fixes #7.

    opened by fredden 0
  • Remove orphaned attributes

    Remove orphaned attributes

    • Create a list of all product attributes from the database
    • Search through the codebase to see whether they are mentioned in InstallData/UpdateData scripts - those are fine
    • Search through phtml files to see whether they are used (also make sure to look for getAttributeName / setAttributeName instead of just attribute_name) - those are fine
    • We should whitelist all core attributes since those have names that occur a lot ('name' / 'sku')
    • The remaining product attributes might be removeable

    This probably can't be done automatically since it'll most likely show too many false positives.

    opened by peterjaap 0
  • Remove orphaned tables

    Remove orphaned tables

    Maybe we should add a command that can identify which tables can be deleted from a Magento 2 install? I'm thinking it should do something like this;

    • Whitelist all core Magento tables
    • Run through the db_schema files and InstallSchema scripts of all extensions to identify table names
    • Compare that list versus the tables actually in the database
    • Interactively ask to drop that table (or generate queries to do it)
    opened by peterjaap 0
Releases(1.5.0)
  • 1.5.0(Jul 28, 2022)

    What's Changed

    • Replace use of fetchAll() for database queries by @fredden in https://github.com/magento-hackathon/module-eavcleaner-m2/pull/18

    Full Changelog: https://github.com/magento-hackathon/module-eavcleaner-m2/compare/1.4.0...1.5.0

    Source code(tar.gz)
    Source code(zip)
  • 1.4.0(Jul 26, 2022)

    What's Changed

    • Add --force option by @fredden in https://github.com/magento-hackathon/module-eavcleaner-m2/pull/17

    Full Changelog: https://github.com/magento-hackathon/module-eavcleaner-m2/compare/1.3.0...1.4.0

    Source code(tar.gz)
    Source code(zip)
  • 1.3.0(Jul 25, 2022)

    What's Changed

    • Remove data from 'wrong' table by @fredden in https://github.com/magento-hackathon/module-eavcleaner-m2/pull/16

    Full Changelog: https://github.com/magento-hackathon/module-eavcleaner-m2/compare/1.2.5...1.3.0

    Source code(tar.gz)
    Source code(zip)
  • 1.2.5(Mar 11, 2022)

    What's Changed

    • Move null value check outside loop by @fredden in https://github.com/magento-hackathon/module-eavcleaner-m2/pull/10

    Full Changelog: https://github.com/magento-hackathon/module-eavcleaner-m2/compare/1.2.4...1.2.5

    Source code(tar.gz)
    Source code(zip)
  • 1.2.4(Mar 10, 2022)

    What's Changed

    • Respect db_prefix by @fredden in https://github.com/magento-hackathon/module-eavcleaner-m2/pull/8
    • Remove config values matching config.xml by @fredden in https://github.com/magento-hackathon/module-eavcleaner-m2/pull/9

    New Contributors

    • @fredden made their first contribution in https://github.com/magento-hackathon/module-eavcleaner-m2/pull/8

    Full Changelog: https://github.com/magento-hackathon/module-eavcleaner-m2/compare/1.2.3...1.2.4

    Source code(tar.gz)
    Source code(zip)
  • 1.2.3(Feb 16, 2022)

    What's Changed

    • Add placeholder path guard to avoid removing placeholder images by @perryholden in https://github.com/magento-hackathon/module-eavcleaner-m2/pull/6

    New Contributors

    • @perryholden made their first contribution in https://github.com/magento-hackathon/module-eavcleaner-m2/pull/6

    Full Changelog: https://github.com/magento-hackathon/module-eavcleaner-m2/compare/1.2.2...1.2.3

    Source code(tar.gz)
    Source code(zip)
  • 1.2.2(Jan 25, 2022)

    What's Changed

    • Ignore static attributes, fixes #3 by @sprankhub in https://github.com/magento-hackathon/module-eavcleaner-m2/pull/4

    New Contributors

    • @sprankhub made their first contribution in https://github.com/magento-hackathon/module-eavcleaner-m2/pull/4

    Full Changelog: https://github.com/magento-hackathon/module-eavcleaner-m2/compare/1.2.1...1.2.2

    Source code(tar.gz)
    Source code(zip)
Owner
FireGento e. V. - Hackathons
Hackathon Repositories — belongs to https://github.com/firegento
FireGento e. V. - Hackathons
SleekwareDB is a NoSQL database storage service. A database storage service that can be used for various platforms and is easy to integrate.

SleekwareDB is a NoSQL database storage service. A database storage service that can be used for various platforms and is easy to integrate. NoSQL API

SleekwareDB 12 Dec 11, 2022
Pure PHP NoSQL database with no dependency. Flat file, JSON based document database.

Please give it a Star if you like the project ?? ❤️ SleekDB - A NoSQL Database made using PHP Full documentation: https://sleekdb.github.io/ SleekDB i

Kazi Mehedi Hasan 745 Jan 7, 2023
Laravel Migrations Generator: Automatically generate your migrations from an existing database schema.

Laravel Migrations Generator Generate Laravel Migrations from an existing database, including indexes and foreign keys! Upgrading to Laravel 5.4 Pleas

Bernhard Breytenbach 3.3k Dec 30, 2022
Adminer database management tool for your Laravel application.

Laravel Adminer Adminer database management tool for your Laravel application. Table of Contents Introduction Features Installation CSRF token middlew

Khalid Moharrum 45 Jul 25, 2022
Seed your database with CSV files

CSV Seeder Seed your database with CSV files This package allows CSV based seeds. Installation Require this package in your composer.json and run comp

null 249 Dec 13, 2022
Connect and work with MySQL/MariaDB database through MySQLi in PHP. This is an introductory project, If you need a simple and straightforward example that takes you straight to the point, you can check out these examples.

First MySQLi PHP Connect and work with MySQL/MariaDB database through MySQLi in PHP. The above exercises are designed for students. This is an introdu

Max Base 4 Feb 22, 2022
A simple library to access and manipulate database records. Built on top of Dibi and hardwired for PostgreSQL.

grifart/tables A simple library to access and manipulate database records. Built on top of Dibi and hardwired for PostgreSQL. This library is develope

GRIFART 5 Nov 11, 2022
This package provides a framework-agnostic database backup manager for dumping to and restoring databases from S3, Dropbox, FTP, SFTP, and Rackspace Cloud

Database Backup Manager This package provides a framework-agnostic database backup manager for dumping to and restoring databases from S3, Dropbox, FT

Backup Manager 1.6k Dec 23, 2022
ORM layer that creates models, config and database on the fly

RedBeanPHP 5 RedBeanPHP is an easy to use ORM tool for PHP. Automatically creates tables and columns as you go No configuration, just fire and forget

Gabor de Mooij 2.2k Jan 9, 2023
A Redis based, fully automated and scalable database cache layer for Laravel

Lada Cache A Redis based, fully automated and scalable database cache layer for Laravel Contributors wanted! Have a look at the open issues and send m

Matt 501 Dec 30, 2022
A php class for managing and connecting to a database

Query builder class php This class is responsible for creating and executing sql commands and helps you to execute as easily as possible and safely. I

Mohammad Qasemi 39 Dec 11, 2022
A complete, simple and powerful database framework written in PHP

BaseSQL BaseSQL is a complete database framework written in PHP. It was built to accelerate projects development by handle database connections and qu

Willian Pinheiro 2 Sep 21, 2021
The query filter bundle allows you to filter data from QueryBuilder and the Database

The query filter bundle allows you to filter data from QueryBuilder and the Database. you can filter multiple columns at the same time and also you can filter relation fields with two-level deep and without any join in your query builder.

Milad Ghofrani 0 Apr 8, 2022
A link database for discovering cool and interesting websites

Yesterlinks Yesterlinks is a database of links to unique or interesting websites. This is a project, in its infancy, designed to be a directory of lin

Sadness 14 Dec 13, 2022
Low code , Zero Configuration ORM that creates models, config, database and tables on the fly.

?? ARCA ORM ?? Low code , Zero Configuration ORM that creates models, config, database and tables on the fly. ?? ???? Made in India ???? Complete docu

Scrawler Labs 28 Dec 18, 2022
The query sorting bundle allows you to sort data from QueryBuilder and the Database

The query sorting bundle allows you to sort data from QueryBuilder and the Database. you can sort multiple columns at the same time and also you can sort relation fields with two-level deep and without any join in your query builder.

Bugloos 12 Oct 6, 2022
A simple library for managing database connections, results pagination and building queries in PHP

PHP lions-software-database-manager This is a simple library for managing database connections, results pagination and building queries in PHP. Esta é

Lions Software 0 Feb 7, 2022
The Enobrev\ORM library is a small framework of classes meant to be used for simply mapping a mysql database to PHP classes, and for creating simply SQL statements using those classes.

The Enobrev\ORM library is a small framework of classes meant to be used for simply mapping a mysql database to PHP classes, and for creating simply SQL statements using those classes.

Mark Armendariz 0 Jan 7, 2022
The lightweight PHP database framework to accelerate development

The lightweight PHP database framework to accelerate development Features Lightweight - Less than 100 KB, portable with only one file Easy - Extremely

Angel Lai 4.6k Dec 28, 2022