A Laravel package to scrub sensitive information that breaks operational security policies from being leaked on accident or not by developers.

Overview


Laravel Scrubber

GitHub license GitHub stars GitHub issues GitHub forks Packagist Downloads PHPUnit

A Laravel package to scrub sensitive information that breaks operational security policies from being leaked on accident or not by developers.

Installation

install the package via composer:

composer require yorcreative/laravel-scrubber

Publish the packages assets.

php artisan vendor:publish --provider="YorCreative\Scrubber\ScrubberServiceProvider"

Configuration

Adjust the configuration file to suite your application.

return [
    'redaction' => '**redacted**', // Define what you want to overwrite detected information with??
    'secret_manager' => [
        'key' => '44mfXzhGl4IiILZ844mfXzhGl4IiILZ8', // key for cipher to use
        'cipher' => 'AES-256-CBC', 
        'enabled' => false, // Do you want this enabled??
        'providers' => [
            'gitlab' => [
                'enabled' => false,
                'project_id' => env('GITLAB_PROJECT_ID', 'change_me'),
                'token' => env('GITLAB_TOKEN', 'change_me'),
                'host' => 'https://gitlab.com',
                'keys' => ['*'], // * will grab all the secrets, if you want specific variables
                                 //  define the keys in an array
            ],
        ],
    ],
    'regex_loader' => ['*'] // Opt-in to specific regex classes or include all with * wildcard.
];

Usage

The scrubber can be utilized in two ways, the first one being a Log scrubber. A tap is added to detect and sanitize any sensitive information from hitting the log file. The second way is to integrate into your application and utilize the Scrubber directly. This way is particular useful if you, for example, would like to detect and sanitize any messages on a messaging platform.

Logging Detection & Sanitization

') // testing.INFO: **redacted** ">
Log::info('some message', [
    'context' => 'accidental',
    'leak_of' => [
        'jwt' => ''
    ]
])

// testing.INFO: some message {"context":"accidental","leak_of":{"jwt": '**redacted**'}} 

Log::info('')

// testing.INFO: **redacted**  

Direct Usage for Detection & Sanitization

"accidental" // "leak_of" => [ // "jwt" => "**redacted**" // ] // ]; Scrubber::processMessage(''); // **redacted**">
Scrubber::processMessage([
    'context' => 'accidental',
    'leak_of' => [
        'jwt' => ''
    ]
]);
// [
//     "context" => "accidental"
//     "leak_of" => [
//         "jwt" => "**redacted**"
//     ]
// ];

Scrubber::processMessage('');
// **redacted**

Regex Class Opt-in

You have the ability through the configuration file to define what regex classes you want loaded into the application when it is bootstrapped. By default, this package ships with a wildcard value.

Regex Collection & Defining Opt-in

To opt in, utilize the static properties on the RegexCollection class.

 'regex_loader' => [
        RegexCollection::$GOOGLE_API,
        RegexCollection::$AUTHORIZATION_BEARER,
        RegexCollection::$CREDIT_CARD_AMERICAN_EXPRESS,
        RegexCollection::$CREDIT_CARD_DISCOVER,
        RegexCollection::$CREDIT_CARD_VISA,
        RegexCollection::$JSON_WEB_TOKEN
    ],

Secret Manager

This package provides the ability to pull in secrets from external sources. This provides the package the ability to detect leakage and sanitize secrets without needing an exact regex pattern to detect it.

Encryption

For enhanced application security, all secrets that are pulled from any provider are encrypted and only decrypted to run the detection. You can see this in action here.

Gitlab Integration

To utilize the Gitlab Integration, you will need to enable the secret_manager and the gitlab provider in the Configuration file. If you are looking for information on how to add secrets in Gitlab. There is an article on adding project variables.

Extending the Scrubber

Creating new Scrubber Detection Classes

php artisan make:regex-class {name} 

This command will create a stubbed out class in App\Scrubber\RegexCollection. The Scrubber package will autoload everything from the App\Scrubber\RegexCollection folder. You will need to provide a Regex Pattern and a Testable String for the class.

Testing

composer test

Credits

Comments
  • FAILS TO WORK WITH CUSTOMIZATIONS

    FAILS TO WORK WITH CUSTOMIZATIONS

    Issue

    1. THE ARTISAN COMMAND OFFERED TO CREATE Scrubber Detection Classes DOES NOT WORK CORRECTLY

    The artisan command does not scaffold the correct implementation of the interface. it produces a class in the format:

    <?php
    namespace App\Scrubber\RegexCollection;
    use YorCreative\Scrubber\Interfaces\RegexCollectionInterface;
    
    class TrialClass implements RegexCollectionInterface
    {
        public function getPattern(): string
        {
            /**
             * @todo
             * @note return a regex pattern to detect a specific piece of sensitive data.
             */
        }
    
        public function getTestableString(): string
        {
            /**
             * @todo
             * @note return a string that can be used to verify the regex pattern provided.
             */
        }
    }
    

    instead of

    
      <?php
      
      namespace YorCreative\Scrubber\RegexCollection;
      
      use YorCreative\Scrubber\Interfaces\RegexCollectionInterface;
      
      class AuthorizationBasic implements RegexCollectionInterface
      {
          public function getPattern(): string
          {
              return '(?<=basic) [a-zA-Z0-9=:_\+\/-]{5,100}';
          }
      
          public function getTestableString(): string
          {
              return 'basic f9Iu+YwMiJEsQu/vBHlbUNZRkN/ihdB1sNTU';
          }
      
          public function isSecret(): bool
          {
              return false;
          }
      } 
    

    2. ADDING A CUSTOM Scrubber Detection Classes SIMPLY DOES NOT WORK

    with the isSecret() method implemented. however that is a non issue i can correct the implementation in my code however it fails to work, infact it seems not to work at all when when there is any file in that folder it creates

    App\Scrubber\RegexCollection;

    This is despite being used as prescribed and specified in the configuration as shown below, WITH OR WITHOUT THE CHANGES IN CONFIGURATION, the thing does not work in fact it does not even show any logs as long as there is a class in the said folder

    3. TOOL ONLY SEEMS TO WORK WITH ITS STOCK 'single' log channel CANNOT WORK WITH papertrail OR ANY OTHER

    To mitigate the issue, i deleted and added my customizations directly in the code base in the vendor package and added it to the default RegexCollection ok now the tool is working as i expect however now

    The package cannot intercept logs sent over to papertrail and i cannot find a way to handle that. tried to add it as a tap like this in config/logging 'tap' => [ScrubberTap::class], still no success no logs then tried to edit package to add the tap directly like this

    
    $this->app->make('config')->set('logging.channels.papertrail.tap', [
                ScrubberTap::class,
    ]);
    
    
    

    The logs fail to come in and now there seems to be absolutely no hacks that i can do to get past this I really need help.

    bug 
    opened by Whizboy-Arnold 3
  • NOT SCRUBBING AT ALL

    NOT SCRUBBING AT ALL

    It seems after the latest update the tool broke, I thought it was not seeing the parts to redact in the logs yet but no. This is with the vanilla configuration. What is going on well is now i can get my logs on any channel and especially, papertrail, Yes this is good: but now it seems that it stopped redacting be it on local ie single channel or papertrail. Mark you this time I even tried to directly fix it to the RegexCollection class as a hack like last time, it seems just not getting invoked in the first place. Please check out on it.

    Also am writing a custom reducter This is want to work:

    'regex_loader' => [
            '*',
            "CustomRegexReducter", ///tried this not working
            CustomRegexReducter::class, ///tried this still not working
        ], 
    

    This is tried with a very simple regex as last time:

    <?php
    
    // namespace App\Scrubber\RegexCollection;
    
    namespace YorCreative\Scrubber\RegexCollection;
    
    use YorCreative\Scrubber\Interfaces\RegexCollectionInterface;
    
    class CustomRegexReducter implements RegexCollectionInterface
    {
        public function getPattern(): string
        {
            /**
             * @todo
             * @note return a regex pattern to detect a specific piece of sensitive data.
             */
            return "process";
        }
    
        public function getTestableString(): string
        {
            /**
             * @todo
             * @note return a string that can be used to verify the regex pattern provided.
             */
            return "process";
        }
    
        public function isSecret(): bool
        {
            return false;
        }
    }
    
    

    However I suppose it wont work unless the above is resolved.

    Trying to debug what could be wrong i added several dd through out the code, maybe you could:

    CHECK FOR COVERAGE:

    image

    it seems the code is not even reaching this point, that dd("registering") //check above imageis not called on the callback unless am wrong.

    question 
    opened by Whizboy-Arnold 2
  • SQLite extension is required on non-development environments

    SQLite extension is required on non-development environments

    Running composer install on my test environment (which does not have the sqlite PHP extension), I get the following error:

    $ composer install --no-interaction --prefer-dist --no-scripts -o
    Installing dependencies from lock file (including require-dev)
    Verifying lock file contents can be installed on current platform.
    Your lock file does not contain a compatible set of packages. Please run composer update.
    
      Problem 1
        - yorcreative/laravel-scrubber is locked to version v2.2.1 and an update of this package was not requested.
        - yorcreative/laravel-scrubber v2.2.1 requires ext-pdo_sqlite * -> it is missing from your system. Install or enable PHP's pdo_sqlite extension.
    

    It seems to me that this is caused by including

    "ext-pdo_sqlite": "*",
    

    in composer.json. Please move that to "require-dev"

    bug 
    opened by Magentron 1
  • Clean up README.md

    Clean up README.md

    Config block

    • Removed double question marks
    • Empty defaults are better than 'change-me' being sent

    Usage

    • Minor flow changes

    Regex Collection & Defining Opt-in

    • A link here was pointing to a blob, while it'll always be correct, if the codebase changes, this section would need to be updated anyway.

    Secret Manager

    • Changed title for "About the Scrubber" to be more descriptive
    • Cleaned up sentence flow

    Encryption

    • Cleaned up minor sentence structure
    documentation 
    opened by LorenzoSapora 1
  • Tap Loader Strategy

    Tap Loader Strategy

    • Added strategy to load channel taps.
    • Added 3 loaders: wildcard, specific and multiple channel tap loaders.
    • Added test coverage for tap loader strategy.
    • Added "tap_channels" to config with wildcard default.
    • Added log channel opt-in information to README
    enhancement 
    opened by yordadev 0
  • Log not written

    Log not written

    I cannot see my logs when I have this enabled, No error is thrown but application works correctly. I am on Laravel 9. This happens even with default configuration

    opened by Whizboy-Arnold 0
  • Regex Opt-in Ability

    Regex Opt-in Ability

    • Added regex loader strategy w/ test coverage
    • Added regex class opt in ability.
    • Added RegexCollection class with static strings of all available regex classes.
    • Updated readme with opt-in information.
    enhancement 
    opened by yordadev 0
  • Regex Opt-in Configuration

    Regex Opt-in Configuration

    Problem:

    By default all regex classes are booted up and used to detect and sanitize content. This is not optimal and an application should be able to opt-in to which regex classes are used.

    Expectation

    An application should be able to define in the configuration file what classes to boot. If no classes are opted-in, the application will boot all regex classes.

    enhancement 
    opened by yordadev 0
Releases(v2.3.0)
  • v2.3.0(Dec 17, 2022)

    What's Changed

    • Fixed handling binary data as json_encode() does not like it by @Magentron in https://github.com/YorCreative/Laravel-Scrubber/pull/22

    Full Changelog: https://github.com/YorCreative/Laravel-Scrubber/compare/v2.2.2...v2.3.0

    Source code(tar.gz)
    Source code(zip)
  • v2.2.2(Dec 8, 2022)

    What's Changed

    • Moved "ext-pdo_sqlite" extension to "require-dev" in composer.json by @Magentron in https://github.com/YorCreative/Laravel-Scrubber/pull/20

    New Contributors

    • @Magentron made their first contribution in https://github.com/YorCreative/Laravel-Scrubber/pull/20

    Full Changelog: https://github.com/YorCreative/Laravel-Scrubber/compare/v2.2.1...v2.2.2

    Source code(tar.gz)
    Source code(zip)
  • v2.2.1(Nov 13, 2022)

    What's Changed

    • Regex Loader Updates & README update. by @yordadev in https://github.com/YorCreative/Laravel-Scrubber/pull/18

    Full Changelog: https://github.com/YorCreative/Laravel-Scrubber/compare/v2.2.0...v2.2.1

    Source code(tar.gz)
    Source code(zip)
  • v2.2.0(Nov 5, 2022)

    What's Changed

    • Patched Stub & Extended Regex Loader by @yordadev in https://github.com/YorCreative/Laravel-Scrubber/pull/15
    • Tap Loader Strategy by @yordadev in https://github.com/YorCreative/Laravel-Scrubber/pull/16

    Full Changelog: https://github.com/YorCreative/Laravel-Scrubber/compare/v2.1.2...v2.2.0

    Source code(tar.gz)
    Source code(zip)
  • v2.1.2(Sep 13, 2022)

    README update

    • Config block

      • Removed double question marks
      • Empty defaults are better than 'change-me' being sent
    • Usage

      • Minor flow changes
    • Regex Collection & Defining Opt-in

      • A link here was pointing to a blob, while it'll always be correct, if the codebase changes, this section would need to be updated anyway.
    • Secret Manager

      • Changed title for "About the Scrubber" to be more descriptive
      • Cleaned up sentence flow
    • Encryption

      • Cleaned up minor sentence structure
    Source code(tar.gz)
    Source code(zip)
  • v2.1.1(Aug 31, 2022)

  • v2.1.0(Aug 26, 2022)

    • Added regex loader strategy w/ test coverage
    • Added regex class opt in ability.
    • Added regex collection class with static properties of all available regex classes for easy opt-in ability in the configuration file.
    Source code(tar.gz)
    Source code(zip)
  • v2.0.1(Aug 25, 2022)

  • v2.0.0(Aug 25, 2022)

    Initial Release of v2.0.0

    Changes:

    • [fix] Refactored the RegexRepository and created a singleton of it, as it was not loading all the classes once when the application was bootstrapped. Instead it was loading all the classes every use of the scrubber.
    • [feature] Added the Secret Manager which allows you to pull in secrets from external sources to detect leakage of them.
    • [feature] Gitlab Integration for Secret Manager.
    • [feature] Ability to define what keys of your secrets from external sources that you want via the configuration file.
    Source code(tar.gz)
    Source code(zip)
  • v1.1.0(Aug 24, 2022)

  • v1.0.0(Aug 16, 2022)

Owner
YorCreative
YorCreative focuses on package development for the Laravel community.
YorCreative
A multitool library offering access to recommended security related libraries, standardised implementations of security defences, and secure implementations of commonly performed tasks.

SecurityMultiTool A multitool library offering access to recommended security related libraries, standardised implementations of security defences, an

Pádraic Brady 131 Oct 30, 2022
ergodnc (Ergonomic Desk & Coffee) is an open source Laravel project that's being built live on the official Laravel YouTube Channel

About This Project ergodnc (Ergonomic Desk & Coffee) is an open source Laravel project that's being built live on the official Laravel YouTube Channel

Mohamed Said 248 Dec 26, 2022
Twitter clone project being developed by using PHP Laravel Framework and tailwind.css

Twits! About Twits! We, as enthusiastic learners and new developers, kicked of this project in order to improve our skills and capabilities in PhP Lar

Furkan Meraloğlu 10 Aug 29, 2022
Clean up and prevent empty meta from being saved for Job, Company, or Resume listings in database

=== Empty Meta Cleanup for WP Job Manager === Contributors: tripflex Tags: wp job manager, meta, cleanup, wpjobmanager Requires at least: 5.2 Tested u

Myles McNamara 3 Feb 7, 2022
Laravel Seo package for Content writer/admin/web master who do not know programming but want to edit/update SEO tags from dashboard

Laravel Seo Tools Laravel is becoming more and more popular and lots of web application are developing. In most of the web application there need some

Tuhin Bepari 130 Dec 23, 2022
Your users do not always report errors, LaraBug does. LaraBug is a simple to use and implement error tracker built for the Laravel framework.

Your users do not always report errors, LaraBug does. LaraBug is a simple to use and implement error tracker built for the Laravel framework. This rep

LaraBug 197 Dec 9, 2022
🕵🏻‍♂️  The easiest way to respect the "do not track" header in Laravel

trackable The easiest way to respect the "do not track" header in Laravel Installation composer require s360digital/trackable API Trackable will expos

s360 2 Oct 7, 2022
A simple package that helps PHP developers to generate the QR code signature as per Zakat authority (ZATCA) requirements of Saudi Arabia.

A PHP package that implements the e-invoice QR code signature requirements as designed by the Zakat authority of Saudi Arabia. How to install? compose

Muktar Sayed Saleh 5 Jun 13, 2022
A code generation tool for Laravel developers.

Blueprint is an open-source tool for rapidly generating multiple Laravel components from a single, human readable definition. Watch a quick demo of Bl

Laravel Shift 2.4k Jan 5, 2023
A nice GUI for Laravel Artisan, ready out of the box, configurable and handy for non-CLI experienced developers.

Artisan UI A nice GUI for Laravel Artisan, ready out of the box, configurable and handy for non-CLI experienced developers. Supported commands must be

Pablo Leone 1 Dec 3, 2021
Fully customizable and tests supported Laravel admin dashboard for developers.

Laravel Admin dashboard Like Laravel Jetstream but built with Hotwire Turbo + additional perks. Tools used: tailwindcomponents/dashboard Hotwire Turbo

null 12 Nov 1, 2022
Added Laravel functionality to Enlightn Security Checker.

Added Laravel functionality to Enlightn Security Checker. Adds a command to check for, and optionally emails you, vulnerabilities when they affect you.

Jorijn Schrijvershof 184 Oct 27, 2022
Added Laravel functionality to Enlightn Security Checker

Added Laravel functionality to Enlightn Security Checker. Adds a command to check for, and optionally emails you, vulnerabilities when they affect you.

Jorijn Schrijvershof 184 Oct 27, 2022
Add Server-Timing header information from within your Laravel apps.

Laravel Server Timings Add Server-Timing header information from within your Laravel apps. Installation You can install the package via composer: comp

Beyond Code 498 Dec 15, 2022
Remita Clone for PHP-CURL Developers

REMITA - Empower your Customers to pay you easily as you grow Description Remita clone app (PHP) was build to ease programmers to integerate remita pa

Abdurrahim Yahya Muazu 4 Mar 31, 2022
Collection of scripts, thoughts about CSP (Content Security Policy)

CSP useful, a collection of scripts, thoughts about CSP I'm testing and using CSP (Content Security Policy), and here are some thoughts, resources, sc

Nicolas Hoffmann 417 Jan 3, 2023
Component for reading phar.io manifest information from a PHP Archive (PHAR)

Manifest Component for reading phar.io manifest information from a PHP Archive (PHAR). Installation You can add this library as a local, per-project d

null 7.1k Jan 9, 2023
Get the thumbnail of youtube and vimeo videos from the url. The returned information is ID and URL of the thumbnail

Video Thumbnail URL Get the thumbnail of youtube and vimeo videos from the url. The returned information is ID and URL of the thumbnail Installation I

Fernando Valler 2 Jan 22, 2022
How to get cookies from users' browser and send the information to your email address and telegram bot

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

MAXWELL 3 Dec 3, 2022