PHP Discord Webcrawler to log all messages from a Discord Chat.

Daniel Reis

Last update: Sep 21, 2022

Related tags

Overview

Disco the Ripper

Disco the Ripper was created to rip all messages from a Discord specific channel into JSON via CLI and help people to investigate some servers who has awkward channels before they get deleted.

This project was designed with Laravel Zero.

Installation

1. Install the dependencies

Use the package manager composer to install Disco the Ripper.

Go to the project folder after install Composer and run the command below:

composer install

2. Setup your credentials

Authorization Token

After everything get installed you should create or make copy the .env.example and rename it to .env.

The env file should contain one key called AUTHORIZATION and you should assign his value with your Discord Web Authorization Token. To get this token, open your Discord App or on the Browser and open the Inspection Page.

To open Inspection Tab on Discord press CTRL + SHIFT + I
To open Inspection Tab on Browser, Press F12

There you will click on Network tab and filter all XHR requests:

After that you will select any request (red square) to get the Authorization Header:

If you got your Authorization Token, put it on your .env file and you should see something like:

User ID

Go to your Discord Settings and enable the Developer Mode.

After that, send any message in any chat on your Discord account and click with the Right button and select the *Copy ID option.

If you paste (CTRL + V) some numbers should appears and should be like: 866058219859214336.

Your env should have this two tokens to run the project.

USERID=866058219859214336
AUTHORIZATION="ODY2MDU4MjE5ODU5MjE0MzM2.xxxxx.6fazt4YSRCQ6vlWMJwEY7pO5hUM"

Usage

To start the project, run the command below and select which server/channel you want to retrieve.

php artisan ripper:start

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

License

MIT

You might also like...

PHP scraper for ZEE5 Live Streaming URL's Using The Channel ID and Direct Play Anywhere

It can scrape ZEE5 Live Streaming URL's Using The Channel ID and Direct Play Anywhere

1 Mar 24, 2022

PHP library to Scrape website into entity easily

Scraper Scraper can handle multiple request type and transform them into object in order to create some API. Installation composer require rem42/scrap

5 Dec 18, 2021

Roach is a complete web scraping toolkit for PHP

🐴 Roach A complete web scraping toolkit for PHP About Roach is a complete web scraping toolkit for PHP. It is heavily inspired (read: a shameless clo

1.1k Jan 3, 2023

A Discord bot that rewards players in Minecraft for sending messages on Discord

MCRewards-Discord-Bot A Discord bot that rewards players in Minecraft for sending messages on Discord How to setup: Download this git as a .zip, or cl

3 Dec 26, 2021

Sends log messages to the Logentries log management service

Phalcon Logentries Phalcon library to connect and make log entries using Logentries. You can adapt it to your own needs or improve it if you want. Ple

10 Apr 15, 2019

Laravel Rocket chat Debugher, push all logs to rocket chat channels

4 Jun 13, 2022

An easy code to send messages on a discord text channel with webhook.

Status Webhook-Discord An easy code to send messages on a discord text channel with webhook. Don't forget to check the latest version of Webhook-Disco

1 Dec 3, 2021

DiscordLookup | Get more out of Discord with Discord Lookup! Snowflake Decoder, Guild List with Stats, Invite Info and more...

DiscordLookup Get more out of Discord with Discord Lookup! Snowflake Decoder, Guild List with Stats, Invite Info and more... Website Getting Help Tool

69 Dec 23, 2022

Allow any Discord user to sign in to your website and save their discord user information for later use.

Simple Discord SSO ( Single Sign-On ) Requires at least: 5.0 Tested up to: 5.8.3 Stable tag: 1.0.2 Requires PHP: 7.4 License: GPLv2 or later License U

2 Oct 7, 2022

Remove messages from the error log that are from a certain folder

Error-Focus A package to stay focused on relevant entries in your error-log Scope This package allows you to declare folders that are not under your d

7 Jul 25, 2022

A live chat which utilises Socket.io and validates messages sent against a MySQL Table.

Socket.io Shoutbox with MySQL Validation The purpouse of this demonstration is to allow users to enter messages in either a public or private chat. Th

9 Oct 14, 2022

One-to-one plugin for editing world chat messages.

WorldChat One-to-one plugin for editing world chat messages. Supports English and Turkish language To set a new world chat format /worldchat new "worl

12 May 20, 2023

Laravel Authentication Log is a package Log user authentication details and send new device notifications.

Laravel Authentication Log is a package which tracks your user's authentication information such as login/logout time, IP, Browser, Location, etc. as well as sends out notifications via mail, slack, or sms for new devices and failed logins.

540 Jan 5, 2023

A Laravel package to output a specific sql to your favourite debugging tool. The supported log output is Laravel Telescope, Laravel Log, Ray, Clockwork, Laravel Debugbar and your browser.

Laravel showsql A Laravel package to output a specific sql to your favourite debugging tool, your browser or your log file. Use case You often want to

196 Dec 28, 2022

Laravel User Activity Log - a package for Laravel 8.x that provides easy to use features to log the activities of the users of your Laravel app

9 Dec 14, 2022

PocketMine-MP To Discord Chat Logging.

Welcome To PocketCord PocketCord is a PocketMine-MP plugin designed to link discord and your PocketMine-MP Server with quick and easy setup! Installat

2 Sep 15, 2022

A PHP Chat based on LE CHAT

General Information: This is a PHP Chat based on LE CHAT v.1.14. An up-to-date copy of this script can be downloaded at https://github.com/DanWin/le-c

137 Dec 17, 2022

Online personal and group chat application using HTML, CSS, JS, PHP,SQL and AJAX that allows users to create own groups, chat

Online personal and group chat application using HTML, CSS, JS, PHP,SQL and AJAX that allows users to create own groups, chat, make friends, check new notifications and edit functionality. Chat automatically updates using AJAX to allow for real-time chatting with other users collected from the database.

2 Aug 5, 2022

Twill is an open source CMS toolkit for Laravel that helps developers rapidly create a custom admin console that is intuitive, powerful and flexible. /// Chat with us and others on Spectrum: https://spectrum.chat/twill

About Twill Twill is an open source Laravel package that helps developers rapidly create a custom CMS that is beautiful, powerful, and flexible. By st

3k Jan 6, 2023

Comments

Dealing with Rate Limits
Disclaimer: I don't understand this topic well so please keep this in mind :)

Discord has rate limits and ideally disco-the-ripper would account for them. I fear it's complicated but hopefully a simple solution could cover most related issues.

As mentioned in Issue #4, I ran into a rate limit issue. The error was similar to:

Client error: `GET https://discord.com/api/v9/channels/...&limit=50` resulted in a `429 Too Many Requests` response: { "global": false, "message": "The resource is being rate limited.", "retry_after": 0.142 }

Ideally disco-the-ripper would detect the error, evaluate retry_after, and then continue/retry accordingly. While fetchAllMessagesFromChannel() contains usleep(5000) it doesn't solve this issue but it might relate to a solution.

I don't know if retry_after increases over time (repeated requests) or if it's a discord server|user specific value. However, it appears X-RateLimit-* headers could be evaluated early and the code could act accordingly.
opened by philip 0
Save results in case of error

A throttling error stopped my fetching at x messages (~1000) and no channel messages were saved. While rate limits is a larger topic (a separate future request could address it), in this case the file saving mechanism could have saved results despite the exception.

Not sure if it should execute saveMessages() when an exception is detected (in my case it was from GuzzleHttp\Exception\ClientException) or instead save more frequently. I suspect saving more frequently is preferred as some channels are enormous so the buffer may grow too large but either (or a different) approach is fine.

opened by philip 0
Missing storage/app/channels/ directory

The saveMessages() method saves to storage/app/channels/ but the channels/ subdirectory does not exist despite it being listed in .gitignore. I expect it to either exist in the repo or be created by the installation process. Or, perhaps it's a simple documentation issue.

opened by philip 0