MISP - Threat Intelligence Sharing Platform

Overview

MISP - Threat Intelligence Sharing Platform

logo

Latest Release GitHub version
CI Action
Gitter
Twitter
Localization
Contributors
License

MISP is an open source software solution for collecting, storing, distributing and sharing cyber security indicators and threats about cyber security incidents analysis and malware analysis. MISP is designed by and for incident analysts, security and ICT professionals or malware reversers to support their day-to-day operations to share structured information efficiently.

The objective of MISP is to foster the sharing of structured information within the security community and abroad. MISP provides functionalities to support the exchange of information but also the consumption of said information by Network Intrusion Detection Systems (NIDS), LIDS but also log analysis tools, SIEMs.

MISP, Malware Information Sharing Platform and Threat Sharing, core functionalities are:

  • An efficient IOC and indicators database allowing to store technical and non-technical information about malware samples, incidents, attackers and intelligence.
  • Automatic correlation finding relationships between attributes and indicators from malware, attack campaigns or analysis. The correlation engine includes correlation between attributes and more advanced correlations like Fuzzy hashing correlation (e.g. ssdeep) or CIDR block matching. Correlation can also be enabled or event disabled per attribute.
  • A flexible data model where complex objects can be expressed and linked together to express threat intelligence, incidents or connected elements.
  • Built-in sharing functionality to ease data sharing using different model of distributions. MISP can automatically synchronize events and attributes among different MISP instances. Advanced filtering functionalities can be used to meet each organization's sharing policy including a flexible sharing group capacity and an attribute level distribution mechanisms.
  • An intuitive user-interface for end-users to create, update and collaborate on events and attributes/indicators. A graphical interface to navigate seamlessly between events and their correlations. An event graph functionality to create and view relationships between objects and attributes. Advanced filtering functionalities and warning lists to help the analysts to contribute events and attributes and limit the risk of false-positives.
  • storing data in a structured format (allowing automated use of the database for various purposes) with an extensive support of cyber security indicators along fraud indicators as in the financial sector.
  • export: generating IDS, OpenIOC, plain text, CSV, MISP XML or JSON output to integrate with other systems (network IDS, host IDS, custom tools), Cache format (used for forensic tools), STIX (XML and JSON) 1 and 2, NIDS export (Suricata, Snort and Bro/Zeek) or RPZ zone. Many other formats can be easily added via the misp-modules.
  • import: bulk-import, batch-import, import from OpenIOC, GFI sandbox, ThreatConnect CSV, MISP standard format or STIX 1.1/2.0. Many other formats easily added via the misp-modules.
  • Flexible free text import tool to ease the integration of unstructured reports into MISP.
  • A gentle system to collaborate on events and attributes allowing MISP users to propose changes or updates to attributes/indicators.
  • data-sharing: automatically exchange and synchronize with other parties and trust-groups using MISP.
  • delegating of sharing: allows a simple pseudo-anonymous mechanism to delegate publication of event/indicators to another organization.
  • Flexible API to integrate MISP with your own solutions. MISP is bundled with PyMISP which is a flexible Python Library to fetch, add or update events attributes, handle malware samples or search for attributes. An exhaustive restSearch API to easily search for indicators in MISP and exports those in all the format supported by MISP.
  • Adjustable taxonomy to classify and tag events following your own classification schemes or existing classification. The taxonomy can be local to your MISP but also shareable among MISP instances.
  • Intelligence vocabularies called MISP galaxy and bundled with existing threat actors, malware, RAT, ransomware or MITRE ATT&CK which can be easily linked with events and attributes in MISP.
  • Expansion modules in Python to expand MISP with your own services or activate already available misp-modules.
  • Sighting support to get observations from organizations concerning shared indicators and attributes. Sighting can be contributed via MISP user-interface, API as MISP document or STIX sighting documents.
  • STIX support: import and export data in the STIX version 1 and version 2 format.
  • Integrated encryption and signing of the notifications via GnuPG and/or S/MIME depending on the user's preferences.
  • Real-time publish-subscribe channel within MISP to automatically get all changes (e.g. new events, indicators, sightings or tagging) in ZMQ (e.g. misp-dashboard) or Kafka publishing.

Exchanging info results in faster detection of targeted attacks and improves the detection ratio while reducing the false positives. We also avoid reversing similar malware as we know very fast that other teams or organizations have already analyzed a specific malware.

MISP 2.4 overview

A sample event encoded in MISP:

MISP event view

Website / Support

Checkout the website for more information about MISP software, standards, tools and communities.

Information, news and updates are also regularly posted on the MISP project twitter account or the news page.

Documentation

MISP user-guide (MISP-book) is available online or as PDF or as EPUB or as MOBI/Kindle.

For the installation guide see the INSTALL or download section.

Contributing

If you are interested to contribute to the MISP project, review our contributing page. There are many ways to contribute and participate to the project.

Please see our Code of conduct.

Feel free to fork the code, play with it, make some patches and send us the pull requests via the issues.

Feel free to contact us, create issues, if you have questions, remarks or bug reports.

There is one main branch:

  • 2.4 (current stable version): what we consider as stable with frequent updates as hot-fixes.

and features are developed in separated branches and then regularly merged into the 2.4 stable branch.

License

This software is licensed under GNU Affero General Public License version 3

  • Copyright (C) 2012-2022 Christophe Vandeplas
  • Copyright (C) 2012 Belgian Defence
  • Copyright (C) 2012 NATO / NCIRC
  • Copyright (C) 2013-2022 Andras Iklody
  • Copyright (C) 2015-2022 CIRCL - Computer Incident Response Center Luxembourg
  • Copyright (C) 2016 Andreas Ziegler
  • Copyright (C) 2018-2022 Sami Mokaddem
  • Copyright (C) 2018-2022 Christian Studer
  • Copyright (C) 2015-2022 Alexandre Dulaunoy
  • Copyright (C) 2018-2022 Steve Clement
  • Copyright (C) 2020-2022 Jakub Onderka

For more information, the list of authors and contributors is available.

Comments
  • STIX or CyBox library not installed correctly

    STIX or CyBox library not installed correctly

    | Type of issue | Support | OS version (server) | Ubuntu 1404 | PHP version | 5.4, 5.5, 5.6, 7.0, 7.1... | MISP version / git hash | 2.4.53 | Browser | If applicable

    Hi,

    I have an issue with Cybox and STIX, this is the message what I see, such as:

    _Mitre's STIX and Cybox python libraries have to be installed in order for MISP's STIX export to work. Make sure that you install them (as described in the MISP installation instructions) if you receive an error below. If you run into any issues here, make sure that both STIX and CyBox are installed as described in the INSTALL.txt file. The required versions are: STIX: CyBox: Other versions might work but are not tested / recommended.

    STIX and Cybox libraries....STIX or CyBox library not installed correctly_

    This is my version Cybox and Stix STIX: 1.1.1.4 CyBox: 2.1.0.12

    So I was installing it and I wasn't anything issue.

    Can you help me, please? Let me know if you need more info.

    Regards!

    T: support 
    opened by santi10 106
  • Password Issue

    Password Issue

    I have a user trying to reset his password, my settings are default (12 characters and /((?=.\d)|(?=.\W+))(?![\n])(?=.[A-Z])(?=.[a-z]).*$/).

    He is trying to set this and it doesn't work saying not matching complexity requirements: mSW%vuc@ot85>?

    and

    aA9!$!@#$@#aeuc<.a

    Related, I can't set a specific password for users as admin because it asks me to enter existing password. As admin I should be able to specify a password, right?

    If you would like to report a bug, please fill the template bellow

    Work environment

    | Questions | Answers |---------------------------|-------------------- | Type of issue | Bug | OS version (server) | ubuntu | OS version (client) | XP, Seven, 10, Ubuntu, ... | PHP version | 5.4, 5.5, 5.6, 7.0, 7.1... | MISP version / git hash | 2.4.78 | Browser | If applicable

    Expected behavior

    Actual behavior

    Steps to reproduce the behavior

    Logs, screenshots, configuration dump, ...

    T: bug 
    opened by bambenek 44
  • Error: STIX or CyBox or mixbox library not installed correctly New Ubuntu 16 Install

    Error: STIX or CyBox or mixbox library not installed correctly New Ubuntu 16 Install

    This template is meant for bug reports, if you have a feature request, please be as descriptive as possible and delete the template

    If you would like to report a bug, please fill the template bellow

    Work environment

    | Questions | Answers |---------------------------|-------------------- | Type of issue support... | OS version (server) ubuntu | OS version (client) Mac OS -X | PHP version 7.0.18 | MISP version / git hash | 2.4.77 | Browser Chrome

    Expected behavior

    Stix, Cybox, Mixbox report ok in sever diagnostics

    Actual behavior

    Mitre's STIX and Cybox python libraries have to be installed in order for MISP's STIX export to work. Make sure that you install them (as described in the MISP installation instructions) if you receive an error below. If you run into any issues here, make sure that both STIX and CyBox are installed as described in the INSTALL.txt file. The required versions are: STIX: 1.1.1.4 CyBox: 2.1.0.12 mixbox: 1.0.2 Other versions might work but are not tested / recommended.

    STIX and Cybox libraries....STIX or CyBox or mixbox library not installed correctly

    Steps to reproduce the behavior

    I have logged into the web portal, and click Administration, Server Settings Diagnostics

    Logs, screenshots, configuration dump, ...

    /var/www/MISP/app/tmp/logs# more exec-errors.log Traceback (most recent call last): File "/var/www/MISP/app/files/scripts/misp2stix_framing.py", line 2, in from misp2cybox import * File "/var/www/MISP/app/files/scripts/misp2cybox.py", line 1, in from cybox.core import Object, Observable, ObservableComposition ImportError: No module named cybox.core

    T: support 
    opened by krypto29s 44
  • Synchronisation issues

    Synchronisation issues

    I'm testing the synchronisation again with two other remote instances, one running 2.4.5 and the other one running the latest commit (2.4.17). I've created an event with distribution "Connected communities" and clicked on publish and it was properly pushed to the instance running 2.4.5 but not to the one running 2.4.17. In my logs I see the following:

    $ tail -f resque-worker-error.log resque-2016-02-12.log resque-scheduler-2016-02-05.log error.log debug.log error.log -n 20 | grep '2016-02-12 10' 2016-02-12 10:02:33 Warning: Invalid argument supplied for foreach() in [/var/www/MISP/app/Model/Event.php, line 678] 2016-02-12 10:02:33 Warning: Invalid argument supplied for foreach() in [/var/www/MISP/app/Model/Event.php, line 678] 2016-02-12 10:02:33 Warning: Invalid argument supplied for foreach() in [/var/www/MISP/app/Model/Event.php, line 678] 2016-02-12 10:02:33 Notice: Undefined variable: remoteId in [/var/www/MISP/app/Model/Event.php, line 685] [2016-02-12 10:02:33] main.INFO: got {"queue":"default","id":"7b18a9c56f2ecfaf0c36adb92185c3cf","class":"EventShell","args":[["publish","2133",null,"5522","1"]]} {"type":"got","args":"[object](Resque_Job: {"queue":"default","id":"7b18a9c56f2ecfaf0c36adb92185c3cf","class":"EventShell","args":[["publish","2133",null,"5522","1"]]})","worker":"misp.my.org:1593"} [] [2016-02-12 10:02:33] main.INFO: Processing ID:7b18a9c56f2ecfaf0c36adb92185c3cf in default {"type":"process","worker":"misp.my.org:1593","job_id":"7b18a9c56f2ecfaf0c36adb92185c3cf"} [] [2016-02-12 10:02:36] main.INFO: got {"queue":"email","id":"5e4e4327c1978bd9130e042a69022fd1","class":"EventShell","args":[["alertemail","1","5521","2133"]]} {"type":"got","args":"[object](Resque_Job: {"queue":"email","id":"5e4e4327c1978bd9130e042a69022fd1","class":"EventShell","args":[["alertemail","1","5521","2133"]]})","worker":"misp.my.org:1633"} [] [2016-02-12 10:02:36] main.INFO: Processing ID:5e4e4327c1978bd9130e042a69022fd1 in email {"type":"process","worker":"misp.my.org:1633","job_id":"5e4e4327c1978bd9130e042a69022fd1"} [] [2016-02-12 10:02:36] main.INFO: done ID:7b18a9c56f2ecfaf0c36adb92185c3cf {"type":"done","job_id":"7b18a9c56f2ecfaf0c36adb92185c3cf","time":3119,"worker":"misp.my.org:1593"} [] [2016-02-12 10:02:36] main.INFO: done ID:5e4e4327c1978bd9130e042a69022fd1 {"type":"done","job_id":"5e4e4327c1978bd9130e042a69022fd1","time":129,"worker":"misp.my.org:1633"} [] 2016-02-12 10:02:33 Warning: Invalid argument supplied for foreach() in [/var/www/MISP/app/Model/Event.php, line 678] 2016-02-12 10:02:33 Warning: Invalid argument supplied for foreach() in [/var/www/MISP/app/Model/Event.php, line 678] 2016-02-12 10:02:33 Warning: Invalid argument supplied for foreach() in [/var/www/MISP/app/Model/Event.php, line 678] 2016-02-12 10:02:33 Notice: Undefined variable: remoteId in [/var/www/MISP/app/Model/Event.php, line 685] 2016-02-12 10:02:33 Warning: Invalid argument supplied for foreach() in [/var/www/MISP/app/Model/Event.php, line 678] 2016-02-12 10:02:33 Warning: Invalid argument supplied for foreach() in [/var/www/MISP/app/Model/Event.php, line 678] 2016-02-12 10:02:33 Warning: Invalid argument supplied for foreach() in [/var/www/MISP/app/Model/Event.php, line 678]

    I've also created another event using a sharing group involving the remote instance running 2.4.17 and published it. I see no errors in my logs but I don't see the event on the remote instance either.

    $ tail -f resque-worker-error.log resque-2016-02-12.log resque-scheduler-2016-02-05.log error.log debug.log error.log -n 20 | grep '2016-02-12 10:15' [2016-02-12 10:15:06] main.INFO: got {"queue":"default","id":"3ccde8d1c0c2517d97799f6aa28002c0","class":"EventShell","args":[["publish","2134",null,"5524","1"]]} {"type":"got","args":"[object](Resque_Job: {"queue":"default","id":"3ccde8d1c0c2517d97799f6aa28002c0","class":"EventShell","args":[["publish","2134",null,"5524","1"]]})","worker":"misp.my.org:1593"} [] [2016-02-12 10:15:06] main.INFO: got {"queue":"email","id":"8085f0eab777c0810cce62b8c8778284","class":"EventShell","args":[["alertemail","1","5523","2134"]]} {"type":"got","args":"[object](Resque_Job: {"queue":"email","id":"8085f0eab777c0810cce62b8c8778284","class":"EventShell","args":[["alertemail","1","5523","2134"]]})","worker":"misp.my.org:1633"} [] [2016-02-12 10:15:06] main.INFO: Processing ID:3ccde8d1c0c2517d97799f6aa28002c0 in default {"type":"process","worker":"misp.my.org:1593","job_id":"3ccde8d1c0c2517d97799f6aa28002c0"} [] [2016-02-12 10:15:06] main.INFO: Processing ID:8085f0eab777c0810cce62b8c8778284 in email {"type":"process","worker":"misp.my.org:1633","job_id":"8085f0eab777c0810cce62b8c8778284"} [] [2016-02-12 10:15:06] main.INFO: done ID:8085f0eab777c0810cce62b8c8778284 {"type":"done","job_id":"8085f0eab777c0810cce62b8c8778284","time":132,"worker":"misp.my.org:1633"} [] [2016-02-12 10:15:08] main.INFO: done ID:3ccde8d1c0c2517d97799f6aa28002c0 {"type":"done","job_id":"3ccde8d1c0c2517d97799f6aa28002c0","time":2389,"worker":"misp.my.org:1593"} []

    On the side of the instance running 2.4.17, the following can be seen in the logs:

    2016-02-12 10:14:43 Error: [MethodNotAllowedException] Invalid Sharing Group or not authorised. (Sync user is not contained in the Sharing group) Request URL: /events Stack Trace: #0 [internal function]: EventsController->add() #1 /var/www/MISP/app/Lib/cakephp/lib/Cake/Controller/Controller.php(490): ReflectionMethod->invokeArgs(Object(EventsController), Array) #2 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(185): Controller->invokeAction(Object(CakeRequest)) #3 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(160): Dispatcher->_invoke(Object(EventsController), Object(CakeRequest), Object(CakeResponse)) #4 /var/www/MISP/app/webroot/index.php(92): Dispatcher->dispatch(Object(CakeRequest), Object(CakeResponse)) #5 {main}

    2016-02-12 10:02:10 Notice: Notice (8): Undefined index: distribution in [/var/www/MISP/app/Model/Event.php, line 1702] Trace: Event::__captureObjects() - APP/Model/Event.php, line 1702 Event::_add() - APP/Model/Event.php, line 1807 EventsController::add() - APP/Controller/EventsController.php, line 893 ReflectionMethod::invokeArgs() - [internal], line ?? Controller::invokeAction() - APP/Lib/cakephp/lib/Cake/Controller/Controller.php, line 490 Dispatcher::_invoke() - APP/Lib/cakephp/lib/Cake/Routing/Dispatcher.php, line 185 Dispatcher::dispatch() - APP/Lib/cakephp/lib/Cake/Routing/Dispatcher.php, line 160 [main] - APP/webroot/index.php, line 92

    2016-02-12 10:02:10 Notice: Notice (8): Undefined index: distribution in [/var/www/MISP/app/Model/Event.php, line 1713] Trace: Event::__captureObjects() - APP/Model/Event.php, line 1713 Event::_add() - APP/Model/Event.php, line 1807 EventsController::add() - APP/Controller/EventsController.php, line 893 ReflectionMethod::invokeArgs() - [internal], line ?? Controller::invokeAction() - APP/Lib/cakephp/lib/Cake/Controller/Controller.php, line 490 Dispatcher::_invoke() - APP/Lib/cakephp/lib/Cake/Routing/Dispatcher.php, line 185 Dispatcher::dispatch() - APP/Lib/cakephp/lib/Cake/Routing/Dispatcher.php, line 160 [main] - APP/webroot/index.php, line 92

    On all instances I'm synchronising with, the organisation name and UUID is the same and match the ones defines on my own instance. Funny enough, those two remote instances are getting events from my instance with distribution "All communities". My instance is running 2.4.17 with the latest commit (f81960f)

    T: support T: potential bug WaitingAnswer 
    opened by h122015 43
  • Support: New Correlations Engine first recorrelate failed

    Support: New Correlations Engine first recorrelate failed

    Support Questions

    We installed 2.4.160 yesterday. After approx. 6 hours the job was seen as failed with WebUI Jobs. Prior to the failure we were seeing default feed ingestions failing with db deadlock (see sample excerpt from error.log below). Similar HTTP 500 were seen in some ron jobs custom PyMISP ingestion scripts.

    1. How can we share more information with you to try to isolate root cause?
    2. We've started a Recorrelate job for the active (new) engine via the WebUI and will monitor, and report back in this issue with results.
    3. The failure was seen in the WebUI jobs; however, the corresponding row in the db jobs table still had a status: 0. Is there a mysql query that will reveal the failure in the jobs table? THis is a much easier way to find the failure vs. traversing the WebUI jobs.

    MISP version

    2.4.160

    Operating System

    RedHat

    Operating System version

    8.4

    PHP version

    7.4.28

    Browser

    No response

    Browser version

    No response

    Relevant log output

    120030 [PDOException] SQLSTATE[40001]: Serialization failure: 1213 Deadlock fou\
    nd when trying to get lock; try restarting transaction
    120031 Stack Trace:
    120032 #0 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Datasource/DboSource.php\
    (502): PDOStatement->execute()
    120033 #1 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Datasource/DboSource.php\
    (468): DboSource->_execute()
    120034 #2 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Datasource/Database/Mysq\
    l.php(459): DboSource->execute()
    120035 #3 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Model.php(2848): Mysql->\
    delete()
    120036 #4 /var/www/MISP/app/Model/Behavior/DefaultCorrelationBehavior.php(173):\
     Model->deleteAll()
    120037 #5 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/BehaviorCollection.php(2\
    38): DefaultCorrelationBehavior->runBeforeSaveCorrelation()
    120038 #6 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Model.php(829): Behavior\
    Collection->dispatchMethod()
    120039 #7 /var/www/MISP/app/Model/Correlation.php(261): Model->__call()
    120040 #8 /var/www/MISP/app/Model/Attribute.php(418): Correlation->beforeSaveCo\
    rrelation()
    120041 #9 /var/www/MISP/app/Lib/Tools/BetterCakeEventManager.php(21): Attribute\
    ->afterSave()
    120042 #10 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Model.php(1970): Better\
    CakeEventManager->dispatch()
    120043 #11 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Model.php(1754): Model-\
    >_doSave()
    120044 #12 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Model.php(2352): Model-\
    >save()
    120045 #13 /var/www/MISP/app/Model/Feed.php(1303): Model->saveMany()
    120046 #14 /var/www/MISP/app/Model/Feed.php(1189): Feed->saveFreetextFeedData()
    120047 #15 /var/www/MISP/app/Console/Command/ServerShell.php(249): Feed->downlo\
    adFromFeedInitiator()
    120048 #16 /var/www/MISP/app/Lib/cakephp/lib/Cake/Console/Shell.php(459): Serve\
    rShell->fetchFeed()
    120049 #17 /var/www/MISP/app/Lib/cakephp/lib/Cake/Console/ShellDispatcher.php(2\
    22): Shell->runCommand()
    120050 #18 /var/www/MISP/app/Lib/cakephp/lib/Cake/Console/ShellDispatcher.php(6\
    6): ShellDispatcher->dispatch()
    120051 #19 /var/www/MISP/app/Console/cake.php(45): ShellDispatcher::run()
    120052 #20 {main}
    

    Extra attachments

    Here's the jobs row that started but then was reported failed in the WebUI Jobs page:

    mysql> select * from jobs where job_type like '%correlation%' order by id asc limit 1 \G
    *************************** 1. row ***************************
               id: 1358491
           worker: default
         job_type: generate correlation
        job_input: All attributes
           status: 0
          retries: 0
          message: Correlating Event 193178 (20 MB used)
         progress: 1
           org_id: 0
       process_id: 27778066884972c4d8decefbc31134b6
     date_created: 2022-08-08 20:28:40
    date_modified: 2022-08-09 02:30:51
    1 row in set (0.02 sec)
    
    mysql>
    

    Code of Conduct

    • [X] I agree to follow this project's Code of Conduct
    needs triage support 
    opened by github-germ 40
  • Support: Why does Event

    Support: Why does Event "Populate from..." trigger CSRF black-holed?

    Support Questions

    When adding a new event and then attempting Populate from... with a freetext list of IPs, the browser responds with

    You have tripped the cross-site request forgery protection of MISP
    

    See error.log this triggers below:

    How can we make this work?

    Thanks!

    MISP version

    2.4.153

    Operating System

    RedHat

    Operating System version

    8.4

    PHP version

    7.4

    Browser

    Chrome

    Browser version

    No response

    Relevant log output

    2022-03-11 23:29:50 Error: [BadRequestException] The request has been black-holed
    Request URL: /events/freeTextImport/292379?_=1647041377363
    Stack Trace:
    #0 /var/www/MISP/app/Lib/cakephp/lib/Cake/Controller/Component/SecurityComponent.php(831): AppController->blackhole()
    #1 /var/www/MISP/app/Lib/cakephp/lib/Cake/Controller/Component/SecurityComponent.php(351): SecurityComponent->_callback()
    #2 /var/www/MISP/app/Lib/cakephp/lib/Cake/Controller/Component/SecurityComponent.php(255): SecurityComponent->blackHole()
    #3 /var/www/MISP/app/Lib/cakephp/lib/Cake/Utility/ObjectCollection.php(129): SecurityComponent->startup()
    #4 /var/www/MISP/app/Lib/cakephp/lib/Cake/Event/CakeEventManager.php(244): ObjectCollection->trigger()
    #5 /var/www/MISP/app/Lib/cakephp/lib/Cake/Controller/Controller.php(683): CakeEventManager->dispatch()
    #6 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(189): Controller->startupProcess()
    #7 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(167): Dispatcher->_invoke()
    #8 /var/www/MISP/app/webroot/index.php(99): Dispatcher->dispatch()
    #9 {main}
    

    Extra attachments

    No response

    Code of Conduct

    • [X] I agree to follow this project's Code of Conduct
    needs triage support 
    opened by github-germ 39
  • Decaying Models

    Decaying Models

    Should we merge #5002 first?

    To Do

    • Improved support of Sightings
      • False positive Sightings should somehow reduce the score
      • Expiration Sightings should mark the attribute as decayed
    • Potential model improvements
      • Instead of resetting the score to base_score once a Sighting is set, the score should be increased additively (based on a defined coefficient); thus prioritizing surges rather than infrequent Sightings
      • Take into account related tags when computing score
    • Increase Taxonomy coverage
      • Users should be able to manually override the numerical_value of tags
    • For specific type, take into account data from other services
      • Could fetch data from BGP ranking for IP/domain/... and based on a weight influence the score

    What does it do?

    Integrate various aspects of decaying models into MISP:

    • Decay score on Attribute based on models
    • Default (from a GitHub repository) Decaying models or per Organization
    • Possibility to share models (import/export) or locally on the instance
    • Tool to fine-tune models and assign them to attribute types
    • Simulation tool presenting how the score for attributes would evolve over time and how their base_score is computed

    Fixes:

    • #4429
    • #1310
    • #445

    Questions

    • [X] Does it require a DB change?
    • [ ] Are you using it in production?
    • [X] Does it require a change in the API (PyMISP for example)?

    Release Type:

    • [ ] Major
    • [X] Minor
    • [ ] Patch
    functionality topic: IDS topic: taxonomies topic: sighting topic: expiration S: not reviewed 
    opened by mokaddem 39
  • PyMISP.add_hashes() Error: [PDOException] SQLSTATE[40001]: Serialization failure: 1213 Deadlock

    PyMISP.add_hashes() Error: [PDOException] SQLSTATE[40001]: Serialization failure: 1213 Deadlock

    2.4.90 mariaDB 5.5.56

    has this been seen before? any idea on how to resolve is appreciated.

    this occurs in error.log when calling PyMISP.add_hashes()

    2018-05-21 03:06:50 Error: [PDOException] SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction
    Request URL: /attributes/add/11404
    Stack Trace:
    #0 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Datasource/DboSource.php(472): PDOStatement->execute(Array)
    #1 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Datasource/DboSource.php(438): DboSource->_execute('UPDATE `misp`.`...', Array)
    #2 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Datasource/Database/Mysql.php(424): DboSource->execute('UPDATE `misp`.`...')
    #3 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Model.php(1926): Mysql->update(Object(Event), '`id` = 11404, `...', Array)
    #4 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Model.php(1760): Model->_doSave(Array, Array)
    #5 /var/www/MISP/app/Model/Attribute.php(570): Model->save(Array, Array)
    #6 /var/www/MISP/app/Model/Attribute.php(610): Attribute->__alterAttributeCount('11404')
    #7 /var/www/MISP/app/Lib/cakephp/lib/Cake/Event/CakeEventManager.php(241): Attribute->afterSave(true, Array)
    #8 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Model.php(1970): CakeEventManager->dispatch(Object(CakeEvent))
    #9 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Model.php(1754): Model->_doSave(Array, Array)
    #10 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Model.php(2352): Model->save(Array, Array)
    #11 /var/www/MISP/app/Controller/AttributesController.php(276): Model->saveMany(Array)
    #12 [internal function]: AttributesController->add('11404')
    #13 /var/www/MISP/app/Lib/cakephp/lib/Cake/Controller/Controller.php(491): ReflectionMethod->invokeArgs(Object(AttributesController), Array)
    #14 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(193): Controller->invokeAction(Object(CakeRequest))
    #15 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(167): Dispatcher->_invoke(Object(AttributesController), Object(CakeRequest))
    #16 /var/www/MISP/app/webroot/index.php(92): Dispatcher->dispatch(Object(CakeRequest), Object(CakeResponse))
    #17 {main}
    
    T: support topic: PyMISP 
    opened by github-germ 38
  • Adding Misp Module

    Adding Misp Module

    I have installed MISP in CentOS Linux release 7.3.1611 (Core) Everything works fine

    I want to enable some modules

    when I try to use enrichment http://127.0.0.1:6666 nothing is loading

    I have configured in the server setting

    Priority Setting Value Description Error Message Critical Plugin.Enrichment_services_enable true Enable/disable the enrichment services Critical Plugin.Enrichment_hover_enable true Enable/disable the hover over information retrieved from the enrichment modules Recommended Plugin.Enrichment_timeout 10 Set a timeout for the enrichment services Recommended Plugin.Enrichment_hover_timeout 5 Set a timeout for the hover services Recommended Plugin.Enrichment_services_url http://127.0.0.1 The url used to access the enrichment services. By default, it is accessible at http://127.0.0.1:6666 Recommended Plugin.Enrichment_services_port 6666 The port used to access the enrichment services. By default, it is accessible at 127.0.0.1:6666

    Is there anything else I need to configure ??? My python version is

    [root@localhost ~]# python -V Python 2.7.5

    T: support 
    opened by abdulshemeer166 38
  • Getting

    Getting "An Internal Error Has Occurred." when trying to reach the MISP Web UI.

    Work environment

    | Questions | Answers |---------------------------|-------------------- | Type of issue | Support | OS version (server) | Ubuntu | OS version (client) | 18.04 | PHP version | 7.2 | MISP version / git hash | 2.4.100 | Browser | Chrome

    Expected behavior

    A functioning webserver

    Actual behavior

    Webpage loads, with "An Internal Error Has Occurred." on the page. Nothing else works

    Steps to reproduce the behavior

    Apache with certificate works, no errors there.

    Logs, screenshots, configuration dump, ...

    The error.log file in /var/www/MISP/app/tmp/logs isn't very descriptive. It gives the following error: 2019-01-28 15:47:58 Error: [ParseError] syntax error, unexpected '' (T_NS_SEPARATOR), expecting function (T_FUNCTION) or const (T_CONST) Request URL: / Stack Trace: #0 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/ConnectionManager.php(83): ConnectionManager::_init() #1 /var/www/MISP/app/Controller/AppController.php(110): ConnectionManager::getDataSource('default') #2 /var/www/MISP/app/Controller/EventsController.php(34): AppController->beforeFilter() #3 /var/www/MISP/app/Lib/cakephp/lib/Cake/Event/CakeEventManager.php(243): EventsController->beforeFilter(Object(CakeEvent)) #4 /var/www/MISP/app/Lib/cakephp/lib/Cake/Controller/Controller.php(677): CakeEventManager->dispatch(Object(CakeEvent)) #5 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(189): Controller->startupProcess() #6 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(167): Dispatcher->_invoke(Object(EventsController), Object(CakeRequest)) #7 /var/www/MISP/app/webroot/index.php(92): Dispatcher->dispatch(Object(CakeRequest), Object(CakeResponse)) #8 {main}

    Looking at the debug.log file, a lot of "undefined variable" errors appear: 2019-01-28 16:04:44 Notice: Notice (8): Undefined variable: me in [/var/www/MISP/app/View/Layouts/default.ctp, line 109] Trace: ErrorHandler::handleError() - APP/Lib/cakephp/lib/Cake/Error/ErrorHandler.php, line 230 include - APP/View/Layouts/default.ctp, line 109 View::_evaluate() - APP/Lib/cakephp/lib/Cake/View/View.php, line 971 View::_render() - APP/Lib/cakephp/lib/Cake/View/View.php, line 933 View::renderLayout() - APP/Lib/cakephp/lib/Cake/View/View.php, line 546 View::render() - APP/Lib/cakephp/lib/Cake/View/View.php, line 481 Controller::render() - APP/Lib/cakephp/lib/Cake/Controller/Controller.php, line 963 ExceptionRenderer::_outputMessage() - APP/Lib/cakephp/lib/Cake/Error/ExceptionRenderer.php, line 292 ExceptionRenderer::error500() - APP/Lib/cakephp/lib/Cake/Error/ExceptionRenderer.php, line 260 ExceptionRenderer::render() - APP/Lib/cakephp/lib/Cake/Error/ExceptionRenderer.php, line 190 ErrorHandler::handleException() - APP/Lib/cakephp/lib/Cake/Error/ErrorHandler.php, line 127 [main] - [internal], line ??

    2019-01-28 16:04:44 Notice: Notice (8): Undefined variable: baseurl in [/var/www/MISP/app/View/Layouts/default.ctp, line 100] Trace: ErrorHandler::handleError() - APP/Lib/cakephp/lib/Cake/Error/ErrorHandler.php, line 230 include - APP/View/Layouts/default.ctp, line 100 View::_evaluate() - APP/Lib/cakephp/lib/Cake/View/View.php, line 971 View::_render() - APP/Lib/cakephp/lib/Cake/View/View.php, line 933 View::renderLayout() - APP/Lib/cakephp/lib/Cake/View/View.php, line 546 View::render() - APP/Lib/cakephp/lib/Cake/View/View.php, line 481 Controller::render() - APP/Lib/cakephp/lib/Cake/Controller/Controller.php, line 963 ExceptionRenderer::_outputMessage() - APP/Lib/cakephp/lib/Cake/Error/ExceptionRenderer.php, line 292 ExceptionRenderer::error500() - APP/Lib/cakephp/lib/Cake/Error/ExceptionRenderer.php, line 260 ExceptionRenderer::render() - APP/Lib/cakephp/lib/Cake/Error/ExceptionRenderer.php, line 190 ErrorHandler::handleException() - APP/Lib/cakephp/lib/Cake/Error/ErrorHandler.php, line 127 [main] - [internal], line ??

    T: support 
    opened by mik5z 36
  • HTTP 503 MISP to PyMISP client

    HTTP 503 MISP to PyMISP client

    In https://github.com/MISP/MISP/issues/3293#issuecomment-393712251 I observed many HTTP 503 error codes from the MISP server in responses back to requests from a multi-threaded PyMISP based client.

    Client performs ingestion of a few thousand events, some with many attributes. The 503s happen most often with PyMISP.add_hashes() calls, but have also been seen with a thread's attempt to instantiate pymisp.PyMISP().

    For example, for one ingestion of about 1000 events, about 500 add_hashes() triggered a 503.

    The client runs on the same server as MISP. This is an IBM 3650 with 32 cpu cores, and 64GB RAM.

    Questions/Brainstorming

    1. Is it correct to assume that the request made with a 503 reply did not transact?

    2. Is there a uniform was to handle non 200 HTTP responses in PyMISP clients?

    3. Is there a limit to the number of active MISP client connections that might be causing the 503s?

    4. Each new thread instantiates a PyMISP. Is there any call that will drop the connection before the thread completes and exits to perhaps reduce the number of active connections?

    5. I've not seen an errors in the error.log nor entries in debug.log when a 503 occurs. Is there another way to monitor this condition?

    6. Might there be some tuning in MISP, the PyMISP client, or perhaps in Apache or the OS, that would help?

    WaitingAnswer 
    opened by github-germ 36
  • Support: No Enrichment Options In MISP

    Support: No Enrichment Options In MISP

    Support Questions

    I am very new to MISP and I want to enrich the events. I have attached my Plug-ins-> enrichment section below. I have tried to run misp-modules.service but still nothing is showing, although I can see all the enrichment files in my /usr/local/src/misp-modules/misp-modules folderand misp-modules.service is in /etc/systemd/system folder of ubutu. Regards

    MISP version

    2.4.165

    Operating System

    Ubuntu

    Operating System version

    22.04

    PHP version

    7.0

    Browser

    FireFox

    Browser version

    No response

    Relevant log output

    No response

    Extra attachments

    02 01

    Code of Conduct

    • [X] I agree to follow this project's Code of Conduct
    needs triage support 
    opened by MU-03 0
  • AAD Plugin - Nested Groups

    AAD Plugin - Nested Groups

    Would it be possible to update the query to the following in order to allow for nested groups. As far as I can see, the output is the same, so no others queries should be needed.

    https://learn.microsoft.com/en-us/graph/api/user-list-transitivememberof?view=graph-rest-1.0&tabs=http

    https://github.com/MISP/MISP/blob/faa1fc5300fc8e1e6e98c72112801625e583a3fb/app/Plugin/AadAuth/Controller/Component/Auth/AadAuthenticateAuthenticate.php#L308

    opened by ThijsLecomte-TC 0
  • Support: Turn off Event Blocklists and Remove the ones currently in there?

    Support: Turn off Event Blocklists and Remove the ones currently in there?

    Support Questions

    For space limitations I have a script that goes through and deletes any indicator over 30 days...I guess everytime it runs it creates another 'event blocklist'? Hah, well now I have a TON of event blocklists.

    Is there a way to delete all the blocklists? Is there a way to turn this off?

    MISP version

    2.4.167

    Operating System

    Ubuntu

    Operating System version

    20.04

    PHP version

    7.4

    Browser

    Firefox

    Browser version

    108.0.1

    Relevant log output

    2023-01-04 11:01:08 Warning: Could not add event '54f8688d-9444-4033-82fb-3c19950d210b' from feed 1: Blocked by blocklist
    2023-01-04 11:01:09 Warning: Could not add event '54f9a0ef-0ebc-414d-88ab-f094950d210b' from feed 1: Blocked by blocklist
    2023-01-04 11:01:09 Warning: Could not add event '5500579e-e1b4-43fe-b7c5-73da950d210b' from feed 1: Blocked by blocklist
    2023-01-04 11:01:10 Warning: Could not add event '55014406-fd90-4fc1-a814-4638950d210b' from feed 1: Blocked by blocklist
    2023-01-04 11:01:11 Warning: Could not add event '551427fe-47ac-4247-93f0-c906950d210b' from feed 1: Blocked by blocklist
    

    Extra attachments

    image

    Code of Conduct

    • [X] I agree to follow this project's Code of Conduct
    needs triage support 
    opened by mathurin68 0
  • Bug: MISP doesn't build on Ubuntu Jammy do-release-upgrade from 16.04

    Bug: MISP doesn't build on Ubuntu Jammy do-release-upgrade from 16.04

    Expected behavior

    Expect the Ubuntu 20.04 script to work. Tried both upgrading and building and neither worked.

    Actual behavior

    It freezes at:

        Uninstalling pymisp-2.4.167:
          Successfully uninstalled pymisp-2.4.167
    Successfully installed pymisp-2.4.167
    Reading package lists...
    Building dependency tree...
    Reading state information...
    libcaca-dev is already the newest version (0.99.beta19-2.2ubuntu4).
    liblua5.3-dev is already the newest version (5.3.6-1build1).
    cmake is already the newest version (3.22.1-1ubuntu1.22.04.1).
    0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
    

    And the process list shows:

    www-data    9105    9104  0 13:23 ?        00:00:00 php ./bin/resque
    www-data    9123       1  0 13:23 ?        00:00:00 bash -c cd '/apps/var/www/MISP/app/Vendor/kamisama/php-resque-
    www-data    9124    9123  0 13:23 ?        00:00:00 php ./bin/resque
    www-data    9141       1  0 13:23 ?        00:00:00 bash -c cd '/apps/var/www/MISP/app/Vendor/kamisama/php-resque-
    www-data    9142    9141  0 13:23 ?        00:00:00 php ./bin/resque
    www-data    9167       1  0 13:23 ?        00:00:00 bash -c cd '/apps/var/www/MISP/app/Vendor/kamisama/php-resque-
    www-data    9168    9167  0 13:23 ?        00:00:00 php ./bin/resque
    www-data    9185       1  0 13:23 ?        00:00:00 bash -c cd '/apps/var/www/MISP/app/Vendor/kamisama/php-resque-
    www-data    9186    9185  0 13:23 ?        00:00:00 php ./bin/resque
    www-data    9207       1  0 13:23 ?        00:00:00 bash -c cd '/apps/var/www/MISP/app/Vendor/kamisama/php-resque-
    www-data    9208    9207  0 13:23 ?        00:00:00 php ./bin/resque-scheduler.php
    

    Though the Vendor directory is not, yet built!

    Steps to reproduce

    Do a do-release-upgrade from 16.04, then run the Ubuntu 20.04 upgrade/build script.

    Version

    2.4.51

    Operating System

    Ubuntu

    Operating System version

    20.04 Jammy

    PHP version

    7.4

    Browser

    All

    Browser version

    No response

    Relevant log output

    Getting a lot of Module 'curl' already loaded in Unknown on line 0
    
    [Tue Jan 03 13:23:22.168928 2023] [core:notice] [pid 8953] AH00094: Command line: '/usr/sbin/apache2'
    root@UBTU-CS-P-2566:/var/log/apache2#  tail error.log
    [Tue Jan 03 13:23:21.824197 2023] [ssl:error] [pid 8909] AH02604: Unable to configure certificate {fQDN}:443:0 for stapling
    PHP Warning:  Module 'intl' already loaded in Unknown on line 0
    
    
    [Tue Jan 03 13:23:21.885795 2023] [ssl:error] [pid 8911] AH02604: Unable to configure certificate misp.local:443:0 for stapling
    [Tue Jan 03 13:23:22.114573 2023] [ssl:warn] [pid 8951] AH01906: misp.local:443:0 server certificate is a CA certificate (BasicConst
    raints: CA == TRUE !?)
    [Tue Jan 03 13:23:22.114637 2023] [ssl:warn] [pid 8951] AH01909: misp.local:443:0 server certificate does NOT include an ID which ma
    tches the server name
    [Tue Jan 03 13:23:22.114758 2023] [ssl:error] [pid 8951] AH02217: ssl_stapling_init_cert: can't retrieve issuer certificate! [subjec
    t: emailAddress={email},C=US / issuer: emailAddress={email},C=US / serial: 7E05D8538D04FDFDC7353F257C8CF1E6119EC42A / notbefore: Jan  3 18:23:21 2023 GMT / notafter
    : Jan  3 18:23:21 2024 GMT]
    [Tue Jan 03 13:23:22.114777 2023] [ssl:error] [pid 8951] AH02604: Unable to configure certificate misp.local:443:0 for stapling
    [Tue Jan 03 13:23:22.161084 2023] [ssl:warn] [pid 8953] AH01906: misp.local:443:0 server certificate is a CA certificate (BasicConst
    raints: CA == TRUE !?)
    [Tue Jan 03 13:23:22.161117 2023] [ssl:warn] [pid 8953] AH01909: misp.local:443:0 server certificate does NOT include an ID which ma
    tches the server name
    [Tue Jan 03 13:23:22.161279 2023] [ssl:error] [pid 8953] AH02217: ssl_stapling_init_cert: can't retrieve issuer certificate! [subjec
    t: emailAddress={email},C=US / issuer: emailAddress={email},CN={},C=US / serial: 7E05D8538D04FDFDC7353F257C8CF1E6119EC42A / notbefore: Jan  3 18:23:21 2023 GMT / notafter
    : Jan  3 18:23:21 2024 GMT]
    [Tue Jan 03 13:23:22.161289 2023] [ssl:error] [pid 8953] AH02604: Unable to configure certificate misp.local:443:0 for stapling
    root@UBTU-CS-P-2566:/var/log/apache2#
    

    Extra attachments

    No response

    Code of Conduct

    • [X] I agree to follow this project's Code of Conduct
    needs triage 
    opened by smclinden 0
  • Bug: timestamp is not honored in /attributes/restSearch

    Bug: timestamp is not honored in /attributes/restSearch

    Expected behavior

    When I issue the following HTTP POST request to /attributes/restSearch, I get old attributes:

    {
        "returnFormat": "json",
        "timestamp": "2023-01-02",
        "published": 1,
        "page": 0,
        "limit": 100
    }
    

    Which will return e.g.

    {
        "id": "1855",
        "event_id": "4",
        "object_id": "0",
        "object_relation": null,
        "category": "Other",
        "type": "comment",
        "to_ids": false,
        "uuid": "cd70624e-79a2-4066-aaec-fc511e0b1xxx",
        "timestamp": "1573635698",
        "distribution": "5",
        "sharing_group_id": "0",
        "comment": "Imported from STIX header description",
        "deleted": false,
        "disable_correlation": false,
        "first_seen": null,
        "last_seen": null,
        "value": "FOO (November 1 to November 8)",
        "Event":
        {
        }
    }
    

    And this timestamp translates into Nov 13 2019 09:01:38 which is 3 years ago.

    Actual behavior

    Get old attributes, regardless of timestamp parameter.

    Steps to reproduce

    Send any request with a timestamp parameter set to e.g. yesterday.

    Version

    2.4.166

    Operating System

    Unsure

    Operating System version

    Unsure

    PHP version

    Unsure

    Browser

    HTTP

    Browser version

    No response

    Relevant log output

    No response

    Extra attachments

    No response

    Code of Conduct

    • [X] I agree to follow this project's Code of Conduct
    needs triage 
    opened by hazcod 1
  • Bug: content-type application/json returns no attributes on /attributes/restSearch

    Bug: content-type application/json returns no attributes on /attributes/restSearch

    Expected behavior

    When sending a POST request to /attributes/restSearch, no attributes are ever returned if I specify a HTTP request Content-Type: application/json header. Things do work if I ommit it or use text/javascript.

    The result will be along the lines of this, regardless of the request:

    {
    "Response": { "Attributes": [] }
    }
    

    Actual behavior

    Get attributes as normal.

    Steps to reproduce

    Send any POST request to /attributes/restSearch with content-type: application/json.

    Version

    2.4.166

    Operating System

    Unsure

    Operating System version

    Unsure

    PHP version

    Unsure

    Browser

    HTTP

    Browser version

    No response

    Relevant log output

    No response

    Extra attachments

    No response

    Code of Conduct

    • [X] I agree to follow this project's Code of Conduct
    needs triage 
    opened by hazcod 1
Releases(v2.4.167)
  • v2.4.167(Dec 26, 2022)

    We are pleased to announce the immediate availability of MISP v2.4.167 with new features and fixes, bugs fixed and a security fix.

    New features

    Timeline improvements for large events

    Timeline is a convenient way to display the different attributes and objects over the time. Events with a large set of attributes or objects (more than 500) cannot display a human readable timeline. Nevertheless there are still a lot of valuable information in such event especially concerning the occurences over time. A new feature has been added in 2.4.167 to display the overall occurrences over the time and display the overall sighting trend.

    Taxonomy highlight

    For MISP users and organisations, it's important to show the important contextualised information and especially the taxonomies which are important to your use-case. We introduced a new feature to highlight the important taxonomy in a MISP instance.

    The site admin user can select the important taxonomies in the taxonomy list:

    and then the taxonomy namespace will appear in a visible box:

    Create objects from free-text import

    The free-text import in MISP is very nifty for analysts willing to enter quickly new attributes. This functionality was initially used for attributes only. In 2.4.167, MISP objects can be created from the free-text import directly too.

    API

    • A new endpoint session kill-switch has been added for the support and integration with MeliCERTes project.

    UI

    • Clarify the exclusivity issue in the UI when exclusive tags are used in the TLP namespace.
    • [dashboard] sort dashboard widgets.

    Many UI improvements and a special thank to Jakub Onderka for the attention to details in the UI.

    Security fix

    A security XSS vulnerability has been fixed in this release and tracked under CVE-2022-47928. We recommend every users to update to the latest version.

    A huge thanks to all the contributors and supporters of the MISP project. This release won't be possible by all the organisations and people supporting us to make MISP a reality.

    For more details about changes in the MISP core software.

    Other updates and changes

    MISP Objects

    • New thaicert-group-cards, Palantir ADS and persnona.
    • Invalid UUID object templates fixed including mactim-timeline-analysis and fail2ban.

    MISP Galaxy

    • New threat-actor such as TAG-53, Malteiro and others added.
    • RAT group updated.
    • Ransomware groups updated.

    MISP taxonomies

    Don't forget to follow us on Mastodon

    MISP projet has its own Mastodon server misp-community.org and don't forget to follow @[email protected] on the fediverse. Core contributors of MISP can sign-up if you want an account.

    Source code(tar.gz)
    Source code(zip)
  • v2.4.166(Nov 30, 2022)

    Workflow screenshot

    We are pleased to announce the immediate availability of MISP v2.4.166 with new features and fixes, including two critical security fixes.

    TAXII 2.1 server push integration

    With the collaboration of CISA and MITRE, we have included the first version of the TAXII integration in MISP, allowing administrators to configure their MISPs to push content to TAXII 2.1 servers. For more informatia new dedicated will be posted soon. On server side, the taxii2-client Python library is required to be installed. The conversion is performed by the wonderful and efficient misp-stix library.

    Logging rework

    The logging of MISP has been severely reworked by Jakub Onderka, including a separate Access log subsystem as well as multiple improvements and cleanups to the system at large.

    Security fixes

    Two critical vulnerabilities have been patched allowing for the tampering with data shared in the community via galaxy clusters and tags. It is HIGHLY recommended to update to 2.4.166 as soon as possible to avoid information tampering. We also encourage everyone to consider informing peered MISP instance owners to do the same. CVEs have been requested and are pending for both. Thanks to Jakub Onderka for discovering and fixing the vulnerabilities.

    Allowing for working around the edge cases introduced by TLP v2.0

    Even though TLP 2.0 has been supported by MISP for a while, in order to cope with both tools old and new as well as older information sources, we see the need to often attach both TLP:WHITE and TLP:CLEAR to data points. This has however been blocked by the taxonomy exclusivity rules - something that we've now added exceptions for.

    Let's hope that we can avoid similar surprises in the future.

    For more details about changes in the MISP core software.

    Other updates and changes

    MISP Objects

    • [passport object] Updated to include passport-creation field.

    MISP Galaxy

    • MITRE ATT&CK updated and fixing the missing reference
    • Many improvements and fixes in all the meta fields
    • Tool galaxy updated
    • Ransomware groups updated
    Source code(tar.gz)
    Source code(zip)
  • v2.4.165(Nov 22, 2022)

    We are pleased to announce the immediate availability of MISP v2.4.165 with many improvements to the workflow subsystem along with various performance improvements.

    Improvements

    • [workflow] Module to toggle/remove the to_ids flag
    • [workflow] Added generic module to support attribute edition
    • [workflow] [triggers:event_after_save_new] Added 2 new triggers for new events and new events from pull.
    • [workflow:execute_module] Allow to ignore format conversion before executing module.
    • [workflows:triggers] Added filtering capability on the index
    • [CLI] Feed management added
    • [CLI] Pretty and JSON output added in list and view feeds
    • [Auth] OpenID connect improved
    • [freetext] Fetch security vendor domains from warninglist
    • [UI] Allow to disable PGP key fetching
    • [UI] Show warning if user don't have permission to use API
    • [tool:evengraph] Include relationships when using pivot key
    • [UI] Show servers where event will be pushed

    Performance improvements

    • [feed] Store freetext feed compressed in cache
    • [internal] Store some data in Redis compressed to save memory
    • [correlation] Do not correlate over correlating value again for full correlation
    • [internal] Add support for simdjson extension
    • [warninglist] Load warninglist from Redis for TLDs and security vendors

    Bugs fixed

    • [tags] not passing name, filter, search all together would lead to the search not working

    Security issues

    • [security] Permission for tag collections
    • [security] Check user permission when attaching clusters

    We strongly recommend MISP administrators to update to this latest version.

    For a more detailed changelog, please see the online Changelog.

    New workflow blueprints available

    New workflow blueprints were added to support new use-cases.

    New MISP modules

    • [expansion] Added extract_url_components module to create an object from an URL attribute.
    • [expansion] New crowdsec expansion module added.
    • [expansion] New VARIoT IoT exploits database expansion module added.
    • [expansion] Updates on hyasinsight expansion module.

    MISP taxonomies

    • new misp-workflow taxonomy to have a consistent tag message for the MISP workflow.
    • Taxonomy in support of integrating MISP with Sentinel. Sentinel indicator threat types added.

    For more details.

    MISP galaxy

    • Many updates to the threat actor database.
    • Update to the MITRE ATT&CK framework to version 12.0.

    For more details.

    MISP objects

    • New object to describe Telegram bots.
    • Updated exploit object.

    For more details.

    Social network - Mastodon

    MISP project is also now reachable via Mastodon. Feel free to follow us at @[email protected]

    Source code(tar.gz)
    Source code(zip)
  • v2.4.164(Oct 10, 2022)

    We are pleased to announce the immediate availability of MISP v2.4.164 with a new tag relationship features, many improvements and a security fix.

    New tag relationship feature

    Relationship can now be added to any attribute tag or event tag. This works with tags and galaxy clusters. The new feature is available in event view.

    The tag relationship feature is also exposed in the API under the endpoint /tags/modifyTagRelationship/[scope]/[id] where scope is the attribute/event and id is the id of the EventTag / AttributeTag object.

    Improvements and bug fixes

    • [periodic_report] Added security recommendations section showing course of actions related to attack techniques.
    • [workflow] add support for local and relationship in workflow.
    • [API/galaxyCluster/restSearch] Allow multiple filtering conditions to be used at once.
    • [EventGraph] Added entity comment in the graph as tooltip and support of comment in searches.
    • [UI] Many improvements and optimisation.

    CVE-2022-42724

    This release fix a security vulnerabilities (CVE-2022-42724) which allows org admin to discover role names which should have been restricted to site admin.

    We strongly recommend MISP administrator to update to this latest version.

    For a more detailed changelog, please see the online Changelog.

    Source code(tar.gz)
    Source code(zip)
  • v2.4.163(Sep 26, 2022)

    We are pleased to announce the immediate availability of MISP v2.4.163 with an updated periodic notification system and many improvements.

    Updated periodic notification system

    • A new option has been added to set the number of days for the trending calculation.
    • New correlation are now showed in the periodic notification.
    • Only the top 10 MITRE ATT&CK techniques are displayed and sorted by number of occurrences.
    • Layout has been improved in the UI and also in the static email rendering.
    • Only show data in the chart for tags having changes over time.

    For more information, check out the Periodic summaries - Visualize summaries of MISP data blog.

    Fixes

    • MISP OpenAPI description file has been improved.
    • [community] Clarification concerning the NATO process.
    • [ssdeep] Check if the ssdeep contains newline characters.
    • Many code clean-up and speed-up included.
    • Improvements and bugs fixed in the correlation engine.
    • Many bugs fixed.

    Thanks to all the contributors and users reporting bugs to make the software better.

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements in MISP core.

    misp-stix v.2.4.163

    misp-stix has been released too and now in-line with the MISP release schedule. The full changelog is available.

    Many improvements in the MISP galaxy and especially the threat-actor galaxy, 360.net Threat Actors added. There is a detailed changelog.

    New financial taxonomy and many other taxonomies. There is a detailed changelog.

    Multiple objects were updated and added, for more details.

    Various fixes in misp-modules for more details.

    Source code(tar.gz)
    Source code(zip)
  • v2.4.162(Sep 13, 2022)

    We are pleased to announce the immediate availability of MISP v2.4.162 with a new periodic notification system, workflow updates and many improvements.

    In addition to the MISP v2.4.162 release, misp-guard has been released which is a mitmproxy addon that inspects the events that MISP is attempting to synchronize with external MISP instances via PUSH or PULL and applies a set of customizable rules defined in a JSON file. This is a complementary tool to support MISP users having to interconenct MISP instances between highly sensitive networks.

    Periodic notification system

    As of version 2.4.162, MISP includes a periodic summary feature allowing users to consult a summary based on a requested time-frame for data the user has access to.

    Currently, the summaries can be generated for 3 different periods: daily, weekly and monthly and then sent to all users that subscribed one of these periods.

    In addition to choose which period users want to subscribed to, they can also specify filtering options such as tags or distribution level to be used to generate the summary. The summary can be sent via email in addition to the User-Interface view.

    Periodic summary Periodic summary

    For more information, check out the Periodic summaries - Visualize summaries of MISP data blog.

    Workflow improvements

    • Added diagnostic support and support of arbitrary URL for webhook module.
    • New Microsoft teams module based on the webhook module.
    • New email notification module to send email to a list of MISP users including Jinja templating.
    • Tag name can now be used in workflows.

    For more details about MISP Workflow, check out the training materials.

    MISP core improvements

    • Allow option to delete tags on event sync prior to soft-delete tag implementation.
    • API/[Event:restSearch] Added option event_tags to filter for eventTag only.
    • API/RestSearch - Added support of static parameter to produce a static HTML output.
    • Syslog/logging for certain log entries vital information was omitted by the syslog. If no custom message is specifically set for the log entry, the change field is included.
    • Enforce UUIDs uniqueness on MISP data back-end.

    Bugs fixed

    • [correlations] save the distribution state of the event before/after saving it, fixes #8528.
    • [attribute tags] removal broken, fixes #8567.
    • Class 'Folder' not found #8544.
    • Create unique SIDs for email attributes in NIDS export.

    Thanks to all the contributors and users reporting bugs to make the software better.

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements in MISP core.

    Many improvements in the MISP galaxy and especially the threat-actor galaxy. There is a detailed changelog.

    Improvement in the false-positive taxonomy and many other taxonomies. There is a detailed changelog.

    Multiple objects were updated and added, for more details.

    Source code(tar.gz)
    Source code(zip)
  • v2.4.161(Aug 11, 2022)

    We are pleased to announce the immediate availability of MISP v2.4.161.

    Small improvements

    • A new option added to log the last API request of an API key. (Thanks to Tom King for the contribution)
    • Overcorrelation features have some new improvements such as:
      • A new tool to generate occurrence counts (real numbers this time)
      • A hook to truncate the over-correlating value table on recorrelation
      • We no longer store the partial counts as occurrences when generating correlations
    • Performance improvements in event fetching
    • Various performance tuning in the new correlation engine including the full recorrelation

    Bugs fixed

    • tlp:amber+strict and tlp:clear are now valid tags
    • [stix2 import] Better external_references parsing for attack patterns objects

    Thanks to all the contributors and users reporting bugs to make the software better.

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements in MISP core.

    Source code(tar.gz)
    Source code(zip)
  • v2.4.160(Aug 8, 2022)

    We are pleased to announce the immediate availability of MISP v2.4.160. With the August summer-holiday season kicking into high gear, we have a very special release for you all, containing a long list of major new features, improvements and general quality of life improvements.

    Unlike we do normally, this time around we're preparing separate blog posts for some of those major features, so follow the links below to read up on in-depth descriptions of each.

    Workflows

    Something that has been in the works for quite some time now is finally hitting a release version of MISP, as of 2.4.160, we have the first release of the built in workflow system released.

    This system allows you to use an easy to use, yet extremely powerful graphical interface to modify how MISP handles certain tasks such as event publishing, user enrollment, synchronisation, etc., by adding additional logical steps in their respective executions, utilising a module system similar to what was already common to MISP from enrichment subsystems, exports as well as imports.

    This is merely the first step (or leap rather) towards customising and sharing custom workflows, stay tuned for new features, improvements as well as triggers and modules in the near future.

    Head over to the README as well as a nifty slide deck, to find out what this incredibly powerful can do for you and your community. If you would like to see a video demonstration of how it works, narrated by @mokaddem's soothing voice, head over to the vlog entry over at youtube.

    New correlation engine

    One of the biggest pain points as of recently has been our dated and rather bloated correlation engine, which could easily bring a long running MISP instance to its knees when certain highly correlated data sources were synchronised.

    As of 2.4.160, we now have 2 brand new correlation engines at your disposal, with the old engine being retired immediately. Please be aware that upgrading to the current version will regenerate your correlations using the new engine, something that can take quite a long time (on our largest instance it took a whopping 40 hours!). With that said, we can assure you it's well worth the wait and should resolve several long standing performance bottlenecks as well as heavily cut down on the space requirements for your data.

    For more information, on the new engines, their differences, the various new support tools as well as what benefits you should expect, head over to the dedicated blog post.

    STIX 2 library reworks

    There has been a massive amount of work going into the STIX 2.x library rework, bringing us closer and closer to having a full mapping of everything expressable. We're collaborating with CISA and Mitre to ensure that MISP can both express and understand STIX to its fullest extent.

    For more information, head over to the release notes over on the MISP STIX library's repo.

    Mermaid support for Event reports added

    Writing custom reports has become more and more popular, but one annoyance has been the lack of a way to depict graphs and flow charts without relying on external tools to create those (and share them as images for example). Using Mermaid, you now have a nifty tool to build graphs out of simple markdown directly in the event report editor.

    Various other improvements

    A long list of other improvements, affecting the performance and stability of the platform as well as improvements to existing features. Head over to the changelog for a detailed list of changes.

    Acknowledgement

    We would like to thank all the contributors, reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in misp-objects, misp-taxonomies and misp-galaxy.

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements in MISP core. Additional changelogs are available for misp galaxy, misp-taxonomies, misp-objects and misp-modules

    Source code(tar.gz)
    Source code(zip)
  • v2.4.159(May 30, 2022)

    We are pleased to announce the immediate availability of MISP v2.4.159. This releases includes many improvements, bugs fixing and improvement concerning performance on large dataset.

    Performance Improvements

    • [DB] Add MysqlExtended DboSource to support index query hints.
    • [Query] Add new setting to disable taxonomy checks when browsing data.
    • We discovered that some MISP users are still using slow file-based session handling in PHP. Now, we added in the diagnostic, if session is file based. We recommend everyone to use the Redis session.
    • Many additional speed-up and faster functions in the MISP internals.
    • Reduce memory usage when generating all correlations.

    Improvements

    • [Feed] Allow option to disable correlations for all events coming from a feed. This can be useful when correlation requires to be disabled for an imported feed.
    • [UI] Allow to upload MISP event by pasting data to textarea in addition to the file upload.
    • An optional feature clusters:attachMultipleClusters is now available to allow the mirroring of attribute clusters to event.
    • [auditlog] Support for fetch event changes from specific time.
    • [UI] Allow to filter attributes from Related Events box.
    • [UI] Allow to filter attributes from warninglist box.
    • [UI] Many UI improvements to make the interface easier to read.
    • [UI] Disable correlation checkbox for non correlating types.
    • [STIX 2 import] Better Galaxies parsing by looking for the ATT&CK technique id.
    • [API] Enable sharing group filter for Event controller not just attribute.

    Fixes

    • [STIX] Avoiding non RFC-4122 UUIDs to be imported (and therefore skipped)
    • [STIX 1 import] Save process network connections.
    • [STIX 1 import] Fixed galaxy tag_names fetching from TTP names.

    Knowledge Bases

    MISP Taxonomies

    • [dga] First version of the DGA taxonomy based on https://www.usenix.org/system/files/conference/usenixsecurity16/sec16_paper_plohmann.pdf.
    • GrayZone of Active Defense, originaly published by Washington University, v2 created and updated by DCG420
    • Various fixes to existing taxonomies.

    MISP Objects Template

    • A new PaloAlto Threat Event object template has been added.
    • A updated security playbook has been added.
    • A new ransom negotiation object has been added.
    • An improved Passive SSH template object.
    • Various fixes and improvements to different object templates such as email, virustotal-submissions and others.

    MISP Galaxy

    • Improved Cryptominers galaxy.
    • Improved backdoors galaxy.
    • Threat Actor galaxy updated and extended with new threat-actors.
    • MISP Galaxy updated for MITRE ATT&CK v11.2.

    Acknowledgement

    We would like to thank all the contributors, reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in misp-objects, misp-taxonomies and misp-galaxy.

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements in MISP core. Additional changelogs are available for misp galaxy, misp-taxonomies, misp-objects and misp-modules

    Source code(tar.gz)
    Source code(zip)
  • v2.4.158(Apr 20, 2022)

    We are pleased to announce the immediate availability of MISP v2.4.158. This release includes a series of security fixes and as such we highly encourage everyone to update to this version as soon as possible.

    Thanks to Dawid Czarnecki of Zigrin Security for the in-depth penetration test and its findings and thanks to the Luxembourg Army for financing the penetration test. This is the follow up to the Cerebrate penetration test also conducted by Zigrin Security on behalf of the Luxembourg Army, as described here.

    Security fixes

    Several security issues have been resolved, head over to the security page for a detailed break-down of the advisories including the associated CVEs. Whilst most of the vulnerabilities listed are mitigated by requiring compromised high privilege accounts, we nevertheless advise all users to update their instances as soon as possible.

    Announcement of a silent fix of phar deserialisation RCE in a previous release (v2.4.156)

    As of the previous security release (v2.4.156), based on the pentest conducted by Ianis BERNARD of the NATO Cyber Security Centre, a high criticality vulnerability was also identiefied. We have opted for a silent fix to the critical vulnerability whilst upgrading the announced criticality of the other security fixes included in the release.

    This is an extreme measure that we take whenever we want to ensure that the community is both aware that they do need to update as soon as possible whilst not drawing attention to the actual critical vulnerability. If you have followed our guidance over the past month to update you are already safe - if you are running a MISP instance below 2.4.156 we highly encourage you to update to the latest version as soon as possible.

    Custom email templates

    Added the ability to override some of the standard e-mail templates with custom ones, just drop the templates mirroring the naming convention of the existing ones in /var/www/MISP/app/View/Email/text and /var/www/MISP/app/View/Email/html into /var/www/MISP/app/View/Email/text/Custom/ and /var/www/MISP/app/View/Email/html/Custom/. Currently supported templates: alert, password_reset.

    RestSearch improvements

    Fixing a baffling oversight on our side, thanks to Tom King we can now search by sharing groups besides just distribution levels.

    A long list of refactors and bugfixes

    Massive thanks to Jakub Onderka for the continuous refactoring, simplifying and cleaning up of the code-base. For a full list of all the improvements that are part of this herculean effort, refer to the changelog

    Acknowledgement

    We would like to thank all the contributors, reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in misp-objects, misp-taxonomies and misp-galaxy.

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements in MISP core.

    Source code(tar.gz)
    Source code(zip)
  • v2.4.157(Mar 25, 2022)

    We are pleased to announce the immediate availability of MISP v2.4.157, following a series of bug fixes as a quick follow up to 2.4.156.

    As a reminder, MISP v2.4.156 included several critical vulnerability fixes, as such, we highly encourage everyone to update to this version as soon as possible. It also brought several new important features that help communities ensure the veracity of their most critical shared data.

    Fixes to the authkey handling

    Manage auth keys of your team as an org admin, until now this feature was broken and org admins had to log in as their automation / sync users in order to generate new keys. This is no longer the case, simply view the user you wish to create a new key for and do it directly from the interface or via the API. Keep in mind that org admins can only create keys for non administrator users.

    Thank you to @oivindoh for pointing this shortcoming out.

    Fix to a breaking bug with event publishing

    Due to a bug introduced by a regression in 2.4.156, publishing events ended up not pushing events with sharing groups to remote instances. This is now resolved and for this in itself we already highly recommend updating to this version. Full instance pushes and pulls were not affected. Neither were events that didn't rely on sharing groups as their distribution model.

    Thank you to @treyka for finding the bug.

    New setting introduced to disable event lock checks

    Sometimes the addition of certain features, whilst having good intentions, ends up being more annoying that useful. In these cases, unless it's something absolutely hindering, we still do not want to modify the default behaviour of MISP over night. Such is the case with the event lock checks, which provide warnings on the event view that another user is also editing the event, a simple warning to users that their event's state may be outdated.

    This functionality is rather verbose when it comes to logging, gets in the way of debugging and can cause session persistence issues in certain cases. As such we've introduced a new setting to disable the functionality and unless you or your community are especially attached to it, we recommend heading over to the server settings and disabling it via the MISP.disable_event_locks setting.

    Thanks to @github-germ and @packet-rat for pointing the annoying nature of this feature out.

    Acknowledgement

    We would like to thank all the contributors, reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in misp-objects, misp-taxonomies and misp-galaxy.

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements in MISP core.

    Source code(tar.gz)
    Source code(zip)
  • v2.4.156(Mar 18, 2022)

    We are pleased to announce the immediate availability of MISP v2.4.156 - a release bringing several new features and fixes two critical vulnerabilities. We highly encourage everyone to update to this version as soon as possible.

    Protected mode - cryptographic signing of synchronisation

    With the current tensions, information assurance in many ways is becoming more and more important across the different MISP communities. Whilst foul play is often quickly discovered and leads to the ejection from a sharing community, leading to an inherent self-healing mechanism of the different networks, in some cases due to information's criticality, more active measures are needed.

    By design, MISP's sharing mechanisms rely on trust relationships between the different interconnected nodes in the various MISP networks. This means that in a mesh network of MISP nodes, information can travel via trusted synchronisation users, the information's veracity being ensured by the various site administrators of the different instances.

    In some cases this is not enough, especially when exchanging data that is meant to be adhered to blindly in a highly automated fashion. Vetted block lists for example affecting large constituencies and the automatic blocking of traffic for service providers for example.

    To support this use-case, MISP as of v2.4.156 has a new mechanism that allows event creators to attach a set of PGP instance signing keys to an event, which are used to sign the events on each hop of the synchronisation. This allows recipient MISPs to discard any updates coming from nodes that cannot produce a valid signature with one of the initial signing keys.

    An example

    Alice and Bob each have their own MISP instances, with Alice feeding Bob with critical information. Bob trusts this information immediately and blindly. Eвa, wanting to remove data points or diluting the information from Alice's stream, is also part of their broader network.

    Traditionally, Alice sharing an event to the network would propagate to both Bob's and Eвa's instance. Eвa could in this case abuse her administrative privileges to modify the event, perhaps injecting disinformation and removing valid data. By synchronising this back to Bob, Bob's instance would see an incoming synchronised edit, which in a mesh network could be legitimate and as such it would accept the change. Propagating it further back to Alice would be blocked by MISP's protection against remote modifications to data at origin.

    unprotected_sync_mode

    With protected mode enabled, this situation changes drastically. Alice could add her own signing key as well as Bob's to the event, ensuring that the only parties able to relay modifications to the event would be Alice and Bob. When leaving Alice's instance, the event would get signed with Alice's signing key. Since the event contains both Alice's and Bob's key, any subsequent modifications from Alice would be accepted by Bob's instance. Incoming edits would be signed by Alice's key, meaning that Bob would validate the package with its locally stored public key of from the initial exchange.

    This means that Eвa modifying the event and attempting to share it with Bob would get rejected, as Eвa, lacking the private keys of Alice and Bob, can only sign it with her own key, which Bob's instance would immediately flag as suspicious and ultimately reject it.

    protected_sync_mode

    Usage

    To get started with the feature, simply use the new protected mode field in the event view, you can convert any event into protected mode:

    protected1

    At which point you can start adding individual keys:

    protected2

    Keep in mind that you can add multiple instance signing keys if you wish for your trusted partners (or your own instances, for example if you have an internal and a sharing MISP in the DMZ).

    protected3

    As a caveat, keep in mind that this mechanism limits the distribution of data inadvertently. Even if the distribution level would allow it, the synchronisation will be limited by who can sign the event for further propagation, so use this new functionality when the use-case really calls for it.

    A massive thank you to our good friend Trey Darley (@treyka) of Cert.be for the brainstorming session that lead to the implementation of this feature!

    Context summary export

    A new export format was added that generates an HTML representation of a summary of all context information from a set of filtered data. One could for example use restsearch to generate all context from any event that is attributed to a threat actor. The resulting HTML will include the Mitre ATT&CK matrix of all leveraged techniques in the selected events as well as any other labelling and context.

    Event warning system

    The new warning system warns users about potential improvements to an event they could be making, such as resolving tagging issues, improving the quality of the event, etc. The system comes with a plugable module system, easily build and deploy your own warning system.

    warning_system

    Internal reworks

    @JakubOnderka continues his massive crusade against ugly spaghetti code, with a continuous stream of refactorings, this time massively improving the code-base of the synchronisation mechanism.

    Pentest - Several security issues resolved

    We would like to thank Ianis BERNARD of NATO Cyber Security Centre. Based on the findings of their pentest we were able to resolve several security vulnerabilities and as such we highly encourage everyone to update to v2.4.156 ASAP.

    Security fixes resolved

    Four security vulnerabilities were fixed in this release. We strongly recommend everyone to install this version as soon as possible.

    LinOTP auth improvements

    Thanks to the lovely work submitted by @andurin, the LinOTP authentication subsystem now includes several improvements, amongst others the ability to conveniently manage and disable the subsystem directly via the system settings.

    Originally, the only way to disable the LinOTP authentication was to purge the related settings from the configuration files. In order not to break the expected functionality for users that already have LinOTP configured, the default behaviour for the new "LinOTP.enable" setting behaves a bit different from other similar settings: When no value was assigned by an administrator, the module is enabled by default if the LinOTP configuration keys exist in the configuration file. That means, if you've had it configured from before, by default it will be enabled. Other than that it will be disabled.

    Confirming the setting as either enabled or disabled by an administrator will override this behaviour with the selected setting.

    A long list of other improvements

    We have received a massive list of pull requests for enhancements and fixes. Make sure you check out the changelog for further details.

    Acknowledgement

    We would like to thank all the contributors, reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in misp-objects, misp-taxonomies and misp-galaxy.

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements in MISP core.

    Source code(tar.gz)
    Source code(zip)
  • v2.4.155(Mar 18, 2022)

    This release is a rapid follow up to v2.4.154, addressing several rather annoying issues

    Bugfixes

    • Various bugfixes to the sharing group blueprint system (especially to it being more restrictive than intended)
    • Updating the DB schema to avoid the diagnostics complaining
    • Fixed an issue with organisation meta fields defaulting to null rather than '' (causing the blueprint issue mentioned above)
    • Rework of the DB schema dumper
    • Fixes to the Kali Linux installer

    Acknowledgement

    We would like to thank all the contributors, reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in misp-objects, misp-taxonomies and misp-galaxy.

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements in MISP core.

    Source code(tar.gz)
    Source code(zip)
  • v2.4.154(Mar 18, 2022)

    MISP 2.4.154 released with a host of new features and fixes, including some new tools that help us navigate the current geo-political landscape when sharing information.

    Sharing group blueprints

    Difficult times often call for radical measures, with the recent world events we've seen more and more communities rapidly reorganising as well as new large communities being established. Sharing information with only subsets of communities has become ever more important and whilst we've had the tools to facilitate this in MISP for a long time, rapidly managing different, often overlapping groups has been difficult.

    Sharing group blueprints allow us to programmatically define reusable blueprints for generating sharing groups, based on inheritance and various filters to automate the task of maintaining the groups.

    Sharing group blueprints accept JSON objects based on which they generate a sharing group each, where various filters can be set for the decision making. The syntax allows for boolean operators as well as the use of organisation metadata and existing sharing group inheritance. This can also be used to create derivative groups with certain members being excluded, for example the below would be such an example:

    {
       "AND": {
           "OR": {
               "org_sector": "Financial",
               "sharing_group_id": 127
           },
           "NOT": {
               "org_nationality": [
                   "Russia",
                   "Russian Federation",
                   "Belarus",
                   "Republic of Belarus"
               ]
           }
       }
    }
    

    The above would generate a sharing group out of all organisations present in sharing group 127, any organisation that has "Financial" as its type, but excluding any of the specifically negated countries' orgnaisations.

    This system thrives on well maintained organisation lists, so make sure that you put in the extra effort of contextualising your organisations!

    Once a blueprint is created, you can review the organisations to be included and if you are satisfied, create the actual sharing group by clicking on (re)generate sharing group.

    sharing-group-blueprint

    One of the advantages of this system is that the regeneration can be run at any time, for a single sharing group or for all, via the interface or the API. This means that creating a cron job that updates all sharing groups based on the rules regularly is trivial, ensuring that for example inherited organisations via updated child sharing groups are updated continuously.

    Populate events using MISP JSON elements

    There's a new way to populate an individual, existing event: by uploading a JSON file containing MISP elements (such as attributes, objects, tags, galaxies, etc), one can now easily paste JSON blobs into a form that an be accessed by clicking on "Populate from..." and selecting "Populate using a JSON file containing MISP event content data".

    Improvements to the OIDC authentication

    A host of improvements and fixes, including the switch to a new library, developed by Jakub Onderka.

    Acknowledgement

    We would like to thank all the contributors, reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in misp-objects, misp-taxonomies and misp-galaxy.

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements in MISP core.

    Source code(tar.gz)
    Source code(zip)
  • v2.4.153(Feb 4, 2022)

    MISP 2.4.153 released

    • MISP UI translation in Thai added.
    • Improved the debugging of the synchronisation, including more meaningful messages in debug logs.
    • Significant improvements in the misp-stix library, to support additional import coverage of files along with improvements to the STIX export.
    • Improved debugging in the TLS handshake for synchronisation.
    • Additional CLI tests for security.
    • Markdown-IT library updated to the latest version, including security fixes to version 12.3.2.
    • Improvements in the various MISP install scripts.

    Many internal improvements and bug fixes.

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements in MISP core.

    MISP Modules

    The MISP modules changelog is available.

    MISP Taxonomies

    MISP Taxonomies changelog is available.

    MISP Galaxy

    • New surveillance group added "Cytrox".
    • New threat-actor such as SideCopy, AQUATIC PANDA and others.
    • Many updates.

    MISP Galaxy changelog

    MISP Objects

    • New social and personal relationships for MISP objects based on FOAF relationships.
    • Probabilistic data structure object added and describes a space-efficient data structure such as Bloom filter or similar structure.
    • Many improvements in GTP, diameter and SS7 attack template objects.
    • New STIX 2.1 objects such artifact and identity available as MISP template object.
    • Many improvements to different MISP object templates.

    MISP objects changelog

    Acknowledgement

    We would like to thank all the contributors, reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in misp-objects, misp-taxonomies and misp-galaxy .

    Source code(tar.gz)
    Source code(zip)
  • v2.4.152(Dec 22, 2021)

    MISP 2.4.152 released

    MISP 2.4.152 released with timeline improvements, optional filtering on sync, LinOTP improvements and more.

    The LinOTP authentication module has been improved to include a mixed mode where both OTP and MISP's usual password authentication can be used together.

    The timelining has been improved in several ways, such as the inclusion of images from objects, as well as various improvements in the timeline's sighting view. Several bugs were affecting this feature have also been fixed.

    A new optional synchronisation filtering has been added to allow for the removal of specific attribute or object types when syncing. The functionality is meant to be used by the final recipient organisations of a synchronisation chain, in order to filter out specific types of information due to legal or specific internal policies. The filtering feature is disabled by default and needs to be enabled in the general configuration. This feature is for ISACs or consumer organisations, not redistributing information to other MISP communities.

    A new STIX 1 and 2 export for attribute restSearch has been added in complement to the existing event export in STIX 1 and 2. The export works just like the other event level STIX export, all you need to do is specify the given STIX format as the return type when querying the attribute restSearch endpoint.

    Many internal improvements and bugs fixed.

    MISP Modules

    The MISP modules changelog is available.

    MISP Taxonomies

    MISP Taxonomies changelog is available.

    MISP Galaxy

    MISP Galaxy changelog

    MISP Objects

    • New Concordia intrusion set object.
    • New temporal event object.
    • Many improvements in user, person, postal-address, email object.
    • New relationships added such as found-in, works-with, drives.

    MISP objects changelog

    Acknowledgement

    We would like to thank all the contributors, reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in misp-objects, misp-taxonomies and misp-galaxy .

    Source code(tar.gz)
    Source code(zip)
  • v2.4.151(Nov 23, 2021)

    MISP 2.4.151 released

    MISP 2.4.151 released including a host of bug fixes and a bunch of new features.

    New features

    • New background processor by @righel
    • Improvements to the CLI tools
    • Bug fixes and improvements

    New background processor

    • MISP has been using CakeResque for its background jobs for the better part of a decade. Whilst it has served us well, the library has been stale for a long time and carries a (for us) unnecessary complexity and is generally the most difficult part of the application to debug
    • Luciano "@righel" Righetti has implemented a completely new, compatible background processing engine using Supervisord
    • Queue and execute jobs the same way as you are used to from before, monitor worker progress via the tools provided by supervisord in addition to MISP
    • No scheduling capabilities, these were an unnecessary overhead for us before as we relied on corn jobs as our preferred scheduling mechanism anyway
    • Expect more improvements to this library over the course of the next months, but feel free to switch to using it already now
    • Currently it is completely optional and the old background processor will still be supported for a while
    • Be aware that manual setup steps are required to get the new processor working, refer to the upgrade guide on the procedure, if you decide to start using it already now

    Various CLI changes

    • Jakub Onderka has been doing a fair bit of refactoring and improvement of the CLI libraries
    • additional administrative tools added to help monitor and manage your MISP instance (such as redis memory diagnostics, mysql table optimisation tool, etc)

    Option to move the system settings to the database

    • Traditionally all system config settings were stored in the config.php file, with a new configuration thanks to Jakub Onderka's implementation the settings can be moved to the database rather than the file.
    • This should help with persistence for containerised installations

    Various improvements

    • The previous version introduced a new STIX library as a replacement for the old one. This change did end up causing some update issues for some installations, the built in updater is now aware of this change and should allow you to easily update via the UI/API updater, with the new STIX library working as intended
    • A long list of improvements, thanks to all contributors! For a detailed list of changes, head over to the changelog

    MISP Modules

    The MISP modules changelog is available.

    MISP Taxonomies

    MISP Taxonomies changelog is available.

    MISP Galaxy

    • Updated to MITRE ATT&CK version 10.
    • Multiple updates in malpedia, threat actor galaxy and Office 365 techniques.

    MISP Galaxy changelog

    MISP Objects

    • New JA3 server object added.
    • New Security playbook object added.
    • New submarine object added
    • New Passive SSH object added.
    • Updated device object.
    • New hashlookup object added.
    • New edr-report object added.

    MISP objects changelog

    Acknowledgement

    We would like to thank all the contributors, reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in misp-objects, misp-taxonomies and misp-galaxy .

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements.

    Source code(tar.gz)
    Source code(zip)
  • v2.4.150(Nov 23, 2021)

    MISP 2.4.150 released

    MISP 2.4.150 released, including a new CA bundle to combat the issues with the Letsencrypt root CA expiration. This is a follow-up release to 2.4.149 and has no other major changes besides pointing to our own repository of the framework that includes the new CA bundle.

    Sync issues due to the expiration of a Letsencrypt root CA

    As described in their blog post, Letsencrpyt had to retire an old Root CA, meaning that that SSL connections when synchronising MISP with other instances would fail if the remote side used letsencrypt. This update includes a new CA bundle that should help you avoid any issues with this.

    Acknowledgement

    We would like to thank all the contributors, reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in misp-objects, misp-taxonomies and misp-galaxy .

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements.

    Source code(tar.gz)
    Source code(zip)
  • v2.4.149(Oct 12, 2021)

    MISP 2.4.149 released (Autumn care-package - STIX 2.1 support and Cerebrate integration)

    MISP 2.4.149 released including many bugs fixed along with some new and improved functionalities

    New features

    • First stage of a massive rework of our STIX integration
    • Various improvements to the integration with Cerebrate

    New STIX libraries

    • The first version of a long ongoing project to rework our entire STIX integration has finally been merged, thanks to the tireless work of @chrisr3d
    • Our converter libraries have embarked on a path of their own, becoming a standalone repository included by default in MISP, but also serving as a useful tool for anyone looking for a clean way of converting between the MISP standard format and various STIX versions (1.1.1, 1.2, 2.0, 2.1).
    • The libraries are still work in progress, but continuously improved, follow misp-stix
    • Included is also a detailed documentation, which also serves as a knowledge base for the mapping between the two formats, available under the documentation sub-directory
    • From this release on, you have more control over which STIX version is used when exporting STIX data from MISP, by specifying the "stix_version" to be returned (supported versions for STIX 1: 1.1.1 and 1.2. For STIX 2: 2.0 and 2.1)

    Cerebrate integration

    • Allow the fetching of sharing group data from Cerebrate instances, our new open source tool in development aiming to solve a host of issues revolving around community management and orchestration. Our first official release of the tool is scheduled for the MISP summit coming up this month
    • To follow the cerebrate project, head over to its github page
    • For the MISP summit to be held on the 21st of October, don't forget to watch the misp-summit. You can still apply for the Call-for-Presentation.

    mail2misp release 1.0

    First official release 1.0 of mail2misp, it's a tool to connect your mail infrastructure to MISP to create events based on the information contained within mail. The solution can be also used to feed MISP instance with honeypot receiving emails.

    Various improvements

    • A long list of improvements, massive thanks to @JakubOnderka for the continuous stream of improvements and quality of life changes
    • Thanks to the work of @righel, our OpenAPI documentation is becoming more and more complete, now covering a long list of the more exotic endpoints and options

    Acknowledgement

    We would like to thank all the contributors, reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in misp-objects, misp-taxonomies and misp-galaxy .

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements.

    Source code(tar.gz)
    Source code(zip)
  • v2.4.148(Aug 9, 2021)

    MISP 2.4.148 released

    MISP 2.4.148 released including many bugs fixed along with security fixes. This release fixes CVE-2021-37742 and CVE-2021-37743.

    New feature

    • added option to block organisation changes at login on ApacheShibbAuth
    • Open data export has been refactored
    • Fix Suricata export concerning sticky buffers
    • ZMQ now includes misp_json_warninglist topic in the pub-sub channels

    Acknowledgement

    We would like to thank all the contributors, reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in misp-objects, misp-taxonomies and misp-galaxy .

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements.

    Source code(tar.gz)
    Source code(zip)
  • v2.4.147(Jul 27, 2021)

    MISP 2.4.147 released

    MISP 2.4.147 released including a massive number of small improvements, bug and security fixes. We strongly recommend all MISP users to upgrade as soon as possible. This release fixes CVE-2021-37534.

    Sync improvements

    Many improvements were done in the synchronisation such as:

    • When saving sightings, only push the new sightings.
    • Filter out existing sightings if remote sever supports that method.
    • Check if event exists before pushing.
    • Check event existence before pushing sightings.
    • Optimise event filtering.

    API/CLI

    Many improvements in the API and CLI.

    This release also includes refactoring of various forms to support future major improvements in MISP.

    Acknowledgement

    We would like to thank all the contributors, reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in misp-objects, misp-taxonomies and misp-galaxy .

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements.

    Source code(tar.gz)
    Source code(zip)
  • v2.4.146(Jul 5, 2021)

    MISP 2.4.145 and 2.4.146 released (Improved warning-lists)

    MISP 2.4.145 and 2.4.146 released including a massive update to the MISP warning-lists, various improvements and security fixes.

    MISP warning-lists improvements.

    Warning lists system has been significantly improved (thanks to Jakub Onderka).

    • Custom warning lists can be created and managed in the MISP user-interface
    • Warning lists can be now imported via the API
    • Warning lists changes are exported in the ZMQ channel
    • Warning lists include new categories to describe the scope

    New features

    Summary email notification

    Email notifications have received a new configuration setting: New event summaries only. This feature publishes the normal alert reports excluding attributes and objects, thereby only describing a summary of the alert. This can be used when encryption cannot be enabled and organisations still require email alerting.

    Documentation

    New documentation has been added to describe the session and cookie handling in MISP.

    API

    • Thanks to a new feature, you can now create read only authentication keys (don't forget to enable the advanced authentication key feature for this to work).

    Security Fixes

    • Various fixes regarding XSS and potential escaping issues including CVE-2021-35502.

    Thanks to the reporters including Nicolas Vidal from TEHTRIS.

    Various improvements

    • [OpenAPI] - Missing return formats added to the documentation
    • [server caching] only push data to redis / logs if there's something to push
    • [attribute] validation tightened for empty strings. A value containing only control characters will now be blocked from entry.
    • [feeds] Added 3 daily feeds (ssh bruteforce, telnet bruteforce, URLs seen) from the APNIC Community Honeynet Project

    Acknowledgement

    We would like to thank all the contributors, reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in misp-objects, misp-taxonomies and misp-galaxy .

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements.

    Source code(tar.gz)
    Source code(zip)
  • v2.4.145(Jul 5, 2021)

    MISP 2.4.145 and 2.4.146 released (Improved warning-lists)

    MISP 2.4.145 and 2.4.146 released including a massive update to the MISP warning-lists, various improvements and security fixes.

    MISP warning-lists improvements.

    Warning lists system has been significantly improved (thanks to Jakub Onderka).

    • Custom warning lists can be created and managed in the MISP user-interface
    • Warning lists can be now imported via the API
    • Warning lists changes are exported in the ZMQ channel
    • Warning lists include new categories to describe the scope

    New features

    Summary email notification

    Email notifications have received a new configuration setting: New event summaries only. This feature publishes the normal alert reports excluding attributes and objects, thereby only describing a summary of the alert. This can be used when encryption cannot be enabled and organisations still require email alerting.

    Documentation

    New documentation has been added to describe the session and cookie handling in MISP.

    API

    • Thanks to a new feature, you can now create read only authentication keys (don't forget to enable the advanced authentication key feature for this to work).

    Security Fixes

    • Various fixes regarding XSS and potential escaping issues including CVE-2021-35502.

    Thanks to the reporters including Nicolas Vidal from TEHTRIS.

    Various improvements

    • [OpenAPI] - Missing return formats added to the documentation
    • [server caching] only push data to redis / logs if there's something to push
    • [attribute] validation tightened for empty strings. A value containing only control characters will now be blocked from entry.
    • [feeds] Added 3 daily feeds (ssh bruteforce, telnet bruteforce, URLs seen) from the APNIC Community Honeynet Project

    Acknowledgement

    We would like to thank all the contributors, reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in misp-objects, misp-taxonomies and misp-galaxy .

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements.

    Source code(tar.gz)
    Source code(zip)
  • v2.4.144(Jun 9, 2021)

    MISP 2.4.144 released

    MISP 2.4.144 released including a massive update to the documentation along with CyCAT.org integration, improvements and fixes including security related fixes.

    OpenAPI integration

    We have a new core team member at MISP Project, Luciano (@righel), who kicked off his tenure with an impressive mapping of all the most important endpoints of MISP to OpenAPI. As of this release, the API documentation is directly available in MISP, along with example payloads and responses. You can also find this information directly on the misp-project website. To all integrators and developers wrangling with the API, we highly recommend you take a look at the API menu in MISP and we wish you happy and headache-free hacking!

    New diagrams and descriptions

    Thanks to the thorough investigations of @mokaddem, we now have the entire synchronisation and authentication flows of MISP mapped in an easy to understand graph - both of these are included as of now directly in your MISP installation, so if you're in doubt about what's going on under the hood, but don't feel adventurous enough to replace your night time reading materials with a hefty chunk of PHP code, have a look at the new graphs!

    CyCAT integration v1

    MISP and CyCAT integration

    CyCAT is a new initiative built by a group of individuals with the aim of cataloguing all the techniques and libraries around cyber-security, mostly with the selfish desire to make their own confusing lives easier (along with all those that are in a similar situation). As of this release, you'll be able to enable a first version of the CyCAT integration in MISP directly, allowing you to directly see relations to your galaxy clusters via CyCAT's own relationship system, giving you an extra layer of background information with the clusters already in use.

    If you are interested in CyCAT and what it can do for you, head over to the CyCAT website.

    To enable the CyCAT integration, got to the Plugin settings and enable the feature.

    Improvements

    • Various quality of life improvements and bug fixes, related to synchronisation, sharing groups, event reports and more!
    • A security fix that would under certain circumstances result in attributes of an object being misassociated to the wrong sharing group after synchronisation. A massive thank you to Jeroen Pinoy for his diligent work in uncovering this issue!

    Acknowledgement

    We would like to thank all the contributors, reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in misp-objects, misp-taxonomies and misp-galaxy .

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements.

    Source code(tar.gz)
    Source code(zip)
  • v2.4.143(May 19, 2021)

    MISP 2.4.143 released

    MISP 2.4.143 released including a new audit subsystem, various quality of life improvements and bug fixes.

    10 year anniversary

    MISP has, as of the 15th of May, turned 10, to celebrate the occasion we have a celebratory MISP logo acting as a temporary replacement of the usual one for the duration of this release.

    It has been a long road since Christophe Vandeplas released the initial version of CyDefsig (later renamed to MISP) in 2011. We would hereby like to thank all contributors and supporters for making MISP what it is today. Looking back at how the tooling and the communities evolved over the decade, we can see how threats and threat intelligence has changed and evolved over the years, molding the platform in the process. Here's to at least another 10 years of active sharing and bringing communities together!

    New audit system

    Thanks to @JakubOnderka, we now have a whole new audit system, storing relevant audit logs in a more concise yet easily machine-parsable way (all changes will be logged as JSON objects). This feature is disabled by default and needs to be enabled in the server settings, though keep in mind that it will not convert existing entries. Especially for new instances, we highly recommend switching to the new system!

    Event republish-alert flood protection

    As our communities grow and we all build our own internal tooling for processing data in MISP, the more likely it is to run into some slightly frustrating issues. One such issue we've encountered recently came from a tool that seems to have regularly (and frequently!) modified certain events and republished them consecutively. This in itself is not an issue, however, it can generate a lot of noise in terms of alert emails. We have now added a protective measure to counter this, make sure you have a look at the appropriate settings to create lockout timers for alerts that can be issued for a single event.

    Improvements

    • Event report hints autocomplete while typing in the Markdown has been improved
    • Server rules element improved
    • MISP modules results now point to the original object itself

    MISP Modules

    Two new MISP modules were introduced:

    • cof2misp module to allow the import of Passive DNS in JSON COF Format into MISP
    • An improved onyphe module to do expansion in MISP with full MISP object support

    Acknowledgement

    We would like to thank all the contributors, reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in misp-objects, misp-taxonomies and misp-galaxy .

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements.

    Source code(tar.gz)
    Source code(zip)
  • v2.4.142(Apr 28, 2021)

    MISP 2.4.142 released

    MISP 2.4.142 released including many new features, a security fix and a long list of quality of life improvements.

    Correlation changes

    One of the most annoying bottlenecks in how we use MISP currently is caused by low quality correlations, both in terms of usability and having a clear view on relevant relationships among data-points. These very often come from either sub-optimal strategies chosen on data creation/ingestion for certain types of attributes, but very often also on edge cases.

    With the current release we've included two main tools to combat this:

    Correlation exclusions

    We can now remove individual values from ever correlating again, so if you come across some typical noisy values (such as empty file hashes, registry values of 000000, internal IPs recurrinly encoded by your sandbox), you can add those to the exclusion list.

    Once added, you can execute the cleaning of the existing correlations, to retroactively execute your exclusion rules. This is a background processed task and depending on the amount of correlations you have may take quite some time (it took us around 30 minutes on 25M correlations), so just fire it off and check back later whether the job has completed.

    You can also comment your reason for removing an entry. In the future we plan on publishing community maintained default exclusion lists.

    Correlation exclusion in MISP

    Top correlations

    List the most correlating values in your instance - in order to evaluate which the most problematic correlations are, simply have a look at the most noisy correlations. We've had some surprising entries in our communities, so perfect time to do some spring cleaning.

    Just hit the delete button on a correlation and it will add a rule to your correlation exclusion list - just don't forget to run the historic cleanup from the correlation exclusion index to remove already existing correlations matching your newly added rules.

    Server sync rule management rework

    MISP server sync rule management

    One of the more painful aspects of managing servers has been the historically bad UI used to manage filter rules. This has now been completely revamped, both with a new look but familiar look and feel as well as some clever new tools to make it more usable.

    For example, when creating pull filters, your instance will now attempt to contact the remote instance to retrieve a list of available tags, so that you no longer have to manually enter all of the filters when creating pull rules. The JSON rule field allowing custom filters now also uses a handy JSON parsing text entry, allowing you to avoid potential mistakes.

    New dashboard widgets

    Thanks to Jeroen Pinoy, we have some new dashboard widgets meant to give you better oversight over how your instance is being used, showing some usage statistics as well as tools to monitor the growth of the user base of the community.

    A bunch of other fixes including security fixes

    We have also a security issue (CVE-2021-31780) causing a potential misalignment of sharing groups on synced attributes, so we highly encourage everyone to update their MISP instance.

    Besides that we have introduced a long list of quality of life improvements as well as many fixes.

    Acknowledgement

    We would like to thank all the contributors, reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in misp-objects, misp-taxonomies and misp-galaxy . The MISP galaxy includes a major update in the Ransomware galaxy which now includes more than 1600 documented ransomware.

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements.

    Source code(tar.gz)
    Source code(zip)
  • v2.4.141(Apr 19, 2021)

    MISP 2.4.141 released

    MISP 2.4.141 released including many improvements from email notification, UI, API and installation scripts.

    User-Interface

    • [UI] Render galaxy cluster description as markdown.
    • [UI] Show threat level icons on event index.
    • [eventgraph:viewPicture] Allow access to saved picture from the eventgraph history.
    • [eventGraph] Improved object coloring strategy.
    • [UI] fix debugon for debug = 1. fix #7131.
    • [UI] Show number of items in freetext feed.
    • [UI] Make feed event preview nicer.
    • [UI] It is 2021! Removed -moz and -webkit specific CSS properties.
    • [UI] Make some parts of MISP nicer.
    • [UI] Nicer pivots.
    • [UI] Simplify keyboard-shortcuts.js.
    • [UI] Use Page Visibility API.

    and many more updates check the changelog for details.

    Email notification

    Email notification has been significantly improved and now support HTML emails.

    • [email] New setting MISP.event_alert_metadata_only.
    • [email] Command for testing generated alert email.
    • [email] Allow to set email subject from template.
    • [email] Back-end support for sending HTML emails.

    This release includes many updates in the local and translation of the user-interface.

    New default feeds were included in MISP such as the newest DataPlane.org feeds.

    Installation scripts and guides

    Many improvement in the RHEL7, 7.9 and CentOS8Stream. We thank all the users reporting issues with RHEL.

    Acknowledgement

    We would like to thank all the contributors, reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in misp-objects, misp-taxonomies and misp-galaxy .

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements.

    Source code(tar.gz)
    Source code(zip)
  • v2.4.140(Mar 10, 2021)

    MISP 2.4.140 released

    We have released 2.4.140, the latest release for MISP, introducing a host of new features, including integrations with various authentication systems, various improvements to the handling of objects, CLI improvements as well as a package containing general bug fixes, along with the usual update of the JSON libraries.

    Manage my identity!

    MISP already had a host of integration options with various IDPs, but this release will give you some additional options, in the shape of OpenID Connect authentication and Azure Active Directory Authentication integrations. Have a look at the various authentication plugins' configuration in the MISP/app/Plugin directory.

    Built in security report of your MISP instance

    As of this release, you can get some guidance on the security posture and potential security impacting misconfigurations and best practices via the new security audit tool, locate in the diagnostics section of the server settings. Make sure you go through the tools findings and make any changes you find appropriate from the suggestions offered. When in doubt, feel free to start a discussion on the support chat hosted on gitter.

    The audit also gives you a sanity check over your CSP posture, used in junction with the new settings and tightened security measures.

    Massive kudos to @JakubOnderka for all this work!

    Cross referencing objects across extended events

    Whilst extended events were the most flexible way of creating counter analyses in MISP as well as being able to provide additional information to a report, we were always lacking a crucial component to make this feature truly shine: The ability to build connected graphs of the data points contained in a set of events extending one another. This has now been added to MISP as of 2.4.140.

    CLI improvements

    We want to make scripting and using the CLI in general a bit more straight-forward. Since the phasing out of the build in task scheduler, we find that there has been a massive uptick in the usage of these tools so expect more improvements in the future. For now, we have added tools to list the connect servers directly from the CLI, to be able to automate the sync process per connected server.

    Additionally, a new set of CLI tools is being built for developers, to ease our lives when trying to modify MISP. The first tool for this toolkit allows us to massage the direct feed description dumps to the expected format for easier modification.

    New types added in MISP

    New full-name, dkim and dkim-signature attribute types were added to MISP. Associated to DKIM objects were included to support tools such as Farsight Security dnsdb to add DKIM information in your investigations.

    Security Vulnerability

    An issue was discovered in app/Model/SharingGroupServer.php in MISP 2.4.139. In the implementation of Sharing Groups, the "all org" flag sometimes provided view access to unintended actors. Thanks to Jeroen Pinoy for the report. The vulnerability has CVE-2021-27904 assigned.

    Acknowledgement

    We would like to thank all the contributors, reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in misp-objects, misp-taxonomies and misp-galaxy .

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements.

    Source code(tar.gz)
    Source code(zip)
  • v2.4.139(Feb 16, 2021)

    MISP 2.4.139 released (Quality of life and bugfix release)

    We have released 2.4.139, the latest release for MISP squashes a set of pretty annoying bugs, whilst also adding some shiny new features to play with, along with the usual update of the JSON libraries.

    Besides that, several usability and performance issues have been resolved along with a host of small improvements, additional API improvements, etc. Make sure that you read the detailed changelog to see all the improvements.

    MISP modules are now Event Report aware!

    The Event Reports are the hot new feature of the past few weeks and we are working on ensuring that analyst reports are becoming the standard companions of the classic event format. For anyone that hasn't played with them before, have a look at the blog post describing how you can create rich, interlinked reports to accompany your events.

    The main update to the Event Report system is its inclusion in the module system as of this version, so if you are building integrations with MISP or simply want to build a convenient way to incorporate reports from your favourite information sources, this feature will make your life much easier.

    MISP modules can impose options for the event fetcher

    Want to restrict what parts of an event your module should receive from MISP? Would you like to include the decay score in your module? Pass parameters back to the fetcher so it can prepare an event that better fits your module's needs!

    EventStream widget

    The built in Dashboard system in MISP has been underutilised since its inception, partially due to its initial focus on a non CTI use-case. We have been working on remedying this over the past few months, including the addition of new widgets to monitor your instance's health as an administrator, to gain high level insights into your sharing community's sharing practices, etc.

    Something that has been missing for a while though was the ability to monitor ongoing trends based on your own interests, such as any new events coming in that relate to a topic that you are interested in. The EventStream widget aims to solve that, by offering a customisable event index widget.

    Users can set their interests in terms of organisation sources and applied tags (such as threat actor, tool and other names) to show the most recent additions that touch on the given subjects.

    This widget also brings a flexible reusable UI layer with itself that widget developers can reuse for a host of other use-cases.

    Acknowledgement

    We would like to thank all the contributors, reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in misp-objects, misp-taxonomies and misp-galaxy .

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements.

    Source code(tar.gz)
    Source code(zip)
  • v2.4.138(Feb 10, 2021)

    MISP 2.4.138 released

    We have released 2.4.138, the latest release for MISP along with an update of the JSON libraries.

    Besides that, several usability and performance issues have been resolved along with a host of small improvements, additional API improvements, etc. Make sure that you read the detailed changelog to see all the improvements. Improvements include the use of the threat level for the alert filtering, many bugs fixed in the event graph and many others.

    Nested Galaxy Element generator (CISA.gov/AIS dynamic marking)

    We have a new tool that allows you to take nested JSON documents and convert it to galaxy cluster elements using a dot delimited format. If you ever want to quickly encoding existing nested data for your custom galaxies, this should make your life easier. This functionality was integrated for the support of the Automated Indicator Sharing (AIS) from DHS/CISA.gov to include dynamic marking. The functionality can be reused for many different use-cases.

    RSIT galaxy added with MITRE ATT&CK

    Reference Security Incident Taxonomy Working Group, is a joint initiative for CSIRTs to produce a reference taxonomy for the CSIRT community. A new version of RIST has been integrated into MISP along with a complete set of relationships with MITRE ATT&CK, thanks to the galaxy 2.0 feature in MISP. Thanks to Koen Van Impe for this new updated galaxy.

    Acknowledgement

    We would like to thank all the contributors, reporters and users who have helped us in the past months to improve MISP and information sharing at large. This release includes multiple updates in misp-objects, misp-taxonomies and misp-galaxy .

    As always, a detailed and complete changelog is available with all the fixes, changes and improvements.

    Source code(tar.gz)
    Source code(zip)
Owner
MISP Project
MISP Project - Open Source Threat Intelligence Platform & Open Standards For Threat Information Sharing
MISP Project
A dumb sharing site for photos and videos, made by me, using a bit of borrowed code.

ShitShare A dumb sharing site for photos and videos, made by me, using a bit of borrowed code. Information This was made on Windows, so when video upl

null 1 Dec 14, 2022
ProjectSend is a free, clients-oriented, private file sharing web application.

ProjectSend is a free, open source software that lets you share files with your clients, focused on ease of use and privacy. It supports clients groups, system users roles, statistics, multiple languages, detailed logs... and much more!

ProjectSend 994 Jan 7, 2023
This is a plugin written in PHP programming language and running on the PocketMine platform that works stably on the API 4.0.0 platform. It allows you to query some other server information

QueryServer This is a plugin written in PHP programming language and running on the PocketMine platform that works stably on the API 4.0.0 platform. I

Thành Nhân 1 Jul 6, 2022
The platform allows you to manage articles, comments, tags, categories, and users for a blogging platform.

Laravel Blogging Platform The platform allows you to manage articles, comments, tags, categories, and users for a blogging platform. The project was w

Khaled Farhat 6 Oct 2, 2022
A platform to create documentation/wiki content built with PHP & Laravel

BookStack A platform for storing and organising information and documentation. Details for BookStack can be found on the official website at https://w

BookStackApp 10.6k Jan 3, 2023
Create Your Own Broadcast Network With AVideo Platform Open-Source. OAVP OVP

Audio Video Platform AVideo is a term that means absolutely nothing, or anything video. Since it doesn't mean anything the brand simply is identifiabl

World Wide Broadcast Network 1.7k Jan 8, 2023
Satu platform demo ringkas untuk rujukan Basic PHP

praktisphpmysql Satu platform demo ringkas untuk rujukan Basic PHP Demo https://legoom.biz.my/praktisphpmysql/ Belajar Koding Kemasukan Mei dan Septem

RB 11 Jun 21, 2022
Centreon is a network, system and application monitoring tool. Centreon is the only AIOps Platform Providing Holistic Visibility to Complex IT Workflows from Cloud to Edge.

Centreon - IT and Application monitoring software Introduction Centreon is one of the most flexible and powerful monitoring softwares on the market;

Centreon 14 Dec 16, 2022
Known: a social group platform

Known: a social group platform Installation One-click Known sites If you want to install on your own web space, we recommend Reclaim Hosting, which in

Known 972 Dec 28, 2022
CTFx is a CTF Platform forked from mellivora, that focuses on low memory footprint and low server CPU usage

CTFx is a CTF Platform forked from mellivora, that focuses on low memory footprint and low server CPU usage. It has a futuristic interface that's optimized for slower hardware, meaning that there is no bulky Javascript running in the background, nor length CSS stylesheets. CTFx improves on the mellivora CTF engine by the UI redesign and the addition of new features.

Milkdrop 7 Dec 22, 2022
Donation Platform for WooCommerce unleashes the power of WooCommerce for your online fundraising, crowdfunding & crowdsponsoring

=== Donation Platform for WooCommerce === Contributors: flinnn Tags: donation, donations, crowdfunding, fundraising, woocommerce Requires at least: 5.

Jonas Höbenreich 15 Dec 21, 2022
The API & platform builder, build your apps 10x faster even more, it's open source & 100% free !

The API & platform builder, build your apps 10x faster, even more. It's open source & 100% free ! Try live demo Why badaso ? 100% FREE - No need for e

Uasoft 1k Jan 2, 2023
A Blogging Platform with a built-in Feed Aggregator. Built with AngularJS and Laravel.

ReMark ReMark is an open source publishing platform built with the informed content creator in mind. It works as: A blogging platform A feed aggregato

Ren 4 Nov 1, 2019
A markdown based blogging platform

OpenPub V2 OpenPub is a markdown based blogging system. The original version of OpenPub was too experimental in the way it handled everything. This ma

Logan Bailey 4 Jun 17, 2021
This is a clone of Angry birds's homepage built using Wordpress, it is configured to run on the Pantheon platform.

Angry-Birds-UI-Clone Description This is a clone of Angry birds's homepage built using Wordpress, it is configured to run on the Pantheon platform. Li

Abir Bouhriz Daidj 3 Oct 20, 2021
A platform to create documentation/wiki content built with PHP & Laravel

BookStack A platform for storing and organising information and documentation. Details for BookStack can be found on the official website at https://w

BookStackApp 10.6k Dec 29, 2022
Implements RMA process in Sylius ecommerce platform

Madcoders Sylius RMA Plugin Sylius RMA (Return Merchandise Authorization) plugin by Madcoders enables customer to create return form and submit return

MADCODERS 4 Jun 24, 2022
Platform for Cryptocracy II, an online cryptic hunt.

Cryptocracy II Platform for Cryptocracy II, the second iteration of an independent cryptic hunt organised by students from DPS Dwarka and DPS RK Puram

Angad Singh 30 May 16, 2022
Self-hosted platform to keep and share your content: web links, posts, passwords and pictures.

Shaark is a self-hosted platform to keep and share your content: web links, posts, passwords and pictures. All of your data can be private, public or

MarceauKa 482 Dec 30, 2022