Searchable DataObjects is a module that permit to include DataObjects into frontend search

Overview

Searchable DataObjects

Searchable DataObjects is a module that permit to include DataObjects into frontend search.

Introduction

Pages are not always the better way to implement things. For example site news can grow rapidly and the first side effect would be a big and difficult to manage SiteTree. DataObjects help maintaining things clean and straight, but unfortunately they are not included in frontend search. This module let you insert DataObject in search.

Requirements

  • SilverStripe 4.1
  • g4b0/htmlpurifier

For SilverStripe 3.1 usage please referr to version 3.0 and below. For SilverStripe >3.1 & <4.0 usage please referr to version 4.x.

Installation

Install the module through composer:

composer require g4b0/searchable-dataobjects
composer update

Make the DataObject (or Pages) implement Searchable interface (you need to implement Link(), getSearchFilter(), getTitleFields(), getContentFields()):

Note: getSearchFilterByCallback() is an optional filter. If you don't plan on calculating any value to determine a returned true or false value it is suggested you don't add this function to your DataObject or Page type.

use g4b0\SearchableDataObjects\Searchable;

class DoNews extends DataObject implements Searchable {

    private static $db = array(
        'Title' => 'Varchar',
        'Subtitle' => 'Varchar',
        'News' => 'HTMLText',
        'Date' => 'Date',
    );
    private static $has_one = array(
        'Page' => 'PghNews'
    );

    /**
     * Link to this DO
     * @return string
     */
    public function Link() {
        return $this->Page()->Link() . 'read/' . $this->ID;
    }

    /**
     * Filter array
     * eg. array('Disabled' => 0);
     * @return array
     */
    public static function getSearchFilter() {
        return array();
    }

    /**
     * FilterAny array (optional)
     * eg. array('Disabled' => 0, 'Override' => 1);
     * @return array
     */
    public static function getSearchFilterAny() {
        return array();
    }

    /**
     * FilterByCallback function (optional)
     * eg. function($object){
     *  return ($object->StartDate > date('Y-m-d') || $object->isStillRecurring());
     * };
     * @return array
     */
    public static function getSearchFilterByCallback() {
        return function($object){ return true; };
    }

    /**
     * Fields that compose the Title
     * eg. array('Title', 'Subtitle');
     * @return array
     */
    public function getTitleFields() {
        return array('Title');
    }

    /**
     * Fields that compose the Content
     * eg. array('Teaser', 'Content');
     * @return array
     */
    public function getContentFields() {
        return array('Subtitle', 'Content');
    }
}

Here you are a sample page holder, needed to implement the Link() function into the DataObject:

class PghNews extends Page {

    private static $has_many = array(
        'News' => 'DoNews'
    );

    public function getCMSFields() {
        $fields = parent::getCMSFields();

        /* News */
        $gridFieldConfig = GridFieldConfig_RelationEditor::create(100);
        // Remove unlink
        $gridFieldConfig->removeComponentsByType('GridFieldDeleteAction');
        // Add delete
        $gridFieldConfig->addComponents(new GridFieldDeleteAction());
        // Remove autocompleter
        $gridFieldConfig->removeComponentsByType('GridFieldAddExistingAutocompleter');
        $field = new GridField('Faq', 'Faq', $this->News(), $gridFieldConfig);
        $fields->addFieldToTab('Root.News', $field);

        return $fields;
    }
}

class PghNews_Controller extends Page_Controller {

    private static $allowed_actions = array(
        'read'
    );

    public function read(SS_HTTPRequest $request) {
        $arguments = $request->allParams();
        $id = $arguments['ID'];

        // Identifico la faq dall'ID
        $Object = DataObject::get_by_id('DoNews', $id);

        if ($Object) {
            //Popolo l'array con il DataObject da visualizzare
            $Data = array($Object->class => $Object);
            $this->data()->Title = $Object->Title;

            $retVal = $this->Customise($Data);
            return $retVal;
        } else {
            //Not found
            return $this->httpError(404, 'Not found');
        }
    }
}

Extend Page and the desired DataObjects through the following yaml:

Page:
  extensions:
    - g4b0\SearchableDataObjects\SearchableDataObject
DoNews:
  extensions:
    - g4b0\SearchableDataObjects\SearchableDataObject

Run a dev/build and then populate the search table running PopulateSearch task:

sake dev/build "flush=all"
sake dev/tasks/PopulateSearch

Enjoy the news into the search results :)

Modifying

Set the number of search results per page

Setting the g4b0\SearchableDataObjects\CustomSearch.items_per_page config setting you can define, how many search results per page are shown. Default is 10

By default the search result is shown at the same page, so if you're searching e.g. on the /about-us/, the results are shown on /about-us/SearchForm/?s=foo. If you don't like that, you can define any Page or Controller class in the g4b0\SearchableDataObjects\CustomSearch.search_controller setting. If you set this setting to this, the current page will be used. Defaults to SearchPage and falls back to the current page if no SearchPage is found.

g4b0\SearchableDataObjects\CustomSearch:
  items_per_page: 15
  search_controller: g4b0\SearchableDataObjects\SearchPage #page type to show the search

Note

Searchable DataObjects module use Mysql NATURAL LANGUAGE MODE search method, so during your tests be sure not to have all DataObjetcs with the same content, since words that are present in 50% or more of the rows are considered common and do not match.

From MySQL manual entry [http://dev.mysql.com/doc/refman/5.1/en/fulltext-search.html]:

A natural language search interprets the search string as a phrase in natural human language (a phrase in free text). There are no special operators. The stopword list applies. In addition, words that are present in 50% or more of the rows are considered common and do not match. Full-text searches are natural language searches if the IN NATURAL LANGUAGE MODE modifier is given or if no modifier is given.

TODO

  • Add other search method in configuration

Suggested modules

Comments
  • SS 3.3.1: tons of

    SS 3.3.1: tons of "Title" indexes

    see #23

    On SS 3.3.1 i see tons of "Title" indexes after running dev/build. It creates 20 new indexes when running dev/build once.

    I'm on dev-master 7c0220f

    opened by wernerkrauss 9
  • Force case insensitiveness on search

    Force case insensitiveness on search

    MATCH...AGAINST uses the actual collation, i.e. if your table has e.g. utf8_unicode_ci the search will be case insensitive but if you have utf8_bin (or any other _bin collation) the comparation will be case sensitive.

    Although SilverStripe says otherwise (utf8_general_ci), the tables are created preserving the default collation of the database. Tested experimentally with 3.3.0.

    opened by ntd 7
  • SearchableDataObject DB table has multiple keys/indexes

    SearchableDataObject DB table has multiple keys/indexes

    I am getting the following error when attempting to /dev/build with the silverstripe-searchable-dataobjects module installed:

    ALTER TABLE SearchableDataObjects ADD FULLTEXT (`Title` ,`Content`) | Too many keys specified; max 64 keys allowed
    

    screen shot 2015-10-07 at 11 08 00

    Based on the error I am assuming the issue is in someway related to the following line in SearchableDataObject::augmentDatabase():

    DB::query("ALTER TABLE SearchableDataObjects ADD FULLTEXT (Title,Content)");

    I am guessing this query is being run multiple times and a new index/key is being created in the DB each time it is run.

    As you can see from the following screenshots (taken in Navicat as MySQL DB management tool) after initially installing the module running /dev/build and /dev/tasks/PopulateSearch I have 24 indexes, then some time later (possibly having changed the current locale for the site) I run another /dev/build and this attempts to increase to > 63 indexes which is what triggers the SQL error.

    After initial set up: screen shot 2015-10-06 at 19 11 28

    After a subsequent /dev/build some time later (having switched locale in my config files): screen shot 2015-10-06 at 18 59 13

    I am not familiar with the augmentDatabase() method or where and when it is called otherwise I would investigate further and propose a fix.

    This article may be of some help, at very least it helped me track down the issue itself - https://www.safaribooksonline.com/blog/2012/11/06/avoiding-too-many-keys-in-mysql/

    Any ideas?

    opened by HARVS1789UK 7
  • Use ORM to create field specs

    Use ORM to create field specs

    Use the DB connector to define the field specs so collation settings are included.

    • the fixed ID field spec was left as is (unsigned) but could be reviewed in the future
    opened by rotassator 5
  • Upgrade to SS4

    Upgrade to SS4

    First of all, thanks alot for writing this module.

    Our web agency maintains a modified fork of your module. As we need our version of the module to be SS4 compatible I thought I'd PR this back to the original.

    I'm using SS 4.1 to build this.

    Some notes about the upgrade:

    • Moved the code folder to src to match the SS4 module standards
    • Updated composer.json to SS4. It may work with SS3 but I haven't tested this.
    • I've added namespacing to all classes (assummed Zirak/SearchableDataObjects would be fine here but this can be changed)
    • Updated any code that broke due to changes with the SS4 API
    • Instead of dropping the SearchableDataObjects table, I truncate it. Notes here https://github.com/g4b0/silverstripe-searchable-dataobjects/commit/ae4a0d8edfb07346e60c52c24f754c1b36342cec
    • Tests do not work in SS 4.1 - I believe this bug is to blame https://github.com/silverstripe/silverstripe-framework/issues/7978. Once this is fixed the tests will need to be looked at again.
    • FulltextSearchable::enable() adds the ContentControllerSearchExtension extension and overrides your CustomSearch extension. I've gotten around this but wonder if there is a better way of doing this. Notes here https://github.com/g4b0/silverstripe-searchable-dataobjects/commit/bd5de0ba1160f6f37b0dda25c00fbad8952d7d89
    • This code also references a namespaced HTMLPurifier - I can add another PR in your other repo if you want.

    The other thing I noticed in my testing was the PageId column in SearchableDataObjects was always 0. Not sure if this is correct?

    Happy to make any changes you want. Thanks!

    opened by antons- 4
  • Return 1 Page when Multiple DataObject ComeBack

    Return 1 Page when Multiple DataObject ComeBack

    I'm using the modules to search content blocks on my site, but it returns a seperate entry for each dataobject, I'd like to only return the page once if the word is found. Is this possible?

    opened by thezenmonkey 4
  • readme.md example DataObject class code bug

    readme.md example DataObject class code bug

    Hi Gabriele,

    Just getting to grips with your module which looks good so far, one thing that has had me stuck trying to get it working for some time was that the readme.md example code of how to set up a DataObject to be searchable contains the following:

        /**
         * FilterByCallback function (optional)
         * eg. function($object){
         *  return ($object->StartDate > date('Y-m-d') || $object->isStillRecurring());
         * };
         * @return array
         */
        public static function getSearchFilterByCallback() {
            return function($object){};
        }
    

    having investigated, I can see that the PopulateSearch class (which seems to be run via /dev/tasks/PopulateSearch and onAfterWrite of any DataObject which is being made searchable) contains the following conditional statement for each DataObject marked as being searchable:

    if(method_exists($class, 'getSearchFilterByCallback')){
                    $dos = $dos->filterByCallback($class::getSearchFilterByCallback());
    }
    

    Although I do appreciate your comments above the getSearchFilterByCallback() method do provide an example of a callback method, I had assumed that the default function in your example ( function($object){}; ) would have ben sufficient, but the lack of any return statement means that this example effectively returns false every time it is run.

    This means that none of my DataObjects were being stored in the SearchableDataObjects database table (as getSearchFilterByCallback() was excluding them all) and therefore none were showing up in the search.

    I would suggest making the example method contain return true; would be a logical addition and hopefully save others some time and not following the same path as I did. For example:

        /**
         * FilterByCallback function (optional)
         * eg. function($object){
         *  return ($object->StartDate > date('Y-m-d') || $object->isStillRecurring());
         * };
         * @return array
         */
        public static function getSearchFilterByCallback() {
            return function($object){return true;};
        }
    
    opened by HARVS1789UK 4
  • BUG SQL where statement breaks extended data objects

    BUG SQL where statement breaks extended data objects

    I have a dataobject called Interaction that implements the SearchableLinkable class, and then I have other dataobjects that extend Interaction like so:

    class Interaction extends DataObject implements SearchableLinkable {
    
    }
    
    class Exhibition extends Interaction {
    
    }
    

    Currently if I save an Exhibition data object in the CMS I get this error:

    ERROR [User Error]: Couldn't run query: 
    SELECT DISTINCT "Interaction"."ClassName", "Interaction"."Created", "Interaction"."LastEdited", "Interaction"
    ."Title", "Interaction"."Content", "Interaction"."DateStart", "Interaction"."DateEnd", "Interaction"
    ."Type", "Interaction"."SortID", "Interaction"."URLSegment", "Interaction"."FeaturedImageID", "Interaction"
    ."VeeziURLID", "Interaction"."PageID", "Interaction"."ID", CASE WHEN "Interaction"."ClassName" IS NOT
     NULL THEN "Interaction"."ClassName" ELSE 'Interaction' END AS "RecordClassName"
    FROM "Interaction"
    WHERE (`Exhibition`.`ID`=1) AND ("Interaction"."ClassName" IN ('Exhibition'))
    ORDER BY "Interaction"."SortID" ASC
    LIMIT 1 
    
    Unknown column 'Exhibition.ID' in 'where clause'
    IN POST /om/admin/pages/edit/EditForm/field/Exhibitions/item/1/ItemEditForm
    Line 598 in /var/www/om/framework/model/MySQLDatabase.php
    

    The problem is that when the SearchableDataObject->onAfterWrite() is called on an extended data object that doesn't inherit SearchableLinkable directly, $this->owner->class is the incorrect value to use, and instead it should return $this->owner->parentClass as data objects that extend another are actually stored in the same table as the parent data object in the database.

    My PR corrects the where clause in this particular scenario by first checking if the data object has a parent class, and if so, use that instead of the data object's class.

    opened by antons- 3
  • Prevent PHP Strict Notice in CustomSearch::getSearchResults()

    Prevent PHP Strict Notice in CustomSearch::getSearchResults()

    Prevent PHP Strict Notice when DataObject::get_by_id() fails to return a valid DataObject in CustomSearch::getSearchResults().

    CustomSearch::getSearchResults() previously assumed that all rows of the SearchableDataObjects DB table will successfully return a valid DataObject when executing the following:

    $do = DataObject::get_by_id($row['ClassName'], $row['ID']);

    There are scenarios when this will return false (meaning $do contains the value of false) and then the following two lines will throw a PHP Strict Notice (as you cannot use the ->property syntax on a variable which contains anything but an object and false !== an object):

    screen shot 2015-10-07 at 17 28 56

    I identified this issue as I had upgraded my SS blog module to a newer version where the BlogEntry class was replaced with BlogPost. My database therefore contained a number of outdated values for the DB's SiteTree table ClassName fields (which read BlogEntry instead of BlogPost). This in turn meant my SearchableDataObjects DB table was full of rows containing ClassName values of BlogEntry.

    This meant numerous calls to DataObject::get_by_id("BlogEntry", 123); which returns false, presumably because the BlogEntry class no longer exists in the code base.

    I expect there will be other instances where peoples databases will contain outdated, inaccurate data and this will end up in the SearchableDataObjects table, causing the same issue.

    opened by HARVS1789UK 3
  • mysql error on a fresh install

    mysql error on a fresh install

    tryting to deploy my website i get the following error on the very first dev/build on a clean database:

    [User Error] Couldn't run query: INSERT INTO SearchableDataObjects (ID, ClassName, Title, Content) VALUES (2, 'Page', 'Über uns', 'Sie können diese Seite mit Ihren eigenen Inhalten füllen, oder sie löschen und Ihre eigenen Seiten erstellen.') ON DUPLICATE KEY UPDATE Title='Über uns', Content='Sie können diese Seite mit Ihren eigenen Inhalten füllen, oder sie löschen und Ihre eigenen Seiten erstellen.' Table 'silverstripe.SearchableDataObjects' doesn't exist

    Guess it's cause onAfterPublish hook tries to update the non existent table. Any way to check if the table exists before trying to update?

    opened by wernerkrauss 3
  • SearchForm results() function exposed as controller action

    SearchForm results() function exposed as controller action

    Browsing to a page ending in ~/results causes the following error:

    [Error] in searchable-dataobjects/code/CustomSearch.php:110 (http://testdomain.com/page/results)
    

    The results() function is being exposed to the controller as an $allowed_action, but should only be available to the form processor.

    Using SilverStripe 3.6.5.

    Issue also exists in SilverStripe CMS: https://github.com/silverstripe/silverstripe-cms/issues/2125

    opened by rotassator 2
  • Error : Class 'HTMLPurifier_Config' not found

    Error : Class 'HTMLPurifier_Config' not found

    Running Task Populate Search

    [Emergency] Uncaught Error: Class 'HTMLPurifier_Config' not found GET /dev/tasks/PopulateSearch

    any help? OK. I found out: Thats more an issue (?) on Purifier.php

    i had to add a require_once right after 'namespace g4b0\HtmlPurifier;':

    namespace g4b0\HtmlPurifier;
    require_once '../vendor/ezyang/htmlpurifier/library/HTMLPurifier.auto.php';
    

    sepp.

    opened by seppzzz 1
  • Possibility to search substrings / use wildcard

    Possibility to search substrings / use wildcard

    At the moment mysql only returns full words found.

    In German language we have a lot of combined words, like "Audittermine" which is combined by "Audit" and "Termine"

    It would be great if a search for "Audit" would also match "Audittermine", or search for singular "Termin" would also match "Termine", maybe configurable?

    From a short research we'd need a wildcard in the query and switch to BOOLEAN MODE, see http://stackoverflow.com/a/9284693/4137738

    opened by wernerkrauss 0
  • Possibility to filter by language or Subsite?

    Possibility to filter by language or Subsite?

    Any chance to get additional fields in the search table, e.g. for subsite/multisite environment or for locale?

    How could we easily alter the seach DB? Should we use a DataObject instead? You could still query it manually... Or just with extensions hooks when we generate the table / insert / search ???

    opened by wernerkrauss 3
Owner
Gabriele Brosulo
Full stack web developer @zirak.it
Gabriele Brosulo
Silverstripe-ideannotator - Generate docblocks for DataObjects, Page, PageControllers and (Data)Extensions

silverstripe-ideannotator This module generates @property, @method and @mixin tags for DataObjects, PageControllers and (Data)Extensions, so ide's lik

SilverLeague 44 Dec 21, 2022
This Magento 2 module adds the option to use Flagpack icons in your Hyvä frontend.

Siteation - Hyva Icon Pack - Flags This Magento 2 module adds the option to use Flagpack icons in your Hyvä frontend. This requires that you have a wo

Siteation 5 Jun 12, 2023
search non profitable charity or organization through api search

Non Profile Charity Search Search non profitable organization or get the details of an organization Installation Require the package using composer: c

Touhidur Rahman 5 Jan 20, 2022
Nova Search is an open source search engine developed by the Artado Project.

Loli Search Loli Search açık kaynak kodlu bir arama motorudur ve yalnızca kendi sonuçlarını değil, diğer arama motorlarının sonuçlarını da göstermekte

Artado Project 10 Jul 22, 2022
Doogle is a search engine and web crawler which can search indexed websites and images

Doogle Doogle is a search engine and web crawler which can search indexed websites and images, and then use keywords to be searched later. Written pri

Zepher Ashe 9 Jan 1, 2023
Search faster into Magento DevDocs and Magento StackExchange! 🔍⚡️

VS Code - Magento DevSearch Search faster into Magento DevDocs and Magento StackExchange! ?? ⚡️ Magento DevSearch is a VS Code extension that allows y

Rafael Corrêa Gomes 12 Oct 18, 2022
This is the information I prepared for the PHP interview.The notes include PHP, MySql, Linux, etc.

PHP面试准备的资料 这个项目是自己准备面试整理的资料。可能包括PHP、MySQL等资料。方便自己以后查阅,会不定期更新,如果错误,请指出,谢谢。欢迎大家提交PR,谢谢大家的star 可以通过https://xianyunyh.gitbooks.io/php-interview/预览。欢迎有精力的朋

Troy 1.2k Dec 24, 2022
A super simple, clean and pretty error handler that replace the default error handler of PHP. You need only include this file!

php-custom-error-handler A super simple, clean and pretty error handler that replace the default error handler of PHP. You need just include only this

null 6 Nov 7, 2022
Magento 2 Blog Extension is a better blog extension for Magento 2 platform. These include all useful features of Wordpress CMS

Magento 2 Blog extension FREE Magento 2 Better Blog by Mageplaza is integrated right into the Magento backend so you can manage your blog and your e-c

Mageplaza 113 Dec 14, 2022
Laravel & MySQL, jQuery, Ajax, Bootstrap. Also, it's include email send function without any API.

Rewards-Dacor Laravel & MySQL, jQuery, Ajax, Bootstrap. Also, it's include email send function without any API. [Live site link] ( https://rewardsdaco

Professional Developer 13 Dec 26, 2022
Magento 2 Module for Search Engine Optimization

Magento 2 Search Engine Optimization Magento 2 Module to Improve Search Engine Optimization (SEO) on your Magento site. Installation Install the modul

Stämpfli AG 100 Oct 7, 2022
A first party module to integrate Elastic App Search in Magento 2.

A first-party Magento integration for building excellent, relevant search experiences with Elastic App Search. ⚠️ This is a beta version of the client

elastic 25 Apr 22, 2022
This AddOn allows you to search for module names when adding a block

Modulsuche und Modulvorschau für REDAXO 5 Dieses AddOn ermöglicht die Suche nach Modulnamen, wenn man einen Block hinzufügt. Dies kann sehr hilfreich

Friends Of REDAXO 15 Nov 30, 2022
The MX_PhinxMigrations module integrates Phinx database migrations into Magento 2

MX Phinx Migrations About The MX_PhinxMigrations module integrates Phinx database migrations into Magento 2 as a replacement for the built-in setup:up

Inviqa 34 Jul 30, 2021
A SilverStripe module for conveniently injecting JSON-LD metadata into the header of each rendered page in SilverStripe

A SilverStripe module for conveniently injecting JSON-LD metadata into the header of each rendered page in Silver

null 4 Apr 20, 2022
Magento 2 Module Experius Page Not Found 404. This module saves all 404 url to a database table

Magento 2 Module Experius Page Not Found 404 This module saves all 404 urls to a database table. Adds an admin grid with 404s It includes a count so y

Experius 28 Dec 9, 2022
Experimenting with Zabbix frontend modules

Modules Request for new example can be left as issue for this repository, will do my best to implement if it is possible. Extending host popup menu Ac

null 1 Sep 19, 2022
Cbe frontauth - A Textpattern plugin to manage backend connections from frontend and protect content from non-logged users

cbe_frontauth This client-side plugin lets your users (or you) manage backend connection from frontend, i.e. connect and disconnect as they (you) woul

null 4 Jan 31, 2020