Elasticsearch driver for Laravel Scout

| Software | Version | --------------| --------------- | PHP | 7.3.22 | Elasticsearch | 7.9.2 | Laravel | 6.20.8 | Laravel Scout | 8.6.1

Describe the bug When I try to use makeAllSearchableUsing function , I understand that this feature aims to eager load relations for mass indexing. If I test it without queue system, It works like a charm. But when I add queue system and try to load 10 millions rows it seems it does 30 millions sql queries. In fact it does not eager load relations anymore.

I don't know if it is a scout bug, but I also reported it there : #https://github.com/laravel/scout/issues/462 In case I do things the right way, it is a major bug as the makeAllSearchableUsing feature is not usable for large set of data.

@babenkoivan What do you think ?

I'm trying to do a join by setting a parent/child relationship and I must set the rooting with the parent id in the URL to do it as explained in the documentation of ElasticSearch[1]. I've seen that ElasticAdapter only take body parameter before calling bulk function and no parameter is provided for rooting[2]. So is it possible to do it ? I can fork your work and try to resolve the problem by making a commit in few time if you want.

[1] https://www.elastic.co/guide/en/elasticsearch/reference/current/parent-join.html [2] https://github.com/babenkoivan/elastic-adapter/blob/master/src/Documents/DocumentManager.php#L34

When deleting models from search, we do not need to provide Elasticsearch with the full model data, just the "_id" field so that Elasticsearch knows which document to delete. Calling toSearchableArray on models that have been deleted from the database can cause issues if relations etc cannot be hydrated, and also creates an unnecessary database call. This commit adds an optional deleteMode argument to DocumentFactory. If this argument is true, then DocumentFactory no longer calls toSearchableArray on the models.

Add a CustomRouting interface that exposes a getRoutingKey method to be implemented by developers. The routing key provided by this method is then provided to Documents and used in index and delete operations.

This pr is linked to the elastic-adapter pr https://github.com/babenkoivan/elastic-adapter/pull/9 and we should first merge the latter. The changes on elastic-adapter should be backwards compatible and so no major version upgrade should be required.

| Software | Version | --------------| --------------- | PHP | 7.4.19 | Elasticsearch | 7.10.0 | Laravel | 8.44.0 | Laravel Scout | 9.1.0

Describe the bug Created a user record get the error: "One or more operations in the bulk request did not complete successfully" on saving

To Reproduce Fresh docker image of elastic search 7.10 (cluster status: green - 3 nodes)

php artisan scout:index user_index

Current behavior Error when saving

exception: "ElasticAdapter\\Exceptions\\BulkRequestException"
file: "<-snip->/vendor/babenkoivan/elastic-adapter/src/Documents/DocumentManager.php"
line: 51
message: "One or more operations in the bulk request did not complete successfully"

User model has trait Searchable and these methods:

  public function searchableAs()
   {
       return 'users_index';
   }

   public function toSearchableArray()
   {
       $array = $this->toArray();

       // Customize the data array...

       unset($array['password']);
       unset($array['remember_token']);

       return $array;
   }


   public function getScoutKey()
   {
       return $this->email;
   }

   public function getScoutKeyName()
   {
       return 'email';
   }

Record hits db but doesn't end up in the ES index

Expected behavior Expected save to be successful and record stored in both DB & ElasticSearch

Hi there!

I'm wondering if you may accept a PR removing the final declarations? I'm unable to extend anything in this package (or the scout-driver-plus package) without copying and pasting the whole class over and binding onto the interface -- negating the benefits of having this package be open source and community supported, since I can't utilize any of their base methods.

Is there a particular purpose for all classes to be final?

Thanks for your time!

Hey!

Thanks for writing some really great software here.

Just wondering if there are plans to upgrade to ES8? Will help me figure out if I need to write my own adapters or not as we are planning on upgrading.

I havent tested with ES8 yet. Its quite possible that things still work. But thought Id reach out as a first step to get your thoughts.

Cheers Lee

I need to make the following query but it appears as though I cannot do this in the base package.

            [
                'multi_match' => [
                    'query' => $queryText,
                    'type' => 'cross_fields',
                    'operator' => 'and',
                    'fields' => ['*']
                ],
            ];

I am aware of the "plus" driver, but the builders generated by that package do not create Scout Builder interfaces that allow it to work with Nova.

I have a fork in which I allowed array queries to be interpreted as raw searches and it appears to work fine for my use case.

// SearchRequestFactory.php
    protected function makeQuery(Builder $builder): array
    {
        if (is_array($builder->query)) {
          return $builder->query;
        }

        $query = [
            'bool' => [],
        ];

        if (!empty($builder->query)) {
            $query['bool']['must'] = [
                'query_string' => [
                    'query' => $builder->query,
                ],
            ];
        } else {
            $query['bool']['must'] = [
                'match_all' => new stdClass(),
            ];
        }

        if ($filter = $this->makeFilter($builder)) {
            $query['bool']['filter'] = $filter;
        }

        return $query;
    }

Description

Currently this scout driver throws an error when adding documents to an index that doesn't exist. By implementing this feature this elastic driver would create an empty index (without custom configuration) before populating it instead of throwing the error that the index doesn't exist. It would be fine to add a configuration option in the config/elastic.client.php that allows this.

This comment https://github.com/babenkoivan/elastic-scout-driver/issues/5 suggests to use elastic migrations. But an automatic solution would - in my use case - be better. It would only need to create an empty index before populating it. I don't need any custom configuration for my indices. Adding an extra package isn't needed for me since that package is better suited for projects that need custom index configurations.

Why

In our use case we have multiple projects (with multiple environments per project) creating indices on the fly. On average we have 5 indices per project per environment (we have 3 environments per project, so 15 indices per project on average). Currently we need to manually setup each index before we can use it. It would be a lot simpler if this package would create them on the fly, since we do not need a custom configuration, instead of throwing an error if it doesn't exist.

Also since other Laravel scout packages automatically create indices on the fly, it would therefore be logical that this package also does that since developers have a certain expectation of scout drivers.

I allow users to save events on my site. Everything works great unless the ES nodes go down. Then if a user tries to save something it sends back the error no alive nodes on your server and wont let them do anything. I figure this is how its supposed to work, but is there a way to allow the app to continue to save to the database even if the ES save doesn't work because the server is down?

| Software | Version | --------------| --------------- | PHP | 7.3.19 | Elasticsearch | 7.1.0 | Laravel | 6.* | Laravel Scout | 8.0

Describe the bug hello , im using at mongodb from jenssegers/mongodb package .

after run command : php artisan scout:import "App\product"

show result : Imported [App\product] models up to ID: 5e35949da49f04517147be86 Imported [App\product] models up to ID: 5e35949da49f04517147c07c Imported [App\product] models up to ID: 5e35949ea49f04517147c277 . . . Imported [App\product] models up to ID: 5ee4ac112407e012ce3bed42 Imported [App\product] models up to ID: 5ee89e5588d2b237f6052402 Imported [App\product] models up to ID: 5eeb3ac95083517cc7581892 All [App\product] records have been imported

but command : curl 'localhost:9200/_cat/indices?v'

show result:

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size yellow open products RIuEwO2_T6SBIwJviIurBA 1 1 0 0 283b 283b

( docs.count = 0 ) :(

my model :

use Illuminate\Database\Eloquent\Model; use Laravel\Scout\Searchable; use Jenssegers\Mongodb\Eloquent\Model as Eloquent;

`class products extends Eloquent { use Searchable; protected $collection = 'products';

public function shouldBeSearchable()
{
    return count($this->toSearchableArray()) > 0;
}

public function toSearchableArray()
{
    $array = $this->toArray();

    // Customize array...

    return $array;
}

}`

scout driver :

'driver' => env('SCOUT_DRIVER', 'elastic'),

my code in web.php :

Route::get('/', function () { $res = App\products::search('Asus')->get(); dd($res); });

But show result :

Illuminate\Database\Eloquent\Collection {#270 ▼ #items: [] }

please help me :(