I seem to be having some trouble getting my first index crawled. I've installed meilisearch in an azure cloud deployment. In my local laravel app, I've required this package and meilisearch-php sdk package.

I created the index without issue and updated the table row to include {"meilisearch":{"url":"https:\/\/[SUBDOMAIN INSERTED].azurewebsites.net","apiKey":"[ADMIN KEY INSERTED]"}} where the two variables are populated with the correct information for my configuration. I have tested the Admin API key successfully in meilisearch on the web interface.

Initially when I ran php artisan site-search:crawl I received an error about casting (an array was expected for array_map but one of the functions was returning an array as contract model).

Doing some quick work to see if I could temporarily solve the problem with a simple cast, the issues just kept cascading down into other lines of code, leading me to believe that my installed versions were not meeting requirements.

I had installed the latest version of meilisearch, but I noticed a suggested version much lower. So I reverted back to meilisearch-php v. 0.17.2 and the casting issues went away.

However, now the crawl command is still throwing an error, saying that the Authorization header is missing. I'm at a loss at this point. I believe I've implemented all as instructed. Do you have any recommendations for me on where things might be going wrong?

UPDATE: Poking at this a little further and I can see that this is still due to the meilisearch-php compatibility. v0.17.2 uses the old authorization header which is deprecated in the docker instance I deployed. If I edit the meilisearch-php vendor files directly to update the header then the error pushes on to another compatibility issue.

So I guess what I need to know is for laravel-site-search v.1.2.0, what version of meilisearch-php is required? Because the requirements spec'd on the documentation site don't give any information on versioning, but there are clearly some compatibility issues that need to be resolved. Or perhaps you can update to allow compatibility with the newest meilisearch-php release?

This PR introduces the possibility to modify the URL of a page before it is indexed. This makes it possible to strip away unwanted query parameters for example.

Resolves #21

I've got an error saying directory does not exist in this package vendor directory after running php artisan view:cache. The log looks like the following:-

[2022-01-18 14:46:42] local.ERROR: The "/home/sail/example-app/vendor/spatie/laravel-site-search/src/../resources/views" directory does not exist. {"exception":"[object] (Symfony\\Component\\Finder\\Exception\\DirectoryNotFoundException(code: 0): The \"/home/sail/example-app/vendor/spatie/laravel-site-search/src/../resources/views\" directory does not exist. at /home/sail/example-app/vendor/symfony/finder/Finder.php:592) [stacktrace] #0 /home/sail/example-app/vendor/laravel/framework/src/Illuminate/Foundation/Console/ViewCacheCommand.php(67): Symfony\\Component\\Finder\\Finder->in() #1 /home/sail/example-app/vendor/laravel/framework/src/Illuminate/Foundation/Console/ViewCacheCommand.php(36): Illuminate\\Foundation\\Console\\ViewCacheCommand->bladeFilesIn() #2 /home/sail/example-app/vendor/laravel/framework/src/Illuminate/Collections/Traits/EnumeratesValues.php(242): Illuminate\\Foundation\\Console\\ViewCacheCommand->Illuminate\\Foundation\\Console\\{closure}() #3 /home/sail/example-app/vendor/laravel/framework/src/Illuminate/Foundation/Console/ViewCacheCommand.php(37): Illuminate\\Support\\Collection->each() #4 /home/sail/example-app/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): Illuminate\\Foundation\\Console\\ViewCacheCommand->handle() #5 /home/sail/example-app/vendor/laravel/framework/src/Illuminate/Container/Util.php(40): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}() #6 /home/sail/example-app/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\\Container\\Util::unwrapIfClosure() #7 /home/sail/example-app/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(37): Illuminate\\Container\\BoundMethod::callBoundMethod() #8 /home/sail/example-app/vendor/laravel/framework/src/Illuminate/Container/Container.php(653): Illuminate\\Container\\BoundMethod::call() #9 /home/sail/example-app/vendor/laravel/framework/src/Illuminate/Console/Command.php(136): Illuminate\\Container\\Container->call() #10 /home/sail/example-app/vendor/symfony/console/Command/Command.php(298): Illuminate\\Console\\Command->execute() #11 /home/sail/example-app/vendor/laravel/framework/src/Illuminate/Console/Command.php(121): Symfony\\Component\\Console\\Command\\Command->run() #12 /home/sail/example-app/vendor/symfony/console/Application.php(1005): Illuminate\\Console\\Command->run() #13 /home/sail/example-app/vendor/symfony/console/Application.php(299): Symfony\\Component\\Console\\Application->doRunCommand() #14 /home/sail/example-app/vendor/symfony/console/Application.php(171): Symfony\\Component\\Console\\Application->doRun() #15 /home/sail/example-app/vendor/laravel/framework/src/Illuminate/Console/Application.php(94): Symfony\\Component\\Console\\Application->run() #16 /home/sail/example-app/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(129): Illuminate\\Console\\Application->run() #17 /home/sail/example-app/artisan(37): Illuminate\\Foundation\\Console\\Kernel->handle() #18 {main} "}

When rendering the search results and trying to print the $hit->highlightedSnippet() nothing gets returned but $hit->snippet() does return something (unhighlighted of course).

Am I missing something?

I'm not sure why it's trying to request 127.0.0.1:7700

My .env file has MEILISEARCH_HOST=http://meilisearch:7700

In a sail shell, curl http://meilisearch:7700 works correctly. curl http://127.0.0.1:7700 does not.

I can also access the Meilisearch UI in my browser at http://localhost:7700.

Dispatching job to crawl `http://localhost:8085`

   MeiliSearch\Exceptions\CommunicationException 

  cURL error 7: Failed to connect to 127.0.0.1 port 7700 after 0 ms: Connection refused (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for http://127.0.0.1:7700/indexes

  at vendor/meilisearch/meilisearch-php/src/Http/Client.php:164
    160▕ 
    161▕         try {
    162▕             return $this->parseResponse($this->http->sendRequest($request));
    163▕         } catch (NetworkExceptionInterface $e) {
  ➜ 164▕             throw new CommunicationException($e->getMessage(), $e->getCode(), $e);
    165▕         }
    166▕     }
    167▕ 
    168▕     private function buildQueryString(array $queryParams = []): string

As others have noticed in a previous issue / discussion (https://github.com/spatie/laravel-site-search/issues/14 and https://github.com/spatie/laravel-site-search/discussions/16 respectively) that highlightedSnippet() was returning null but snippet() was returning content.

Looking at both the Meilisearch docs (https://docs.meilisearch.com/reference/features/search_parameters.html#attributes-to-highlight) and the properties of Spatie\SiteSearch\SearchResults\Hit, the highlighted properties will actually be within an array called _formatted, whereas this package was previously looking for highlightedDescription, highlightedEntry or highlightedH1 (assuming the default MeiliSearchDriver was in use).

The function itself will still either return null (if either _formatted isn't an array (which would only be the case if the attributesToHighlight search parameter is empty from what I can see) or the property name cannot be found in _formatted) or the string from the _formatted array.

Thanks

If for some reason fetching the robots.txt throws and exception the crawler stops. Setting ignoreRobots has no effect because fetching the file happens before checking if we want to ignore it or not.

Maybe we can catch any exception when trying to fetch the robots.txt file and then check if we want to honor it and it is not null?

What do you think @freekmurze?

• The IndexedUrlEvent was missing the visibility keywords for constructor property promotion.
• Added a missing space between the visibility and the property type.