A Laravel package to fetch Open Graph data of a website.

Hi Team, Before I speak about the issue, I would first like to say that the package does a very good job on almost all urls. Great job !

To speak about the issue I am facing, am having troubles only with retrieving og data for FB posts url. The result appears empty. However this is not the case when I use this package with twiiter urls, blogs, instagram urls. So I am here to seek some help in case am missing something in the way am accessing it.

TIA

Fixes a bug related to Issue #83 on main repository where after verifying the problem it was determined that SSL marked as Wildcard ( * ) can cause problems with "get_headers" function in php.

It was assumed previously that "FILTER_VALIDATE_URL" was being too strict, but later determined that SSL verification was failing.

Describe the bug verify_image_url Function is assuming twitter post images with ":orig", ":large" ":medium" or ":small" as invalid images and thus is sending empty image value on the fetch return array.

Generated example:

Array
(
    [0] => Array
        (
            [robots] => NOODP
            [msapplication-TileImage] => //abs.twimg.com/favicons/win8-tile-144.png
            [msapplication-TileColor] => #00aced
            [facebook-domain-verification] => moho2ug7zs57jijiywrewd8wb5a08h
            [swift-page-name] => permalink
            [swift-page-section] => permalink
            [al:ios:url] => twitter://status?id=1524828581536440320
            [al:ios:app_store_id] => 333903271
            [al:ios:app_name] => Twitter
            [al:android:url] => twitter://status?status_id=1524828581536440320
            [al:android:package] => com.twitter.android
            [al:android:app_name] => Twitter
            [type] => article
            [url] => https://twitter.com/MexicanSkynet/status/1524828581536440320
            [title] => SKYNΞT | 🔴 twitch.tv/mexicanskynet on Twitter
            [image] =>  
            [image:user_generated] => true
            [description] => “Hoy tenemos el mas pintoresco y tradicional torneo M7GP a las 9:00 pm CUU / 10:00 pm CDMX

Recuerda unirte al chat de voz: https://t.co/GGyxQJteMB
y recuerda sintonizarnos por: https://t.co/L0KmK5iRHt

#MarioKart8Deluxe #mariokart #nintendoswitch #twitchaffiliate”
            [site_name] => Twitter
            [fb:app_id] => 2231777543
        )

)

After commenting line 56 trough line 51 in src/OpenGraph.php it successfully generated the image url on this example:

Array
(
    [0] => Array
        (
            [robots] => NOODP
            [msapplication-TileImage] => //abs.twimg.com/favicons/win8-tile-144.png
            [msapplication-TileColor] => #00aced
            [facebook-domain-verification] => moho2ug7zs57jijiywrewd8wb5a08h
            [swift-page-name] => permalink
            [swift-page-section] => permalink
            [al:ios:url] => twitter://status?id=1524828581536440320
            [al:ios:app_store_id] => 333903271
            [al:ios:app_name] => Twitter
            [al:android:url] => twitter://status?status_id=1524828581536440320
            [al:android:package] => com.twitter.android
            [al:android:app_name] => Twitter
            [type] => article
            [url] => https://twitter.com/MexicanSkynet/status/1524828581536440320
            [title] => SKYNΞT | 🔴 twitch.tv/mexicanskynet on Twitter
            [image] => https://pbs.twimg.com/media/FSlHM2bWIAAQn-8.jpg:large
            [image:user_generated] => true
            [description] => “Hoy tenemos el mas pintoresco y tradicional torneo M7GP a las 9:00 pm CUU / 10:00 pm CDMX

Recuerda unirte al chat de voz: https://t.co/GGyxQJteMB
y recuerda sintonizarnos por: https://t.co/L0KmK5iRHt

#MarioKart8Deluxe #mariokart #nintendoswitch #twitchaffiliate”
            [site_name] => Twitter
            [fb:app_id] => 2231777543
        )

)

Expected behavior Url array key should be populated if a twitter post has an image.

** System ** Laravel 8 PHP 7.4

Side note anyone having problems getting metadata from socialmedia please use the following user agent set on this example: $opg_array = OpenGraph::fetch('URL', true, null, null, 'facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)');

Hello how are you? I started getting this error when using...

[2022-03-18 21:45:05] local.ERROR: Using $this when not in object context {"exception":"[object] (Error(code: 0): Using $this when not in object context at [...]back-end\vendor\shweshi\opengraph\src\OpenGraph.php:12)

Laravel 8.x My code: if (!$validator->fails()) { $data_group = OpenGraph::fetch($request->link); $validator->after(function ($validator) use ($data_group) { if(empty($data_group['title'])) { $validator->errors()->add('link', 'Error'); } }); } I don't understand its origin or how to solve it...

It doesn't seem to parse the open graph data from Netflix correctly.

When you fetch the open graph data from Netflix (any page on netflix.com) using OpenGraph::fetch('view-source:https://www.netflix.com/nl-en/title/70180387', true)

it only returns:

[
    "viewport" => "width=device-width, initial-scale=1",
]

while the tags seem to be present in the source:

<meta property="og:title" content="Homeland | Netflix"/>
<meta property="og:description" content="CIA analyst Carrie Mathison struggles with mental health and family issues while leading the war against terrorism in this political thriller series."/>
<meta property="og:url" content="https://www.netflix.com/nl-en/title/70180387" id="meta-url"/>
<meta property="og:image" content="https://occ-0-435-769.1.nflxso.net/dnm/api/v6/E8vDc_W8CLv7-yMQu8KMEC7Rrr8/AAAABecpPMdDJbTCAHnkTPNz5_MTnRfgAQ8NuGi1h2O_FZ5y-IcWYq-PZWQ9GhQ0zreFSDbmz48qepCON5qbqtFDq_wJn7dJ.jpg?r=286"/>
<meta property="al:ios:url" content="nflx://www.netflix.com/title/70180387?locale=en-NL"/>
<meta property="al:ios:app_store_id" content="363590051"/>
<meta property="al:ios:app_name" content="Netflix"/>
<meta property="al:android:url" content="nflx://www.netflix.com/title/70180387?locale=en-NL"/>
<meta property="al:android:package" content="com.netflix.mediaclient"/>
<meta property="al:android:app_name" content="Netflix"/>
<meta name="twitter:card" content="summary_large_image"/><meta name="twitter:site" content="@netflix"/>

Retrieve the error by using libxml_get_errors() perform logging or return the list of issues to the caller.

$html = $this->curl_get_contents($url, $lang);
        /**
         * parsing starts here:.
         */
        $doc = new DOMDocument();

        $libxml_previous_state = libxml_use_internal_errors(true);
        $doc->loadHTML('<?xml encoding="utf-8" ?>'.$html);

        //catch possible errors due to empty or malformed HTML
        Log::warning(libxml_get_errors())

        libxml_clear_errors();
        // restore previous state
        libxml_use_internal_errors($libxml_previous_state);

Using Log::warning(libxml_get_errors()) is not compatible with PHP 5.6

The package seems not to work with some website. for exemple this one : https://www.udemy.com/formation-a-la-photographie-pour-les-debutants/

I tried with other package and it seems to be a 403 error ... any clue on this ?

added fourth argument which can be used to pass additional options to LoadHTML() method, including options for suppressing warnings and errors; loadHTML() second argument is 0 (LIBXML_ERR_NONE flag) #64

using try...catch wasn't best idea since if there are a lot of DOMDocument::loadHTML() everything gets really slow.

the libxml_use_internal_errors(true) indicates that you're going to handle the errors and warnings ourselves and so they don'to mess up the output / slow down things.

This is not the same as the @ operator. The warnings get collected behind the scenes and afterwards you can retrieve them by using libxml_get_errors() in case you wish to perform logging or return the list of issues to the caller.

Whether or not you're using the collected warnings you should always clear the queue by calling libxml_clear_errors().

Some meta tag may use value instead of content property

Added get_meta_value to handle this kind of case, also when both content nor value property is present, will return null

Example case of meta value property: Medium read time meta data

<meta data-rh="true" name="twitter:label1" value="Reading time" />
<meta data-rh="true" name="twitter:data1" value="3 min read" />

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior: Run a test from cli that calls... $data = OpenGraph::fetch($url, true); // where url returns a 404 error. See error: shweshi\OpenGraph\Exceptions\FetchException. The requested URL returned error: 404 Not Found

Expected behavior I would have expected the error to have been failed gracefully, returning the 404 message to the calling method.

Desktop (please complete the following information):

• OS: MacOS with Docker/Laradock
• Browser n/a
• Version current/latest

Additional context I'm actually calling it with... but still not getting past the fetch error thrown.

try {
            $data = OpenGraph::fetch($url, true);
        } catch (shweshi\OpenGraph\Exceptions\FetchException $e) {
            $message = $e->getMessage();
            $data = $e->getData();
        }

Suggestion: I think, it will be a great addition if you add support for CURL requests through proxy, passed as parameter to fetch()

Reason: Sometimes, URL access is blocked or is slow from the server's country/location. In that case it's easier to setup/use a proxy server instead of creating a custom php solution from scratch.

In my case I fetch page html for further use (searching for other data) but also need to get open graph data, I could use this package if it allowed to pass html string instead of url.

Although not technically a meta tag, the title tag would be a handy addition as it is an arbitraty informative label too.

With a tiny addition between the $metadata decleration the $tags as $tag loop to create a$metadata['title'] key val pair, one could add the given title too.

        if($allMeta){
            $title = $doc->getElementsByTagName('title');
            $metadata['title'] = $title->length > 0 ? $title->item(0)->textContent : null ;
        }

By putting it here, should there be a metata tag called title too, it would override this value for the preffered one.

Good Morning,

I'm writing you because I have had a bug when obtaining the openGraph of a url, it does not get the image because it has escaped characters and it is not validated.

When using openGraph there are certain urls from which the image is not obtained correctly, and it is because a decode of the image should be added in the verify_image_url function.

Page example: https://www.forbes.com/sites/forbescoachescouncil/2017/04/19/networking-or-job-boards-nine-strategies-to-find-your-next-role/

Image url: https://thumbor.forbes.com/thumbor/fit-in/1200x0/filters%3Aformat%28jpg%29/https%3A%2F%2Fblogs-images.forbes.com%2Fforbescoachescouncil%2Ffiles%2F2017%2F04%2FNetworking-or-Job-Boards-Nine-Strategies-To-Land-Your-Next-Role.jpg

Solution: $url = urldecode ($url);

Thanks in advance,

Mirian