rah_sitemap

Packagist | Issues | Donate

Sitemap plugin for Textpattern CMS. Generates Sitemaps.org XML sitemaps for your site, which help Google and other search engines to index your valuable content. Rah_sitemap maps your categories, sections, articles and even custom URLs of your choosing, and what is best, none of it requires diving into code. All configuration is done from a clean graphical user-interface.

Install

Using Composer:

$ composer require rah/rah_sitemap

Or download an installer package.

Basics

Rah_sitemap generates a sitemap for your Textpattern website, listing all of its section-, article category- and article-pages. The generated sitemap follows the XML based Sitemap protocol format and is targeted to search engines, opposed to your visitors. The sitemap is meant to help search engines to index your site as it grows and gets more and more various nested pages.

The Sitemap can be configured directly from Textpattern’s Preferences panel, making rah_sitemap very easy to setup and use. The plugin itself takes care of the rest.

Accessing the sitemap

The generated sitemap becomes publicly accessible from the site’s root. The sitemap can be accessed from two URLs, depending on the site’s permanent link mode. If the site is configured to use clean URLs, the sitemap can be accessed using a clean path like http://example.com/sitemap.xml, where the example.com would be the site’s URL. Additionally a query version, http://example.com/?rah_sitemap=sitemap, is available in both the messy and the clean URL modes, and can be used if the other one isn’t available.

Immediate boost to search engine visiblity?

Not exactly. Providing Sitemap is to help crawling, finding your site’s pages that could otherwise be hard to discover. Sitemaps itself do not boost content’s visibility.

For a simple, almost a static website with just few easily discoverable pages, a sitemap may not be necessary at all. If search engines already can get to your pages, you do not really need rah_sitemap, or Sitemaps in general. Once a page is indexed, that’s where sitemap’s work ends.

Rah_sitemap is particularly useful when your site has pages that are hard to discover due to being loaded using Ajax, there is distinct content with little linking, or pages are nested deep in a complex page structures. Rah_sitemap can also help your site to start up when it has very few external links pointing to its pages.

It’s common misconception to think that Sitemaps guarantee that pages will be indexed. This is not the case. Sitemap is a map. Whether a place is marked on a map doesn’t mean someone will actually go there, or when. Like any map, the map is used to find and learn, to increase the future knowledge. Sitemap’s update interval also helps to estimate when your site is updated next and when it should be crawled again.

Normally, you will benefit from submitting a sitemap, but its just one piece in the puzzle. In no case will you get penalized from Sitemap or including wrong content in it.

Configuration

After rah_sitemap is installed, you may want to configure it to fit your site. For instance, you may want to exclude certain irrelevant articles or sections. The plugin’s settings can be configured from Textpattern’s Preferences panel, organized under its own Sitemap section. Sections and Categories can be excluded from the sitemap from their respective editors.

Sending the sitemap to search engines

Once you have a sitemap up and running, you may want to inform search engines about its existence. There are few ways you can do it: Webmaster Tools for Google, you could use a robots.txt directive or search vendor specific pinging.

The recommended way is by using a robots.txt file. To get robots.txt up and running, you will have to add robots.txt file at root of your site’s domain, so that its accessible from https://example.com/robots.txt. If your Textpattern site has fully functional clean URLs, is installed at the root and you already do not have robots.txt file, rah_sitemap will automatically create the file for you — or well, serve it dynamically.

If not, you will need to create or edit a file named robots.txt at the root of the domain. In that file you would add a Sitemap directive containing an absolute URL to your sitemap:

Sitemap: https://example.com/?rah_sitemap=sitemap

Where the https://example.com/ is your site’s location as defined in Textpattern’s Preferences panel. The directive should be placed on its own line.

Preferences

Rah_sitemap comes with number of preferences which all can be found from your Preferences panel, organized under a Sitemap section. Rah_sitemap allows excluding sections, categories and articles from the XML sitemap. Following options will be present.

Exclude articles based on fields

The field can be used to exclude articles from the sitemap based on any article field and its value. The option takes a comma-separated list of articlefield: value pairs, where the field is the database field and the value is the field’s value that will be excluded. Available fields include Title, AuthorID, Body, Excerpt, Category1, Category2, Section, Keywords, url_title, custom_1 to custom_10 and Image.

Values used in the option support two wildcard characters. An underscore (_) matches exactly one character, and a percent sign (%) matches zero or more characters.

If you wanted to exclude articles posted to sections named as notes and private or by a user mailer, you could use the following in the field:

Section: notes, Section: private, AuthorID: mailer

Additional URLs

Comma-separated list of additional local site URLs added to the sitemap. Note that a Sitemap only allows local URLs, meaning that any URL used, needs to link to the same domain as where the website itself is located. If a URL is relative and doesn’t start with a HTTP or HTTPS protocol, the site’s URL is prepended to the URL.

Include future articles?

If set to Yes, articles with future publishing date are visible in the sitemap. Please note that by default the article tag doesn’t display future articles, unless its time attribute is explicitly set to future or any.

Include published articles?

If set to Yes, published articles are visible in the sitemap. If both this option and Include future articles? are set to No, no articles will be visible in the sitemap.

Include expired articles?

If set to No, expired articles are not visible in the sitemap.

Exclude sticky articles?

If set to Yes, sticky articles are not visible in the sitemap.

For developers

Rah_sitemap offers small set of tools for developers. These tools allow other Textpattern plugins to extend rah_sitemap’s functionality by adding new URLs to the sitemap. The plugin is packaged in a class structure that can be extended if needed, and introduces new Textpattern callback events.

Callback

Rah_sitemap introduces a new public-facing callback event to the Textpattern’s event library named rah_sitemap.urlset. The event is fired before a sitemap is printed out. The callback event can be used with the API to add new URLs to the sitemap.

As with other callback events in Textpattern, hooking to rah_sitemap’s event happens using Textpattern’s callback handling functions, mainly register_callback.

register_callback('abc_function', 'rah_sitemap.urlset', 0, $urls);

fuction abc_function($event, $step, $void, $urls) {
    $urls['http://example.com/foo/bar'] = '2013-03-04 10:06:30';
}

Custom URL functions

If you are supplying a custom URL function for Textpattern, please note that the URLs the function generates need to meet RFC 3986 and RFC 3987. All URLs should also be entity escaped from special syntax characters using Textpattern’s txpspecialchars function. All URLs Textpattern itself generates follow these specifications, and so should your custom URL plugin.

As rah_sitemap integrates well with Textpattern’s core, it uses the same URL functions as Textpattern. If an URL given to the sitemap doesn’t meet those specification, the sitemap will become invalid.