Making Wagtail pages more SEO friendly with Wagtail Metadata
Introduction
Wagtail pages are great for creating a lot of rich content straight out of the box, but for SEO optimization, they need some tweaking.
Here, I subclass the Page model with some help from the wagtail-metadata plug-in.
This sub-classed model becomes the base for all site pages and holds all the data for og metadata, twitter cards, page description etc..
Creating the SEO Page Model
The metadata tags are the main focus here, but we'll also add two new fields to handle page descriptions and titles both on-site and off, with some pre-save validation to check that both are optimal lengths for SEO.
Installing wagtail-metadata
Begin by installing wagtail-metadata:
$ pip install wagtail-metadataAdd 'wagtailmetadata' to your installed apps.
The plugin provides various mixins to use. Source code and documentation are available on GitHub:
GitHub - neon-jungle/wagtail-metadata: A tool to assist with metadata for social media and search engines.
A tool to assist with metadata for social media and search engines. - neon-jungle/wagtail-metadata
The title, image and description in the above block have been scraped directly from the metadata of that page. I'll show how's that's done in a later blog, but for now, what the wagtail-metadata plug-in will do is allow the same to be created for any page you create.
To see it in action, copy the URL of this page and paste it into a new post on a dynamic site such as Facebook. You'll see the preview card without needing to submit the post.
Creating the mixin
I'm using a slightly modified version of the WagtailImageMetadataMixin class to make a new mixin.
I generally keep critical site-wide functionality like this in a separate core app. I'll have all my site settings, hooks and other customisations here as much as is practical.
from django.conf import settings
from django.db import models
from django.utils.translation import gettext_lazy as _
from wagtailmetadata.models import WagtailImageMetadataMixin
from wagtail.admin.panels import FieldPanel, MultiFieldPanel
from wagtail.models import Page
def get_image_model_string():
try:
image_model = settings.WAGTAILIMAGES_IMAGE_MODEL
except AttributeError:
image_model = 'wagtailimages.Image'
return image_model
class SEOPageMixin(WagtailImageMetadataMixin, models.Model):
search_image = models.ForeignKey(
get_image_model_string(),
null=True,
blank=False,
related_name='+',
on_delete=models.SET_NULL,
verbose_name=_('Search Image'),
help_text=_("The image to use on previews of this page on external links and search results. \
This will also be the image used for blog posts on the index pages.")
)
summary = models.TextField(
null=False,
blank=False,
help_text=_("A summary of the page to be used on index pages and on-site searches.")
)
content_panels = Page.content_panels + [
FieldPanel('summary'),
]
promote_panels = [
MultiFieldPanel([
FieldPanel('slug'),
ImageChooserPanel('search_image'),
FieldPanel('seo_title'),
FieldPanel('search_description'),
FieldPanel('show_in_menus'),
], _('SEO page configuration')),
]
def get_meta_url(self):
return self.full_url
def get_meta_title(self):
return self.seo_title or self.title
def get_meta_description(self):
return self.search_description or self.summary
def get_meta_image(self):
return self.search_image
class Meta:
abstract = True
def clean(self):
super().clean()
errors = {}
len_search_title = len(self.seo_title or self.title)
if len_search_title < 15 or len_search_title > 70:
if len_search_title==0:
msg = _("empty")
else:
msg = _(f"{len_search_title} character{'s' * bool(len_search_title>1)}")
if self.seo_title:
errors['seo_title'] = ErrorList([
_(f'Title tag is {msg}. It should be between 15 and 70 characters for optimum SEO.')
])
else:
errors['seo_title'] = ErrorList([
_(f'Page title is {msg}. Create a title tag between 15 and 70 characters for optimum SEO.')
])
len_search_description = len(self.search_description or self.summary)
if len_search_description < 50 or len_search_description > 160:
if len_search_description==0:
msg = _("empty")
else:
msg = _(f"{len_search_description} character{'s' * bool(len_search_description>1)}")
if self.search_description:
errors['search_description'] = ErrorList([
_(f'Meta Description is {msg}. It should be between 50 and 160 characters for optimum SEO.')
])
else:
errors['search_description'] = ErrorList([
_(f'Summary is {msg}. Create a meta description between 50 and 160 characters for optimum SEO.')
])
if errors:
raise ValidationError(errors)The SEOPage mixin creates 'summary' and 'search_image' fields and adds them to the Content and Promote panels respectively.
The get_meta methods will be used later in the template to render all the metadata:
- The
meta titlewill come from the optional Title Tag (seo_title) on the Promote tab if filled in, otherwise from the page title. - The
meta descriptionwill come from the optional Meta Description (search_description) on the Promote tab if filled in, otherwise the new Page Summary field will be used.
It's a good idea to use the Title Tag and Meta Description fields on the Promote tab.
The Title Tag gives you the opportunity to add in common modifiers that can help in ranking (such as How to, Review, Best, Tips, Top, Find, Buy, Free, Easy) that you might not want to appear in the content title.
The custom Summary field I've added provides an opportunity to give a meaningful page brief in your website (such as internal search results and product or blog page listing cards), while Meta Description allows you to enter a more brief, more Search Engine friendly summation - this becomes the description snippet on Google for instance.
Finally, the custom clean() method provides a check to make sure the title and description are optimised for search engines:
- SEO title should be between 15 and 70 characters. The code checks the length of
seo_titleif present, otherwise the pagetitlelength is validated. - SEO description should be between 50 and 160 characters. Similarly, the length is validated from
search_descriptionif present, otherwisesummaryis checked.
Adding the SEOPage abstract class
We're not quite there as all we have so far is the abstract mixin class. We need to make the new SEOPage abstract class from this:
class SEOPage(SEOPageMixin, Page):
search_fields = Page.search_fields + [
index.SearchField('summary'),
]
class Meta:
abstract = TrueNote that I'm adding the new summary field to the internal search index. This is entirely optional and can be replaced with a simple pass if not desired.
Your page models should all now inherit SEOPage (either directly or indirectly) so that, for example, your home page model might look like:
class HomePage(SEOPage):
--- class property definition ---
content_panels = SEOPage.content_panels + [
--- class panels definition ---
]Templates
Adding the metadata
Finally, in the <head> section of your base.html (or in your head.html if you configured your site this way), load seo_metadata_tags and the appropriate code to place all of your metadata.
{% load static seo_metadata_tags %}
....
{% if request is not None %} {# 500 error has no request #}
{% meta_tags as meta_tags %}
{% if meta_tags is not None %}{{ meta_tags }}{% endif %}
{% endif %}
....For this site (and others), I added a few more tags that may or may not help with ranking - the shifting sands of Google algorithms might have made some of these obsolete already, but they're included anyway:
<meta property="og:type" content="website">
<meta name="robots" content="index, follow, archive, imageindex, noodp, noydir, snippet, translate, max-snippet:-1, max-image-preview:large, max-video-preview:-1">
<meta name="target" content="all">
<meta name="audience" content="all">
<meta name="coverage" content="Worldwide">
<meta name="distribution" content="Global">
<meta name="rating" content="safe for kids">og:type is wanted by Facebook in case you're using something like Facebook OAuth (as this site is), along with the following:
<meta property="fb:app_id" content="{{ your-fb-app-ID }}">The meta name="robots" supplies directives to search engines on how your page should be crawled. You could also supply this on a per-page basis if necessary. Google Developers has a good article describing the use of this. I also discuss this a bit further in another blog and explain how to create this tag and robots.txt dynamically.
I suspect the others in the list are redundant these days, but there is nothing that will impact SEO negatively there.
Add a Canonical Tag
It's worth having a read of the Google Search Console documentation for this if you're not familiar with canonical tags. In a nutshell, you may have multiple URL's pointing to the same page. For instance, if you selected the 'seo' filter on the blog index page then came to this page, you'd arrive with the url
- https://enzedonline.com/en/tech-blog/making-wagtail-pages-more-seo-friendly-with-wagtail-metadata/?tag=seo
Google would index this as a separate page. In fact you can potentially end up with hundreds or thousands of indexed 'pages' from just a couple of dozen canonical pages. This is splitting your page hits and affecting your ranking. You'll also start to get errors on Search Console where Google has selected what its algorithms decide is the canonical page for you.
In this case, I just want to tell Google that the url without the query string is the canonical page. For wagtail pages, this is just page.full_url. I can ignore the case for non-Wagtail pages since I'm not allowing indexing of those (see next blog on an explanation and how-to).
To cater for this, I add a template tag to my core_tags.py:
@register.simple_tag(takes_context=True)
def canonical(context):
page = get_context_var_or_none(context, 'self')
if not page:
return ''
else:
return mark_safe(f'')Now you can just add {% canonical %} directly underneath your call to meta_tags in your base/head template.
{% load static seo_metadata_tags core_tags %}
....
{% if request is not None %} {# 500 error has no request #}
{% meta_tags as meta_tags %}
{% if meta_tags is not None %}{{ meta_tags }}{% endif %}
{% endif %}
{% canonical %}
....Example head output
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="robots" content="index, follow, archive, imageindex, noodp, noydir, snippet, translate, max-snippet:-1, max-image-preview:large, max-video-preview:-1">
<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:title" content="How to Make Wagtail Pages More SEO Friendly with Wagtail Metadata">
<meta name="twitter:description" content="Use sub-classing to create SEO optimized og metadata, twitter cards and page description for your Wagtail site with the wagtail-metadata plug-in.">
<meta name="twitter:image" content="https://enzedonline.com/media/images/adding_wagtail_seo_metadata.original.png">
<meta property="og:url" content="https://enzedonline.com/en/tech-blog/making-wagtail-pages-more-seo-friendly-with-wagtail-metadata/">
<meta property="og:title" content="How to Make Wagtail Pages More SEO Friendly with Wagtail Metadata">
<meta property="og:description" content="Use sub-classing to create SEO optimized og metadata, twitter cards and page description for your Wagtail site with the wagtail-metadata plug-in.">
<meta property="og:site_name" content="Enzed Online">
<meta property="og:image" content="https://enzedonline.com/media/images/adding_wagtail_seo_metadata.original.png">
<meta property="og:image:width" content="688">
<meta property="og:image:height" content="272">
<meta itemprop="url" content="https://enzedonline.com/en/tech-blog/making-wagtail-pages-more-seo-friendly-with-wagtail-metadata/">
<meta itemprop="name" content="How to Make Wagtail Pages More SEO Friendly with Wagtail Metadata">
<meta itemprop="description" content="Use sub-classing to create SEO optimized og metadata, twitter cards and page description for your Wagtail site with the wagtail-metadata plug-in.">
<meta itemprop="image" content="https://enzedonline.com/media/images/adding_wagtail_seo_metadata.original.png">
<title>How to Make Wagtail Pages More SEO Friendly with Wagtail Metadata</title>
<meta name="description" content="Use sub-classing to create SEO optimized og metadata, twitter cards and page description for your Wagtail site with the wagtail-metadata plug-in.">
<meta property="og:type" content="website">
<meta name="target" content="all">
<meta name="audience" content="all">
<meta name="coverage" content="Worldwide">
<meta name="distribution" content="Global">
<meta name="rating" content="safe for kids">
<link rel="canonical" href="https://enzedonline.com/en/tech-blog/making-wagtail-pages-more-seo-friendly-with-wagtail-metadata/">Setting the Image Filter
This is an undocumented but very useful feature of wagtail-metadata that I found out about only after picking through the code to see if I could do something else.
The default is to serve up your search_image full size. If, for some reason, your search image has been uploaded at 3000x4000px, then this is what is getting served each time a page requests your open graph summary for instance. That's quite a bit of overhead.
In your site settings, you can assign an image filter to the site settings variable WAGTAILMETADATA_IMAGE_FILTER which will create a formatted rendition and set this to your image tags above. A good idea if your search_image is large format or not a suitable aspect ratio.
Image filter in this case means a string that you would supply to image.getrendition() which takes all the usual operations you might use with the {% image %} template tag, with the difference that multiple operations are separated by the pipe | character rather than spaces.
If you're interested in making a custom filter, I go through creating thumbnails with preserved images (i.e. resized without cropping or stretching) in another article.
The og:image width and height is 688x272 so I set the filter to use my custom thumbnail operation to match this and output to png. I use the thumbnail image filter I describe in the blog mentioned above. You can use this or another valid Wagtail image filter (such as fill).
In your base.py, add the following setting (with appropriate filter):
WAGTAILMETADATA_IMAGE_FILTER = "thumbnail-688x272|format-png"Redirect www on the Server
For the same reasons to supply Google a canonical, Google will also treat www.example.com/home and example.com/home as two separate pages and will split rankings etc and arbitrarily decide which is the canonical.
Make sure your server is forwarding requests with 301 (permanent redirect) for www onto the base domain address, a CNAME record isn't sufficient for this.
How you do this depends on your server of course. On NGINX, you might add the following to /etc/nginx/sites-available/site-name
server {
# Redirect all HTTP requests to HTTPS
# Redirect all www requests to domain root
listen 80;
listen [::]:80;
server_name www.example.com example.com;
return 301 https://example.com$request_uri;
}
server {
listen 443 ssl http2;
listen [::]:443 ssl http2 ipv6only=on;
server_name example.com;
# Redirect all www requests to domain root
if ($host = www.example.com) {
return 301 https://example.com$request_uri;
}
.... other config ....
}Bypass Analytics for Unpublished Content
While we're in the <head> section and talking about SEO, this is a good time to make sure your dev site and page previews aren't adding to your Google Analytics page views with a check for either debug mode or request.is_preview:
{% if not debug and not request.is_preview %}
<!-- Analytics -->
<script async src="https://www.googletagmanager.com/gtag/js?id={{ settings.site_settings.Tokens.google_analytics }}"></script>
<script src="{% static 'js/analytics.js' %}"></script>
{% endif %}Be sure to have your internal IP addresses added to INTERNAL_IPS in your settings file. If you're only running your development site locally, then you'll just need:
INTERNAL_IPS = ['127.0.0.1',]Summary
From the perspective of site development, there's obviously a lot more that can be done to boost search engine rankings. If you're interested in boosting your search engine rankings further, SE Ranking has some very useful blog posts on the subject. It's a dark art that needs constant adaptation as search engines tweak their algorithms.
In this blog, we created a page model with some SEO building blocks, including generating dynamic metadata, adding a search page and preview card image and ensuring the search engine title and description meet the recommended sizes.
We also added a canonical tag and added a permanent redirect for www requests.
Finally, we added a filter to prevent analytics being measured for views in debug mode, or page previews.