A guide to creating an XML sitemap, robots.txt, metadata and setting URL rewrite rules for a NextJS site.
Published: by Hannah
NextJS was created with SEO in mind meaning your site will cover many SEO elements well naturally such as having a built in Metadata API, or at least, be fairly easy to optimise for search engines. Increased awareness of SEO also means that many developers provide basics like metadata components in templates. Whether you've started your NextJS site from scratch or a template, it is still a good idea to check best practices to ensure your site has these essential technical SEO elements set up properly.
Enforcing redirect protocols for your site is important in ensuring you don't create duplicate content and dilute page "authority" or organic "ranking" potential for SEO.
It's easy to overlook the fact that your site has "www" and non "www" domain versions and "trailing slash" vs non-trailing slash versions. For example, you create the page "Penguins" - then it can be accessed via:
A simple redirect or domain URL rewrite rule to a single version of the several possible combos will eliminate content duplication issues.
So how do you decide which version is best? In short, there's no right or wrong since search engines can respect any and won't penalise you for one or the other. So this can be an aesthetic-based decision but it's worth taking the time to decide if you want to keep to the traditional 'www' or take the tech view on trailing slashes indicating a folder in the information architecture or just the shorter possible version altogether.
1. Locate your "next.config.mjs" or "next.config.js" file
2. Insert an 'async redirects' key. For example, if you want to set a protocol for the "https://www.example.co.uk/penguins" version:
async redirects() {
return [
{
source: "/:path*",
has: [{ type: "host", value: "example.co.uk" }],
destination: "https://www.example.co.uk/:path*",
permanent: true,
}
]
}
Go to https://nextjs.org/docs/app/api-reference/next-config-js/redirects for further documentation.
Good news again, Next.js provides a plugin for these. In case you're wondering what these files are good for. The main thing to know is that search engines like Google and Bing use these to more quickly understand which parts of your site you want served in the search engine page results. And if you haven't already, create a Google Search Console and Bing Webmaster accounts to claim your site and submitting your XML sitemap.
1. Install the 'next-sitemap' plugin from the root of your project. (see my previous blog for tips on using npm in Terminal as a MacOS user:
sudo npm i next-sitemap
)
2. Create a new file in the root of your project, named "next-sitemap.config.js":
/** @type {import('next-sitemap').IConfig} */
module.exports = {
siteUrl: 'https://www.example.co.uk/',
changefreq: 'daily',
priority: 0.7,
sitemapSize: 5000,
generateIndexSitemap: false,
generateRobotsTxt: true,
transform: async (config, path) => {
return {
loc: path, // => this will be exported as http(s)://<config.siteUrl>/<path>
changefreq: config.changefreq,
priority: config.priority,
lastmod: config.autoLastmod ? new Date().toISOString() : undefined,
alternateRefs: config.alternateRefs ?? [],
}
},
robotsTxtOptions: {
policies: [
{
userAgent: '*',
allow: '/',
},
],
},
}
3. Test before publishing. If you're following my previous guide as a MacOS user, run the following command in Terminal:
sudo npm run postbuild
Tip: For larger and more complex sites, you may need to organise your sitemaps into more than one and have an 'index sitemap' directory for them. Read further documentation on https://www.npmjs.com/package/next-sitemap
It's a good idea to create the capability to write your own metadata per each page and have a fallback auto-generated metadata. At minimum you should be considering meta and OG: titles and descriptions for each unique page on your site and image alt tags for all images including icons. These are useful for how your content appears on search engine result pages and for meeting user Accessibility needs. This is all made easier with the Metadata and Image API that Next.js already comes with to streamline meta: titles, descriptions, OG tags, favicons, robots directive tags and image alt tags.
1. Define shared fields in your CMS schema for titles and descriptions that will be used for both backend meta/og/image titles/tags and descriptions.
For example, in a Sanity page schema file "...schema/page.ts":
Tip: Set character limits for meta titles e.g. 50-70 characters including spaces, and meta descriptions e.g. 120-170 characters including spaces. And validation restraints to remove line spacings or prevent odd characters that won't be translated into Unicode for search engines to display on search engine result pages. Also be mindful to adapt the character limits for different languages that may naturally have longer/shorter word lengths.
import { DocumentIcon, ImageIcon } from '@sanity/icons'
import { defineArrayMember, defineField, defineType } from 'sanity'
export default defineType({
type: 'document',
name: 'page',
title: 'Page',
icon: DocumentIcon,
liveEdit: true,
fields: [
defineField({
type: 'string',
name: 'title',
title: 'Title',
validation: (rule) => rule.required(),
}),
...
defineField({
name: 'overview',
description:
'Used both for the <meta> description tag for SEO, and the personal website subheader.',
title: 'Overview',
type: 'array',
of: [
// Paragraphs
defineArrayMember({
lists: [],
marks: {
annotations: [],
decorators: [
{
title: 'Italic',
value: 'em',
},
{
title: 'Strong',
value: 'strong',
},
],
},
styles: [],
type: 'block',
}),
],
validation: (rule) => rule.max(155).required(),
}),
...
defineField({
type: 'image',
icon: ImageIcon,
name: 'image',
title: 'Image',
options: {
hotspot: true,
},
preview: {
select: {
imageUrl: 'asset.url',
title: 'caption',
},
},
fields: [
defineField({
title: 'Caption',
name: 'caption',
type: 'string',
}),
defineField({
name: 'alt',
type: 'string',
title: 'Alt text',
description:
'Alternative text for screenreaders. Falls back on caption if not set',
}),
],
}),
...
2. Create a 'global' component that can be used in other page header or Image box templates. This could be organised in a subfolder in your root directory like "...components/global/SiteMeta.tsx" and make use of the Next "head-elements" library. In my case, using a Sanity CMS you can see an example of the tsx at https://github.com/sanity-io/sanity-template-vercel-visual-editing/blob/main/components/global/SiteMeta.tsx
Look for other applicable examples in the Next GitHub.
Tip: it's a good idea to seperate the global and individual componenets per different page templates because you may want to arrange the site meta differently e.g. "Product Title | Product Category | Brand name" for a PDP vs "Brand name - Article Title" for a press release, or grab snippets for a description from different parts of a blog page vs a product page.
3. Call the declared (meta or image) components in each page header layout and define the arrangement of how you want metadata to be generated. In my case, for a Sanity project "…/components/pages/page/PageHead.tsx".
You can always read further documentation at https://nextjs.org/learn/dashboard-app/adding-metadata
https://nextjs.org/learn/dashboard-app/improving-accessibility
Up Next...