Marketing & SEO

Sitemaps in VibeReference

A sitemap is a file that provides information about the pages, videos, and other files on your website, and the relationships between them. In VibeReference,...

Sitemaps in VibeReference

A sitemap is a file that provides information about the pages, videos, and other files on your website, and the relationships between them. In VibeReference, sitemaps are crucial for helping search engines discover and crawl your website efficiently, which can improve your SEO performance.

Why Sitemaps Matter

Sitemaps serve several important purposes:

  • Improved Discoverability: Help search engines find all the important pages on your site
  • Better Indexing: Provide metadata that helps search engines index your content properly
  • Faster Discovery: Speed up the discovery of new or updated content
  • Enhanced SEO: Support your overall search engine optimization strategy
  • User Navigation: Can also be used to help users navigate complex websites

Types of Sitemaps

XML Sitemaps

The most common format, used primarily for search engines:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://yourvibereferenceapp.com/</loc>
    <lastmod>2023-09-10</lastmod>
    <changefreq>weekly</changefreq>
    <priority>1.0</priority>
  </url>
  <url>
    <loc>https://yourvibereferenceapp.com/features</loc>
    <lastmod>2023-09-05</lastmod>
    <changefreq>monthly</changefreq>
    <priority>0.8</priority>
  </url>
  <url>
    <loc>https://yourvibereferenceapp.com/pricing</loc>
    <lastmod>2023-09-01</lastmod>
    <changefreq>monthly</changefreq>
    <priority>0.9</priority>
  </url>
</urlset>

Sitemap Index

For large sites with multiple sitemaps:

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <sitemap>
    <loc>https://yourvibereferenceapp.com/sitemap-pages.xml</loc>
    <lastmod>2023-09-10</lastmod>
  </sitemap>
  <sitemap>
    <loc>https://yourvibereferenceapp.com/sitemap-products.xml</loc>
    <lastmod>2023-09-08</lastmod>
  </sitemap>
  <sitemap>
    <loc>https://yourvibereferenceapp.com/sitemap-blog.xml</loc>
    <lastmod>2023-09-12</lastmod>
  </sitemap>
</sitemapindex>

HTML Sitemaps

Human-readable sitemaps designed for website visitors:

<div class="sitemap">
  <h2>Site Navigation</h2>
  <ul>
    <li><a href="/">Home</a></li>
    <li>
      <a href="/products">Products</a>
      <ul>
        <li><a href="/products/feature1">Feature 1</a></li>
        <li><a href="/products/feature2">Feature 2</a></li>
      </ul>
    </li>
    <li><a href="/pricing">Pricing</a></li>
    <li><a href="/blog">Blog</a></li>
    <li><a href="/contact">Contact</a></li>
  </ul>
</div>

Image Sitemaps

Specifically for image content:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
        xmlns:image="http://www.google.com/schemas/sitemap-image/1.1">
  <url>
    <loc>https://yourvibereferenceapp.com/gallery</loc>
    <image:image>
      <image:loc>https://yourvibereferenceapp.com/images/sample1.jpg</image:loc>
      <image:title>Sample Image 1</image:title>
      <image:caption>This is a caption for sample image 1</image:caption>
    </image:image>
    <image:image>
      <image:loc>https://yourvibereferenceapp.com/images/sample2.jpg</image:loc>
      <image:title>Sample Image 2</image:title>
      <image:caption>This is a caption for sample image 2</image:caption>
    </image:image>
  </url>
</urlset>

Video Sitemaps

For websites featuring video content:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
        xmlns:video="http://www.google.com/schemas/sitemap-video/1.1">
  <url>
    <loc>https://yourvibereferenceapp.com/videos/tutorial</loc>
    <video:video>
      <video:thumbnail_loc>https://yourvibereferenceapp.com/thumbnails/tutorial.jpg</video:thumbnail_loc>
      <video:title>Getting Started with VibeReference</video:title>
      <video:description>A beginner's guide to using VibeReference</video:description>
      <video:content_loc>https://yourvibereferenceapp.com/videos/tutorial.mp4</video:content_loc>
      <video:duration>360</video:duration>
    </video:video>
  </url>
</urlset>

Implementing Sitemaps in Next.js

Static Sitemap Generation

For websites with static routes, create a sitemap.xml file in the public directory:

// scripts/generate-sitemap.ts
import fs from 'fs';
import { globby } from 'globby';
import prettier from 'prettier';

async function generateSitemap() {
  const prettierConfig = await prettier.resolveConfig('./.prettierrc.js');
  const pages = await globby([
    'pages/**/*.tsx',
    'pages/*.tsx',
    '!pages/_*.tsx',
    '!pages/api',
  ]);

  const baseUrl = 'https://yourvibereferenceapp.com';
  const currentDate = new Date().toISOString();

  // Create sitemap items for each page
  const sitemap = `
    <?xml version="1.0" encoding="UTF-8"?>
    <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
      ${pages
        .map((page) => {
          const path = page
            .replace('pages', '')
            .replace('.tsx', '')
            .replace('/index', '');
          const route = path === '' ? '' : path;
          
          return `
            <url>
              <loc>${baseUrl}${route}</loc>
              <lastmod>${currentDate}</lastmod>
              <changefreq>monthly</changefreq>
              <priority>${route === '' ? '1.0' : '0.8'}</priority>
            </url>
          `;
        })
        .join('')}
    </urlset>
  `;

  const formatted = prettier.format(sitemap, {
    ...prettierConfig,
    parser: 'html',
  });

  fs.writeFileSync('public/sitemap.xml', formatted);
  console.log('Sitemap generated successfully!');
}

generateSitemap();

Add this script to your package.json:

{
  "scripts": {
    "build": "next build",
    "generate-sitemap": "ts-node --project tsconfig.json ./scripts/generate-sitemap.ts",
    "prebuild": "npm run generate-sitemap"
  }
}

Dynamic Sitemap Generation

For websites with dynamic routes, create an API endpoint:

// pages/api/sitemap.xml.ts
import { NextApiRequest, NextApiResponse } from 'next';
import { createClient } from '@supabase/supabase-js';

const supabase = createClient(
  process.env.NEXT_PUBLIC_SUPABASE_URL!,
  process.env.NEXT_PUBLIC_SUPABASE_ANON_KEY!
);

export default async function handler(req: NextApiRequest, res: NextApiResponse) {
  // Set the response type
  res.setHeader('Content-Type', 'text/xml');
  
  try {
    // Fetch your dynamic page data
    const { data: products, error } = await supabase
      .from('products')
      .select('slug, updated_at')
      .order('updated_at', { ascending: false });
    
    if (error) throw error;
    
    // Create the XML sitemap
    const sitemap = `<?xml version="1.0" encoding="UTF-8"?>
      <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
        <url>
          <loc>https://yourvibereferenceapp.com/</loc>
          <lastmod>${new Date().toISOString()}</lastmod>
          <changefreq>daily</changefreq>
          <priority>1.0</priority>
        </url>
        ${products.map(product => `
          <url>
            <loc>https://yourvibereferenceapp.com/products/${product.slug}</loc>
            <lastmod>${new Date(product.updated_at).toISOString()}</lastmod>
            <changefreq>weekly</changefreq>
            <priority>0.8</priority>
          </url>
        `).join('')}
      </urlset>`;
    
    // Send the sitemap
    res.status(200).send(sitemap);
  } catch (error) {
    console.error('Error generating sitemap:', error);
    res.status(500).send('Error generating sitemap');
  }
}

Then configure your next.config.js to rewrite requests:

// next.config.js
/** @type {import('next').NextConfig} */
const nextConfig = {
  reactStrictMode: true,
  async rewrites() {
    return [
      {
        source: '/sitemap.xml',
        destination: '/api/sitemap.xml',
      },
    ];
  },
};

module.exports = nextConfig;

Using next-sitemap Package

For an easier implementation, use the next-sitemap package:

npm install next-sitemap

Create next-sitemap.config.js in your project root:

/** @type {import('next-sitemap').IConfig} */
module.exports = {
  siteUrl: 'https://yourvibereferenceapp.com',
  generateRobotsTxt: true,
  changefreq: 'weekly',
  priority: 0.7,
  exclude: ['/admin/*', '/dashboard/*'],
  robotsTxtOptions: {
    additionalSitemaps: [
      'https://yourvibereferenceapp.com/server-sitemap.xml',
    ],
  },
};

Add to package.json:

{
  "scripts": {
    "build": "next build",
    "postbuild": "next-sitemap"
  }
}

Sitemap Best Practices

  1. Keep Your Sitemap Updated: Regenerate regularly as you add or modify content
  2. Limit Size: Keep each sitemap under 50,000 URLs and 50MB
  3. Use Sitemap Index: For large sites, break into multiple sitemaps
  4. Include Important Pages Only: Focus on high-quality, crawlable pages
  5. Be Consistent with URLs: Use the same protocol (https) and domain as your website
  6. Prioritize Properly: Use the priority attribute thoughtfully
  7. Submit to Search Engines: Register your sitemap with Google Search Console and Bing Webmaster Tools
  8. Include in robots.txt: Add a reference to your sitemap in robots.txt

Example robots.txt with sitemap reference:

User-agent: *
Allow: /

Sitemap: https://yourvibereferenceapp.com/sitemap.xml

Validating Your Sitemap

Before submitting your sitemap to search engines, validate it:

  1. XML Validation: Ensure it follows the sitemap protocol
  2. URL Check: Verify all URLs are accessible and return 200 status codes
  3. Mobile Friendliness: Confirm mobile URLs are correct if using mobile sitemaps
  4. Canonical Verification: Ensure URLs match your canonical URLs

Sitemap Analytics and Monitoring

Track how search engines are crawling your sitemap:

  1. Google Search Console: Monitor coverage and indexing issues
  2. Server Logs: Check for search engine bot activity
  3. Regular Testing: Periodically test URLs in your sitemap

Sitemap Design Patterns for VibeReference

Multi-tenant SaaS Pattern

For VibeReference applications with multiple customer accounts:

// Separate sitemaps for public content and customer-specific content
async function generateSitemapIndex() {
  const publicSitemap = 'https://yourvibereferenceapp.com/public-sitemap.xml';
  const customerSitemaps = await getCustomerDomains().map(
    domain => `https://${domain}/customer-sitemap.xml`
  );
  
  return `
    <?xml version="1.0" encoding="UTF-8"?>
    <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
      <sitemap>
        <loc>${publicSitemap}</loc>
        <lastmod>${new Date().toISOString()}</lastmod>
      </sitemap>
      ${customerSitemaps.map(url => `
        <sitemap>
          <loc>${url}</loc>
          <lastmod>${new Date().toISOString()}</lastmod>
        </sitemap>
      `).join('')}
    </sitemapindex>
  `;
}

Localized Content Pattern

For multi-language VibeReference applications:

// Generate language-specific sitemaps
const languages = ['en', 'fr', 'es', 'de'];

const languageSitemaps = languages.map(lang => {
  return `
    <sitemap>
      <loc>https://yourvibereferenceapp.com/sitemaps/sitemap-${lang}.xml</loc>
      <lastmod>${new Date().toISOString()}</lastmod>
    </sitemap>
  `;
}).join('');

const sitemapIndex = `
  <?xml version="1.0" encoding="UTF-8"?>
  <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    ${languageSitemaps}
  </sitemapindex>
`;

Resources

Ready to build?

Go from idea to launched product in a week with AI-assisted development.