Metadata Filtering: Combining Tags and Vector Search

Metadata filtering allows you to restrict similarity search queries based on structured tag fields. Let us review the filter operators.

1. Why Use Metadata Filtering?

Consider a customer support system. If you run a search query for "How to reset password", a normal similarity search might return relevant help documents but could also return out-of-date documentation or resources for the wrong platform version.

By attaching metadata (such as { "status": "active", "platform": "ios" }), you can filter the search to target only active iOS documentation.

2. Pinecone Metadata Filter Operators

Pinecone supports MongoDB-style filter operators:

• $eq: Equal to.
• $ne: Not equal to.
• $in: Exists in an array.
• $nin: Does not exist in an array.
• $gt / $gte: Greater than / Greater than or equal to.
• $lt / $lte: Less than / Less than or equal to.

3. Querying with Metadata Filters

Define a filter object inside the query options:

import { pc } from "../lib/pinecone";

async function searchFilteredDocuments(queryVector: number[]) {
  const index = pc.index("knowledge-base");

  const response = await index.query({
    vector: queryVector,
    topK: 5,
    includeMetadata: true,
    // Define the metadata filtering rules
    filter: {
      category: { $eq: "tutorials" },
      year: { $gte: 2024 },
      status: { $in: ["published", "featured"] },
    },
  });

  return response.matches;
}