Skip to content
grepvine.aiJoin preview

Best practices

Keys sort in byte order, listings fold them at a delimiter, and the batch helpers and gv record each key as the path field. A key prefix therefore works as a folder, a listing and a filter at once. Put the part you narrow by most often first:

Index Key layout Narrow with
Support tickets tickets/2026/10/T-48213.md path STARTSWITH 'tickets/2026/10/'
Several repositories payments/services/refund.go path STARTSWITH 'payments/'
Incident logs logs/checkout/2026-10-14T09.log path STARTSWITH 'logs/checkout/2026-10-'

Zero-pad numbers in keys, as in 2026/09/, so byte order matches numeric order.

Agents pick an index by its description. The gv agent skill tells agents to read it before they search. Say what the index holds, where it comes from, how keys are laid out, which metadata fields exist, and how often it changes:

# Support tickets
Every ticket from the helpdesk since 2024, one Markdown file each, refreshed hourly.
- Keys: `tickets/{yyyy}/{mm}/{id}.md`
- Metadata: `product` (string), `priority` (integer, 1 is highest),
`status` (`open`, `pending` or `closed`), `opened_at` (timestamp)

Put what you search together in one index.

Section titled “Put what you search together in one index.”

A search covers one index. Keep documents that you search together in the same index, and tell them apart with key prefixes and metadata. Put corpora you never search together, such as source code and support tickets, in separate indexes, so a broad pattern over one doesn’t return matches from the other.

Filters compare whole values, ranges and prefixes. ENDSWITH and CONTAINS aren’t supported, so derive the fields you need when you write:

from pathlib import PurePosixPath
metadata = {
"ext": PurePosixPath(key).suffix.lstrip("."), # ext = 'go'
"repo": key.split("/", 1)[0], # repo = 'payments'
}
  • Keep one type per field. A comparison only matches values of the literal’s type. If priority is a string in some documents, priority <= 2 skips them. In TypeScript, 4 is stored as an integer and 4.5 as a number. Tag a field that can hold either: { type: "number", value: 4 }.
  • Store times as timestamps. A timestamp field works with CURRENT_DATE - INTERVAL '7 days'. A date stored as a string only compares in byte order.
  • Use names a filter can reference. A field name with a space or a / can be stored but not filtered on. Searching gives the rule.

Each document holds up to 64 fields and 64 KiB of metadata.

Write throughput comes from sending many documents per request; tradeoffs explains why.

  • Use the batch helpers. put_many, putMany and gv upload fill each request to 2,048 documents or 16 MiB. A loop of single writes waits for one acknowledgment per document.
  • Run batches in parallel. gv upload keeps 16 batches in flight and gzips them. --concurrency sets the count, from 1 to 64. Over REST, send batches with Content-Encoding: gzip.
  • Make imports restartable. Pass an idempotency key derived from the import, such as support-import-2026-10-14, and rerun a failed call with the same key and documents. Check each document’s error in the results.
  • Leave 429 to the client. The SDKs and gv already wait for Retry-After and retry.

An index only reflects what you last wrote to it. Deleted or renamed source files keep matching until you remove their documents.

  • Mirror a repository with gv sync. Run it on every push to the default branch. With full git history, each run uploads only the files changed since the last synced commit.
  • Collect from many sources with gv upload or the batch helpers. They never delete, so delete documents yourself when their source goes away.
# .github/workflows/grepvine-sync.yml
name: Sync acme/monorepo
on:
push:
branches: [main]
concurrency: grepvine-sync
jobs:
sync:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- run: |
curl -fsSL https://get.grepvine.ai/install.sh | sh
echo "$HOME/.local/bin" >> "$GITHUB_PATH"
- run: gv sync . acme/monorepo --json
env:
GREPVINE_API_KEY: ${{ secrets.GREPVINE_API_KEY }}

concurrency runs one sync at a time. Without fetch-depth: 0, the recorded commit is missing from the clone and every run compares the whole directory with the whole index.

A query is narrowed to candidate documents by the literal text in its pattern and by its filter, before any text is checked. Tradeoffs gives the rule: a pattern needs a literal run of at least three bytes.

Slower Faster Why
\d+ms timeout after \d+ms Adds a literal.
[A-Z]{3}-\d+ INC-\d+ Names the prefix you’re looking for.
A search over the whole index, keeping results under services/billing/ --filter "path STARTSWITH 'services/billing/'" Narrows candidates by metadata.

Every search returns every match. A pattern such as error over a log index can match on most lines of most documents, and an agent that reads all of them uses up its context. Cap the results, then widen if the first ones aren’t enough:

from itertools import islice
query = QueryRequest(pattern="OrderNotFound", mode=QueryRequestMode.LITERAL, lines_before=2, lines_after=2)
hits = list(islice(grepvine.paginate(
client.search.query, organization="acme", index="monorepo", body=query
), 50))
  • Start specific. Begin with a literal and a filter, and widen the pattern only when it finds too little.
  • Return locations with context. Give the model key:line: text and two or three context lines. Fetch a byte range of the document only when it needs more; searching shows how.
  • Treat no match as an answer. gv search exits with status 1 when nothing matched. A tool built on it should pass that to the model as an empty result.