Create an Index

An index turns documents into searchable parts and embeddings. The original files remain in their project folder.

To create one, open Data → Indexes and select Create Index.

⤢Expand

Screenshot previewThe current Create Index form for a standard index

×

Standard index settings

Use a Standard Index for the normal document-to-embedding workflow.

Setting	What it does
Index Mode	Selects Standard Index or Projection Index.
Index Name	Identifies the index in Data and in flow configuration.
Data Location	Selects the project folder whose documents the index reads. / selects the project root.
Embeddings	Selects the connection and embedding model used to encode document parts and search queries.
Dimension	Shows or accepts the vector dimension supported by the selected embedding model. The query model and stored vectors must use the same dimension.
Chunking Strategy	Selects By Character, By Token, or Custom.
Max Chunks per Document	Limits how many parts one document can create. Leave blank for no limit.
Chunk Size	Target size for each character- or token-based part.
Chunk Overlap	Repeats this amount between adjacent parts to preserve context across a boundary.

BotDojo configures the index storage automatically. The current form defaults to By Character, chunk size 2000, chunk overlap 20, and no maximum number of chunks. Confirm that those values fit your content before creating a production index.

Select embeddings

The Embeddings list is populated from the project’s compatible connections. Select the provider and model that will also be available when the index is searched.

Dimension affects storage, model compatibility, and retrieval behavior. BotDojo fills it from models with a fixed dimension and validates it when a model exposes a configurable value.

Select a chunking strategy

• By Character uses a recursive text splitter and measures Chunk Size and Chunk Overlap in characters.
• By Token measures them using the selected tokenizer.
• Custom selects a flow exported as a BotDojo chunker. A custom chunker can rewrite parts or add part-level metadata before embedding.

See Chunking for selection guidance and every custom setting.

Projection indexes

A Projection Index derives a searchable subset from an existing parent index instead of reading documents directly from a folder. Its top-level settings are:

Setting	What it does
Index Name	Names the derived index.
Parent Index	Selects the existing index whose part rows are the source.
Materialized Columns	Opens the parent-row filter, typed metadata columns, and column indexes.

The optional Parent Filter is a SQL WHERE expression applied to rows copied from the parent. Test Filter validates the expression and reports how many parent parts match before you create or update the projection.

Use a projection when you need a governed subset of an existing index—for example, a region, language, product, or document class—without chunking the source documents again.

Configure materialized columns

Materialized columns turn selected JSON metadata values into typed columns for filtering and metadata queries.

• For a standard index, create and populate the index first. Then select Edit → Materialized Columns → Configure.
• For a projection index, configure them while selecting the parent index and optional parent filter.

See Materialized Columns for column expressions, types, query examples, column indexes, and rebuild behavior.

Create and populate the index

Select Create Index after all required settings are complete. Then open the index:

• If automatic indexing is enabled, new and modified documents are queued for processing.
• Otherwise, select Run Index Update.

Wait until the index reports Ready, Need Embeddings is 0, Out of Sync is 0, and Errors is 0. Then select Query to verify semantic retrieval and metadata before connecting the index to a production flow or agent.