Using the Indexes Tab Interface¶

New Interface Available

The Indexes Tab is a new interface available in the Next environment as part of Release 1.7.2. This dedicated interface provides a comprehensive way to create, manage, and search indexes directly within the Toolkit Configuration. For general indexing information, see the Indexing Overview.

Overview¶

The Indexes Tab is a dedicated interface within Toolkit Configuration that provides a centralized location for managing all indexing operations for supported toolkits. It offers an intuitive sidebar-based layout with three main sections:

Indexes Sidebar: Displays all created indexes with their status and metadata
Index Management: Provides detailed controls for selected indexes
Search & Chat: Enables running search tools and viewing results in a chat interface

Key Features:

Visual Status Indicators: Easy identification of index states (success, in-progress, failed)
Multiple Search Tools: Access to Search Index, Stepback Search Index, and Stepback Summary Index
Real-time Progress Monitoring: Live updates during index creation and updates

Prerequisites¶

Before using the Indexes Tab interface, ensure the following requirements are met:

Project-Level Configuration¶

PgVector Configuration: Vector storage must be configured at the project level
- Navigate to Settings → AI Configuration
- Configure PgVector connection settings
- Verify connection is active
Embedding Model Configuration: An embedding model must be selected
- Navigate to Settings → AI Configuration
- Select and configure an embedding model
- Test model availability

Access Control

The Indexes tab is automatically disabled if PgVector and Embedding Model are not properly configured at the project level. You must complete the project-level setup before the tab becomes available.

Toolkit Configuration¶

Supported Toolkit: Ensure you're working with a toolkit that supports indexing:

Category	Supported Toolkits
Repos	ADO Repos, Bitbucket, GitHub, GitLab
Wikis	ADO Wiki, Confluence, SharePoint
Issues	ADO Boards, ADO Plans, Jira
Files	Artifact, SharePoint
Designs	Figma
Tests	TestRail, Xray Cloud, Zephyr Enterprise, Zephyr Essential, Zephyr Scale

Toolkit Tools: Enable indexing tools in your toolkit configuration:

Tool	Status	Description
Index Data	Required	For creating indexes
Search Index	Required	For search functionality
Stepback Search Index	Optional	For advanced search
Stepback Summary Index	Optional	For summarized search results
List Collections	Optional	For viewing available indexes
Remove Index	Optional	For index cleanup

Indexing Tab Availability

The Indexes tab is automatically disabled if the Index Data tool is not enabled in your toolkit configuration. You must enable the Index Data tool before the Indexes tab becomes available.

Accessing the Indexes Tab¶

Navigate to Toolkits: Go to Toolkits in the main navigation
Select Toolkit: Choose a toolkit that supports indexing from your list(e.g., Github)
Open Indexes Tab: Click on the Indexes tab in the toolkit detail view

If the Indexes tab is disabled or not visible, verify that your project-level prerequisites are properly configured.

¶

Creating a New Index¶

Step 1: Initiate Index Creation¶

Click the + Icon: In the Indexes sidebar, click the + Create New Index button
New Index Form: The center panel will display the new index creation form

Step 2: Configure Index Parameters¶

Required Fields:

Index Name: Provide a unique name for your index (collection suffix)
- Must be unique within the toolkit
- Maximum 7 characters
- Alphanumeric characters recommended

Toolkit-Specific Parameters:

Configure parameters specific to your toolkit type. Common parameters include:

Parameter	Description	Example
Collection Suffix	Unique identifier for the index	`docs`, `prod`, `v1`
Progress Step	Progress reporting interval (0-100)	`10`
Clean Index	Remove existing data before indexing	✓ or ✗
Chunking Config	Document chunking configuration	`{}` (default)

Toolkit-Specific Settings

Different toolkits require different parameters. For example:

GitHub: Repository name, branch, file patterns
Confluence: Space key, page filters
Jira: JQL queries, field extraction settings
TestRail: Project ID, suite filters

Refer to toolkit-specific documentation for detailed parameter information.

Step 3: Validate and Start Indexing¶

Form Validation: The Index button remains inactive until all mandatory fields are filled
Review Configuration: Verify all parameters are correct
Start Indexing: Click the Index button to begin the process

Managing Existing Indexes¶

Selecting an Index¶

Click Index Card: Select any index from the left sidebar
View Details: Index information and management options appear in center panel
Access Tools: Available actions depend on index status

Index Information Panel¶

When an index is selected, the index card displays:

Index Name: Current name/collection suffix
Last Updated: Timestamp of most recent update
Document Count: Number of indexed documents

Manual Index Updates¶

Trigger Manual Update:

Select Index: Click on the index you want to update
Click Update: Use the Update button in the index information panel
Monitor Progress: Watch real-time updates in the center and right panels
Review Results: Check for successful completion or error messages

Update Scenarios:

Incremental Updates: Add new content since last indexing
Full Refresh: Complete re-indexing of all content (use Clean Index option)
Parameter Changes: Modify indexing parameters before updating

Deleting Indexes¶

Delete Process:

Select Index: Choose the index to delete from the sidebar
Click Delete: Use the Delete button in the index information panel
Confirm Deletion: Enter the index name in the confirmation modal
Permanent Removal: Index and all associated data are permanently deleted

Deletion Warning

Index deletion is permanent and cannot be undone. All indexed data, search history, and configurations are permanently removed.

¶

Using Search Tools¶

Accessing Search Functionality¶

Prerequisites for Search:

Successful Index: Only completed indexes support search operations
Selected Tools: Search tools must be enabled in toolkit configuration

Available Search Tools:

Search Index: Basic semantic search across indexed content
Stepback Search Index: Advanced search that breaks down complex questions
Stepback Summary Index: Search with automatic summarization of results

Search Tool Selection¶

Navigate to Run Tab: Click the Run tab in the center panel
Tool Dropdown: Select search tool from the dropdown menu
Configure Parameters: Set search parameters and LLM model settings

Basic Search (Search Index)¶

Configuration:

Select Tool: Choose "Search Index" from tool dropdown
Enter Query: Provide search query in the text field (e.g., How do I create secrets in Elitea and what are the best practices for managing sensitive configuration data?)
Configure Model: Select LLM model and adjust settings if needed
Optional Parameters:
- Filter: Apply content filters to narrow search scope (e.g., file_type:markdown, author:john.doe)
- Cut Off (0 - 1): Relevance threshold for search results (e.g., 0.7 for high relevance, 0.3 for broader results)
- Search Top: Maximum number of top results to return (e.g., 10, 25, 50)
- Full Text Search: Enable comprehensive text-based search (✓ enabled for detailed content search)
- Extended Search: Activate advanced search algorithms (✓ enabled for semantic similarity)
- Reranker: Use AI-powered result reranking for improved relevance (✓ enabled for better result ordering)
- Reranking Config: Configure reranking algorithm parameters (e.g., model settings, weight adjustments)

Execute Search:

Activate Run Button: Button becomes active when query is provided
Click Run: Execute the search operation
View Results: Results appear in the right panel chat interface

Advanced Search (Stepback Search)¶

Configuration:

Select Tool: Choose "Stepback Search Index" from dropdown
Enter Complex Query: Provide detailed or multi-part query
Model Settings: Configure LLM for query decomposition
Search Parameters:
- Messages: Number of conversation messages to include in context
- Filter: Apply content filters to narrow search scope (e.g., file_type:markdown, author:john.doe)
- Cut Off (0 - 1): Relevance threshold for search results (e.g., 0.7 for high relevance, 0.3 for broader results)
- Search Top: Maximum number of top results to return (e.g., 10, 25, 50)
- Full Text Search: Enable comprehensive text-based search (✓ enabled for detailed content search)
- Extended Search: Activate advanced search algorithms (✓ enabled for semantic similarity)
- Reranker: Use AI-powered result reranking for improved relevance (✓ enabled for better result ordering)
- Reranking Config: Configure reranking algorithm parameters (e.g., model settings, weight adjustments)

Summarized Search (Stepback Summary)¶

Configuration:

Select Tool: Choose "Stepback Summary Index" from dropdown
Enter Query: Provide query requiring summarized response
Summary Settings: Configure summarization parameters
Model Selection: Choose appropriate LLM for summarization
Search Parameters:
- Messages: Number of conversation messages to include in context
- Filter: Apply content filters to narrow search scope (e.g., file_type:markdown, author:john.doe)
- Cut Off (0 - 1): Relevance threshold for search results (e.g., 0.7 for high relevance, 0.3 for broader results)
- Search Top: Maximum number of top results to return (e.g., 10, 25, 50)
- Full Text Search: Enable comprehensive text-based search (✓ enabled for detailed content search)
- Extended Search: Activate advanced search algorithms (✓ enabled for semantic similarity)
- Reranker: Use AI-powered result reranking for improved relevance (✓ enabled for better result ordering)
Reranking Config: Configure reranking algorithm parameters (e.g., model settings, weight adjustments)

Summary Parameters: - Summary Length: Short, medium, or long summaries - Include Citations: Reference source documents in summary - Focus Areas: Specific aspects to emphasize in summary

Search Results Management¶

Result Display:

Structured Output: Results appear as organized chat messages
Source References: Links to original indexed documents
Relevance Scores: Confidence ratings for search matches
Metadata: Document types, dates, and other contextual information

Result Actions:

Copy Results: Copy search output for external use
Export Data: Save results in various formats
Refine Search: Modify parameters and search again
Follow-up Questions: Continue conversation with additional queries

Viewing Index Configuration¶

Configuration Tab Access¶

Select Index: Choose any index from the sidebar
Navigate to Configuration: Click the Configuration tab in center panel
Review Settings: All configuration parameters are displayed in read-only format

Configuration Information¶

Displayed Parameters:

Creation Settings: Original parameters used during index creation
Toolkit-Specific Config: Parameters unique to the toolkit type
Processing Options: Chunking, filtering, and processing configurations
Timestamp Information: Creation date, last modified date
Version Information: Index format version and compatibility

Configuration Categories:

Category	Information Displayed
Basic Settings	Index name, collection suffix, toolkit type
Data Source	Source location, filters, scope parameters
Processing	Chunking configuration, content extraction settings
Advanced	Custom parameters, optimization settings

Understanding Configuration Details¶

Data Source Parameters:

Source Location: Repository, space, project, or file location
Scope Filters: Branches, labels, file patterns, date ranges
Access Credentials: Associated credential information (name only)

Processing Configuration:

Chunking Strategy: How documents are split for indexing
Content Extraction: File types and content elements included
Filtering Rules: Content exclusion patterns and rules

Read-Only Display

All configuration information is read-only and cannot be modified from this tab. To change configuration, create a new index or update the existing index with new parameters.

Troubleshooting¶

Common Issues and Solutions¶

Tab Not Available¶

Symptoms:

Indexes tab is missing or disabled
Cannot access indexing interface

Solutions:

Verify Prerequisites: Ensure PgVector and Embedding Model are configured
Check Toolkit Support: Confirm toolkit supports indexing
Review Permissions: Verify user has access to indexing features
Refresh Browser: Clear cache and reload the page

Index Creation Failures¶

Symptoms:

Index creation process fails
Error notifications during indexing
Stuck in "in progress" state

Solutions:

Check Credentials: Verify toolkit credentials are valid and accessible
Review Parameters: Ensure all required parameters are provided
Data Source Access: Confirm data source is accessible and contains data
Resource Limits: Check if data size exceeds system limits
Network Connectivity: Verify stable internet connection

Common Error Messages:

Error	Possible Cause	Solution
"Authentication failed"	Invalid credentials	Update toolkit credentials
"Data source not found"	Incorrect source parameters	Verify repository/space/project names
"Insufficient permissions"	Limited access rights	Grant appropriate permissions to credential
"Processing timeout"	Large dataset or slow connection	Reduce scope or increase timeout settings

Search Tool Issues¶

Symptoms:

Search tools not available
Run button remains disabled
No search results returned

Solutions:

Index Status: Verify index is successfully completed
Tool Selection: Ensure search tools are enabled in toolkit
Query Format: Check search query syntax and format
Model Configuration: Verify LLM model is properly configured
Collection Access: Confirm index collections are accessible

Performance Optimization¶

Large Dataset Handling¶

Strategies:

Incremental Indexing: Use progressive updates instead of full re-indexing
Scope Filtering: Limit indexing scope to relevant content
Chunking Optimization: Adjust chunk sizes for optimal processing
Batch Processing: Process large datasets in smaller batches

Search Performance¶

Optimization Tips:

Specific Queries: Use specific search terms instead of broad queries
Result Limits: Set appropriate limits on result counts
Model Selection: Choose appropriate LLM models for search tasks
Collection Targeting: Search specific collections instead of all indexes

Getting Additional Help¶

Documentation Resources: - Indexing Overview: General indexing concepts - Indexing Tools: Detailed tool documentation - Toolkit-specific guides for detailed configuration

Best Practices¶

Index Naming and Organization¶

Naming Conventions:

Descriptive Names: Use meaningful collection suffixes (docs, prod, test)
Version Control: Include version indicators for time-based indexes (v1, 2024q1)
Environment Separation: Distinguish between environments (dev, staging, prod)
Purpose Indication: Reflect the index purpose (onboard, support, api)

Organization Strategies:

Logical Grouping: Group related indexes by purpose or team
Lifecycle Management: Implement retention policies for old indexes
Access Control: Consider who needs access to which indexes
Documentation: Maintain documentation of index purposes and usage

Efficient Index Management¶

Creation Best Practices:

Start Small: Begin with limited scope and expand as needed
Test First: Use test environments before production indexing
Validate Data: Ensure data quality before indexing
Monitor Resources: Track system resource usage during indexing

Update Strategies:

Incremental Updates: Prefer incremental over full updates when possible
Scheduled Maintenance: Use off-peak hours for large updates
Change Detection: Implement change detection to trigger targeted updates
Rollback Plans: Maintain ability to revert to previous index versions

Search Optimization¶

Query Design:

Specific Queries: Use specific terms for better accuracy
Context Awareness: Leverage conversation context for follow-up questions
Tool Selection: Choose appropriate search tools for different use cases
Result Validation: Verify search results against known information

Model Configuration:

Model Selection: Choose appropriate LLMs for different search types
Parameter Tuning: Adjust temperature and token limits based on use case
Cost Management: Balance result quality with computational costs
Performance Monitoring: Track search performance and optimize accordingly

Maintenance and Monitoring¶

Regular Maintenance:

Index Health Checks: Regularly verify index integrity and performance
Cleanup Operations: Remove unused or outdated indexes
Performance Reviews: Analyze search performance and user satisfaction
Security Audits: Review access permissions and credential management

Monitoring Practices:

Usage Analytics: Track index usage patterns and popular searches
Error Monitoring: Monitor for indexing and search failures
Resource Tracking: Monitor system resource consumption
User Feedback: Collect feedback on search quality and interface usability