Building Enterprise RAG Pipelines

Python
# Basic RAG pipeline setup
import nvidia_rag_blueprint as rag

# Initialize pipeline with default configuration
pipeline = rag.Pipeline(
    embedding_model="llama-nemotron-embed-1b-v2",
    generation_model="nemotron-3-super-120b-a12b",
    vector_db="elasticsearch",
    storage="seaweedfs"
)

# Ingest multimodal documents
pipeline.ingest(
    data_path="./corporate_docs",
    file_types=["pdf", "docx", "audio", "video"],
    extract_tables=True,
    extract_images=True
)

# Query the knowledge base
response = pipeline.query(
    question="What are our Q3 safety protocols?",
    enable_reranking=True,
    reasoning_budget="medium"
)

Progress:

• Architecture Design - Define components (ingestion, retrieval, generation)
• Data Pipeline Setup - Configure multimodal extraction and storage
• Vector Database - Set up Elasticsearch or Milvus with GPU acceleration
• Model Integration - Deploy NIM microservices (embedding, generation, reranking)
• Agent Capabilities - Enable MCP server, query decomposition, metadata filtering
• Observability - Implement monitoring, evaluation (RAGAS), and telemetry
• Production Deployment - Scale with Kubernetes/OpenShift, enable GPU sharing

1. Configure Multimodal Ingestion

YAML
# ingestion-config.yaml
extraction:
  text: nemotron-ocr-v1
  tables: nemotron-table-structure-v1
  charts: nemotron-graphic-elements-v1
  images: nemotron-page-elements-v3
  audio: nvidia-riva-asr
metadata:
  custom_fields: ["department", "confidentiality", "date"]
  auto_tagging: true

2. Set Up Hybrid Search

Python
# Configure dense + sparse retrieval
search_config = {
    "dense_weight": 0.7,
    "sparse_weight": 0.3,
    "rerank_top_k": 10,
    "enable_vlm_rerank": True,
    "collections": ["docs", "images", "audio"]
}

3. Enable Agent Integration

Python
# MCP server for agent communication
mcp_server = rag.MCPServer(
    pipeline=pipeline,
    capabilities=["summarize", "query_decomposition", "metadata_filter"],
    reasoning_budget_levels=["low", "medium", "high"]
)

Example 1: Enterprise Document Search Input: "Find all safety incidents in manufacturing from Q3 2024 with images" Output:

JSON
{
  "documents": [
    {"title": "Q3 Safety Report", "confidence": 0.94, "page": 15},
    {"title": "Incident Log 2024-09", "confidence": 0.87, "page": 3}
  ],
  "images": ["safety_chart_q3.png", "incident_photo_092024.jpg"],
  "reasoning_trace": ["Filtered by date range", "Searched safety keywords", "Retrieved images"]
}

Example 2: Multimodal Content Generation Input: Chart image + "Explain this quarterly performance data" Output: VLM-generated analysis with image understanding and contextual explanation

Example 3: Agent Workflow Input: Complex query requiring decomposition Output:

JSON
{
  "sub_queries": [
    "What is our current inventory level?",
    "What were last quarter's sales figures?", 
    "What is the projected demand?"
  ],
  "synthesized_response": "Based on current inventory of 10K units..."
}