# Architecture Overview Source: https://docs.vantedge.run/concepts/architecture-overview VantEdge system architecture and deployment model ## System Architecture VantEdge provides an intelligent deployment architecture that separates the control plane from your data plane, ensuring complete data isolation while providing centralized management. VantEdge deployment architecture showing control plane orchestration, customer Kubernetes clusters, and integration with legacy systems ## Architecture Components ### Control Plane (VantEdge Infrastructure) The control plane is hosted VantEdge infrastructure that handles orchestration and deployment workflows: **Key components:** * **VE Dashboard + Backend**: Web interface and API for managing deployments * **Control Plane Queue**: Asynchronous task queue (Celery) handling deployment requests with retry logic * **Audit/Logging DB**: Complete audit trail of all deployment activities and system events The control plane manages deployments via: * `cloud_service_provision()` - Provisions infrastructure resources * `deploy_ai_service()` - Deploys AI agents and models ### Data Plane (Customer Infrastructure) Your data and workloads remain in your infrastructure with complete isolation: **VE Agents**: Lightweight agents deployed in your Kubernetes clusters that: * Execute deployment commands from the control plane * Report status and metrics back to control plane * Run alongside your AI workloads * Access your data sources directly **AI Workloads**: Your agents, models, and applications that: * Run in containerized environments * Access your databases and data sources * Process data locally without sending to VantEdge * Maintain sub-100ms latency through data locality ### Integration Layer VantEdge connects with your existing infrastructure: **Legacy Systems** * PostgreSQL databases * MongoDB and NoSQL stores * S3 data lakes * On-premises data sources **SaaS Tools** * Slack, Gmail, Microsoft Teams * Salesforce, HubSpot * Jira, Asana, Linear * Google Drive, Notion, Confluence ## Data Locality & Performance The architecture enables intelligent, data-aware deployment: ``` Enterprise Data Infrastructure ├── Region 1 (US-East) │ ├── PostgreSQL Database │ ├── S3 Data Lake │ └── Agent Deployment │ ├── Pods: 3-10 (auto-scaled) │ ├── Local Data Cache │ └── Direct DB Connections ├── Region 2 (EU-West) │ ├── Regional Data Replica │ └── Agent Deployment │ └── Edge-optimized workloads └── Edge Locations ├── IoT Data Streams └── Real-time Processing Agents ``` **Performance Benefits:** * **Sub-100ms latency** for data access operations * **Reduced egress costs** by minimizing cross-region data transfers * **Improved reliability** through local data availability * **Better compliance** with data residency requirements ## Intelligent Orchestration The deployment system analyzes workload requirements and determines optimal placement: **Analysis Factors:** * CPU, memory, and GPU requirements * Data access patterns and frequency * Network latency requirements * Data residency and compliance needs **Automated Management:** * Lifecycle management (scaling, updates, recovery) * Dynamic resource adjustment based on usage * Health monitoring and automatic failover * Zero-downtime rolling updates The orchestration layer automatically analyzes your data topology and places agents to optimize performance, cost, and compliance outcomes. ## Security & Isolation **Complete Data Isolation** * Your data never leaves your infrastructure * VantEdge control plane only manages orchestration * No data flows through VantEdge systems **Secure Communication** * Encrypted channels between control plane and VE Agents * RBAC for deployment permissions * Secrets management for credentials * Audit logging for all operations *** This architecture enables VantEdge to provide centralized management and intelligent orchestration while keeping your data secure and isolated in your own infrastructure. # Cloud Integrations Source: https://docs.vantedge.run/concepts/integrations Connect cloud providers to manage resources across workspaces ## What are Cloud Integrations? Cloud integrations connect your organization to external cloud providers (AWS, Google Cloud, Azure), enabling VantEdge to provision and manage cloud resources on your behalf across multiple workspaces. ## How Integrations Work **Organization-Level Scope** * Integrations belong to your organization, not individual workspaces * One integration can be shared across multiple workspaces * Centralized credential management with distributed usage **Cross-Workspace Usage** ``` Organization: "ML Startup" ├── Integration: "AWS Production Account" │ ├── Used by: "Model Training" workspace │ ├── Used by: "API Deployment" workspace │ └── Used by: "Analytics" workspace └── Integration: "GCP Development Account" ├── Used by: "Development" workspace └── Used by: "Experiments" workspace ``` ## Supported Providers ### Amazon Web Services (AWS) * **Authentication**: IAM roles or access keys * **Resources**: EKS clusters, EC2 instances, S3 storage, Secrets Manager * **Features**: Multi-account support, cross-region deployments ### Google Cloud Platform (GCP) * **Authentication**: Service account JSON keys * **Resources**: GKE clusters, Compute Engine, Cloud Storage, Secret Manager * **Features**: Workload identity, project-based organization ### Microsoft Azure * **Status**: Coming soon * **Planned**: AKS clusters, Virtual Machines, Blob Storage, Key Vault ## Integration Features **Connection Management** * Real-time status monitoring * Automatic credential validation * Connection testing and diagnostics * Smart retry mechanisms **Resource Organization** * Environment tagging (dev/staging/prod) * Multi-region support * Cost tracking and allocation * Resource lifecycle management ## Common Integration Patterns ### Single Account Pattern ``` Small Team Setup: └── AWS Integration: "Company Account" ├── Workspace: "Web App" ├── Workspace: "Mobile API" └── Workspace: "Analytics" ``` **Best for**: Small teams, cost optimization, simple setups ### Environment Separation ``` Enterprise Setup: ├── AWS Integration: "Development Account" │ └── Dev/staging workspaces └── AWS Integration: "Production Account" └── Production workspaces only ``` **Best for**: Security requirements, compliance, large teams ### Multi-Cloud Strategy ``` Advanced Setup: ├── AWS Integration: "Primary Workloads" ├── GCP Integration: "ML & Analytics" └── Azure Integration: "Legacy Systems" ``` **Best for**: Vendor diversification, specialized workloads *** Cloud integrations provide the bridge between VantEdge and your cloud infrastructure, enabling seamless resource management across your entire organization. # Organizations Source: https://docs.vantedge.run/concepts/organizations Understanding VantEdge's multi-tenant architecture and organization isolation ## What is an Organization? An **organization** in VantEdge is a completely isolated tenant environment. Each organization gets its own database schema, unique domain, and dedicated resources, ensuring complete data separation between different teams or companies. ## Multi-Tenant Architecture VantEdge operates as a **multi-tenant platform** where each organization is a separate tenant: ``` Organization: "Acme Corp" ├── Domain: acme.vantedge.run ├── Database Schema: org_acme_db ├── Isolated Resources: Complete separation └── Team Access: Only Acme Corp members Organization: "Beta Inc" ├── Domain: beta.vantedge.run ├── Database Schema: org_beta_db ├── Isolated Resources: Complete separation └── Team Access: Only Beta Inc members ``` ## Key Features **🔒 Complete Data Isolation** * Each organization has its own database schema * No cross-tenant data access possible * Encrypted data at rest and in transit **🌐 Unique Domains** * Every organization gets a custom subdomain * Clean, branded experience for your team * Easy bookmark and sharing within organization **👥 Team Management** * Organization-level user management * Role-based access controls * Invitation and member onboarding system **📊 Resource Separation** * Cloud integrations belong to organizations * Workspaces are organization-scoped * Complete billing and usage isolation ## Organization Types ### Personal Organizations * **Automatically created** when you sign up * **Single-user** environment for individual work * **Domain**: Usually `firstname-lastname.vantedge.run` * **Use cases**: Personal projects, learning, experimentation ### Team Organizations * **Manually created** for teams and companies * **Multi-user** collaborative environment * **Custom domain**: `company-name.vantedge.run` * **Use cases**: Team projects, production workloads, enterprise usage ## Organization Hierarchy ``` Organization: "Enterprise Corp" ├── Members: Users with different roles ├── Workspaces: Project containers │ ├── "AI Training - Production" │ ├── "Web App - Staging" │ └── "Data Pipeline - Development" ├── Integrations: Cloud provider connections │ ├── AWS Production Account │ ├── AWS Development Account │ └── GCP Analytics Account └── Resources: Distributed across workspaces ├── Kubernetes clusters ├── Model deployments ├── AI gateways └── Secrets and configurations ``` ## Organization Roles **Owner** * Full control over organization * Billing and subscription management * Can delete organization * Manage all members and workspaces **Admin** * Manage workspaces and members * Configure integrations and settings * Cannot delete organization or manage billing * Full access to all organization resources **Member** * Access to assigned workspaces * Can create resources within permitted workspaces * Limited admin functions * Cannot manage organization-level settings ## Getting Started **For Individuals:** 1. Sign up → Personal organization created automatically 2. Start with your default personal workspace 3. Add cloud integrations and begin building **For Teams:** 1. Create or join a team organization 2. Set up shared cloud integrations 3. Create project-specific workspaces 4. Invite team members with appropriate roles *** Organizations provide the foundation for VantEdge's secure, multi-tenant architecture, ensuring your team's data and resources remain completely isolated. # Workspaces Source: https://docs.vantedge.run/concepts/workspaces Project containers that organize resources within your organization ## What are Workspaces? Workspaces are **project containers** within your organization that provide resource isolation and team collaboration boundaries. Think of them as dedicated environments for specific projects, applications, or deployment stages. ## Key Benefits **🔒 Resource Isolation** Resources in different workspaces are completely separate - no cross-workspace interference. **🏗️ Project Organization** Group related deployments, experiments, and configurations in logical containers. **👥 Team Collaboration** Share workspaces with specific team members and assign granular permissions. **🔄 Environment Separation** Use different workspaces for development, staging, and production environments. ## Workspace Structure ``` Organization: "AI Startup" ├── Workspace: "Model Training" │ ├── Kubernetes clusters for training │ ├── Experiment configurations │ └── Training datasets and secrets ├── Workspace: "Production API" │ ├── Model deployment services │ ├── AI gateways for traffic routing │ └── Production monitoring and alerts └── Workspace: "Research & Development" ├── Experimental models ├── A/B testing configurations └── Prototype deployments ``` ## Workspace Types ### Default Workspaces * Every organization has **exactly one** default workspace * Personal orgs: "Personal" workspace * Team orgs: "Default" workspace (can be renamed) * Cannot be deleted (organization safety) ### Project Workspaces * Created manually for specific projects * Custom names and descriptions * Can be deleted when no longer needed * Full team collaboration features ### Environment Workspaces * Separate workspaces for different stages * Common pattern: dev/staging/production * Isolated secrets and configurations * Different access controls per environment ## Workspace Roles | Role | Create Resources | Manage Secrets | Add Members | Delete Workspace | | ---------- | ---------------- | -------------- | ----------- | ---------------- | | **Viewer** | ❌ | ❌ | ❌ | ❌ | | **Member** | ✅ | ✅ | ❌ | ❌ | | **Admin** | ✅ | ✅ | ✅ | ❌ | | **Owner** | ✅ | ✅ | ✅ | ✅ | ## Managing Workspaces ### Workspace Overview The Workspaces page shows all your available workspaces with key information: Workspaces overview showing workspace cards with member counts and roles Each workspace card displays: * **Workspace name and type** (Default, Personal, Active) * **Member count** and your role (owner, admin, member, viewer) * **Resource counts** (integrations, clusters, etc.) * **Quick actions** (switch workspace, settings) ### Workspace Settings Click the settings icon to manage workspace configuration: Workspace settings dialog showing name, description, and danger zone options **Available settings:** * **Name and Description**: Update workspace details * **Member Management**: Add/remove team members * **Danger Zone**: Delete workspace (only if no resources exist) ## What Goes in a Workspace? **Deployments** * AI model deployments on Kubernetes * Web applications and APIs * Microservices and containers **Infrastructure** * Kubernetes clusters (dedicated or shared) * AI gateways for traffic management * MCP servers for AI protocols **Configuration** * Experiment definitions and A/B tests * Environment-specific secrets * Deployment configurations **Monitoring** * Performance metrics and alerts * Resource usage tracking * Error monitoring and logging *** Workspaces provide the organizational structure that makes complex AI projects manageable, secure, and scalable across your team. # Agent Deployment Source: https://docs.vantedge.run/features/agent-deployment Deploy AI agents and workloads to Kubernetes with intelligent orchestration ## What is Agent Deployment? VantEdge provides an intelligent deployment architecture for AI agents, models, and workloads across Kubernetes environments. Our proprietary orchestration layer enables agents to run where your data lives, minimizing latency and maximizing performance through data locality. Built on award-winning research in edge computing and distributed stream processing, our deployment infrastructure brings proven techniques from real-time data systems to AI agent orchestration. Learn more about VantEdge's system architecture in the [Architecture Overview](/concepts/architecture-overview). ## Key Features **🎯 Data-Aware Deployment**\ Deploy agents in proximity to your data sources using intelligent placement algorithms. Minimize latency and transfer costs with cross-region and edge deployment support. **☁️ Multi-Cloud Orchestration**\ Unified management across AWS EKS, Google GKE, and Azure AKS. Deploy hybrid strategies with seamless workload migration between providers. **📊 Auto-Scaling & Optimization**\ Horizontal pod autoscaling based on load with resource-aware scheduling. Optimized for both GPU and CPU workloads with efficient utilization. **🔍 Monitoring & Observability**\ Real-time performance metrics and resource tracking. Automated health checks with centralized logging for debugging. ## Container & Agent Runtime Agents run in containerized environments with: * **Isolated execution contexts** for security and resource management * **Automatic dependency management** and version control * **Configurable resource limits** (CPU, memory, GPU allocation) * **Health monitoring** with automatic restart on failure * **Secrets injection** for secure credential management ## Supported Deployment Types **AI Agents** * Conversational agents and chatbots * Voice agents with real-time processing * Multi-agent systems with inter-agent communication * Tool-calling agents with API integrations **AI Models** * Language models (GPT, BERT, T5, custom LLMs) * Vision models (classification, detection, generation) * Embedding models and vector search * Custom models from HuggingFace or private registries **Processing Workloads** * Stream processing and real-time analytics * Batch inference pipelines * Data transformation and ETL * Multi-stage agent workflows ## Cluster Management ### Kubernetes Infrastructure VantEdge supports multiple cluster types: * **AWS EKS** - Managed Kubernetes with AWS integration * **Google GKE** - Autopilot and standard cluster modes * **Azure AKS** - Azure-native Kubernetes service * **Self-managed** - Custom K3s and edge deployments The platform provides one-click cluster provisioning with automated version updates and security patches. Node pool management includes auto-scaling, and the system manages add-ons like CSI drivers and monitoring tools. ### Scaling Strategies ```yaml theme={null} # Example: Auto-scaling Configuration agent_deployment: min_replicas: 2 max_replicas: 20 scaling_metrics: - cpu_utilization: 70% - memory_utilization: 80% - custom_metric: "agent_queue_depth" data_locality: prefer_same_az: true prefer_same_region: true max_latency_ms: 100 ``` The platform scales in multiple dimensions: * **Horizontal scaling** adjusts replica count based on metrics * **Vertical scaling** modifies resource requests/limits dynamically * **Cluster autoscaling** adds or removes nodes based on demand * **Predictive scaling** uses patterns to scale proactively ## Deployment Workflow **1. Define Deployment**\ Select your agent or model type, configure resource requirements (CPU, memory, GPU), set scaling policies, and define data access patterns. **2. Infrastructure Selection**\ Choose target Kubernetes clusters and regions. Configure networking, ingress rules, monitoring, and alerting thresholds. **3. Optimize Placement**\ The orchestration layer analyzes data access requirements, identifies optimal locations, configures caching strategies, and sets up low-latency networking. **4. Deploy & Monitor**\ Zero-downtime rolling updates with automated health checks. Real-time dashboards provide visibility, while cost tracking identifies optimization opportunities. ## Performance Optimization **Resource Efficiency**\ Automatically right-size containers based on usage patterns. Leverage spot instances for cost savings while optimizing storage and network performance. Minimize idle resource consumption. **Latency Optimization**\ Deploy agents near data sources with in-memory caching for hot data. Optimized network paths and pre-warmed database connections eliminate cold-start penalties. **Cost Management**\ Track per-deployment costs with detailed breakdowns. Identify optimization opportunities like right-sizing and unused resource cleanup. Leverage reserved capacity discounts where available. *** VantEdge Agent Deployment provides enterprise-grade orchestration that combines the power of Kubernetes with intelligent, data-aware placement strategies for optimal AI agent performance. # Model Deployment Source: https://docs.vantedge.run/features/model-deployment Deploy specialized SLMs and finetuned models alongside agents for ultra-low latency inference **Alpha Release** - Model Deployment is currently in active development. Features and APIs may change. Not recommended for production use. ## What is Model Deployment? Model Deployment enables you to deploy Small Language Models (SLMs) and specialized finetuned models directly alongside your AI agents. By co-locating models with agents and data, you can achieve lower inference latency while eliminating external API dependencies and costs. ## Why Co-Located Models? **Latency Reduction**\ Deploying models where your agents run eliminates network roundtrips to external APIs. Local inference removes the overhead of external API calls. **Cost Efficiency**\ Avoid per-token API pricing by running your own optimized models. For high-volume workloads, self-hosted models can significantly reduce inference costs. **Data Locality**\ Models deployed near your data sources access context faster and more securely. No data leaves your infrastructure, improving both performance and compliance. **Model Specialization**\ Deploy domain-specific finetuned models optimized for your use cases. Smaller, specialized models often outperform larger general-purpose models for specific tasks. **When to Use Co-Located Models:** * High-volume inference workloads * Latency-sensitive applications (voice agents, real-time chat) * Domain-specific tasks where SLMs perform well * Data privacy and compliance requirements * Cost optimization for predictable workloads ## Supported Model Types **Small Language Models (SLMs)** * Phi-3, Llama-3 8B, Mistral 7B, and other compact models * Domain-specific finetuned variants * Quantized models for efficient inference * Multi-lingual and task-specific models **Specialized Models** * Custom finetuned models from your training pipelines * Embedding models for vector search and retrieval * Classification and sentiment analysis models * Named entity recognition and extraction models **Model Sources** * HuggingFace model hub * Custom model registries * Private model repositories * Local model artifacts ## Deployment Architecture ``` Agent + Model Co-Location ├── Kubernetes Pod │ ├── Agent Container │ │ ├── Business Logic │ │ ├── Tool Calling │ │ └── Data Access Layer │ ├── Model Container │ │ ├── SLM Inference Engine │ │ ├── Model Weights (8GB) │ │ └── Local Cache │ └── Shared Resources │ ├── Unix Domain Sockets │ ├── Shared Memory │ └── Local Network (localhost) └── Data Sources ├── PostgreSQL (same AZ) ├── Vector Store (co-located) └── S3 Cache (regional) ``` **Co-location benefits:** * Reduced network overhead through localhost communication * No egress costs as data and inference stay within the same pod/node * Efficient resource sharing between agents and models * Simplified networking without complex service mesh configuration ## Deployment Configuration ### Basic Model Deployment ```yaml theme={null} model_deployment: name: customer-support-phi3 model: source: microsoft/Phi-3-mini-4k-instruct quantization: int8 max_tokens: 4096 resources: gpu: nvidia-t4 memory: 16Gi replicas: 2 co_location: agent: support-agent prefer_same_pod: true prefer_same_node: true ``` ### Advanced Configuration ```yaml theme={null} model_deployment: name: legal-document-analyzer model: source: private-registry/legal-llama-7b optimization: onnx batch_size: 8 scaling: min_replicas: 1 max_replicas: 10 inference: framework: vllm tensor_parallel: 2 kv_cache_size: 8Gi ``` ## Model Optimization **Quantization**\ Deploy models with 8-bit or 4-bit quantization to reduce memory footprint with minimal accuracy loss. Ideal for SLMs running on CPU or smaller GPUs. **Inference Frameworks**\ Choose from optimized inference engines like vLLM (high throughput), TGI (tensor optimization), or Ollama (developer-friendly). Each framework offers different performance tradeoffs. **Resource Allocation**\ SLMs typically require 8-16GB memory and 4-8 CPU cores. GPU acceleration (T4, A10) provides faster inference for high-throughput workloads. **Caching Strategies**\ Implement KV-cache for repeated prompts and prompt caching for common prefixes. For agent workflows with consistent system prompts, caching can significantly reduce latency. ## Integration with Agents Models deployed alongside agents communicate via: * **HTTP/REST**: Standard inference API * **gRPC**: Efficient binary protocol * **Unix Domain Sockets**: Fastest option for same-pod communication * **Shared Memory**: Direct memory access for maximum performance The agent deployment system automatically configures the optimal communication pattern based on co-location settings. ## Future: Continuous Model Learning **Coming Soon**: Support for continuous model finetuning with real-time data streams is under development. This will enable models to learn from production data and improve over time without retraining cycles. The model deployment architecture is designed to support future continuous learning capabilities: * Real-time data ingestion from agent interactions * Incremental model updates without downtime * A/B testing between model versions * Automated evaluation and rollback This positions VantEdge to support the next generation of adaptive AI systems that continuously improve from production data. *** Model Deployment brings inference closer to your agents and data, enabling the performance required for production AI applications. # Secrets Management Source: https://docs.vantedge.run/features/secrets-management Secure storage for API keys, credentials, and sensitive configuration ## What is Secrets Management? VantEdge provides secure, encrypted storage for sensitive information like API keys, database passwords, and service credentials. Secrets are stored in cloud-native secret managers (AWS Secrets Manager, GCP Secret Manager) and automatically injected into your deployments. ## Key Features **🔒 Cloud-Native Security**\ Secrets stored in your cloud provider's managed services with encryption at rest and in transit. No secrets stored on VantEdge infrastructure. **👥 Role-Based Access**\ Workspace-level isolation with granular permissions. Only admins and owners can retrieve secret values—members can create and reference secrets without seeing actual values. **🔄 Automatic Injection**\ Secrets automatically injected into agent deployments, model containers, and applications as environment variables or mounted files. **📊 Complete Audit Trail**\ Every secret access, creation, and modification is logged for compliance and security monitoring. ## Secret Types **API Tokens**\ Service-specific API keys for OpenAI, HuggingFace, Anthropic, and other platforms. **Database Credentials**\ Connection strings, passwords, and authentication details for PostgreSQL, MongoDB, Redis, and other data stores. **Service Credentials**\ OAuth tokens, JWT secrets, and other service authentication credentials. **Generic Secrets**\ Any sensitive configuration values, encryption keys, or custom credentials. ## Creating Secrets ```yaml theme={null} # Example secret configuration name: "openai-api-key" type: "api-token" value: "sk-proj-..." description: "OpenAI API key for GPT-4 models" workspace: "production" ``` Secrets are created through the platform UI or API and immediately encrypted in your cloud provider's secret manager. ## Using Secrets in Deployments **Environment Variables** ```yaml theme={null} # Automatic injection into containers env: - name: OPENAI_API_KEY valueFrom: secretKeyRef: name: openai-api-key - name: DATABASE_URL valueFrom: secretKeyRef: name: postgres-connection ``` **File Mounts** ```yaml theme={null} # Mount secrets as files volumes: - name: api-credentials secret: secretName: service-account-key volumeMounts: - name: api-credentials mountPath: /etc/secrets readOnly: true ``` Deployments automatically receive the latest secret values without manual updates. ## Permission Model | Role | Create | View List | Retrieve Values | Update | Delete | | ---------- | ------ | --------- | --------------- | ------ | ------ | | **Viewer** | ❌ | ✅ | ❌ | ❌ | ❌ | | **Member** | ✅ | ✅ | ❌ | ✅ | ✅ | | **Admin** | ✅ | ✅ | ✅ | ✅ | ✅ | | **Owner** | ✅ | ✅ | ✅ | ✅ | ✅ | **Security Features:** * Write-only by default (values hidden after creation) * Admin-only value retrieval prevents accidental exposure * Complete audit logging tracks all access * Encryption using cloud provider services ## Workspace Isolation ``` Organization: "AI Startup" ├── Workspace: "Production" │ ├── openai-api-key (prod key) │ ├── database-password (prod DB) │ └── stripe-api-key (live mode) └── Workspace: "Development" ├── openai-dev-key (dev key) ├── test-db-password (dev DB) └── stripe-test-key (test mode) ``` Secrets in different workspaces are completely isolated—no cross-workspace access is possible. ## Integration with Deployments Secrets automatically integrate with: * **Agent deployments** for API access and authentication * **Model deployments** for inference service credentials * **Data sources** for database and storage authentication * **Tool integrations** for Slack, Gmail, and other services Reference secrets by name in your deployment configurations, and VantEdge handles the secure injection automatically. ## Cloud Provider Support **AWS Secrets Manager**\ Encryption with AWS KMS, IAM-based access control, automatic rotation support, and multi-region replication. **Google Cloud Secret Manager**\ Google-managed encryption keys, IAM policies, version management, and global access. **Azure Key Vault**\ Coming soon with Azure Active Directory integration and FIPS 140-2 compliance. *** Secrets Management provides secure credential storage with automatic injection into your AI deployments, ensuring sensitive data is protected while remaining accessible to authorized applications. # Welcome to VantEdge Source: https://docs.vantedge.run/index Data-aware AI infrastructure for enterprise agents and voice applications ## Data-Aware AI Infrastructure VantEdge is the **complete platform for enterprise AI agents** - providing intelligent data access, model deployment, and orchestration optimized for real-time applications. Built on award-winning research in distributed systems and stream processing. ## Platform Overview Universal data access layer with intelligent caching and tool integration for sub-100ms queries Deploy agents near data sources with intelligent orchestration across multi-cloud environments Co-locate SLMs with agents for sub-50ms inference latency and 60-80% cost reduction Complete isolation with dedicated infrastructure and unique domains per organization ## Core Capabilities ### Context Management **Universal data access** across heterogeneous sources with intelligent query routing and translation. Access PostgreSQL, MongoDB, vector databases, and SaaS tools (Slack, Gmail, Salesforce) through a unified interface. **Key features:** * Sub-100ms data access with multi-tier caching * Tool calling for external API integration * Semantic query understanding and optimization * Real-time data ingestion for voice agents ### Agent & Model Orchestration **Data-aware deployment** that places agents and models near data sources for optimal performance. Automatic scaling, health monitoring, and zero-downtime updates. **Key features:** * Co-located SLMs for ultra-low latency inference * Multi-cloud Kubernetes management (AWS, GCP, Azure) * Intelligent placement based on data topology * Horizontal and vertical auto-scaling ### Voice Agent Optimization **Purpose-built for voice applications** requiring real-time data access and sub-second response times. Optimized caching, failover strategies, and context continuity. **Key features:** * Sub-100ms data queries for real-time conversations * Cached responses for common queries and FAQs * Context persistence across multi-turn conversations * Healthcare, customer support, and sales use cases ## Architecture Principles **Data Locality First**\ Deploy agents and models where your data lives. Minimize latency and egress costs through intelligent placement. **Intelligent Caching**\ Multi-tier caching achieves 70-90% cache hit rates with sub-10ms response times for common agent queries. **Research-Backed**\ Built on award-winning research in edge computing, distributed stream processing, and real-time data systems. *** **Getting Started**\ Explore [Context Router](/solutions/context-router) for data access, [Agent Deployment](/features/agent-deployment) for orchestration, or learn about [Organizations](/concepts/organizations) and workspace management. # Context Router Source: https://docs.vantedge.run/solutions/context-router Universal data access layer enabling AI agents to query heterogeneous data sources with intelligent caching and tool integration **Alpha Release** - Context Router is currently in active development. Features and APIs may change. ## What is Context Router? Context Router is a **universal data access layer** that enables AI agents to interact with diverse data sources through a unified interface. It transforms fragmented organizational data across databases, SaaS tools, and APIs into an accessible, governed data fabric optimized for agent workflows. Designed specifically for voice agents and real-time applications, Context Router delivers sub-100ms data access with intelligent caching and query optimization. ## Core Problem AI agents need access to organizational data, but that data is scattered across: **Data Systems** * Relational databases (PostgreSQL, MySQL, Snowflake) * NoSQL stores (MongoDB, DynamoDB, Redis) * Vector databases (Pinecone, Weaviate, Qdrant) * Knowledge graphs (Neo4j) **SaaS Tools & APIs** * Communication platforms (Slack, Gmail, Microsoft Teams) * CRM systems (Salesforce, HubSpot) * Project management (Jira, Asana, Linear) * Document stores (Google Drive, Notion, Confluence) Each system has its own query language, authentication model, rate limits, and access patterns. Context Router provides a unified interface while handling the complexity of routing, translation, caching, and tool calling. ## Key Features ### Natural Language Queries Developers interact with data using natural language—Context Router handles the complexity of translating queries to the appropriate backend languages internally. ```python theme={null} from vantedge import VantEdgeClient client = VantEdgeClient( context_router_url="http://localhost:8000", api_key="your_api_key" ) # Natural language queries - no SQL required result = client.context_router.query("Show me all pending orders for building 123") result = client.context_router.query("What's the work center capacity at plant 5?") result = client.context_router.query("Find recent emails about the Q4 roadmap") ``` ### Intelligent Query Planning Context Router uses LLM-powered query planning to: * Understand natural language intent * Determine which data sources to query * Generate optimized backend queries (SQL, API calls, etc.) * Merge results from multiple sources when needed ### Semantic Caching Multi-tier caching system with semantic understanding: * Similar queries share cached results (e.g., "Show recent tickets" and "Display latest support requests") * Configurable TTL and similarity thresholds * Sub-10ms response time for cached queries * 70-90% cache hit rate for common queries ### Multi-Source Data Access Query across heterogeneous data sources through a single interface: * **Databases**: PostgreSQL, MySQL, and more * **SaaS Tools**: Gmail, Slack, and other integrations * **APIs**: REST endpoints and custom connectors ### Data Synchronization Sync data from external sources to local storage for faster querying: * Schedule automated syncs (e.g., sync last 30 days of emails daily) * Query synced data with sub-millisecond latency * Manage sync jobs through the Management API *** ## Python Client SDK The VantEdge Python Client provides a comprehensive SDK for interacting with Context Router, including querying data and managing configuration. ### Installation ```bash theme={null} pip install vantedge-client ``` ### Basic Usage ```python theme={null} from vantedge import VantEdgeClient # Initialize the client client = VantEdgeClient( context_router_url="http://localhost:8000", api_key="your_api_key", timeout=30, max_retries=3 ) # Natural language query result = client.context_router.query( query="Show me all pending orders for building 123", user_id="agent_001" ) print(f"Found {result.count} results") print(f"Sources used: {result.sources_used}") print(f"Latency: {result.latency_ms}ms") print(f"Cache hit: {result.cache_hit}") # Access individual results for item in result.data: print(item.data) # Health check health = client.context_router.health_check() # Get available data sources sources = client.context_router.get_sources() # Close when done client.close() ``` *** ## Management API The Management API enables remote configuration of Context Router components including connectors, cache, LLM settings, and data synchronization. ### Connector Management Manage data source connectors at runtime. ```python theme={null} from vantedge import VantEdgeClient, PostgresConfig client = VantEdgeClient( context_router_url="http://localhost:8000", api_key="your_api_key" ) # List all connectors connectors = client.management.list_connectors() for conn in connectors: print(f"{conn.name}: {conn.status} ({conn.type})") # Get specific connector details connector = client.management.get_connector("orders") # Create a new PostgreSQL connector config = PostgresConfig( host="db.example.com", port=5432, database="orders_db", user="readonly_user", password="secure_password", description="Orders database" ) new_connector = client.management.create_postgres_connector("orders", config) # Update connector configuration updated = client.management.update_connector( name="orders", timeout=60 ) # Get connector schema (tables and columns) schema = client.management.get_connector_schema("orders") # Delete a connector client.management.delete_connector("old_db") ``` ### Cache Management Configure and manage the semantic caching layer. ```python theme={null} # Get current cache settings settings = client.management.get_cache_settings() print(f"TTL: {settings.ttl_seconds}s, Threshold: {settings.semantic_threshold}") # Update cache settings new_settings = client.management.update_cache_settings( ttl_seconds=600, # 10 minute TTL semantic_threshold=0.9 # Higher similarity required ) # Get cache statistics stats = client.management.get_cache_stats() print(f"Cached queries: {stats.total_keys}") print(f"Memory used: {stats.memory_used_bytes} bytes") # List cached entries entries = client.management.list_cache_entries(limit=50, offset=0) for entry in entries['entries']: print(f"Hash: {entry.query_hash}, TTL remaining: {entry.ttl_remaining}s") # Clear entire cache client.management.invalidate_cache() # Clear cache by pattern client.management.invalidate_cache(pattern="orders") # Clear cache for specific source client.management.invalidate_cache(source="gmail") ``` ### LLM Configuration Configure the LLM provider used for query planning. ```python theme={null} from vantedge import LLMProvider # Get current LLM settings llm_settings = client.management.get_llm_settings() print(f"Provider: {llm_settings.provider}") print(f"Model: {llm_settings.model}") # Switch to Groq (faster inference) new_settings = client.management.update_llm_settings( provider=LLMProvider.GROQ, model="llama-3.3-70b-versatile", temperature=0.1 ) # Switch to Anthropic client.management.update_llm_settings( provider=LLMProvider.ANTHROPIC, model="claude-sonnet-4-20250514" ) # Test LLM connectivity result = client.management.test_llm(test_query="What tables exist?") if result.success: print(f"LLM responding in {result.latency_ms}ms") else: print(f"LLM error: {result.error}") ``` ### Data Sync Synchronize data from external sources (e.g., Gmail) to local PostgreSQL for faster querying. ```python theme={null} # List all sync jobs jobs = client.management.list_sync_jobs() # Create a sync job for Gmail sync_config = { "days_back": 30, "filters": { "labels": ["INBOX", "SENT"], "from_address": "team@company.com", "subject_contains": "invoice" }, "include_attachments": False, "max_items": 500 } schedule = { "enabled": True, "cron": "0 2 * * *" # Daily at 2am } job = client.management.create_sync_job( name="daily-email-sync", source_connector="gmail", target_connector="postgres", sync_config=sync_config, schedule=schedule ) print(f"Created sync job: {job.job_id}") # Get sync job details job_info = client.management.get_sync_job(job.job_id) # Update sync job updated_job = client.management.update_sync_job( job_id=job.job_id, sync_config=SyncConfig(days_back=7), enabled=True ) # Run sync manually execution = client.management.run_sync_job( job_id=job.job_id, days_back=7, # Override config force_full=False # Incremental sync ) print(f"Sync started: {execution.execution_id}") # Check sync status status = client.management.get_sync_status(job.job_id) print(f"Status: {status.status}") print(f"Progress: {status.progress_pct}%") print(f"Items synced: {status.items_synced}") # Get sync history history = client.management.get_sync_history(job.job_id, limit=10) for exec in history['executions']: print(f"{exec.started_at}: {exec.status} - {exec.items_synced} items") # Cancel running sync if status.status == "running": client.management.cancel_sync_job(job.job_id) # Get synced data statistics stats = client.management.get_synced_data_stats(job.job_id) print(f"Total items: {stats.total_items}") print(f"Date range: {stats.date_range_start} to {stats.date_range_end}") # Purge old synced data result = client.management.purge_synced_data( job_id=job.job_id, older_than_days=90, confirm=True # Required to execute ) print(f"Purged {result['deleted_count']} old records") # Delete sync job client.management.delete_sync_job(job.job_id) ``` ### Planner Hints Inject domain-specific knowledge into the query planner to guide how it interprets natural language queries. ```python theme={null} # List all hints hints = client.management.list_hints() # Create hints to guide the planner client.management.create_hint( name="building-terminology", content="When users ask about 'buildings', query the 'facilities' table using facility_id", category="terminology", priority=10 ) client.management.create_hint( name="order-status", content="Order status values are: 'pending', 'processing', 'shipped', 'delivered', 'cancelled'", category="business_rules", priority=5 ) client.management.create_hint( name="date-format", content="Dates in the database are stored in UTC. Convert user timezone references accordingly.", category="schema", priority=8 ) # Update a hint client.management.update_hint( hint_id="abc123", content="Updated hint content", enabled=True ) # Preview how hints appear in the system prompt preview = client.management.preview_hints_prompt() print(f"Active hints: {preview['enabled_hint_count']}") print(preview['prompt_section']) # Delete a hint client.management.delete_hint("abc123") # Clear all hints client.management.clear_hints() ``` *** Context Router enables AI agents to access organizational data with sub-100ms latency while maintaining security, governance, and compliance.