Model Deployment
8 min read
Serving Open-Source LLMs with vLLM on Kubernetes
A practical guide to deploying open-source LLMs with vLLM on Kubernetes — covering GPU sizing, request routing, autoscaling, batching, and safe rollouts.
AI & MLOps Blog
AI Infrastructure Insights & Production Lessons
We share everything we learn — real use cases, real production lessons. Technical deep-dives on MLOps, model deployment, AI reliability, and more.
📝 Building in public
Posts authored by the Resilio Tech Team. More in-depth tutorials and case studies coming soon.
🚀
3+
Occasional Updates
Quality content on DevOps, Cloud & MLOps
8 articles found
Category: Model Deployment
Browse Categories
Model Deployment
6 min read
Model Canary Releases: Shadow Traffic, Rollbacks, and Safe Promotion
A practical guide to rolling out ML models safely in production using shadow traffic, canary promotion, quality gates, and fast rollback paths.
Model Deployment
6 min read
GPU Autoscaling: Right-Sizing Inference Clusters Without Over-Provisioning
How to autoscale GPU-backed inference clusters without wasting money, including queue-based scaling, warm capacity, and right-sizing by workload profile.
Model Deployment
7 min read
How We Cut GPU Costs by 40% Without Sacrificing Model Performance
Practical strategies for reducing GPU infrastructure costs — covering spot instances, GPU scheduling, model optimization, and right-sizing — without degrading inference quality.
Model Deployment
5 min read
Multi-Model Serving: Running Dozens of Models on Shared Infrastructure
How to serve many ML models on shared infrastructure without noisy-neighbor problems, unpredictable latency, or runaway GPU spend.
Model Deployment
5 min read
Load Testing LLM Endpoints: Why Traditional Tools Don't Work
Why standard API load-testing assumptions break for LLM inference, and how to design tests that reflect token generation, concurrency, and real serving bottlenecks.
Model Deployment
5 min read
LLM Token Economics: Tracking and Controlling Inference Spend
How to measure token-level inference spend in production and add practical controls around prompt size, output limits, routing, caching, and tenant budgets.
Model Deployment
4 min read
Spot Instances for Training Workloads: Checkpointing and Fault Tolerance
How to run ML training workloads on spot or preemptible capacity safely, with checkpointing, interruption handling, retry policy, and pipeline design for fault tolerance.
Browse by Category
Trending Topics
Latest Posts
Production RAG Systems: A Reliability Checklist
3/30/2026 • 6 min read
Serving Open-Source LLMs with vLLM on Kubernetes
3/29/2026 • 8 min read
Why Your ML Models Fail in Production (And How to Fix It)
3/28/2026 • 6 min read
AI Observability: Metrics and Dashboards That Actually Matter
3/27/2026 • 5 min read