gpu-management

Heterogeneous GPU Sharing on Kubernetes

Open-source, self-hosted AI gateway for multi-tenant orgs: OpenAI-compatible LLM routing, RAG & vector stores, MCP hub, GPU fleet (MIG slicing), sandboxed coding agents, AI red-teaming & guardrails, and cost optimization — all in one console.

TensorFusion landing page and product docs

The Ollama developer experience with the vLLM production power. Deploy local LLMs via Docker with smart and automatic GPU VRAM management.

MCP-based GPU coordination for AI agents, with human oversight across multiple servers.

Ultra-low-latency speech-to-text with intelligent GPU management - Enhanced version with lazy loading, auto resource release, and modern multi-language UI

A lightweight GPU/node resource locking bot for teams — manage exclusive & shared access to cluster nodes and devices via chat commands or web UI.

🎙️ High-quality Text-to-Speech system based on Llasa-8B with intelligent GPU memory management. Features: 96% memory savings, Web UI + REST API + MCP, auto GPU selection, Docker deployment.

An intelligent NVIDIA GPU task scheduler with real-time monitoring and automatic job execution. Features persistent task queues, daemon scheduling, and multi-notification support for efficient GPU resource management.

A simple tool to expose only specified number of GPUs with desired memory to Tensorflow

ML Framework and CUDA Checker is a Python-based GUI application for checking PyTorch, TensorFlow, and CUDA installations. It provides detailed system specs, compatibility checks, advanced GPU management, and offers options to view instructions, export logs, and update machine learning frameworks.

A lightweight and portable suite of Windows batch scripts to start, monitor, update, and completely stop Ollama to release GPU VRAM and system memory.

GPU management on multi-GPU platforms

Multi-server GPU slot management for LM Studio and other OpenAI-compatible inference backends