inference-aiops

This plugin provides root-cause analysis and operational governance for GPU inference clusters running vLLM, Ray Serve, SGLang, or TGI, with 39 tools for metrics diagnosis, scaling, draining, and model management backed by an audit log and policy engine. It reads Prometheus metrics directly from inference endpoints and the Ray dashboard, performs latency and utilization diagnostics, and records all operations (reads and writes) to a local audit database with risk-tier labels for governance tracking.