The AI Ecosystem: Model Servers, Gateways, Agent Platforms and More
A map of the AI software ecosystem by category: model servers, chat interfaces, agent platforms, gateways, vector databases, MCP servers and cloud services.
Last updated
165 projects across 11 categories. A dash means the entry is listed but not yet described.
| Category | What it covers | Projects |
|---|---|---|
| Model servers | Programs that run an AI model and answer requests. | 32 |
| Chat web interfaces | Browser front ends for talking to a model. | 17 |
| Agent and workflow platforms | Platforms for building agents or AI workflows. | 26 |
| AI gateways and proxies | Sit in front of one or more model providers. | 12 |
| Vector databases and AI memory | Search and memory stores for AI apps. | 12 |
| Training and experiment tools | Fine-tuning, experiment tracking and observability. | 8 |
| Notebooks and image/audio tools | Interactive notebooks and media generation. | 7 |
| MCP servers | Tool servers that agents call. | 11 |
| Cloud-hosted AI services | Managed model and AI platforms. | 11 |
| Personal AI agent apps and bots | Self-hosted assistants and chat bots. | 8 |
| Other / not sure what it is | Not yet classified. | 21 |
Model servers
Programs that run an AI model and answer requests.
| Project | What it is | Project | What it is |
|---|---|---|---|
| ollama | Runs open models locally with one command | powerinfer | Fast inference on consumer GPUs using neuron sparsity |
| vllm | High-throughput inference engine built on PagedAttention | petals | Runs large models across volunteer machines, BitTorrent-style |
| sglang | Fast serving framework for LLMs and multimodal models | fastchat | LMSYS platform for serving and evaluating chat models |
| llama-cpp | C/C++ inference for GGUF models on CPUs and GPUs | xinference | Server for LLM, embedding and multimodal models |
| localai | Self-hosted, OpenAI-compatible API for local models | gpt4all | Desktop app and runtime for local chat models |
| lmdeploy | Compression and serving toolkit from the InternLM team | lm-studio | Desktop app for downloading and running local models |
| lightllm | Lightweight Python LLM inference and serving framework | llmstudio | Unified API and prompt tooling for LLM providers |
| aphrodite-engine | vLLM-based engine for large-scale serving | jan | Open-source offline ChatGPT alternative |
| koboldcpp | Single-file llama.cpp runner with a story and chat UI | nvidia-nim | NVIDIA inference microservices shipped as containers |
| tabbyapi | OpenAI-compatible API server for ExLlama models | bentoml | Framework for packaging and serving ML models |
| text-generation-webui | Gradio web UI for running local models | baseten-truss | Baseten's open-source model packaging format |
| huggingface-tgi | Hugging Face Text Generation Inference server | ray | Distributed compute framework; Ray Serve hosts models |
| tensorrt-llm | NVIDIA's optimized LLM inference library | dynamo | NVIDIA's distributed inference serving framework |
| triton | NVIDIA Triton Inference Server for multi-framework serving | kubeai | Kubernetes operator for serving open models |
| deepspeed-mii | Microsoft DeepSpeed low-latency inference library | instructlab | Tooling for tuning models with synthetic data |
| mlc-llm | Compiles models to run on phones, browsers and GPUs | openai-compatible | Any server that copies the OpenAI API |
Chat web interfaces
Browser front ends for talking to a model.
| Project | What it is | Project | What it is |
|---|---|---|---|
| openwebui | Self-hosted chat UI for Ollama and OpenAI-compatible backends | huggingface-chat-ui | Open-source UI behind HuggingChat |
| librechat | ChatGPT-style UI that supports many providers | gradio | Python library for quick ML web demos |
| lobechat | Modern chat UI with plugins and multi-provider support | chat-langchain | LangChain's reference documentation chatbot |
| nextchat | Lightweight cross-platform ChatGPT-style client | ai-chatbot | Vercel's Next.js chatbot template |
| anythingllm | All-in-one chat with documents, RAG and agents | privategpt | Chat with your documents fully offline |
| sillytavern | Roleplay and character chat front end | h2ogpt | H2O.ai private document Q&A and chat |
| betterchatgpt | Open-source ChatGPT-style UI using your own API key | lollms | Lord of Large Language Models web UI |
| chuanhugpt | Gradio chat UI for many LLMs | qanything | Youdao's RAG question-answering system |
| fastchat-webui | FastChat's Gradio-based web interface |
Agent and workflow platforms
Platforms for building agents or AI workflows.
| Project | What it is | Project | What it is |
|---|---|---|---|
| dify | LLM app platform with visual workflows and RAG | praisonai | Low-code multi-agent framework |
| fastgpt | Knowledge-base Q&A and workflow platform | superagi | Open-source autonomous agent framework |
| flowise | Drag-and-drop LLM flow builder | agentic-flow | — |
| langflow | Visual builder for agents and RAG flows | upsonic | Python agent framework |
| n8n.io | Workflow automation with AI nodes | devika | Open-source agentic software engineer |
| n8n-mcp | MCP server that exposes n8n nodes and workflows to agents | suna | General-purpose open-source AI agent from Kortix |
| maxkb | Knowledge-base Q&A and agent platform | langserve | Deploys LangChain runnables as REST APIs |
| ragflow | RAG engine built around deep document understanding | hayhooks | Serves Haystack pipelines over HTTP |
| weknora | Tencent's document understanding and RAG framework | pipecat | Framework for real-time voice and multimodal agents |
| autogpt | Early autonomous agent platform | hatchet | Task queue and workflow orchestration |
| autogen-studio | No-code UI for Microsoft AutoGen | trigger.dev | Background jobs and workflows platform |
| crewai | Role-based multi-agent framework | nocobase | No-code platform with AI features |
| langroid | Multi-agent Python framework | obot | Platform for hosting MCP servers and agents |
AI gateways and proxies
Sit in front of one or more model providers.
| Project | What it is | Project | What it is |
|---|---|---|---|
| litellm | Unified OpenAI-format proxy for 100+ providers | kong-proxy | Kong API gateway with AI proxy plugins |
| new-api | LLM gateway with key and quota management | envoy-ai-gateway | Envoy-based gateway for AI traffic |
| sub2api | — | cloudflare-ai-gateway | Cloudflare's hosted caching and analytics gateway |
| bifrost | Fast LLM gateway from Maxim AI | tensorzero | Open-source LLM gateway, observability and optimization |
| portkey-ai-gateway | Portkey's open-source AI gateway | omniroute | — |
| helicone | Observability proxy for LLM requests | 9router | — |
Vector databases and AI memory
Search and memory stores for AI apps.
| Project | What it is | Project | What it is |
|---|---|---|---|
| chroma | Open-source embedding database | graphiti | Temporal knowledge graphs for agent memory |
| milvus | Distributed vector database | cognee | Memory and knowledge-graph engine for agents |
| qdrant | Vector search engine written in Rust | lightrag | Graph-based RAG framework |
| weaviate | Open-source vector database with hybrid search | feast | Open-source ML feature store |
| clickhouse | Columnar analytics database with vector search | quivr | Second-brain RAG assistant |
| mem0 | Memory layer for AI agents | memos | — |
Training and experiment tools
Fine-tuning, experiment tracking and observability.
| Project | What it is | Project | What it is |
|---|---|---|---|
| mlflow | Experiment tracking and model registry | llama-factory | Unified fine-tuning UI and CLI |
| kubeflow | ML workflows on Kubernetes | ms-swift | ModelScope's fine-tuning and deployment framework |
| tensorboard | Training metrics visualizer | langfuse | Tracks what an AI app does: traces, evals, prompts |
| axolotl | Fine-tuning framework driven by YAML configs | dask_http | HTTP interface of a Dask cluster |
Notebooks and image/audio tools
Interactive notebooks and media generation.
| Project | What it is | Project | What it is |
|---|---|---|---|
| jupyter-lab | Next-generation Jupyter notebook interface | comfyui | Node-based image generation UI |
| jupyter-notebook | Classic Jupyter notebook server | f5-tts | Flow-matching text-to-speech model |
| jupyter-server | Backend that powers Jupyter front ends | gpt-sovits | Few-shot voice cloning and text-to-speech |
| marimo | Reactive Python notebook |
MCP servers
Tool servers that agents call.
| Project | What it is | Project | What it is |
|---|---|---|---|
| mcp | Model Context Protocol endpoint | meta-ads-mcp | Meta Ads management over MCP |
| mcp-server | Generic MCP server | netlicensing-mcp | NetLicensing license management over MCP |
| mcphub | Hub for managing multiple MCP servers | notebooklm-mcp | Google NotebookLM access over MCP |
| mcp-atlassian | Jira and Confluence over MCP | context7 | Up-to-date library docs for coding agents |
| mcp-documentation-server | Document search over MCP | dbhub | Database access over MCP |
| mcp-searxng | Web search through SearXNG over MCP |
Cloud-hosted AI services
Managed model and AI platforms.
| Project | What it is | Project | What it is |
|---|---|---|---|
| aws-bedrock | Amazon's managed foundation model service | groq | Fast hosted inference on custom LPU chips |
| azure-openai | OpenAI models hosted on Azure | replicate | Run open models through an API |
| vertex-ai | Google Cloud's ML and model platform | modal | Serverless GPU compute |
| databricks-model-serving | Databricks managed model endpoints | salesforce-einstein | Salesforce's built-in AI features |
| fireworks-ai | Fast hosted inference for open models | qnabot-on-aws | AWS solution for conversational Q&A bots |
| together-ai | Hosted open models and fine-tuning |
Personal AI agent apps and bots
Self-hosted assistants and chat bots.
| Project | What it is | Project | What it is |
|---|---|---|---|
| openclaw | Personal AI assistant that runs on your own devices | hermes | — |
| clawdbot | Earlier name of OpenClaw | paperclip | — |
| astrbot | Multi-platform chatbot framework | agenticmail | — |
| langbot | Chatbot platform for messaging apps | junoclaw | — |
Other / not sure what it is
Not yet classified.
| Project | What it is | Project | What it is |
|---|---|---|---|
| ai-agent-config | — | lumiverse | — |
| ai-copilot | — | network-ai | — |
| ai-engine-wordpress | AI Engine plugin for WordPress | omnigent | — |
| blinko | Self-hosted AI note-taking | onyx | Open-source enterprise search and chat |
| boxlite | — | open-genai-stack | — |
| crawl4ai | LLM-friendly web crawler | open-generative-ai | — |
| decepticon | — | pinchtab | — |
| eclipse-theia | Extensible cloud and desktop IDE platform | qwed | — |
| emlog | Blogging CMS | ruflo | — |
| flyto2 | — | trek | — |
| kubepi | Kubernetes management panel |