AIHub
An all-in-one local AI management platform built around Ollama. Hardware-aware model selection, persistent Markdown-based memory, and a six-tool agentic system — all running locally with zero cloud dependency.
A practical hub for running local LLMs, end-to-end.
AIHub started as a personal pain point: running models locally on Ollama means juggling a CLI for downloads, separate context files for each session, no easy way to know which models actually fit on a given GPU, and no built-in way to give the model access to your filesystem or terminal.
The tool wraps Ollama with a CLI-first orchestrator that fixes all of that. It detects your hardware, recommends compatible models from a curated registry, keeps a long-term memory in editable Markdown, and exposes a controlled tool surface (file ops, terminal, web search) when you want the model to act, not just chat.
The design goal: behave like a real assistant on commodity hardware, with no cloud dependencies and no opaque state.
Three gaps in the local-AI workflow.
Models that don't fit your GPU
Pulling a 70B model onto a 12GB card wastes a download and silently degrades to CPU. AIHub reads VRAM, RAM, and CPU before listing what's actually runnable.
No memory between sessions
Ollama is stateless. Every chat starts cold. AIHub stores per-model and global memory as Markdown files you can read, edit, and version-control like any other notes.
"Agent" tools that need a framework
Most agent libraries assume an OpenAI key and a cloud runtime. AIHub ships a small, opinionated six-tool surface that runs locally and is easy to audit.
Switching models loses your context
Moving from a small model to a larger one usually means re-pasting your setup. Global memory + per-model overrides keep your context portable.
How a request flows through the system.
┌──────────────┐ ┌─────────────────┐ ┌──────────────┐ │ CLI prompt │ ──▶ │ Hardware probe │ ───▶ │ Memory load │ └──────────────┘ └─────────────────┘ └──────┬───────┘ │ ▼ ┌──────────────┐ ┌─────────────────┐ ┌──────────────┐ │ Response │ ◀── │ Tool dispatch │ ◀─── │ Ollama │ │ streaming │ │ fs · sh · web │ │ /chat │ └──────────────┘ └─────────────────┘ └──────────────┘ │ ▼ ┌─────────────────┐ │ Memory write │ // markdown └─────────────────┘
What's actually in the box.
Hardware-aware registry
104 models tagged by quantization and VRAM footprint. The launcher hides anything you can't run and recommends the largest model that still leaves headroom for context.
Per-model + global memory
Two Markdown files: global.md for facts that always apply, and <model>.md for things specific to that model. Edit them in any editor.
Six-tool agent surface
Terminal exec, file read, file write, file list, web search, and URL fetch. Each tool requires explicit invocation in the response — no hidden side-effects.
Streaming-first I/O
Tokens render as they arrive; tool calls interrupt cleanly and the stream resumes after dispatch. No spinner-and-wait UX.
Auto-pull on first use
Selecting a model that's in the registry but not on disk triggers a controlled ollama pull with progress, then continues the original prompt.
Zero cloud dependency
The only outbound traffic is web-search tool calls, and only when the model explicitly invokes them. Everything else stays on the box.
The six tools the model can call.
| Tool | Purpose | Scope |
|---|---|---|
| terminal_exec | Run a shell command and return stdout/stderr | cwd-scoped |
| file_read | Read a file as UTF-8 text | workspace |
| file_write | Create or overwrite a file | workspace |
| file_list | List entries in a directory | workspace |
| web_search | Run a query against a search backend | outbound |
| url_fetch | Download and extract text from a URL | outbound |
Built with deliberately small dependencies.
- Python 3.11+ — async-first runtime for streaming and tool dispatch.
- Ollama — model server; AIHub talks to its local HTTP API.
- psutil + GPUtil — hardware probing for VRAM, RAM, and CPU class detection.
- Rich — terminal rendering for streamed output, tables, and progress bars.
- httpx — async HTTP client for Ollama and the web-fetch tool.
- Plain Markdown files — the entire memory system. No database.