JARVIS Setup Guide — alfred makes

Everyone's seen the viral "AI JARVIS" videos. Here's how to actually build one — every step, every prompt. It's a recipe: each step is one job, then the command or prompt. You can also paste this whole guide into Claude or Codex and have it start the project for you.

What you're building

A personal JARVIS that runs your day. Five pieces:

Hermes — the brain (an open-source AI agent, not Claude directly)
An always-on Mac mini — so it runs 24/7 and can control the computer
The talk feature — "Hey Jarvis" in, a British voice out
Sub-agents with jobs — one for ads, one for dev, one for research, that hand work to each other
A local dashboard — the Iron Man HUD on localhost

Docs to keep open: https://hermes-agent.nousresearch.com/docs/

1. Install Hermes (10 min)

Mac/Windows (easiest): download the desktop installer from https://hermes-agent.nousresearch.com/ and run it. (Terminal install script is on that page too.)

Pick a model + add your key (any model with 64k+ context):

hermes model

Start it and talk to it:

hermes --tui

Check everything's healthy:

hermes doctor

Rule: get a plain chat working first. If a normal chat won't run, don't add anything else yet.

2. Make it always-on

Run Hermes on a Mac mini (or any machine you leave on 24/7). That's what lets it answer emails at 3am, keep context, and control the computer it lives on. Jarvis on a laptop that sleeps at night isn't Jarvis.

3. The talk feature (Hey Jarvis + the voice)

Hermes' built-in voice is push-to-talk (Ctrl+B) with free local transcription (faster-whisper). To get a hands-free "Hey Jarvis" that only answers when you call it, you put a wake-word listener in front of Hermes. Here's the actual loop — this is the part most builds get stuck on, so follow the interface exactly:

openWakeWord hears "Hey Jarvis" (its pretrained hey_jarvis model — free, local): https://github.com/dscripka/openWakeWord
Your script records what you say and transcribes it with faster-whisper.
It sends that text to Hermes in one-shot mode: hermes -z "your transcribed text". This is the key interface — hermes -z returns just the agent's final reply as plain text (no banner, no spinner), which is exactly what a script can capture.
It pipes that reply into ElevenLabs and plays it — your British Jarvis voice.
Loop back to listening.

The British voice: set Hermes' TTS to ElevenLabs via hermes config set (provider + your key), or call ElevenLabs directly in the script. Pick any voice in your account for the accent.

(Alternative interface: Hermes can also run as an OpenAI-compatible HTTP endpoint if you'd rather call it over HTTP than the CLI.)

4. Sub-agents that talk to each other

This is what makes it feel like a team, not a chatbot.

Make a separate agent per job — each is its own profile with its own model, skills, and context:

hermes -p bobby (ads) · hermes -p tom (dev) · hermes -p scout (research)

Easier: open the Profile Builder in Hermes' local web dashboard — it sets identity + model + skills + MCP servers in one flow.

Give each one a skill (a SKILL.md file — instructions for its job), then enable it with hermes tools.

How they connect to each other: Hermes runs an orchestrator/worker setup — your main Jarvis agent delegates subtasks to the child agents and they report back. So Jarvis reads the morning brief, hands the PR to Tom, the ad tweak to Bobby, the research to Scout.

Run the team in Slack (so you can see and steer it): connect Hermes to Slack through its messaging gateway — Socket Mode, so no public server needed.

Setup: hermes slack manifest --agent-view --write, create the Slack app from that manifest, install it, and give Hermes the bot token + socket token.

Two ways to run the team in Slack:

Per-channel personas (simplest): one agent behaves differently per channel — a #ads channel loads the ad tools (Bobby), #dev loads GitHub (Tom), #research loads search (Scout). Same agent, different hats.
Separate agent bots: run each profile as its own bot; the main agent hands tasks off in a shared channel and each replies in-thread.

Either way, you watch the hand-offs happen in Slack and jump in when you want.

5. Recommended MCPs

Install + authenticate any MCP server with hermes mcp (it handles the OAuth).

The core four:

MCP	Job
RevenueCat	App revenue & subscriptions
Gmail	Read + reply to customer email
Meta (Facebook) Ads	Run + monitor ad campaigns
Metricool	Post to all social platforms

Add as you need them:

MCP	Job
Slack	Control + coordinate the agents
GitHub	PR review / dev tasks
Supabase (or Postgres)	Your user database (support context)
Firecrawl (or any web-search MCP)	Research / content angles
Stripe	Revenue outside the app
Google Calendar	Scheduling / the daily-brief routine

6. Guardrails (do this before it can spend money or email anyone)

An always-on agent wired to your ad account, Gmail, and computer can do real damage if it acts on its own. Turn on approvals before you connect the dangerous MCPs.

Set the approval mode in ~/.hermes/config.yaml:

approvals.mode: smart (recommended) — Hermes pauses and asks you to confirm before risky actions: spending ad budget, sending an email, deleting files.
manual asks on everything; off asks on nothing (don't).

This is the "say yes before it acts" behavior — the agent prepares the action, shows you a summary, and waits for your one-time approval. Approvals are scoped to that exact action and expire after a few minutes, so an approval to send one email can't be reused for another.

More controls (Hermes' Tirith security layer): per-tool permission levels, a full audit trail of everything the agent did, and rate/cost limits. A handful of highly destructive commands are hard-blocked no matter what.

Also: scope your API keys to the minimum (a read-only stats key is not a spend-money key), use Hermes' user allowlist so only you can trigger it, and for extra isolation run it in the Docker/sandboxed backend.

7. The dashboard (localhost HUD)

Two prompts, back to back (Prompt B then Prompt C below). It runs on localhost:3000 — it's just a local web app, no hosting. The bottom waveform pulses when the voice speaks.

8. The full prompt list

A — Wake word listener + voice loop (Claude Code):

Build a local Python background service. Use openWakeWord's pretrained hey_jarvis model to listen for "Hey Jarvis." On detection, record my speech and transcribe it with faster-whisper. Send the transcript to my Hermes agent in one-shot mode by calling hermes -z "<transcript>" and capturing its plain-text reply. Speak that reply back through ElevenLabs, then return to listening.

B — Dashboard design (Claude Design):

Design an Iron Man "JARVIS" desktop dashboard. Dark navy background, glowing cyan holographic UI, thin monospace labels. Centerpiece: a glowing arc-reactor orb (radial particle sphere) with faint orbit rings around it. Top corner: a live clock + date. Side panels: app stats — revenue and downloads — as glowing figures. Bottom: a horizontal audio waveform strip. All cyan-on-navy, soft glow, cinematic and minimal.

C — Dashboard build (Claude Code):

Take this design and build it as a local web app on localhost:3000. Match the cyan-on-navy HUD style exactly. Animate the arc-reactor orb (slow rotate + pulse) and the orbit rings. Make the bottom waveform audio-reactive: feed the audio output into the Web Audio API AnalyserNode and drive the bar heights off its live frequency data, so it pulses when the JARVIS voice speaks. Pull the revenue/download numbers from a simple local data file for now. Smooth 60fps.

D — Sub-agent skill template (SKILL.md** starter):**

You are [Bobby, the ads agent]. Your job: [monitor and adjust Meta ad campaigns]. Tools: [Meta ads MCP]. When the main agent hands you a task, do it, then report back a one-line summary. Only escalate to me if [spend exceeds $Y or ROAS drops below Z].

Prefer fully open-source?

OpenJarvis, from Stanford — an open agent brain you run locally (add your own voice layer on top): https://scalingintelligence.stanford.edu/blogs/openjarvis/