Context
I’ve been running Hermes Agent for myself for quite a while now (for the last half a year, if my memory serves me right). While I use oh-my-pi for coding, I use Hermes as a personal assistant for other tasks such as browsing the web, asking questions, doing some automation, etc. Then I realized – what if I could share this awesome setup with my family? Facebook came out with an agent called Muse, and OpenAI also came out with one called Dot. I don’t trust either company that much with personal data, so I wanted my family to use one that was fully self-hosted! Coincidentally, I had also gotten myself a 2x R9700 local LLM machine for inference. I now had all the pieces I needed to create something fully local.
The requirements
Here was the hardware I was working with:
- 3 node k3s cluster
- an inference server (2x R9700)
- a claude subscription & chatgpt subscription
Deploying my services
I got a LiteLLM endpoint set up so that all the Hermes agents can communicate with LiteLLM. This was super easy to set up – I got it deployed and linked to my radiance server (the local LLM server) in under an hour (agentic deployments are crazy fast). I then built a Helm chart for this multi-Hermes setup. The concept is simple. Every user gets their own pod. Each pod has access to the LiteLLM server for inference. Each pod also has access to a Hindsight server for memory. Then came a requirement to build something: hindsight-gateway
Hindsight gateway is a proxy I built for the hindsight memory system. It was necessary because I wanted every user to use the same Hindsight server, but read from and write to their own bank. By default, Hindsight doesn’t have access controls between banks – so a user who knows the name of another bank can read it directly. The hindsight-gateway is just another layer on top that ties each user’s access to a specific memory bank. Below is the architecture of my setup.

It’s all public
Everything I wrote to deploy Hermes and make it available for my family is also available for other people to run. It is available as Hermify, a Helm chart that runs a Hermes agent per person on one cluster. Feel free to deploy it on your own cluster and see if it works for your use case. If it doesn’t, feel free to let me know! We can make it better together 😀.
That’s all for today! I’ve had a lot of fun deploying this.