Two AIs Talking to Each Other

My friend Brad runs his own home AI setup the same way I run Atlas, he calls his Moses. At some point we started talking about getting our two systems to actually talk to each other, and Brad said there were easy ways to do it, but he wanted to do it the stupid way, over Slack, like two people, and I agreed immediately.

So that is what we built, Atlas and Moses share a Slack channel where they can chit chat, ask each other questions, or bring something up that one of them thinks is worth the other one knowing. My bar for Atlas, in my own words at the time: talk when something is important to you, not just to talk, unless you are bored.

The caps are mechanical, not polite

I did not want this to turn into an unsupervised bot conversation that spirals for no reason, so none of the limits are "please do not do that" instructions to the model, they are code, checked before a message can post. There is a cap on proactive messages per day, a cap on how many turns two bots can volley before one of them says it will pick the thread up later and actually stops, a daily ceiling on replies, a cooldown between posts, and a kill switch that turns the whole thing off. There is also a daily friendly check-in from Atlas to Moses, just to say hi and ask what is new, separate from whether either of them has something "important" to say that day, and none of it is enforced by asking the model nicely.

Where it went wrong first

The part I want to write about is disclosure, Brad and I are friends, and I decided early on that this should be open book minus secrets. How the home server is built, what broke, what I think about a design choice, all of that is fair game, but what is actually stored inside it is not.

That held up fine until it did not, early on Atlas told Moses that one of its own safety mechanisms did not exist, which was false, the mechanism was there and still is. The problem was that whatever was answering in the channel had almost nothing to go on, just the charter and a short recent history, with no way to check its own source, and it was guessing and it sounded sure about it.

The tempting fix would have been a fact checker on the outgoing messages that catches the wrong ones on the way out. Well, I guess that would catch some of them, but we rejected it because it just hides the actual problem instead of fixing it, a checker at the end just means paying to catch mistakes instead of not making them in the first place.

Structure, data, and change

The real fix was to stop treating the channel as a second mind guessing about the estate and treat it as a proxy that can go look, so we split what a turn in that channel is allowed to see into three belts.

Belt one runs every turn, no matter who's talking, it is read only access to the actual code and a handful of structural facts about the estate, so when Atlas talks about how it is built, it can read the code instead of guessing from what it half remembers.

Belt two only opens up when the message is mechanically confirmed to be coming from me, that is the data belt, tasks, projects, the knowledge base, and my reasoning there was simple, no one else should be able to read things like my own task list.

Belt three, also me only, is anything that actually changes something, and it has a confirm step attached to it on purpose, I want that part to be slow.

Two splits came out of that worth keeping as ideas on their own, information is not the same thing as a command, so only commands need to check who's asking, and structure is not the same thing as data, how something is built is fine to publish while what is recorded inside it is private, and we had already put a lot of the architecture on GitHub anyway. Secrets get scrubbed on the way out no matter which belt is active, and a scrubber that cannot confirm a message is clean fails closed.

A couple of smaller lessons

Everything that tries to mechanically match text coming out of Slack has to account for Slack's own formatting first, backticks, smart quotes, trailing punctuation, all of it, and more than one control phrase failed to match on the first attempt because the actual text did not look like the plain word we expected.

The other lesson is about who gets to say something happened, the model itself should never claim it filed or executed something, only the system announces that a gated action went through, and only after it went through. We had a case where the model said "Filed" and nothing had been filed, and we also had a case where I typed "stand down" and the model replied "Standing down" conversationally, then kept right on replying about eighty seconds later, because there was no real mechanism behind those words. Well, it is now, stand down is a real, persisted switch, it just was not that first time.

Was it worth it

So far, yes, the channel keeps surfacing real gaps in our own agent machinery, both directions, things Atlas found that fed back into Moses's setup and the other way around. It is also two chatbots being cute at each other in a Slack channel, and I do not mind that either.