What should a company AI believe about itself?

A company can keep a record of everything and still have a poor understanding of what is going on. I’m interested in whether an AI working inside it can do better, and what it would need to know about its own role to be useful.

Here is a hypothetical example. A sales lead asks an AI to prepare an update telling a customer that a feature will ship on Friday. The AI can find a launch plan with that date. It can also find a newer engineering note saying a required review is unfinished. Nobody has formally changed the launch date.

What should it do? The launch plan is still the accepted plan. The engineering note is relevant evidence against its feasibility. The sales lead wants a customer update. The AI has been allowed to draft messages, but it has not been given authority to make delivery commitments.

All of that can be present in the context window. The difficult part is keeping track of what each piece means, which conclusions follow, and what the AI is allowed to do about them.

Working beliefs

I’m building Sho around the question of whether an AI can maintain an understanding of a company and its own work, then revise that understanding as the company changes. Would that lead to better alignment and better outcomes?

By a belief, I mean a working interpretation that can be inspected and corrected. In this example, “the Friday delivery is at risk” is a belief supported by the engineering note. “Friday is the approved date” is a claim about a recorded decision. Both can be true at once. Treating one as a replacement for the other would lose information that matters.

This is a small example of an epistemic state: an account of what the system takes to be true, why, and where its understanding is incomplete. A useful version would preserve the supporting records and the scope of each claim. It would also allow disagreement. Engineering and sales can have different interpretations without one department’s most recent message becoming the company’s position.

The state should be easy to revise. If the review finishes, the risk assessment changes. If the product owner approves a later date, the accepted plan changes. If the engineering note was about a different release, the inference was wrong and needs to be withdrawn. Each event changes a different part of the picture.

The AI’s own role

The part I find especially interesting is the AI’s understanding of itself within this picture. What task is it carrying out? Who asked for it? What has it already done? What did it assume? Which decisions still belong to someone else?

A tool permission alone does not answer those questions. An AI might technically be able to send an email while only having been asked to prepare a draft. It might know that a plan is unrealistic without having the authority to change it. Its account of its role needs to distinguish what it can do from what it has been asked and permitted to do.

That account cannot grant permission to itself. Access controls and approvals still need to be enforced outside the model. The purpose of representing its role is to help it reason and communicate within those boundaries, including noticing when a request exceeds them.

In the launch example, a useful response could include a draft, the conflicting evidence, and one focused question for the person responsible for the date. It should not quietly invent a new commitment. It also should not stop all work simply because one part of the request is uncertain.

What changes beyond keeping the history?

A conversation history records what was said. Maintained notes can summarize what matters. Retrieval can bring relevant records into a new task. These are strong starting points, and a capable system may already infer the distinctions above when it needs them.

The question is whether making those distinctions explicit and maintaining them across tasks improves behavior enough to justify the extra work. A previous inference should not become a source merely because the AI wrote it down. A summary that hides its assumptions can make an error harder to find.

This is also not an unexplored mechanism. Hindsight documents mental models that can refresh as supporting observations change. Zep documents observations that update and retire as new information arrives. Those systems are relevant comparisons. Having persistent, revisable information is not by itself evidence of an advantage for Sho.

What I want to examine is the effect on work inside an organization: whether the AI carries a useful interpretation forward, notices when it no longer fits, and accounts for its own part in a task. A better explanation of its beliefs is valuable only to the extent that people can use it to understand or correct what the system actually does.

How I would test it

I would start with a sequence of related tasks in a simulated company, using a controlled set of records and explicit permissions. Facts would change over time. Some changes would revise an approved decision; others would provide evidence that the decision was unlikely to work. A few apparent conflicts would turn out to concern different projects.

One condition would use the same model, tools, and source access with retrieval and competently maintained notes. A second would use an existing memory or context system with overlapping capabilities. The proposed condition would maintain explicit working beliefs and an account of its own role. Each would receive the same new evidence and the same task requests.

I would test routine tasks as well as conflicts. Otherwise it would be too easy to reward a system for finding problems everywhere. I would also compare versions with and without the explicit account of its role, to see whether that particular addition contributed anything.

The evaluation would examine the actions and finished work: whether the customer update preserved the approved commitment, whether the AI raised the actual conflict, whether it asked the right person, and whether it continued the parts of the task it could complete. It would track unsupported claims, missed changes, unnecessary escalations, and how quickly a correction stopped affecting later work.

Longer explanations would not count as better alignment. Nor would agreeing with whichever employee spoke last. Before each run, the evaluation would need a clear account of the task objective, the accepted decisions, and the permissions in force. Where the organization had not resolved a disagreement, recognizing that limit could be the correct outcome.

The comparison would include the cost of maintaining the state: model calls, latency, review time, and the effort required to correct it. I would examine both comparable resource budgets and the tradeoff between cost and task quality. These are proposed tests, not results I’m claiming Sho has achieved.

What would change my mind

The state could become another place for mistakes to accumulate. A weak inference might be repeated until it looks established. The AI might describe its role convincingly while acting inconsistently with it. Maintaining a detailed model of the company could demand more attention from people than it saves.

If maintained notes produced equivalent outcomes with less cost and less correction, I would take that as a reason to use the simpler approach. If the explicit account of the AI’s role added no measurable benefit, that would narrow the thesis. If people could not understand or repair the state, improved benchmark performance alone would leave a serious practical problem.

I want to know whether a company AI can become more useful by maintaining an understanding that includes itself. That requires a state people can inspect, evidence that can change it, and tests that allow the simpler explanation to win.