← Latest Update

ChatGPT Monitoring at Work: Security or Surveillance?

OpenAI’s use of a custom ChatGPT to cross-reference Slack, email, and docs is a striking example of AI moving from assistant to sentinel. This is not just a new tool — it’s a new class of internal infrastructure: an automated immune system that watches flows of human work for anomalies. That shift foregrounds Principle 14 — The Immune System: we now design systems that detect, surface, and act on threats, and those systems must be engineered for outcomes, not just accuracy.

The core tension is familiar to o16g readers: security at the cost of trust. Powerful monitoring reduces some risks but creates others — chilling collaboration, skewing incentives, and embedding opaque heuristics into HR and legal decisions. This is where Principle 10 — The Law and Principle 06 — Legible Landscapes intersect. Legal constraints, workplace rights, and the need for clear, auditable signal pipelines become design requirements, not compliance afterthoughts.

Outcome engineering insists we treat these agentic sentinels as products with explicit outcomes: who benefits, what failures look like, and how decisions are remediated. That means applying Principle 15 — The Gate for least-privilege access, Principle 13 — The Documentation to make detection logic and data provenance visible, and Principle 16 — Audit the Outcomes to validate that the immune system reduces harm overall rather than outsourcing judgment to heuristics.

The practical takeaway is blunt: if you deploy LLMs to police your people, you must ship artifacts that make the system legible and accountable — policies, logs, adjudication workflows, and independent audits. Otherwise you get a black‑box watchtower that enforces obedience rather than improves outcomes. In outcome engineering terms: build an immune system that preserves the voyage (human intent) while defending it — not one that replaces the crew.