Navigating Agency: How AI and Humans Shape Each Other

Artificial intelligence has moved from the realm of science‑fiction to the fabric of everyday experience. From recommendation engines that shape our media consumption to autonomous systems that assist in surgery, AI systems now act in ways that were once the exclusive domain of human judgment. This shift raises a profound question: who holds agency? David Mumford’s recent essay, AIs and Humans with Agency, argues that agency is not a zero‑sum game but a dynamic relationship between designers, users, and the algorithms themselves. Building on his insights, this post explores the nature of agency, the risks of conflating autonomy with authority, and how we can cultivate a collaborative future where both AI and humans thrive.

---

1. Defining Agency in the Age of AI

Traditionally, agency refers to the capacity of an individual to act intentionally and influence outcomes. In philosophy, it is tied to concepts of free will, responsibility, and moral accountability. When we talk about AI, we must distinguish three layers of agency:

1. Design‑time agency – the choices made by developers, data scientists, and policymakers that shape an algorithm’s objectives, constraints, and training data. 2. Run‑time agency – the system’s ability to make decisions in real time, often based on probabilistic inference rather than deterministic logic. 3. User‑mediated agency – the ways in which humans interact with, interpret, and override AI outputs.

Mumford emphasizes that these layers are interdependent. An autonomous vehicle, for example, may exhibit run‑time agency in steering decisions, but its safety envelope is defined by design‑time choices, and a driver’s willingness to intervene reflects user‑mediated agency.

---

2. The Illusion of Full Autonomy

A common narrative in tech marketing portrays AI as fully autonomous agents capable of replacing human judgment. This rhetoric can be misleading for several reasons:

- Opacity: Deep learning models often operate as “black boxes,” making it difficult to trace the reasoning behind a specific output. - Context‑dependence: AI systems excel in narrow, well‑defined tasks but struggle with the nuanced, contextual reasoning humans perform effortlessly. - Value alignment: Algorithms inherit the biases and value judgments embedded in their training data and objective functions.

When we mistake operational autonomy for moral agency, we risk absolving humans of responsibility. As Mumford notes, delegating decisions to an AI without clear accountability structures can erode trust and lead to systemic failures.

---

3. Agency as a Spectrum, Not a Binary

Instead of viewing agency as a binary—either human or machine—we should see it as a spectrum where control is shared and negotiated. This perspective aligns with the concept of human‑in‑the‑loop (HITL) systems, where AI provides recommendations, but humans retain final authority. Benefits include:

- Error mitigation: Human oversight can catch edge‑case failures that models miss. - Skill augmentation: AI can handle repetitive analysis, freeing humans to focus on creative and strategic tasks. - Ethical grounding: Humans can inject societal values that are difficult to encode algorithmically.

A practical illustration is clinical decision support: AI flags potential diagnoses, but physicians validate and contextualize the findings before treatment.

---

4. Designing for Shared Agency

To foster a healthy partnership, designers must embed principles that respect both AI capabilities and human values:

1. Transparency – Provide interpretable explanations for AI outputs (e.g., feature importance, counterfactuals). 2. Controllability – Offer mechanisms for users to adjust parameters, set constraints, or abort actions. 3. Responsibility tracing – Log decision pathways so that accountability can be assigned when outcomes are adverse. 4. Feedback loops – Enable continuous learning from human corrections, ensuring the system evolves with user expectations.

Mumford highlights that these design choices are themselves acts of agency. By consciously shaping how AI interacts with humans, developers exercise a form of meta‑agency that can either empower or constrain downstream users.

---

5. Policy and Governance Implications

Regulators are beginning to recognize the need for frameworks that address AI agency. The European Commission’s AI Act proposes risk‑based classifications and mandates for transparency and human oversight in high‑risk applications. Similar initiatives in the United States and Asia stress accountability and auditability.

Key policy considerations include:

- Defining legal personhood for AI (most experts, following Mumford, argue against it). - Establishing liability regimes that reflect the shared nature of agency. - Requiring impact assessments that evaluate how AI decisions affect human autonomy.

Effective governance will require collaboration between technologists, ethicists, and legislators—a true embodiment of shared agency.

---

6. The Future: Co‑evolution of Minds and Machines

Looking ahead, the relationship between AI and humans is likely to become even more intertwined. Emerging paradigms such as neuro‑symbolic AI, interactive reinforcement learning, and brain‑computer interfaces blur the line between tool and collaborator.

In this co‑evolutionary landscape, Mumford’s central claim—that agency is a relational property—offers a guiding principle: we must design systems that listen to human intent and learn from human values, while also allowing AI to suggest novel pathways that humans might not have imagined.

---

Conclusion

Agency in the age of AI is not a competition but a conversation. By recognizing the multiple layers of agency, rejecting the myth of full autonomy, and embedding transparency, control, and accountability into system design, we can ensure that AI amplifies rather than diminishes human freedom. As David Mumford reminds us, the ultimate test of any intelligent system is not how independently it can act, but how responsibly it can share the stage with its human partners.

---

Takeaway: Embracing a spectrum of agency empowers both humans and machines to co‑create a future where technology serves as a true extension of human intention.

Sources: https://www.dam.brown.edu/people/mumford/blog/2026/AIs%20with%20Agency.html