Refusal isn't one direction. It's a cone.

One influential line of work suggested the picture might be simple: a single refusal direction in the model's activation space, determining whether a request gets refused. If that were right, safety would be a one-dimensional object. Find the direction, defend it, done.

Earlier work located refusal directions by difference-in-means — averaging activations on harmful prompts, averaging on harmless ones, and taking the gap.

This paper takes a gradient-based approach to representation engineering instead, optimizing directly for directions that control refusal behaviour rather than reading them off a statistical contrast. The method finds directions the difference-in-means approach misses, which is what makes the central result possible.

Multiple independent refusal directions, and multi-dimensional concept cones. Not one direction but a structured region of activation space, with several distinct mechanisms driving refusal behaviour.

Orthogonality does not imply independence. Two directions can be mathematically orthogonal and still interfere with each other when you actually intervene on the model. The paper introduces representational independence as the stricter criterion: two directions count as independent only if ablating one leaves the other's representation intact — accounting for non-linear interactions, not just geometric ones.

Mechanistically independent directions exist. Applying that stricter test, the authors identify directions that are genuinely functionally separate, confirming that refusal is governed by more than one mechanism.

Gradient-based direction-finding is a harder engineering problem than activation extraction. It isn't enough to read values out of a forward pass — derivatives have to flow through an intervention, repeatedly, across many candidate directions. Most ad-hoc hooking code isn't built for that.

The paper's implementation notes are unambiguous on where that machinery came from: all of its algorithms and exploratory experiments are implemented using NNsight.

There's a second demand. The argument rests on the finding holding across model families; a refusal geometry that only appeared in one architecture wouldn't support the claim. The authors train refusal directions on models from the Gemma 2, Qwen 2.5, and Llama 3 families, and sweep size within Qwen 2.5 — from 1.5B to 14B parameters — to test whether larger models support higher-dimensional cones. NNsight expresses interventions against any PyTorch model through one API, so replicating an experiment on a different model doesn't mean rewriting the intervention code for a new forward pass.

The practical consequence is uncomfortable and useful in equal measure. Ablating a single refusal direction doesn't remove refusal — it removes one route to it. Any safety intervention that assumes it has found the mechanism is likely defending a narrower target than it thinks.

The methodology generalizes past refusal. Any behaviour suspected of having a linear representation can now be tested the same way, and the authors position the gradient-based approach as groundwork for further study of model internals.