[See also Posthuman Life, pp. 96-103, section 4.3]
Charles Stross’ science fiction novel Accelerando provides a vivid and blackly funny portrayal of a transition from a merely transhuman to a genuinely posthuman world.
In Accelerando, the Singularity has arrived by the 22nd Century (Vinge 1993). The self-improving AI’s that now run the world are “wide human descendants” of human corporations and automated legal systems, which achieved both sentience and a form of legal personhood back in the 21st. As Stross’ narrator observes, the phrase “smart money” has taken on an entirely new meaning.
Eventually, these “corporate carnivores” – known as the “Vile Offspring” – institute a new economics (Economic 2.0) in which supply and demand relationships are computed too rapidly for those burdened by a “narrative chain” of personal consciousness to keep up. Under Economics 2.0 first person subjectivity is replaced “with a journal file of bid/request transactions” between autonomous software agents. E 2.0 is so remorselessly efficient that it comes to dominate not only the Earth but also the majority of the solar system. Whole planets pulverized and diverted to fast-thinking dust clouds of smart matter “blooming” around the sun (Stross 2006, 208-10):
This post-singularity scenario certainly seems bad for humans. Even their souped-up transhuman offspring prove equally incapable of functioning within E 2.0 and can only flee to the outer solar system and beyond as their worlds are “ethnically cleansed”.
At the same time, it is not clear that E 2.0 is really “good” for posthumans in a way that might conceivably outweigh its bad impact on humans.
If the posthuman entities – such as the Wide Descendents eating up the inner solar system of Stross’ novel – lack a linear, narrative consciousness, can their form of existence be worthy of ethical consideration?
Well, it might be argued that any being with conscious awareness – even one that does not involve rational subjectivity or personhood – is worthy of some moral consideration. Most accept that nonhuman animals are conscious of pains and pleasures and it is plausible to argue that their interest in avoiding pains and having pleasures are identical to humans.
However, many humanists claim that the reasoning prowess of humans distinguishes them radically from nonhuman animals. Responsiveness to reasons is both a cognitive and a moral capacity. For Kant, this capacity to choose the reasons for our actions – to form a will, as he puts it – is the only thing that is good in an unqualified way and is the most important distinguishing characteristic of humanity as opposed to animality.
Even humanists for whom the human capacity for self-shaping is one good among many, here, claim that “autonomy” confers a dignity on humans that should be protected by laws and cultivated.
Beings with the capacity for autonomy the moral status that goes with it are commonly referred to as “persons”. Locke defined a person as “a thinking intelligent being that has reason and reflection and can consider itself as itself, the same thinking thing in different times and places”. If Locke is right about the psychological preconditions for personhood, then beings such as the Vile Offspring cannot count as persons because, as Stross puts, their phenomenology lacks the “narrative centre” that a being needs to consider itself the same thing at different times. The practical rationality described in most post-Kantian conceptions of autonomy might not be accessible to a being with non-subjective phenomenology. Such an entity would be incapable of experiencing itself as having a life that might go better or worse for it.
If humanists are right to say that persons have special moral worth and we add to this the claim that there could be no nonpersons with greater or equivalent moral worth than persons, then very weird and very non-human posthumans such as Vile Offspring who lack personal phenomenology would not be as worthy of moral consideration as humans or transhumans.
Posthumans lacking personhood and the capacity for pleasure and pain would not be sources for any kind of moral claim. Posthumans lacking personhood but possessing functional equivalents of pleasure or pain could be granted an equivalent status to non-human animals that also lack the psychological prerequisites of personhood.
Posthuman singularity ethics would then be possible only in an etiolated form though it would not be applicable where our wide human descendants departed radically from human phenomenological invariants.
Perhaps it this is what accounts for the “vileness” of the Vile Offspring. That they are not conscious subjects with plans for life and conceptions of the good but churning clouds of super-intelligent matter driven by inchoate drives – like H P Lovecraft’s blind, idiot God, Azathoth.
However, this analytic of the vile is premature. For it assumes that there is a moral hierarchy mapping onto a psychological or phenomenological hierarchy. But the fact that there are beings – persons – with the distinctive mental properties described by Locke and Kant does not entail that all beings lacking these properties must be morally inferior, or even vile. For it is conceivable that there could be intelligent beings whose experience lacks some perquisites for personhood but have phenomenological attributes that are different but not morally inferior.
We humans might find it hard to conceive what such impersonal phenomenologies could be like (to say of them that they are “impersonal” is not to commit ourselves regarding the kinds of experiences they furnish). However, this difficulty may simply reflect the fact that our phenomenology constrains our grasp of phenomenological possibility and necessity (Metzinger 2004: 213; Roden 13b).
In particular, our phenomenology may be characterized by variable degrees of what Thomas Metzinger calls “autoepistemic closure”.
A phenomenology is autoepistemically closed if the processes that generate it are inaccessible within it. According to Metzinger, human personal experience is a dynamic and temporally situated model of the world, which represents the modeller as a distinct component. The phenomenal world model thus includes a phenomenal self-model or PSM. However, neither model represents the subpersonal cognitive processes that implement them. To borrow a phrase from Michael Tye: the phenomenal world-models and self-models are “transparent” – we seem to look through them into an immediately given world out there and a self-present mental life “in here” (Metzinger 2004 131, 165).
Both immediacies, according to Metzinger, are epistemic illusions generated by the model’s insensitivity to its computational underpinnings. There is no self or subject doing the looking. The experienced self is, rather, the simulated content of the PSM rather than the subpersonal process that generates it.
If, as Metzinger claims, we are not self-intimating Cartesian selves or Kantian transcendental subjects but self-models, it is little wonder that our phenomenology affords limited insight into the space of possible minds. For example, our subjectivity seems to exist in a spatial-temporal pocket: a situated, embodied self and an ever evolving present. It is characterized by a bivalent distinction between self and other, non-mine and mine and a sense of temporal newness – or presentationality – “a virtual window of presence” that gives us a baseline with which to distinguish actuality and simulated possibility (Ibid. 42, 96). But this representational scheme may depend on the fact that our sensory and motor systems are “integrated within the body of a single organism”. Other kinds of life – e.g. “conscious interstellar gas clouds” or (more saliently for us) decentred post-human “swarm” intelligences like the Vile Offspring – might have experiences of a quite different nature (Metzinger 2004: 161).
A physically distributed entity with computing power to burn might support a “multi-threaded” and “multi-level” phenomenology that tracks the adventures of distributed processing sites while providing high-resolution models of its own cognitive processes. Such a distributed consciousness might have a very different functional structure to human consciousness.
A multi-threaded phenomenology might employ different strategies for modelling relationships between the modeller and its environment. We cannot easily imagine what such a phenomenology would be like – but inability to imagine it is not a demonstration of its impossibility.
So it is at least conceivable that a nonhuman phenomenology could be impersonal, but have representational characteristics no less sophisticated than “higher order” moral properties such as autonomy in humans. If personhood and autonomy are not unique “higher-order moral properties” and we are not yet in a position to compare them with posthuman modes of being, then we have no grounds to assume that they trump other candidates for ethical consideration. So we have very weak grounds for believing that persons (or autonomous human subjects) stand at the moral summit or centre of creation.
If that is right, then a person-relativist humanist ethic should be rejected along with a species relativist one. There may be non-personal modes of existence following a singularity (or posthuman-maker) no less valuable than those accessible to persons. This is compatible with the claim that persons have some intrinsic moral worth – though it does not entail this. If this value is genuinely intrinsic it is presumably unaffected by the existence of different modes of existence with their own intrinsic worth.
I think this possibility implies a form of posthuman justice. This is not the postmetaphysical, procedural justice described by Rawls and other liberal anti-perfectionists. Posthuman justice cannot be predicated on “fair terms of co-operation” between citizens of a state since any human-posthuman disconnection would, arguably, preclude a republic of humans and posthumans (Roden 2013a).
Now, we could try to express a formal principle of justice on the basis of the assumption that there could be valuable posthuman forms of existence: for example:
We should give equivalent consideration to such modes of being, whatever they may be.
I use “equivalent” in favour of “identical” since it would be presumptive to describe a nonpersonal intelligence as having identical interests to a personal one.
However, this substitution does not achieve much. It does not tell us how these interests are equivalent or what duties might flow from the principle. As a guide to action or to life, the formal principle is not worth the pixels it is written in.
To invert Rawls’ famous disclaimer: the theory of posthuman justice is metaphysical, not political. It does not tell us what to do or how to coordinate our institutions. It just allows, (for want of countervailing arguments) that potential posthuman lives could support modes of existence that are not less than ours.
We could choose not to acknowledge these potential lives – were it possible to do so – but this refusal to acknowledge posthuman “otherness” would be a kind of failure; equivalent to the claim that something into which our insight is really very limited – “normal” human subjectivity and personhood – has a superior claim over the nonpersonal or vile occupants of posthuman possibility space. This position might be warranted if our place in posthuman possibility space were not under consideration – e.g. if we were comparing the higher order moral properties of actual humans with actual nonhuman animals. But our attitude to our nonhuman Wide Descendants is at issue. Refusal to consider this possibility would be an intellectual failure as well as a kind of injustice.
Now, I think some would object that this capacious metaethical statement simply fails to do justice to the difficulty and danger attending an actual disconnection scenario. How, for example, could it guide us in an alien post-singularity environment of the kind described in Accelerando? There the remaining humans cannot communicate or interpret the “radically other” posthumans eating up the mass of the inner solar system (Near the end of Accelerando, the Vile Offspring start to resurrect every human who ever existed. Nobody finds out why.)
Metzinger, Thomas. 2004. Being No One: The Self-Model Theory of Subjectivity. Cambridge, MA.: MIT Press 2004.
Roden, David (2013a). “The Disconnection Thesis”, in Amnon Eden, Johnny Søraker, Jim Moor, and Eric Steinhart (eds.), The Singularity Hypothesis: A Scientific and Technological Assessment , , Springer-Verlag: Berlin Heidelberg, 281-298.
Roden, David (2013b). ‘Nature’s Dark Domain: An Argument for a Naturalized Phenomenology’, in Human Experience and Nature, Royal Institute of Philosophy Supplement 72. London: Cambridge University Press.
Stross, Charles. 2006. Accelerando. London: Orbit.
Vinge, Vernor. 1993. “The Coming Technological Singularity: How to Survive in the Post-Human Era”, Vision-21:Interdisciplinary Science and Engineering in the Era of Cyberspace. Accessed 8 December 2007. http://www.rohan.sdsu.edu/faculty/vinge/misc/singularity.html.
 In contrast to the transparent multi-modal phenomenology of experience, human verbal thinking is relatively opaque since we are able to recollect earlier stages of processing to represent the syntactic and semantic properties of linguistic symbols.