Does Criminal Law Deter? (Part Two)

Tweet

(Part One)

This is the second in a two-part series looking at Robinson and Darley’s article “Does Criminal Law Deter? A Behavioural Science Investigation”. It is commonly thought that changes to the substantive criminal law (i.e. the rules about counts as a crime and how much punishment should be attached to the commission of crime) can have a deterrent effect. At least, many policy debates and legislative changes proceed on that assumption. But how plausible is it really?

Robinson and Darley argue that it isn’t. While they accept that a criminal justice system, with conspicuous mechanisms of enforcement, can have a deterrent effect, they argue that it is unlikely that the substantive criminal law adds to that effect. And, certainly, the effects are nowhere near as fine-grained or precise as policy debates would typically have us presume.

As reviewed in part one, their argument for this conclusion is fairly simple. They claim that in order for the substantive criminal law to have a deterrent effect, three conditions must be met: (i) the knowledge condition, according to which potential criminals must know what the law demands; (ii) the immediacy condition, according to which potential criminals must be able to bring that knowledge to bear on their crime-relevant decisions; and (iii) the weighting condition, according to which the perceived costs of crime must outweigh the perceived benefits.

Robinson and Darley think it unlikely that all three of these conditions can be met. This means their argument looks something like this:

• (1) In order for the substantive rules of criminal law to have a deterrent effect, three conditions must be met: (i) the knowledge condition; (ii) the immediacy condition; and (iii) the weighting condition.

• (2) It is highly unlikely that the knowledge condition is met.

• (3) It is highly unlikely that the immediacy condition is met.

• (4) It is highly unlikely that the weighting condition is met.

• (5) Therefore, it is highly unlikely that the substantive rules of criminal law have a deterrent effect.

We reviewed the case for premises (2) and (3) the last day. Today we’ll look at the case for premise (4).


1. The Rational Choice Model and the Weighting Condition
The deterrence assumption tends to work with a rationalistic approach to criminal behaviour. That is to say, it tends to presume that criminals make decisions about whether or not to commit a crime based on an analysis of the costs/benefits of the crime. We’ve already looked at ways in which this model of criminal behaviour could be flawed: if the knowledge and immediacy conditions are not met, then potential criminals won’t be able to engage in the kind of cost/benefit analysis required by the deterrence assumption.

Still, there is a distinction to be made between two kinds of rational choice model of criminal behaviour. The first, which focuses on the actual costs and benefits the crime, and the second, which focuses on the perceived costs and benefits. Any model focusing on the former is likely to be well off the mark, since potential offenders may not be able to appreciate the actual costs and benefits. But a model based on the latter could get off the ground, since the perceived costs and benefits are what (presumably) guide any particular decision.

Robinson and Darley’s discussion of the weighting condition focuses on the perceived costs and benefits. They hold that the following inequality is needed if the deterrence assumption is to prove correct:

Perceived Cost of Crime > Perceived Benefit of Crime

Furthermore, and following a suggestion originally put forward by Jeremy Bentham, they argue that the perceived cost will depend on three variables:

Perceived Cost = [Probability of Punishment][Delay of Punishment][Total Amount of Punishment]

In other words, perceived cost is a function of the total amount of punishment, times the probability of being punished, discounted by some delay factor.

Their claim is that behavioural science has revealed that the effect of each of these variables on perceived cost is far more complex and (sometimes) more counterintuitive than is often believed to be the case. They support this by looking at a range of studies. Of course, as they note at the outset, good controlled studies on the effects of each variable on perceived cost are hard to come by: researchers are not permitted to punish research subjects as severely as we punish criminals in the real world. But it is possible to draw some conclusions based on animal studies, experiments involving punishments of moderate intensity (that subjects have consented to) and “natural” experiments.

So let’s look at the evidence.


2. Evidence Relating to the Probability of Punishment
When it comes to assessing the impact of punishment probabilities on behaviour, Robinson and Darley turn to conditioning studies. These are the classic animal-in-a-box experiments, which were so heavily associated with the behaviouralist movement in psychology. The typical set-up in such an experiment is that an animal is trained to perform (or already predisposed to perform) some kind of action (e.g. pressing a button), and this action is then linked to some punishment (e.g. a mild electric shock). In this experimental set-up, it is easy to vary the rate of punishment and see what effect this has on discouraging the action.

Such that have been done on this suggest that once the probability of punishment drops below a certain threshold, it has practically no deterrent effect. For example, in classic studies by Azrin, Holz and Hake it was found that a 10% punishment rate failed to suppress behaviour.

One problem with these studies, however, seems to have been that the punishments were not randomised (i.e. the punishments occurred at regular intervals) and, apparently (I have not confirmed this), studies involving randomised rates of punishment are hard to come by. But, reviewing the few that have been done, Lande found that there was less suppression of behaviour as the probability of punishment declined. Furthermore, he found that there were “response bursts” after punishment. In other words, the experimental subjects engaged in more of the relevant behaviour after having been punished. This may be because they indulged in something akin to the gambler’s fallacy, i.e. they assumed they were much less likely to be punished “the next time round”.

You can imagine where all this is going. Robinson and Darley argue that these studies have worrying implications for the criminal law. Citing actual conviction rates (across all crimes) of 1.3% in the U.S. and single digit punishment rates for even the most serious of crimes, Robinson and Darley suggest that the criminal law may have minimal deterrent effect. This is because the actual punishment rate is below the rate at which it will have a deterrent effect. They also cite anecdotal cases in which a criminal committed a crime just after having been punished, which seems to confirm Lande’s finding of “response bursts”.

So Robinson and Darley seem to support the following argument:

• (6) If the punishment rate drops below a certain threshold (maybe 10%), the criminal law will not have a deterrent effect and may also have perverse effects (e.g. it may lead to “response bursts”).

• (7) The actual punishment rate is below this threshold.

• (8) Therefore, the criminal law will not have a deterrent effect.

There are a several criticisms one could make of this argument. Robinson and Darley only address the most obvious: what if people overestimate the rate of punishment? What if people think the punishment rate is above the relevant threshold? They respond to this critique by suggesting that many potential criminals will suffer from overconfidence bias, and by pointing to David Anderson’s study (discussed in part one) of actual criminals’ attitudes toward punishment. Anderson found that 76% of active criminals and 89% of the most serious offenders either did not think they would be caught or didn’t even consider the possibility of punishment.

I have some other problems with the argument. First, I think it hangs too much on animal conditioning studies. More human data would have been nice (the studies on actual criminals’ attitudes toward punishment are welcome in this respect). Second, the relevant thresholds are little too vague for my liking. And third, I think the argument diverges too much from the original thesis of Robinson and Darley’s article. Their original claim was about the deterrent effect of the substantive criminal law. But this argument really has to do with the effectiveness of the criminal justice system as a whole, not the rules of criminal law. This is surprising given that they originally supported the deterrent effect of the criminal justice system as a whole.


3. Evidence Relating to the Amount of Punishment
One of the assumptions shared by proponents of the deterrence hypothesis is that fine-grained modulation of the total amount of punishment attaching to an offence can make a real difference to how frequently that crime is committed. Thus, for example, increasing the sentence length for theft from five to ten years in prison is thought to make a difference to behaviour. One can see the attraction of this assumption. Simple cost-benefit reasoning tells us that ten years in jail is worse than five years in jail (approximately twice as worse), ergo potential criminals should think twice as hard about committing theft if the sentence is increased to ten years.

As attractive as this reasoning is, there are several problems with it. Again, the all-important factor here is how increases in the duration or amount of punishment are perceived by potential criminals, not what the reality is. Behavioural evidence suggests that these perceptions may be out of line with reality. Robinson and Darley review several strands of evidence in support of this contention.

They start with conditioning studies. The primary conclusion from these studies (they mention a few) is that increases in the severity of punishment do indeed increase deterrence. But there are some interesting nuances. It has been found that animals can adapt to the intensity of punishment. Thus, for example, you could start out giving a pigeon electric shocks below 60 volts and then gradually increase it, even up to 300 volts, and find that the pigeon continued with the behaviour you were trying to deter. This would not be true if you started out punishing at the more intense level.

This doesn’t bode well for the criminal law. It is common for first-time offenders to given more lenient punishments. Indeed, sometimes their punishments are completely suspended. This may create the conditions in which an “adaptation to intensity”-effect can flourish. Furthermore, it’s not just animal studies that support this view. In a famous paper by Brickman and Campbell, it was suggested that human experience of affective changes follows something like a “hedonic treadmill” in which there is gradual adaptation to a new affective baseline. So, for example, what was initially experienced as intensely pleasurable gradually becomes less so as the experiencer adjusts to the new “baseline” level of pleasure.

Robinson and Darley argue that two distinct types of adaptation could occur in the prison environment: (i) neutralisation; and (ii) hardening. The first type of adaptation arises when the prisoner adjusts their affective baseline to cope with what was initially felt to be a highly punitive state. Thus, the punitive effect of, say, prison is neutralised over time. This means that they will continue to experience positive and negative changes to their affective state, but those changes will be relative to the new (lower) baseline. They cite in support of this claim studies showing that the risk of suicide among prisoners is much higher in the first 24 hours of incarceration. “Hardening” is a different process, whereby the prisoner just becomes more immune to changes in their affective environment. Thus, as they go along, they are less affected by positive and negative events. Both of these types of adaptation could undermine the deterrent effect of punishment, particularly imprisonment, for repeat offenders.

Another problem has to do with the discrepancy between remembered pain and the actual duration of a punitive experience. Research by Daniel Kahneman and his colleagues suggests that remembered pain is not simply a direct function of the duration of pain. Instead, it seems to be a function of the most intense pain experienced and the most recent painful experience. The result being that short, but intensely unpleasant, punishment is remembered as being much worse than long, moderately unpleasant, punishment. This lends credence to the deterrent powers of the classic “short, sharp shock” approach to punishment, and detracts somewhat from the contemporary obsession with duration of prison sentence.

The upshot of all this is that there are diminishing deterrent returns to be had by increasing the duration of punishment, and maybe no returns at all to be had from modulating the severity of the punishment in proportion to the gravity of the offence and the offender (due to adaptation effects). This is not to say that there aren’t sound moral reasons for modulating punishment in this manner (proportionality still feels like it is morally justified); but it is to say that such moral reasons are not easily grounded in deterrence.

One criticism of this argument is that it seems to dwell upon the deterrent effect of punishment on those who are undergoing or have undergone it (“special deterrence”) and not on the deterrent effect on the general population. Robinson and Darley wave this objection off by arguing that many of those who commit crimes are repeat offenders. But, unfortunately, that still doesn’t address the criticism. The general population will not have been privy to the kinds of experiences that lead to adaptation effects, so maybe they could be genuinely deterred by increases in the total amount of punishment? The discussion of discounting (below) may provide some response to this criticism.

Robinson and Darley also go on to argue that the social milieu from which many offenders are drawn can affect how they perceive the total amount of punishment. Many are socialised in an environment in which “doing time” is common among their peers, and the potential costs are consequently downplayed. Furthermore, many come from deprived backgrounds and so the costs of punishment may seem much less to them than to someone from a more affluent background. Again, these factors may combine to undermine the deterrent effect.


4. Evidence Relating to the Delay of Punishment
The final factor that affects the perceived cost of punishment is the delay between carrying out the act that is to be punished and the actual administration of punishment. Robinson and Darley are brief on this point since the behavioural evidence seems to be pretty clearcut: a combination of conditioning studies and cognitive psychology experiments indicates that the greater the delay between the act and the punishment, the lesser the deterrent effect.

Perhaps one of the most widely-popularised models of this argues that humans (and other animals) hyperbolically discount the value of future rewards (and losses) relative to more immediate rewards and losses. In other words, they prefer “smaller sooner” rewards to “larger later” rewards. This is thought to account for addictive behaviours like cigarette smoking. Even though people are often aware of the long-term costs of smoking, and the long-term rewards of not-smoking, their internal valuation mechanisms are such that the immediate reward of smoking seems more attractive than the long-term reward of quitting. The diagram below provides a representation of hyperbolic discounting curves for human preferences. Note how the value of the smaller reward exceeds that of the larger reward at a certain point.

The relevance of this to criminal behaviour and punishment should be obvious. Since there is often a significant delay between the commission of a crime and the punishment (if any) of that crime, the short term rewards of the crime will often seem more attractive to a potential criminal than the long-term costs of the punishment. Consequently, the delay undermines the deterrent effect.

I think Robinson and Darley are probably correct about this, however, I can’t help but point out that this, once again, has more to do with the criminal justice system as whole than it does to do with the substantive criminal law. After all, substantive criminal laws tend not to stipulate that there must be a significant delay between the commission of crime and the imposition of punishment.


5. Conclusion
Okay, so that brings us to the end of Robinson and Darley’s argument. To briefly recap, their central thesis is that the substantive criminal law is unlikely to deter criminal behaviour. This is because in order for the substantive law to have a deterrent effect it must satisfy three conditions: (i) the knowledge condition; (ii) the immediacy condition; and (iii) the weighting condition. Drawing from a range of behavioural science research, Robinson and Darley argue that it is unlikely that all three conditions are met.

Their argument is plausible, and brings together some interesting strands in the behavioural science literature, but it still contains points of weakness. The most annoying of which is, perhaps, its tendency to stray from the central thesis — which was supposed to be about the substantive criminal law — into more general concerns about the criminal justice system.

Does Criminal Law Deter? (Part One)

Tweet

My title is drawn from a 2004 article by the criminal law theorists Paul Robinson and John Darley. The article critiques the common view — at least, common among proponents of an economic/consequentialist approach to law — that the criminal law can deter criminal behaviour. Contrary to this view, Robinson and Darley argue that although the criminal justice system as a whole may deter criminal behaviour, the substantive rules of criminal law do not.

Over the course of the next two posts, I want to review and engage with their defence of this thesis. As they subtitle their article “A Behavioural Science Investigation”, it should come as no surprise to learn that many of the reasons they adduce are drawn from the experimental literature. Consequently, this series of posts gives me a good excuse to review some of interesting empirical findings.

The remainder of this post will do two things. First, it will outline the basic structure of Robinson and Darley’s argument. Then it will look at the evidence they present in support of two of the key premises in that argument. Part two will wrap things up by looking at the evidence supporting the third key premise.


1. The Anti-Deterrence Argument
It is often assumed that the substantive rules of criminal law — i.e. the rules that define the elements of a criminal offence and that state the range of punishments that can attach to such offences — can have a deterrent effect. Certainly, many debates among theorists and policy makers proceed on this assumption. They consider whether new rules should be created in order to deter different sorts of behaviour and they try to increase the severity of the punishment in order to increase the deterrent effect of the rules. To be sure, alterations to the substantive rules are proposed for other reasons too (justice, consistency, political optics etc.), but deterrent effect is usually lurking somewhere in the background.

Robinson and Darley claim the assumption of the deterrent effect is flawed. The criminal justice system as whole may indeed have a deterrent effect. That is to say, having a system that polices, enforces, prosecutes and punishes crime may have a deterrent effect. But the role of the substantive rules of criminal law in securing that deterrent effect is rather more questionable. This is because three conditions be met if the substantive rules are going to be capable of deterrence. They are (the names are mine, not those of the original authors):

The Knowledge Condition: The potential offender must know, directly or indirectly, what the substantive rule actually is.

The Immediacy Condition: The potential offender must be able to bring that knowledge to bear on the immediate context in which they are deciding whether or not to commit a crime.

The Weighting Condition: The potential offender must perceive/believe that the expected cost of committing the crime is greater than the expected benefit.

The problem is that it is very difficult for all three of these conditions to be met. Thus, Robinson and Darley’s anti-deterrence argument can be interpreted in the following manner.

• (1) In order for the substantive rules of criminal law to have a deterrent effect, three conditions must be met: (i) the knowledge condition; (ii) the immediacy condition; and (iii) the weighting condition.

• (2) It is highly unlikely that the knowledge condition is met.

• (3) It is highly unlikely that the immediacy condition is met.

• (4) It is highly unlikely that the weighting condition is met.

• (5) Therefore, it is highly unlikely that the substantive rules of criminal law have a deterrent effect.

The remainder of this series will be dedicated to premises (2) - (4). Before getting to them, however, a word or two about premise (1) would seem to be in order. There are two kinds of deterrent effect that the law could have. It could have a general deterrent effect, which means it could deter the general population from committing a crime; or it could have a special deterrent effect, which means it could discourage a particular person or group of persons (who have usually already been subjected to the rules) from committing a crime again in the future.

I’m inclined to think that it is quite difficult to prove general deterrent effects, if only for the simple reason that people who are dissuaded by the law often fail to inform us of this fact. Empirical evidence can, of course, be drafted in to support a deterrent effect. This is typically done by comparing data on different populations living under different legal rules and trying to make some inference from this data. But this is often of limited utility. For example, attempts to do this when trying to prove that the death penalty has a deterrent effect are very messy, despite the fact that the presence or absence of the death penalty is one of the better known facts about the legal system. (Donohue and Wolfers paper).

Establishing a special deterrent effect is more plausible. In fact, many proponents of the deterrence thesis limit their focus to a special group of potential offenders: the hardened or career criminals, who are more likely to engage in the kind of rational cost-benefit analysis that seems to be demanded by the deterrence assumption.

Although Robinson and Darley avoid drawing the general/special distinction (they do mention it at one point in response to another argument), their arguments are often directed to the hardened or career criminals. Thus, their first premise can be interpreted as laying down a plausible set of conditions that would have to be met in order for the substantive criminal law to have an effect on such people. That might be a problem insofar as the evidence adduced sometimes fails to consider the general deterrent effect.


2. Does the Knowledge Condition Fail?
The first condition claims that in order for the substantive rules to have a deterrent effect, the potential criminal must actually know what the rules demand. Which means they either have to look them up for themselves, or they have to imbibe the information from others. The problem with this is twofold. First, people typically do not look up the substantive criminal law, and second their acquisition of the relevant information from others is often misleading. This is true even among the population with the most incentive to learn the substantive rules.

This isn’t merely armchair speculation. Empirical studies of the knowledge condition seem to confirm it. Robinson and Darley report on a study that they themselves carried out. The study investigated people’s knowledge of four different legal rules across five different U.S. states. The rules related to the duty to assist strangers and to report felonies, and the right to use deadly force when retreat was possible and when protecting one’s home. The five states took different positions on each of the rules, some being “majority” positions and some being “minority” positions. The findings were stark. People’s beliefs about the content of the legal rules did not vary if they were living in a state adopting a minority position, but nor did their beliefs track the majority position. Instead, people believed the content of the rule to be that which according with their own moral intuitions.

That study covered a sample of all citizens within a given U.S. state. Other studies have looked at beliefs among people who have been convicted of felonies. Robinson and Darley cite one study in particular from David Anderson, which tested convicted felons knowledge of the punishments that attached to their crimes. It found that only 22% felt that they knew what the punishment was, 18% said they had no idea or were wrong, and 35% didn’t even think about it. The last result is worth remarking on since it feeds into the immediacy condition, which we will discuss below.

Robinson and Darley also cite studies suggesting that indirect acquisition of information relating to criminal law can be misleading. A classic example being the public’s knowledge of the insanity defence. Surveys reveal that people think this defence is far more widely applicable and far more successful than it actually is.

There are a couple of counterarguments to consider here. First, it could be that certain substantive rules are well-known and really could therefore have a deterrent effect. For example, pretty much everybody knows the rules relating to murder, theft and rape (although the last one is certainly more contentious) and might be dissuaded from doing these things as a result. Furthermore, pretty much everybody knows that the severity of punishment increases once you reach a certain age (e.g. 18), and evidence seems to suggest that there is a drop-off in criminality among young offenders when they reach that age. But these, for Robison and Darley are rare exceptions and, indeed, may be explicable because the substantive law tracks intuitive moral norms. Certainly, they don’t provide reason to think that people’s knowledge tracks the fine-grained distinctions often made during policy debates.

A second problem with the argument is that it over-intellectualises deterrence. As the authors themselves put it: a rat can still be deterred by a perceived threat, without having to be consciously aware of the rule implementing that threat. This is undoubtedly true. But as the authors also point out, the perceived threat of punishment has more to do with perceived enforcement practices (e.g. police officers on the beat), not knowledge of the substantive criminal law. Indeed, one could probably increase enforcement, without changing the substantive rules, and achieve a significant deterrent effect.

The only caveat I would add to Robinson and Darley’s argument about the knowledge condition is that errors about the law could still support its deterrent effect if they tend to be systematically biased in a particular direction. To be more precise, if people’s beliefs about the law tend to be more severe and punitive than the reality, this could be highly deterrent. That said, evidence for such a systematic bias is lacking and, indeed, other evidence suggests the opposite may be true.


3. Does the Immediacy Condition Fail?
The second condition for achieving a deterrent effect is the immediacy condition. This condition claims that potential criminals must be able to bring their knowledge of the criminal law to bear on their crime-relevant decisions. It is important to note that this condition only really becomes a factor if we assume that the knowledge condition has already been met. After all, if people’s knowledge of the substantive criminal law is faulty, it doesn’t really matter if they bring it to bear on their decisions or not (unless, as I suggested at the end of the previous section, their errors are biased in the direction of increased severity).

But even if we assume they have the correct knowledge, Robinson and Darley point to a range of factors suggesting that they are unlikely to be able to bring this knowledge to bear on any particular decision to commit a crime. The factors that prevent this from happening break down into two categories: (i) personality-based factors; and (ii) contextual factors.

Personality-based factors concern the people who commit crimes. Robinson and Darley cite a range of research suggesting that potential offenders, as a class, tend to be more risk-seeking and more impulsive than the rest of the population. Consequently they are less likely to engage in the kind of forward-planning demanded by the deterrence assumption. Gottfredson and Hirschi have also theorised that poor self-control and inability to delay gratification is a prominent personality trait among potential offenders.

The problem with impulse control and risk-seeking are exacerbated by recent drug and alcohol abuse, which is common among offenders (though there may be some selection effect here since there are so many drug-related offences). Anderson’s study, mentioned above, found that 66% of offenders reported recent drug abuse. There are also mental disorders that can affect criminal behaviour, such as paranoia and mania.

These personality-based impediments to the use of knowledge are supplemented by other contextual impediments. Some crimes are committed “in the heat of the moment”, in anger or retaliation. These emotions tend to swamp any rational analysis of the costs and benefits of crime. Furthermore, many potential offenders differentially associate with other criminals (in gangs and so forth). This creates a group context in which there is often short-term pressure to commit crimes (in order to win group acceptance) and in which the long-term costs of crime tend to be downplayed.

The net result is that knowledge of the criminal law, even if it is correct, is often obscured in the immediate context of a crime.

We’ll pause at this point and resume in part two by looking at the third condition for deterrence. That condition forces us to consider the rational choice model of criminal decision-making in more detail, and to investigate how potential offenders perceive the costs and benefits of their decisions.

Rule by Algorithm? Big Data and the Threat of Algocracy

Tweet

An increasing number of people are worried about the way in which our data is being mined by governments and corporations. One of these people is Evgeny Morozov. In an article that appeared in the MIT Technology Review back in October 2013, he argued that this trend poses a serious threat to democracy, one that should be resisted through political activism and “sabotage”. As it happens, I have written about similar threats to democracy myself in the past, so I was interested to see how Morozov defended his view.

Unfortunately, Morozov’s article is not written in a style that renders its argumentative structure immediately transparent. I shouldn’t complain: not everyone writes in the style of an analytic philosopher; not everyone aspires to reduce human language to a series of number propositions and conclusions. Nevertheless, I have set myself the task re-presenting Morozov’s argument in a more formal garb, and subjecting it to some critical scrutiny. That's what this blog post is about.

The discussion is broken-down into three sections. First, I’ll talk in general terms about the problem Morozov sees with data-mining technologies. Second, I present what I take to be Morozov’s central argument, which I call the argument from the threat of algocracy. This, I suggest, is similar to an argument found in the work of the political philosopher David Estlund. Finally, I look at the suggested solutions to the problem, noting how Morozov’s solution differs from one that I have defended in the past.


1. The Problem of Invisible Barbed Wire
The way in which our data is being monitored and processed has been well-documented. A recent article by Alice Marwick in the NY Review of Books gives a good overview of the phenomenon. Interested readers should check it out. What follows here is just a summary.

In brief, modern technology has made it possible for pretty much all of our movements, particularly those we make “online”, to be monitored, tracked, processed, and leveraged. We can do some of this leveraging ourselves — as do the proponents of self-experiments and members of the quantified self movement — by tracking our behaviour along various metrics and using this to improve our diets, increase our productivity and so forth. To many, this is a laudatory and valuable endeavour.

But, of course, governments and corporations can also take advantage of these data-tracking and processing technologies and use it to pursue their ends. Governments can do it to monitor terrorist activity, track welfare fraud, and prevent tax evasion. Companies can do it to track consumer preferences, create targetted advertising and manipulate consumer purchasing decisions.

You might think that this is all to the good. Who doesn’t want to stop terrorism, stamp out tax evasion and have a more pleasant shopping experience? But Morozov points out that it has a more sinister side. He worries about the role of data-mining in creating a system of algorithmic regulation, one in which our decisions are “nudged” in particular directions by powerful data-processing algorithms. This is worrisome because the rational basis of these algorithms will not be transparent. As he puts it himself:

Thanks to smartphones or Google Glass, we can now be pinged whenever we are about to do something stupid, unhealthy or unsound. We wouldn’t necessarily need to know why the action would be wrong: the system’s algorithms do the moral calculus on their own. Citizens take on the role of information machines that feed the techno-bureaucratic complex with our data. And why wouldn’t we, if we are promised slimmer waistlines, cleaner air, or longer (and safer) lives in return? 

(Morozov, 2013)

In other words, the algorithms take over from the messy, human process of democratic decision-making. Citizens become beholden to them, unsure of how they work, but afraid to disregard their guidance. This creates a sort of prison of “invisible barbed wire” which constrains our intellectual and moral development, as well as our lives more generally:

[The problem] here is the construction of what I call “invisible barbed wire” around our intellectual and social lives. Big data, with its many interconnected databases that feed on information and algorithms of dubious provenance impose severe constraints on how we mature politically and socially…  

The invisible barbed wire of big data limits our lives to a space that might look quiet and enticing enough, but is not of our own choosing and that we cannot rebuild or expand. The worst part is that we do not see it as such. Because we believe that we are free to go anywhere, the barbed wire remains invisible. 

(Morozov, 2013)

The upshot: big data is undermining democracy by depriving us of our ability to think for ourselves, determine our own path in life, and critically engage with governmental decision-making.


2. The Argument from the Threat of Algo-cracy
Morozov’s argument is clothed in some ominous language and clever metaphors (yes: I like the point about “invisible barbed wire”) but for all that it will be familiar to students of political philosophy. People have long worried about the prospect of epistemic elites taking over governmental decision-making, just as people have also long argued in favour of such takeover (Plato and John Stuart Mill being two who spring to mind).

Morozov’s argument is the argument from the threat of algocracy (i.e. rule by algorithm). At the heart of this argument is the debate about what legitimates governmental decision-making in the first place. What gives governments the right to set policies and impose limits on our behaviour? What makes a system of government just and worthwhile? Generally speaking, there are three schools of thought on the matter:

Instrumentalists: hold that legitimation derives from outcomes. Policies and regulations are designed to accomplish certain goals (in moral terms: they are designed to help us realise fundamental human goods like joy, sociality, friendship, knowledge, freedom etc.). They are legitimate if (and only if) they accomplish those goals.

Proceduralists: hold that legitimation comes from the properties of the decision-making procedures themselves. They defend their view by arguing that we have no idea what the correct mix of regulations is in advance. What matters is that the procedure through which these regulations are adopted is itself just; that it gives people the opportunity to voice their concerns; that it is comprehensible to them; that it gives due weight to their concerns; and so forth.

Pluralists: hold that some combination of just procedures and good outcomes is needed.

Democratic governance is often defended on pluralist grounds, although some theorists emphasise the instrumental goods over the procedural ones, and vice versa. The general idea being that democratic governance has just procedures and leads to good outcomes, at least more so than alternative systems of governance.

The problem with justifying governance along purely instrumentalist lines is that it can give rise to the threat of epistocracy. This threat has been most clearly articulated by David Estlund. As he sees it, if it is really the case that procedure-independent goods legitimate systems of government, it is likely that that some people (the epistemic elites) have better knowledge or foresight of the policies that will lead to those outcomes than others. Indeed, some of these others (the epistemically incompetent) might actively thwart or undermine the pursuit of the procedure-independent goods. Consequently, and following the instrumentalist logic, we should hand over decision-making control to these epistemic elites.

This looks to be an unwelcome conclusion. It seems to reduce the people affected by governmental decisions to little more than moral patients: receptacles of welfare and other positive outcomes, who do not actively shape and determine the course of their own lives. Hence people tend to fall back on pluralist and/or proceduralist approaches to legitimacy, which value the active contribution to and participation in decision-making. (I should add, here, that people can also resolve the problem by appealing to freedom or self-determination as one of the procedure-independent goods toward which governance should be directed).

Morozov’s compaint is essentially the same as that of Estlund. The only difference being that where Estlund is concerned about human epistemic elites (like experts and other technocrats), Morozov is concerned about algorithms, i.e. non-human artificial intelligences that mine and manipulate our data. Morozov couches his argument in terms of the threat to democracy, but what he is specifically talking about are both procedural goods associated with democratic governance (comprehensibility, participation, deliberation) and the other goods that should be protected by such procedures (freedom, self-determination, autonomy etc.). His concern is that over-reliance on data-mining algorithms will undermine these goods, even if at the same time they help us to achieve certain others.

To phrase his argument in slightly more formal terms:

The Argument from the Threat of Algocracy 

• (1) Legitimate democratic governance requires decision-making procedures that protect freedom and autonomy, and that allow for individual participation in and comprehension of those decision-making procedures.  

• (2) Reliance on data-mining algorithms undermines freedom and autonomy and prevents active participation in and comprehension of decision-making procedures.  

• (3) Therefore, reliance on data-mining algorithms is a threat to democratic governance.

This argument relies on a normative premise (1) and a factual/predictive premise (2). I think the normative premise is reasonably sound. Indeed, I would add in the explicit claim — implied in Morozov’s article — that this form of governance is desirable, all else being equal. The only question then is whether all else is indeed equal. Proponents of algocracy — like proponents of epistocracy — could argue that the procedure-independent gains from algorithmic policy-making will outweigh the procedural and autonomy-related costs. Morozov needs it to be the case that the goods singled out by his argument are more important than any of those putative gains (assuming he argues from consequentialism) or that their impingement is blocked by deontological constraints. I think he would have a strong argument to make on consequentialist grounds; I’m less inclined to the deontological view.

I suspect that the factual/predictive premise is going to prove more controversial. Is it true that the widespread use of data-mining algorithms will have the kind of negative impact envisioned by Morozov? It certainly seems to be true that the majority people don’t comprehend the basis on which algorithms make their decisions. Whether this must be the case is less clear to me. I’m not well-versed in how modern algorithms are coded. Still, I suspect there is a good case to be made for this. That is to say: I suspect algorithms throw up many surprises, even for the engineers who create them, and will continue to do so, particularly as they become more complex.

A slightly more tricky issue has to do with the potentially autonomy-enhancing effects of data-mining technologies. The message from fans of self-experimentation and self-quantification, who are often obsessed with mining data about themselves, seems to be that technologies of this sort can greatly enhance the control one has over one’s life. They can lead to greater self-awareness and goal fulfillment, allowing us to pick and choose the behavioural strategies that lead to the best outcomes for us. That would seem to suggest that premise (2) is false in certain instances.

But that’s not to say that it is generally false. Morozov is probably right in thinking that top-down use of such technologies by governments and corporations is a problem. Nevertheless, the experiences of these individual users does suggest that there is a way in which the technology could be harnessed in a positive manner. Morozov is aware of this argument, and resists it by arguing that the forces of capitalism and bureaucracy are such that the top-down uses will tend to dominate the bottom-up uses. This seems to be what is happening right now and this is what is important.


3. Solutions?
So what is to be done about the threat of algocracy? Morozov complains that current solutions to data-mining miss the point. They conceive of the problem in terms of the right to privacy, and hence craft solutions that give people greater control over their data, either through robust legal protections of that data, or market-based systems for owning and trading in that data (the suggestion in the article is that market-based solutions are distinct from legal ones, though this is clearly not the case: the kinds of market-based solution that are being advocated rely on the law of property and contract).

Instead, Morozov — in a section titled “Sabotage the System. Provoke more Questions” — argues for a four-pronged solution (there is some overlap between the prongs):

A. Politicise the problem: we must, as he puts it “articulat[e] the existence — and profound political consequences — of the invisible barbed wire”. We must expose the “occasionally antidemocratic character” of big data.

B. Learn how to sabotage the system: we must arm ourselves by resisting the trend toward self-monitoring and refusing to make money off our own data. “Privacy can then reemerge as a political instrument for keeping the spirit of democracy alive”.

C. Create “provocative digital services”: we must facilitate technologies that allow us to see the negative effects of data-mining. For example “instead of yet another app that could tell us how much money we can save by monitoring our exercise routine, we need an app that could tell us how many are likely to lose health insurance if the insurance industry has as much data as the NSA.”

D. Abandon preconceptions: we must cast-off any preconceptions we may have about how our digital services work and interconnect.

In short, what we need is some good old-fashioned, consciousness-raising and political activism.

I think there is merit to this. I think we could be more critical about the uses of technology. That’s why I find Morozov’s brand of anti-techno-utopianism somewhat refreshing. But I definitely worry that this is all just a bit too idealistic. If the forces of capitalism and bureaucracy are as powerful as Morozov seems to suggest elsewhere in the article, is some bottom-up activism going to be enough to overturn it?

In my article “On the Need for Epistemic Enhancement: Democratic Legitimacy and the Enhancement Project”, I suggested that broader use of enhancement technologies could help stave off the threat of epistocracy. I’m not convinced I’m correct about this, incidentally, but it does prompt the thought: would broader use of enhancement technologies also help stave off the threat of algocracy?

Probably not. Enhancement technologies might help to equalise the balance of power between ordinary humans and epistemically elite humans. Whether they could do the same for the balance of power between humans and algorithms is another matter. The gap is much wider. AIs have significant advantages over humans when it comes to data-mining and processing. The only kinds of enhancement technology that could bridge the gap are the truly radical (and hypothetical) forms. The ones that would give rise to a genuinely posthuman form of existence. Even if this form of existence is possible, concepts like freedom and self-determination may no longer have any meaning in such a world. Now there’s a thought…

Book Recommendation ♯12: Contractualism and the Foundations of Morality by Nicholas Southwood

Tweet

(Series Index)

I’ve been meaning to recommend Nicholas Southwood’s excellent defence of metaethical constructivism — Contractualism and the Foundations of Morality — for some time, but held off because I was hoping to do a few substantive blog posts about it. Unfortunately, it looks like I won’t be able to do any substantive posts about it in the foreseeable future. Fortunately, it looks like this is the ideal time to offer a recommendation as it is due for paperback release this month.

As far as I am concerned, the book represents the best recent attempt to defend a constructivist (specifically contractarian) metaethics. Metaethics is the branch of moral philosophy that is concerned with the ontology and epistemology of moral facts. Constructivist metaethical theories hold that moral facts are constructed out of certain fundamental elements of practical rationality. They are distinct from realist theories insofar as they deem moral facts to be mind-dependent, but they are similar to realist theories in that they believe moral propositions to have truth value. Although I have mixed metaethical views, I am generally inclined toward constructivist theories. Consequently, I am a biased reader of Southwood’s book: I would like for it to be successful, though I am not blind to its shortcomings.

Nevertheless, I think I am correct in saying that Southwood’s book is a masterclass in concision and argumentative acuity. It opens with some general reflections on desiderata for a successful defence of contractarian constructivism. Southwood limits his focus to finding a grounding for moral obligations, thereby setting to one side the grounding for moral values. He then proceeds to diagnose the flaws with two well-established contractarian metaethical theories — Hobbesian and Kantian, respectively — before defending his own preferred approach: deliberative contractarianism.

I don’t have much else to say, except: If you have any interest in metaethics, you should get this book.

Is mind-uploading existentially risky? (Part Three)

Tweet

(Part one, Part two)

This is the third (and final) part in my ongoing series about the rationality of mind-uploading. The series deals with something called Searle’s Wager, which is an argument against the rationality of mind-uploading. The argument was originally developed by Nicholas Agar in his 2011 book Humanity’s End. This series, however, covers a debate between Agar and Levy in the pages of the journal AI and Society. The first two parts discussed with Levy’s critique; this part discusses Agar’s response.

The basic elements of Searle’s Wager are easily summarised: assume we have some putative mind-uploading technology which scans your brain, and then creates an artificial replica of it that can be seamlessly integrated with other digital media (before finally destroying the original copy). To all external appearances, this artificial replica appears to be you: it has the memories and relevant behavioural traits, and claims to be conscious. Is this enough to make uploading rational? No, argues Agar. It could be — following the arguments of John Searle (among others) — that this upload lacks true intrinsic intentionality and consciousness: that it is not really “you” but rather an empty simulacrum of who you once were. In other words, it is possible that uploading entails your death; not your continuation in another form. This would be because we live in a world in which weak AI is true, but not strong AI.

Agar’s argument is based on principles of rational choice. His claim is that the possibility that uploading entails death, coupled with the minimal benefits of uploading from the perspective of one who is likely to face the decision, is enough to determine its irrationality. Levy’s critique challenged Agar on both of these fronts. In brief, Levy argued that:

The non-zero probability of Weak AI is not enough to make uploading irrational: In his original discussion, Agar claimed that the non-zero probability that uploading entailed death was enough to make the Wager work. Levy said that this was false. Non-zero probabilities of death come pretty cheap, and if we tried to avoid them all we wouldn’t be able to perform any actions in the first place. Thus, the principle of rational choice motivating the wager was absurd. 

Agar understates the potential benefits of uploading: Another key part of Agar’s argument was the claim that by the time putative uploading technologies became available, other advanced forms of biological enhancement and life extension would be available. Thus, the benefits of uploading to the person faced with the decision would be minimal: they could already continue their life in a biologically-enhanced form. What good would a digital or artificial form be? Levy responded by arguing that this ignores the role of anchoring points in assessing the benefits of uploading. From our current perspective, there might be little difference between continued biological existence and uploaded existence, but from the perspective of the already-biologically enhanced, things might seem very different.

In his article, Agar responds to both of these arguments. The remainder of this post summarises these responses.


1. The Probabilities Really Do Matter
Responding to the first argument, Agar makes a significant concession to Levy’s critique: the probabilities really do matter. Unlike the Pascalian Wager by which it is inspired, Searle’s Wager deals with finite rewards and finite costs. Consequently, the mere non-zero risk of death is not by itself enough to render uploading irrational. Instead, the risk of death must cross some relevant threshold of reasonability before it is sufficient to render uploading irrational.

Having made that concession, Agar then proceeds to argue that the risk of death from uploading does cross this reasonability threshold. He does not do this by arguing for an arbitrary numerical figure for reasonability (e.g. 5% risk) but instead by arguing for a principle of epistemic modesty. This will be familiar to anyone who has read up on the epistemology of disagreement (something I once covered in relation to religious disagreements). The principle looks like this (there is an unfortunate typo in the official version of Agar’s article; I have removed it from this statement of the principle):

Principle of Epistemic Modesty (PEM): suppose others believe differently from you on a matter of fact. They appear to have access to the fact that is as good as yours and they seem not to be motivated by considerations independent of the fact. You should take seriously the possibility that you are wrong and your opponents are right.

Agar supports this principle, both in relation to strong AI and other philosophical doctrines, with a series of observations and thought experiments. For example, he cites the fact that many reasonable people (e.g. Searle) who have access to all the same facts and arguments, reject strong AI. He also discusses the general problem that many philosophical doctrines — such as the veracity of strong AI over weak AI — are not subject to empirical confirmation. At best, the truth of strong AI could be verified, subjectively, by the person who chooses to undergo uploading. But, of course, such a person would have to take the very risk that Searle’s Wager counsels against to verify the claim.

Another problem mentioned by Agar, in support of PEM, is that many controversial philosophical doctrines like Strong AI are dependent on or connected to other philosophical doctrines that are also quite controversial. He gives the example of physicalism and strong AI to prove his point. Although strong AI is not strictly incompatible with non-physicalism, Agar suggests that the truth of physicalism would raise the probability of Strong AI; whereas its falsity would lower the probability. While he himself is a committed physicalist, he acknowledges that it could be false and this probability should impact upon the overall probability of Strong AI.

For the most part, I don’t have a problem with these particular arguments. I agree that it is difficult to verify certain philosophical doctrines; and I agree that dependency does affect overall probability (though there is a problem here when it comes to dwindling probabilities: if you join together a sufficient number of uncertain propositions you make virtually anything seem improbable). I’m less sure about the reasonability of other people’s beliefs and how these should affect one’s reasoning, but I’ll ignore that problem here (it is discussed in my earlier series on disagreement in the philosophy of religion). Where I do have a problem is with Agar use of a thought experiment to explain what it means to take your opponent’s views seriously.

He asks you to imagine an omniscient being giving you the opportunity to place a bet on all your core philosophical commitments — physicalism, naturalism, consequentialism, Platonism, scientific realism, epistemological externalism and so on. The bet works like this: (i) if it turns out that your commitments are true, you will get a small monetary reward; (ii) if it turns out that they are false, you will die. Would you accept the bet? Agar says that he is pretty sure that physicalism is true, but he wouldn’t bet his life on it. He thinks that our commitment to other controversial philosophical doctrines should entail a similar level of caution.

The problem I have with this thought experiment is the role it plays in the overall dialectic. Agar’s overarching goal is to show that the probability of Strong AI being false is sufficient to warrant caution in the case of Searle’s Wager. But this thought experiment simply restates Searle’s Wager, albeit in a more general form that applies to other philosophical doctrines. It seems like he is just underlining or highlighting the original argument, not presenting a new argument that supports the Wager. Am I wrong? Comments below please.


2. The Rewards of Uploading?
Responding to the second of Levy’s criticisms, Agar tries to argue that the costs of uploading (relative to the putative benefits) are sufficient — even from the perspective of the hypothetical future agent who may face the decision — to render uploading irrational. Agar’s discussion of this point is largely a repeat of his previous claims about the limited gains of uploading, coupled with some responses to Levy’s specific complaints. I’ll run through these briefly here, offering some critical comments of my own along the way.

When it comes to the potential costs of uploading, Levy criticises Agar for claiming that this might entail death simply because it might entail the cessation of consciousness. According to Levy, a “modified” version of the organism view of identity can block this conclusion. Agar, rightly as far as I’m concerned, expresses some puzzlement at this. The organism views maintains that continuation of the same identity depends on the continuation of the underlying biological organism. Given that uploading entails switching from a biological organism to a non-biological one, it is difficult to see how it blocks the conclusion that uploading entails death. There would have to be some serious “modification” of the organism view to make that comprehensible. Furthermore, even if identity was preserved in this manner, the cessation of continued consciousness might make uploading bad enough to be worth avoiding (although see here for a provocative counterargument).

When it comes to the potential benefits of uploading, Levy criticises Agar for not fully appreciating the decision from the perspective of the hypothetical future agent. That person, as you recall, is likely to have many biological enhancements and the potential for an indefinite biological life with those enhancements. Still, might they not be tempted by the additional enhancements of non-biological existence? Agar responds by going into more detail about the alleged benefits of uploading, appealing in the process to the writings of perhaps its most influential advocate: Ray Kurzweil. The notion of mind-uploading appeals to Kurzweil (in part) because it could allow for the exponential expansion of our intellects. Apparently, with uploading we will be able cannibalise the inanimate parts of the universe and incorporate them into our own intelligences:

[u]ltimately, the entire universe will become saturated with our intelligence.... We will determine our own fate rather than having it determined by the current ‘dumb’ simple, machinelike forces that rule celestial mechanics. 

(Kurzweil, The Singularity is Near, p. 29)

Agar suggests that this vastly expanded intellect is less appealing than it first seems. After all, it may well entail a kind of hivemind (or Borg-like) existence, with some significant erosion of individuality or personality. Those things are valuable to us and are likely to continue to be valuable to us in the future. Do we really want to risk losing them? Furthermore, the values of this vastly expanded intellect could be truly alien from our own. Surely it would be better to stick to a comprehensible, more human-like form of existence? (This is actually Agar’s main argument throughout his book: don’t gamble on posthuman values)

Emerging out of this argument is Agar’s response to Levy about the case for investing in mind-uploading technologies in the here and now. As you’ll recall from part one, Levy also criticised Agar for surreptitiously switching back and forth between different timeframes when developing the case for Searle’s Wager: sometimes looking at it from the perspective of a hypothetical future mind-uploader, and sometimes from the perspective of the present. The only decision we have to make in the here and now is whether to invest in mind-uploading technologies or not. Levy suggested that it could make sense to invest in these things, even if we don’t currently value an uploaded existence, because we might come to value them at the relevant time. It would be like a man who tries to learn bridge in his early 50s, even though he has no present interest in it, because he hopes he will value it when he comes to retirement. At that point in time, bridge-playing may be an enjoyable social activity.

Agar responds be arguing that it may indeed make sense for the future retiree to make decisions like this, but that’s only because his future entails a comprehensibly human form of existence. He wants to have an enjoyable retirement, with a rich and rewarding social life, and he sees that bridge-playing might be an essential part of that. This is very different from the biological human gambling on a non-biological form of existence that may not even entail his/her continued existence as an individual.

I think there is some merit to these points. Of course, uploading need not entail the kind of exponential increase in intelligence envisioned by Kurzweil, but in that case the question needs to be asked whether it is any different from an enhanced and indefinite form of biological existence. This does, however, bring me to an aspect of Agar’s argument that I find difficult to follow. Agar repeatedly insists (both in this article and in the original book) that his argument is concerned with the future person who has biological enhancements, that includes something like De Grey’s LEV-technologies. To that person, uploading will be irrational. Agar underscores the centrality of this to his argument by conceding that if such alternative technologies are not available, uploading will seem less irrational. Why so? Because it might be the only plausible means for a terminally ill person to continue their lives.

I get that. But then Agar also insists that his argument does not depend on whether LEV-technologies (or their ilk) are actually available but on whether they are likely to become available before uploading technologies (i.e. he insists that his claim is about the relative likelihood of the technologies coming onstream):

My point here requires only that LEV is likely to arrive sooner than uploading. Uploading requires not only a completed neuroscience, total understanding of what is currently the least well-understood part of the human body, but also perfect knowledge of how to convert every relevant aspect of the brain’s functioning into electronic computation. It is therefore likely to be harder to achieve than LEV. 

(Agar, 2012, p. 435)

This is the bit a I don’t get. I think that in saying this Agar is illegitimately switching back and forth between different timeframes in the manner Levy critiqued. As I see it, there are two ways to interpret the claim about relative likelihood:

Interpretation One: Agar is speaking from the present about the timelines according to which different technologies will be realised. In other words, he is simply claiming that LEV is likely to be achieved before uploading. This might be true. The problem is that it doesn’t affect the rationality of the decision from the perspective of the hypothetical future agent. From that perspective, it’s either going to be the case that LEV and uploading are available or one or the other. If it’s the latter, then it might be rational to opt for uploading if it is the only means of continuing one’s life. The relative likelihood is unimportant. 

Interpretation Two: Agar is speaking from the perspective of the hypothetical future agent, who doesn’t quite have the full suite of LEV technologies available to them, but faces a choice between some temporarily life-extending therapy and uploading. Maybe then the argument is that this person should gamble on the temporarily life-extending technology in the hope that more LEV technologies will come onstream before they die.

The first interpretation is textually supported by what Agar says, but it doesn’t help the argument about future irrationality. The second interpretation is more sensible in this regard, but isn’t supported by what Agar says. It also faces some limitations. For example, there could well be scenarios in which the benefits of a few more months of biological existence are outweighed by the possible benefits of uploading.

So I’m left somewhat puzzled by what Agar means when he talks about the relative likelihood of the different technologies. Which interpretation does he favour? Is there another interpretation that I am missing out on?


3. Conclusion
To briefly sum up, in this series I’ve looked at the debate between Neil Levy and Nicholas Agar over the merits of Searle’s Wager. The wager purports to show that mind-uploading is irrational because of the risk that it entails death. Levy critiqued the wager on the grounds that it relied on an unsustainable principle of rational choice — viz. we should avoid any action that carried a non-zero risk of death — and understated the possible benefits of uploading.

In this post, we’ve looked at Agar’s responses. He agrees with Levy that his original principle of rational choice is unsustainable, but responds by claiming that the risk of death from uploading is still sufficiently high to merit caution. He also defends his cost/benefit analysis of uploading by arguing that it involves gambling on a posthuman form of existence. I’ve raised some questions about the arguments he uses to support both of these responses.

Anyway, I’ll leave it there for now. I’ll be doing some more posts on mind-uploading in the near future.

Is mind uploading existentially risky? (Part Two)

Tweet

(Part One)

This is the second in a series of posts looking at Searle's Wager and the rationality of mind-uploading. Searle's Wager is an argument that was originally developed by the philosopher Nicholas Agar. It claims that uploading one's mind to a computer (or equivalent substrate) cannot be rational because there is a risk that it might entail death. I covered the argument on this blog back in 2011. In this series, I'm looking at a debate between Nicholas Agar and Neil Levy about the merits of the argument. The current focus is on Levy's critique.

As we saw in part one, the Wager argument derives its force from the uncertainty about which of the following two philosophical theses is true: (i) weak AI, which holds that mind-uploading is not possible because a machine cannot be conscious or capable of human-type understanding; or (ii) strong AI, which holds that mind-uploading is possible. Agar submits that because there is some chance that weak AI is true, it cannot be rational to upload your mind since that could entail your self-destruction.

Or, at least, it cannot be rational from the perspective of our future descendants who may face the choice. This is because, by the time they face it, technology is likely to have advanced to the point where radical (biological) life extension is a reality and hence any decision to upload would require forgoing the benefits of radical biological life extension. In part one, we saw how Levy takes Agar to task for his inconsistency about the relevant timeframes in the argument. In today's post we are going to look at the larger issue of probabilities and philosophical risk. These play an important part in Agar's overall argumentative strategy, but Levy thinks Agar's use of probabilities is misleading: if we followed Agar's approach, we would never be able to rationally make any decision.

The remainder of this post is divided into three sections. First, I look at the topic of philosophical risk and the relationship between Searle's Wager and the classic Pascal's Wager. Second, I outline Levy's critique. Third, I offer some brief comments on this critique. I really will be brief in this final section since, in the next post, I will be dealing with Agar's response to Levy and Agar anticipates some of the points I would like to make.


1. Philosophical Risk and Pascal's Wager
I strategically de-emphasised the links between Searle's Wager and Pascal's Wager in part one. But there is no avoiding them now since certain parts of Levy's critique play upon the differences between the two. I apologise in advance to those of you who are familiar with Pascal's Wager and the extremely extensive literature about it.

Pascal's Wager is an argument about the rationality of believing in God. Originally developed by the French mathematician (and polymath) Blaise Pascal, it was supposed to show that anyone who wished to maximise expected utility (or minimise expected loss) should believe in God. This was because the expected reward that would come from doing so was so great -- in fact, infinitely great -- and the expected loss of not doing so was so extreme -- in fact, infinitely extreme -- that believing was clearly the rational thing to do, provided that the probability of God’s existence was greater than zero.

In broad outline, the Pascalian Wager had the following structure:

Pascal's Wager: God may or may not exist, and we each have a choice of whether or not to believe in him. The expected rewards/losses that come from doing so are:

A. Non-belief + God does exist: Punishment forever in hell (i.e. infinite punishment);

B. Non-belief + God does not exist: Some finite utility gain over the course of one lifetime due to forgoing the costs of believing.

C. Belief + God does exist: Reward forever in heaven (i.e. infinite reward).

D. Belief + God does not exist: Some finite utility loss over the course of one lifetime due to the costs of believing.

Now, there are many criticisms of this characterisation of the wager in the literature. For example, people argue that it fails to capture all the possible choices one has to make (which God, which specific doctrinal view etc.), or that it presumes too readily that believing is a voluntary action with certain definite consequences. But if we set these aside, the argument does make an interesting point about relative risks and the role they ought to play in decision-making. You see, Pascal's key observation was that the risks/rewards from believing/not-believing are asymmetrical. In particular, the risks from not-believing, when coupled with the potential gains, were so massively outweighed by the potential gains from believing that the decision to believe dominated the decision to not-believe, so long as God's existence had some non-zero probability attached to it.

Pascal's Wager contains probably the most extreme example of risk asymmetry you’ll ever see — that’s what makes it so potentially compelling. It is unlikely that you'll come across another argument that involves literally infinite gains being compared to infinite losses. Nevertheless, risk asymmetry arguments of this general type are common in the philosophical literature. The recently buoyant field of moral risk, for example, centres around several risk asymmetry arguments. Proponents of these arguments claim that they should lead us to alter our moral choices. For example, there are those that argue that the moral risks associated with aborting a foetus are sufficiently asymmetrical to the moral rewards (or, indeed, moral neutrality) of aborting a foetus to think that one ought to minimise moral risk by avoiding abortion.

Clearly, that's a controversial argument. But Searle's Wager involves something with a similar structure. In essence, the claim being made by Agar is that the risk associated with uploading in a world in which weak AI is true is sufficiently asymmetrical to the reward associated with uploading in a world in which strong AI is true, for not-uploading to be the rational choice. It is this alleged asymmetry, and its supposed affect on rational choice, that Neil Levy challenges in his critique.


2. Too Much Philosophical Doubt in the World?
In making his argument about the risk asymmetries involved in uploading, Agar doesn’t actually offer us any specific estimates of those risks. This is not surprising since any such estimates would be highly controversial. What Agar does do is to make some very general claims about the kinds of probabilities that would be required for his argument to be successful. When he does, Levy latches onto a particular quote that he thinks undermines the risk asymmetry argument. The quote is:

[I]f there is room for rational disagreement you should not treat the probability of Searle being correct as zero...This is all that the Wager requires 

(Agar, 2010, p. 71)

What Agar appears to be saying here is that risk asymmetry between the world in which weak AI is true and the world in which it is false, is so great that so long as there is a non-zero probability that we are living in that world, uploading will always be irrational. Why so? Because if we are living in that world, then there is a non-zero probability that uploading will entail our deaths.

Levy argues that this is much too strong a claim. A non-zero probability might make a difference in the case of Pascal’s Wager, where the risks and rewards are potentially infinite. But when we are dealing with finite gains and losses, this is not true. Consequently, Agar is working with a principle of rational choice that cannot be plausibly sustained. If it was the case that a non-zero probability of death always ruled out a particular choice, then we wouldn’t be able to perform any actions. After all, virtually any action you might like to think of carries a non-zero probability of death.

Levy’s critique can be interpreted as a simple reductio-style argument. As follows (numbering continues from part one):

• (8) Agar’s argument requires only that any action with non-zero probability of death attached to it be avoided.

• (9) If it were the case that any action that carried a non-zero probability of death were to be avoided, many actions (including actions that Agar thinks might be rational) would be ruled out.

• (10) It would be absurd to rule out all of these actions.

• (11) Therefore, the principle guiding Agar’s argument is absurd.

We’ve already seen how premise (8) is textually supported by the quote from Agar’s book. Thus, the other two premises (9) and (10) are the key to Levy’s critique. What can be said in favour of them?

Let’s start with premise (9). Levy gives several examples of the problems that flow from this principle. I’ll just focus on a couple here. The first is the rationality of using neuroprosthetic enhancements. These are artificial devices that are connected up with the rest of your biological brain and enhance some particular cognitive functions. Agar seems to endorse the use of such devices on the grounds that the brain is modular and hence replacing certain parts of it (NB: I don’t think Agar advocates complete replacement) with functionally enhanced analogues would be better than copying and uploading an emulation of the whole brain. But, of course, this is not quite true. As Levy points out, there is a non-zero chance that the modular hypothesis about brain function is wrong and that replacing individual parts of it will actually lead to the cessation of consciousness — i.e. your death. Levy argues that something similar is true of other putative forms of enhancement — e.g. genetic modification — and, indeed, a whole host of other mundane actions — e.g. crossing the road. Non-zero probabilities of death come pretty cheap and infiltrate many of our decisions.

(Levy also points out how Agar’s argument relies on other controversial philosophical claims that have a non-zero probability of falsehood. For example, claims about the badness of death, or about what it means for a person to cease existing.)

The point of all these examples is relatively clear: if we had to start ruling out all these other actions, particularly the mundane ones, we would never be able to act at all. That would be absurd. It follows then that Agar’s motivating assumption about a non-zero probability of death is itself absurd. Levy thinks that Agar must resolve this absurdity by making some more concrete claims about the costs and benefits associated with uploading.


3. Comments and Conclusions
What are we to make of all this? Funnily enough, when I read Agar’s argument a few years back I didn’t latch onto the quote about non-zero probabilities in quite the same way as Levy. I never thought that Agar’s argument relied on the absurd view that any philosophical risk is sufficient to render an action irrational — something I discussed in an earlier series on moral risk. I always thought that it relied on the view that risks above a certain threshold were sufficient to render an action irrational. Or, maybe, that the relative risk of uploading was much higher than that of not-uploading so that the latter was always going to be preferable to the former.

To put it another way, I never thought that premise (8) was a core commitment of Agar’s argument; I thought the motivating principle had something to do with risk thresholds or relative risks. It turns out that Agar agrees with this. After Levy penned his critique, Agar responded and in doing so he clarified the assumptions underlying the wager. We’ll look at what he had to say the next day.

Is mind uploading existentially risky? (Part One)

Tweet

A couple of years ago I wrote a series of posts about Nicholas Agar’s book Humanity’s End: Why we should reject radical enhancement. The book critiques the arguments of four pro-enhancement writers. One of the more interesting aspects of this critique was Agar’s treatment of mind-uploading. Many transhumanists are enamoured with the notion of mind-uploading, but Agar argued that mind-uploading would be irrational due to the non-zero risk that it would lead to your death. The argument for this was called Searle’s Wager, as it relied on ideas drawn from the work of John Searle.

This argument has been discussed online in the intervening years. But it has recently been drawn to my attention that Agar and Neil Levy debated the argument in the pages of the journal AI and Society back in 2011-12. Over the next few posts, I want to cover that debate. I start by looking at Neil Levy’s critique of Agar’s Searlian Wager argument.

The major thrust of this critique is that Searle’s Wager, like the Pascalian Wager upon which it is based, fails to present a serious case against the rationality of mind-uploading. This is not because mind-uploading would in fact be a rational thing to do — Levy remains agnostic about this issue — but because the principle of rational choice Agar uses to guide his argument fails to be properly action-guiding. In addition to this, Agar ignores considerations that affect the strength of his argument, and is inconsistent about certain other considerations.

Over this post and the next, I will look at Levy’s critique in some detail. I start out by returning to wager argument, and summarising its essential features. I then look at an important point Levy makes about the different timeframes involved in the wager. Before getting into all that, however, I want to briefly explain what I mean by the phrase “mind-uploading”, and highlight two different methods of uploading. Doing so, will help to head-off certain misconceptions about what Agar and Levy are saying.

Let’s start with the general concept of mind uploading. I understand this to be the process through which your mind (i.e. the mind in which your self/personality is instantiated) is “moved” from its current biological substrate (the brain) to some non-biological substrate (digital computer, artificial brain etc.). As I see it, there are two methods by which this could be achieved:

Copy-and-Transfer: Your mind/brain is copied and then transferred to some technological substrate, e.g. your brain is scanned and monitored over a period of time, and then a digital replica/emulation is created in a supercomputer. This may also involve the destruction of the original copy.

Gradual Replacement: The various parts of your mind/brain are gradually replaced by functionally equivalent, technological analogues, e.g. your frontal lobe and its biological neurons are replaced by a neuroprosthesis with silicon neurons.

Agar’s argument pertains to the first method of uploading, not the second. Indeed, as we shall see, part of the reason he thinks it would be irrational to “upload” is because we would have various neuroprosthetics available to us as well. More on this later.


1. Searle’s Wager
Since I have discussed Searle’s Wager at length before, what follows is a condensed summary, repeating some of the material in my earlier posts. That said, there are certain key differences between Levy’s interpretation of the argument and mine, and I’ll try to highlight those as I go along.

Wager arguments work from principles of rational choice, the most popular and relevant of which is the principle of expected utility maximisation. This holds that, since we typically act without full certainty of the outcomes of our actions, we should pick whichever option yields the highest probability-discounted payoff. For example, suppose I am purchasing two lottery scratch-cards in my local newsagents’ (something I would probably never do, but suppose I have to do it). Suppose they both cost the same amount of money. One of the cards gives me a one-in-ten chance of winning £100, and the other gives me a one-in-a-hundred chance of winning £10,000. Following the principle of maximising expected utility, I should pick the latter option: it yields an expected utility of £100, whereas the the former only yields an expected utility of £10.

You might actually find that result a bit odd. You might think the one-in-ten chance of £100 is, in some sense, a “safer bet”. This is probably because: (a) you are risk averse in decision-making; and (b) you discount higher values of money once they go beyond a certain threshold (i.e. for you, there are diminishing marginal returns of increasing the value of a bet beyond a certain sum). Deviations from expected utility of this sort are not unusual and Agar uses them to his advantage when defending his argument. I’ll get to that point later. In the meantime, I’ll draft a version of the argument that uses the expected utility principle.

The Searlian Wager argument asks us to look at the risks associated with the choice of whether or not to undergo mind-uploading. It says that this choice can take place in one of two possible worlds: (i) a world in which John Searle’s Chinese Room argument is successful and so, at best, Weak AI is true; and (ii) a world in which John Searle’s Chinese Room argument is false and so Strong AI might be true. I’m not going to explain what the Chinese Room argument is, or talk about the differences between Weak and Strong AI; I assume everyone reading this post is familiar with these ideas already. The bottom line is that if Weak AI is true, a technological substrate will not be able to instantiate a truly human mind; but if Strong AI is true, this might indeed be possible.

Obviously, the rational mind-uploader will need Strong AI to be true — otherwise mind-uploading will not be a way in which they can continue their existence in another medium. And, of course, they may have very good reasons for thinking it is true. Nevertheless, there is still a non-zero risk of Weak AI being true. And that is a problem. For if Weak AI is true, then uploading will lead to their self-destruction and no rational mind-uploader will want that. The question then is, given the uncertainty associated with the truth of Strong AI/Weak AI, what should the rational mind-uploader do? Agar argues that they should forgo mind-uploading because the expected utility of doing so is much higher than the expected utility of doing so. This gives us the Searlian Wager:

• (1) It is either the case that Strong AI is true (with probability p) or that Weak AI is true (with probability 1-p); and you can either choose to upload yourself to a computer (call this “U”) or not (call this “~U”).

• (2) If Strong AI is true, then either: (a) performing U results in us experiencing the benefits of continued existence with super enhanced abilities; or (b) performing ~U results in us experiencing the benefits of continued biological existence with whatever enhancements are available to us that do not require uploading.

• (3) If Weak AI is true, then either: (c) performing U results in us destroying ourselves; or (d) performing ~U results in us experiencing the benefits of continued biological existence with whatever enhancements are available to us that do not require uploading.

• (4) Therefore, the expected payoff of uploading ourselves (Eu(U)) is = (p)(a) + (1-p)(c); and the expected payoff of not uploading ourselves (Eu(~U) is = (p)(b) + (1-p)(d).

• (5) We ought to do whatever yields the highest expected payoff.

• (6) Eu(~U) > Eu(U)

• (7) Therefore, we ought not to upload ourselves.

The logical structure of the decision underlying the argument can also be depicted in a decision tree diagram. In this diagram, the first decision node is taken up by N, which represents nature. Nature determines the probabilities that govern the rest of the decision. The second decision node represents the choices facing the rational mind-uploader. The four endpoints represent the possible outcomes for that decision-maker.

The key to the Searlian Wager argument is premise (6). What reasons does Agar give us for thinking that the expected utility of not-uploading are lower than the expected utility of uploading? We got an initial sense of the argument from the preceding discussion — if you upload, you risk self-destruction — but as it turns out, Agar oscillates between two distinct timeframes when he tries to flesh out this argument. Levy takes him to task for it.


2. The Timeframe Problem
Currently, human beings have a problem: we all die. From our perspective, putative mind-uploading technologies might be worth a gamble. If we are going to die anyway, why not have our minds copied and transferred while we are it. We have nothing much to lose and everything to gain. Right?

Right, but as Agar correctly observes, none of us currently faces Searle’s Wager. After all, the technology for putative mind-reading does not exist. So the only people who will face it are future persons (possibly future versions of ourselves). Consequently, when analysing the risks and benefits associated with the choice, we have to look at it from their perspective. That, unfortunately, necessarily involves some speculation about the likely state of other sorts of technologies at that time.

And Agar does not shy away from such speculation. Indeed, I hinted at the sort of speculation in which he engages in my formulation of the argument in the previous section. Look back at the wording of premises (2) and (3). When describing the benefits of not-uploading, both premises talk about the “benefits of continued biological existence with whatever enhancements are available to us that do not require uploading.” This is an important point. Assuming we continue to invest in a stream of enhancement technologies, we can expect that, by the time putative mind-uploading technologies become available, a large swathe of other, biologically-based, radical enhancement technologies will be available. Among those, could be technologies for radical life extension — maybe something along the lines of Aubrey de Grey’s longevity escape velocity will become a reality — which will allow us to continue our existence in our more traditional biological form for an indefinite period of time. Wouldn’t availing of those technologies be a safer and more attractive bet than undergoing mind-uploading? Agar seems to think so. (He also holds that it is more likely that we get these kinds of technologies before we get putative mind-uploading technologies.)

But how can Agar be sure that forgoing mind-uploading will look to be the safer bet? The benefits of mind-uploading might be enormous. Ray Kurweil certainly seems to think so: he says it will allow us to realise a totally new, hyperenhanced form of existence, much greater than we could possibly realise in our traditional biological substrate. These benefits might completely outweigh the existential risks.

Agar responds to this by appealing to the psychological quirk I mentioned earlier on. It can be rational to favour a low-risk, smaller benefit, gamble over a high-risk high benefit gamble (if we modify the expected utility principle to take account of risk-aversion). For example, (this is from Levy, not Agar) suppose you had to choose between: (a) a 100% chance of £1,000,000; and (b) a 1% chance of £150,000,000. I’m sure many people would opt for the first gamble over the second, even though the latter has a higher expected utility. Why? Because £1,000,000 will seem pretty good to most people; the extra value that could come from £150,000,000 wouldn’t seem to be worth the risk. Maybe the choice between uploading and continued biological existence is much like the choice between these two gambles?

This is where things break down for Levy. The attractiveness of those gambles will depend a lot on your starting point (your “anchoring point”). For most ordinary people, the guarantee of £1,000,000 will indeed be quite attractive: they don’t have a lot of money and that extra £1,000,000 will allow them to access a whole range of goods presently denied to them. But what if you are already a multi-millionaire? Given your current wealth and lifestyle an extra £1,000,000 might not be worth much, but an extra £150,000,000 might really make a difference. The starting point changes everything. And so this is the problem for Agar: from our current perspective, the benefits of biological enhancement (e.g. life extension) might seem good enough, but from the perspective of our radically enhanced future descendants, things might seem rather different. The extra benefits of the non-biological substrate might really be worth it.

In essence, the problem is that in defending a suitably-revised form of premise (6), Agar oscillates between two timeframes. When focusing on the benefits of mind-uploading, he looks at it from our current perspective: the non-radically enhanced, death-engendering human beings that we are right now. But when looking at the risks of mind-uploading, he switches to the perspective of our future descendants: the already radically-enhanced, death-defeating human beings that we might become. He cannot have it both ways. He must hold the timeframes constant when assessing the relevant risks and rewards. If he does so, it may turn out that mind-uploading is more rational than it first appears.

Okay, so that’s the first of Levy’s several critiques. He adds to this the observation that, from our current perspective, the real choice is between investing in the development of mind-uploading technologies or not, and argues that there is no reason to think such investment is irrational, particularly if its gains might be so beneficial to us in the future. I’ll skip over that argument here though. In the next post, I’ll move onto the more substantive aspect of Levy’s critique. This has to do with the impact of non-zero probabilities on rational choice.