Where are the differential effects?

One way to test if the changes in crime rates were caused by the law or by other factors is to look for differential changes in crime. For example, it does not seem likely that the carry law would affect crimes that occur in the home of the victim, since it was already legal to keep a gun in the home for defence before the law.

Effect on robbery should be more than on rape

Lott notes that a Texas poll suggests that 97% of first-time applicants for concealed-weapon permits already owned a handgun (page 28), so the carry law did not result in an increase in gun availability at home. Because the law only affected the carrying of guns in public places we would only expect them to affect crimes carried out in public places.

NCVS statistics indicate 36% of robberies and 10% of rape/sexual assaults occurred ``on street other than near home'' [37]. If the carry law caused a 20% decline in crimes in public places, you would expect a 7% decline in robbery and a 2% decline in rapes. The effect of the law might be something other than a 20% decline, but in general we should expect the effect on robbery to be about 3 1/2 times as great as that on rape. Instead, Lott found a 5% decline in rape and a 2% decline in robbery.

In response to a similar criticism made by Webster [45] Lott (page 133) offers two arguments:

First, that the effect on the slope was larger for robbery, that the robbery rate was increasing before the law and decreasing afterwards. However, this is undercut by Table 4.13 which shows the result of adding data for 1993 and 1994. This causes the change in robbery rates associated with the law to go from negative to positive. This suggests that robbery rates went back up after 1992.

The robbery rate was changing in a way that was not explained by Lott's model. Since we don't know what was making robbery increase, we have no way of knowing when the increasing trend would end (certainly it couldn't go on increasing forever). Assuming that robbery would have continued to increase for the whole period of the study without the carry law seems a little unwarranted.

There is also an element here of shifting the goal posts. If you can find a decrease in the rate, report that. If that doesn't work, look at the trends.

Second, that some robberies were not street crimes and the laws could cause an increase in other robberies by making them relatively more attractive. This argument seems to miss the point. The carry law could equally well cause an increase by substitution in non-street rapes. Lott offers no evidence at all that this supposed effect was different for rapes than for robberies.

Was there substitution from violent crime to property crime?

Lott found that the laws were associated with an increase in property crime, most noticeably with a 7.1% increase in auto theft. Lott argues that this change occurred because
Criminals respond to the threat of being shot while committing such crimes as robbery by choosing to commit less risky crimes that involve minimal contact with the victim.
Unfortunately for this argument, the law was not associated with a significant decrease in robberies. In fact, when data for 1993 and 1994 was included, it was associated with a small (not statistically significant) increase in robberies.

The law was associated with a significant reduction in assaults, but there does not seem to be any reason why criminals might substitute auto theft for assault.

In the second edition of Lott's book table 9.1 shows the results of his latest analysis, using data up to 1996. This table shows that the effect on violent crime (-2.3%) is very similar to the effect on property crime (-1.6substitution effect, but, since it is not plausible that a carry law could cause a greater reduction in auto theft than in murder, there is evidence that both reductions were caused by some other factor. Rather than point this difficulty out to his readers, Lott quietly drops further discussion of the substitution effect.

Can different effects in different states be explained by the number of permits issued?

Table 4.9 shows the effects in each state that adopted a carry law. It is clear that there there are some notable differences in the effects, even though each state adopted a similar law. Only murder and rape show a consistent pattern of reduction. The other crime categories show a fairly even split between states showing an increase and states showing a decrease. It does not seem reasonable to argue that the carry law caused any change, except for the case of murder and rape.

In response to a similar argument made by Black and Nagin [5], Lott writes (page 143):

The difference that did exist across states can be explained by differences in the rate at which handgun permits were issued.

Lott has data on handgun permits for three states (table 4.7). This shows that the percentage of the population with permits in 1994 was 1% in Florida, 1.4% in Oregon, and 4% in Pennsylvania. However, this is exactly the opposite ordering of the changes in the violent crime rate: -4% in Florida, -3% in Oregon, and -1% in Pennsylvania. If differences in the rate at which handgun permits were issued explain this, it can only be if more handgun permits cause more violent crime.

Black and Nagin [5] also observe that the effects on murder and rape depend on the inclusion of Florida in data set--if Florida is excluded, the effect on murder changes from a 9% decrease to a 1% decrease, while the effect on rape changes from 3% decrease to 1% increase.

Lott offers several objections to this argument:

Firstly, he objects to Black and Nagin conducting their analysis using only counties with populations of more than 100,000. This is a strange objection to make, since Lott states that restricting the sample in this way makes the effects of shall-issue law stronger and more significant. In any case, it makes no difference whether the sample is restricted in this way since the effects on murder and rape also vanish when Florida is excluded from a sample containing all counties.

Secondly, Lott reports that when he reran all the regressions in the book without Florida, in only eight out of one thousand did the result change from significant to not significant. This goes some way towards alleviating the concerns about the influence of Florida on the results, but the fact remains that it did make a large difference to the most important regressions.

Thirdly, Lott suggests that Black and Nagin conducted a search for a specification that weakened the results and that traditional statistical tests of significance are based on the assumption that the most favourable (or unfavourable) one out of a number of tests has been chosen. However, it well known that all that is necessary to do in such a situation, is to adjust the level of significance accordingly, and Black and Nagin did so.

Effect on men should be more than on women

Only 20% of permits were issued to women, but the male and female homicides rate went down by the same amount and the reduction in the rape rate was similar to the decrease in assaults. Lott speculates that guns are four times as effective for females. While this is not impossible it seems more likely that the decreases were caused by some other factor that applied to males and females equally. Certainly it seems to be a bit much of a coincidence that the supposed extra effectiveness for women just happened to be exactly equal to ratio between the numbers of male and female permit holders.

Effect on strangers should be more than on family members

As mentioned before, we would expect carry laws to mainly affect crimes committed in public places. Since crimes committed by strangers are more likely to occur in public places than crimes committed by family members, you would expect a greater effect on crimes committed by strangers than on crimes committed by family members. Table 5.2 however, shows just the opposite--the percentage of homicides by strangers went up, while the percentage of homicides by family went down. While these changes were not large enough to be statistically significant, even if there was no change to these percentages, this would still be a failure to find a differential effect, and evidence that the changes in crime rates were caused by something other than the shall issue laws.

In response to a similar argument made by Alschuler [1], Lott argues (page 148) that concealed handguns could defend against estranged family members. While this is possible, it misses the point of the argument, which is not that carry laws would have no effect on in-family homicides, but that the effect would be less than that on stranger homicides.

Effect on adults should be more than on juveniles

Even more surprising is the fact that the carry laws apparently were just as effective in reducing juvenile homicides as they were in reducing adult homicides, even though juveniles were not eligible for permits.

Ludwig [33] used juvenile homicide rates to control for unobserved variables that may vary over time and found that, if anything, the carry laws resulted in an increase in adult homicide rates.

Lott offers two arguments in response to this criticism (page 147).

First, that ``criminals may well tend to leave an area where law-abiding adults carry concealed handguns''. This is contradicted by his own results: a substitution into property crime, and no displacement of violent crime to nearby areas without carry laws.

Second, that gun-carrying adults may be able to protect some youngsters. However, even we make the extremely generous estimate that this happens half the time, the effect of the law on the juvenile homicide rate would only be half that of the adult rate. The effect, as seen above was the same. A more realistic estimate, combined with the substitution effect mentioned on the previous, suggests that there should have been no effect, or a small increase in juvenile homicides.

Were crimes displaced to adjacent counties without carry laws?

Lott claims to have found a differential effect by looking at counties without carry laws that were near counties that adopted such laws. If the carry law is deterring crime in the counties with carry laws, you might expect some criminals to move their activities to nearby counties without carry laws and increase the crime rate there.

His table 4.14 clearly shows that the violent crime in such counties did not change at all when the carry law was passed. However, because three of the four categories of violent crime showed an increase, and one a decrease, Lott argues that this shows that there was a spillover effect.

Did crime reductions exactly coincide with the laws?

On page 131 Lott writes
To illustrate that the results are not merely due to the ``normal'' ups and downs for crime, we can look again at the diagrams in chapter 4 showing crime patterns before and after the adoption of the non-discretionary laws. The declines not only begin right when the concealed handgun laws pass, but the crime rates end up well below their levels prior to the law. Even if laws to combat crime are passed when crime is rising, why would one believe that they happened to be passed right at the peak of any crime cycle?
(You can see an example of one of the diagrams here). This is wrong. Lott's diagrams do not show crime rates at all, but rather plot two quadratic curves that he fitted to the data. It is no surprise that there is a peak when the laws were passed--this is one of the few places where it is possible for the fitted curves to peak. Even if the crime rate started to decline before the laws passed, Lott's diagram could still show a peak coinciding with the law.

I ran some experiments by fitting a similar pair of quadratic curves to a sequence of random numbers. Almost always the curves seemed to show that something had happened at the junction of the two curves, even though nothing had.

If your browser supports Java, the applet below lets you repeat my experiments. Each time you click on the ``Randomize'' button, a curve is fitted to a new set of random numbers. You can also use the left mouse button to move points to new positions to see how the fitted curve changes.

Were crime reductions greatest in counties which issued more permits?

Lott found that crime reductions tended to be greater in high population counties and argued that these were the counties that issued the most permits.

There are two problems here: firstly, he is using a counties' population as a proxy for the number of permits issued without validating that proxy. This is puzzling, since he has county level data for the numbers of permits issued for two states. It would be simple to check to see if relatively more permits were issued in high population counties.

Secondly, high population centres show greater variations in crime rates. The following table shows the standard deviation of the log of the homicide rate for US cities of different sizes over a 20 year period.

City size s.d. of log homicide rate
One million and over 0.15
500k-999k 0.12
250k-499k 0.15
100k-249k 0.10
It follows that when crime goes down it decreases more in places with higher populations. The next table shows the crime trends in cities of each size for the periods before and after 1991 (when the homicide rate peaked).
City size annual change annual change
  in homicide rate in homicide rate
  1976-1991 1991-1997
One million and over 0.5 -1.7
500k-999k 0.3 -0.8
250k-499k 0.2 0.4
100k-249k 0.2 -0.2
That is, if the reduction in crime rates was just the product of the normal increases and decreases in crime it would also follow the same pattern of being greatest in high population centres.

What about multiple victim public shootings?

In his book and in a paper co-authored with Landes [30] Lott argues that carry laws caused an 89% decrease in murders in multiple victim public shootings. For reasons explained earlier we would expect a greater effect of a carry law on crimes that occur in public places.

Lott and Landes also offer another argument why we might expect a greater effect on multiple victim public shootings: since there are more people present in such shootings, the chance of a criminal encountering a permit holder is greater than that for a single-victim crime. However, this argument is flawed, since all the costs of crimes are higher for multiple victim crimes--the criminal is more likely to be caught and convicted, the penalty will be higher, and they are more likely to encounter resistance from non-permit-holders. To put it in economic terms, what is important is not the absolute increase in cost of a crime, but the relative cost. There is no good reason to expect an increase of $2 in the price of something costing $20 to have a greater effect on demand then an increase of $1 in the price of something costing $10.

Furthermore, there is an upper limit to the cost of these crimes. Since the criminal dies in many of these crimes it is hard to see how the cost can increase from this.

For these reasons it is unclear whether you would expect to see a greater deterrent effect on multiple victim public shootings than on other crimes.

As to the question of whether ``more guns'' cause ``less multiple victim public shootings'', there are similar problems to the question as to whether ``more guns'' cause ``less crime''.

Firstly, there weren't significantly more guns.

Secondly, there are doubts about whether there were ``less multiple victim public shootings''. Although there was an 89% reduction when comparing before and after rates in states that introduced shall issue laws, (table 2 of [30]) shows that the average murder and injury rate from multiple victim shootings was 0.042 in states without such laws and 0.029 in states with shall issue laws (just 31% lower). The difference between 89% and 31% is enormous and suggests that something is wrong--the only way that the reduction could really have been 89% is if there was some other mysterious factor operating that would have caused the rate of mass public shootings in shall-issue states to have been much higher than in the other states, were it not for the shall-issue law. Lott and Landes do not identify such a factor.

Also, table 3.2 of the book indicates that the overall murder rate was 9.5 in states without such laws and 5.1 in states with shall issue laws (46% lower). That is, multiple victim shootings were actually relatively more common in states with shall issue laws.

Furthermore, it is absurd to suppose that the carry law could cause a decrease as large 89% in multiple victim public shootings. It is possible that some perpetrators might be deterred, but since many of them die from police weapons or their own weapons it is surely less than 89%. Indeed it is unlikely that as much as 89% of adults are even aware of the carry laws.

Lott and Landes found that that neither the frequency nor the severity of punishment had an effect on public shootings, suggesting that these crimes are not easy to deter. Lott believes that carry laws work to reduce public shootings not by increasing the cost of the crime, but by decreasing the value of the crime to the criminal. Lott writes [27]:

What motivates most of these criminals seems to be the desire for publicity. They want to kill as many people as possible. The possible presence of concealed weapons can limit the carnage, and thus the incentive to begin the attack.
However, the average number of deaths per incident in public shootings in states with shall issue laws was 1.7 (my analysis of table 1 of [30]), almost the same as the 1.8 deaths per incident in states without shall issue laws. This very small difference does not seem like anywhere enough to make such shootings worthless to the perpetrators.

Also, out of the hundreds of cases studied, Lott and Landes fail to present a single case where a concealed handgun was used to limit the carnage in a mass public shooting. The closest they come is the description of two cases where a civilian gun was used for defense, one involving a shotgun and one involving a handgun retrieved from a car.

Lott and Landes present an analysis in table 10 that seems to indicate that the shall-issue law reduced the number of deaths per incident by 2.2, a figure much greater than the number of deaths per incident in states without shall-issue laws. This once again suggests that something is wrong with the model.

Tim Lambert