September 10, 2004

The Elasticity of the Vote for Dummies

Hausmann and Rigobon's argument about the non-randomness of the CNE cold-audit sample is based on something they've called the "elasticity of the vote." The concept is borrowed from economics, where price elasticity is a bedrock conceptual tool. This is my academic turf, so I'll have a stab at explaining it to non-economists.

According to a standard economics textbook, price elasticity of demand is "a measure of the responsiveness of the quantity demanded of a good to changes in price." (Stiglitz and Boadway, 1994).

Very good, but what the hell does it mean?

A caffeinated illustration
Say a cup of coffee in your town costs $1. At that price, you buy 100 cups of coffee a year. Now say coffee prices go down to 75 cents a cup. At that price, how many cups of coffee will you buy per year? Logically, more than 100. But how many more?

Well, maybe you're very sensitive to price changes, and you'll buy 200 cups of coffee a year: your demand is relatively elastic - since a 25% drop in price led to a 100% rise in consumption. Or maybe you're not so concerned about price changes and you'll buy just 105 cups a year - then your demand is relatively inelastic.

Of course, economists don't care how much coffee you personally drink, they care how much coffee a whole bunch of people drink. So the question is not how your demand for coffee will react to a change in price, but how overall demand for coffee will react to a change in price.

Say you do a study and you find that when the cost of coffee goes down 25 cents, on average people will buy 20 cups of coffee more per year than before. The ratio of the coffees-people-used-to-drink to coffees-people-drink-now now is 100:120.

Now, say you take a random sample from that population. How many more cups of coffee per year would you expect the sample to buy?

Well, if the sample is truly random, you would expect it to react in the same way as the overall population - the sample ratio should be 100:120 as well.

But say your sample doesn't behave that way. Say the people in your sample are now buying 130, or 140 cups of coffee a year. What can you conclude then?

Well, at the very least you can say the coffee drinkers in your sample behave differently from the overall population of coffee drinkers. Your sample acts as though they are more sensitive to coffee price changes than the people in the overall population - in economist-talk, their demand is relatively more elastic than the demand of the rest of the population.

So something screwy is going the very least, you have reason to suspect that your sample was not really randomly selected from the total population of coffee drinkers.

The key thing to remember is that statistical methods allow you to estimate the exact probability that your sample, which seems at first glance to be non-random, actually is random - but just happened to give you a screwy-seeming result by chance.

Applying this idea to the cold-audit data
Now, Hausmann and Rigobon's study of the cold audit relies on an adaptation of the same train of thought. They have three data-sets: one on the November 2003 signature gathering drive, one with CNE's total referendum results, and one on the referendum results in the sample selected for the cold audit.

All they're doing is comparing two ratios: the ratio of 2003 signatures to Si-votes in the overall CNE results, and the ratio of 2003 signatures to Si-votes in the audited sample. If the audit sample truly was random, the two ratios should be reasonably close to one another.

They're not.

Say you determine that, for the overall population of voters, every 100 signatures obtained in November 2003 yielded 120 Si-votes. The ratio for the entire population is 100:120. Now cold-audit day comes around, and CNE chooses a supposedly random sample of voting centers. But when you compare the signatures-to-Si-votes ratio in the audited voting centers, you realize that it's different from the signatures-to-Si-votes ratio in the overall population: say 100:140 instead of 100:120.

[Note: these are not the actual numbers - H&R's analysis was rather more sophisticated, including a correction term to account for the growth of the voting population and a log scale transformation - so the numbers I'm using are just meant to illustrate the point.]

What can you conclude from that difference in the ratios?

The exact same thing you could conclude in the case of the coffee drinkers...and for the exact same reason!

The relationship between willingness-to-sign-in-2003 and willingness-to-vote-Si-in-2004 should be the same for a random sample than for the overall population. Yet, for some not-yet-explained reason, each Nov. 2003 signature yields more Si-votes in the audited center than in the non-audited ones.

The question, then, is what are the chances that a gap that big between the two ratios is the product of pure chance? Luckily, as we've seen, statistical methods allow you to estimate this probability quite precisely. In this case, Hausmann and Rigobon peg the chances of these results happening by coincidence at less than 1%.

That means that if you pick a sample at random, more than 99 times out of 100 you'll end up with a ratio less odd than the one CNE happened to get.

Ergo, there's it's more than 99% likely that the audit sample wasn't really random.

Does that make sense?

Join a moderated debate on this post.

The Zenith of the Egghead Brigade

Thanks to Miguel Octavio for linking to this astonishing New York Times article on Benford's law.

Dr. Theodore P. Hill asks his mathematics students at the Georgia Institute of Technology to go home and either flip a coin 200 times and record the results, or merely pretend to flip a coin and fake 200 results. The following day he runs his eye over the homework data, and to the students' amazement, he easily fingers nearly all those who faked their tosses.

"The truth is," he said in an interview, "most people don't know the real odds of such an exercise, so they can't fake data convincingly."


Seems to me like Miguel is really enjoying this moment of maximum relevance and exposure for the Egghead Brigade. Good! There's a kind of poetic rightness to it - the fraud hypothesis will be made or sunk, in the end, not by some rambling CD fourth republic dinosaur, but by a scientist, by some mild mannered academic who's spent the last 30 years working quietly, diligently behind a computer screen. Chavez is powerful. Science, much more so.

Join a moderated debate on this post.

September 8, 2004

Pre-emptive Strike: Anticipating the Counterarguments to Hausmann and Rigobon

I've been reading and re-reading Hausmann and Rigobon's piece to try to find holes in it. I'm finding it hard. The most obvious counterargument doesn't seem to work...and the conspiracy theories needed to explain them seem particularly far-fetched.

Obvious Counterargument 1: JIJO - Junk In, Junk Out.
Chavistas have held that the opposition run exit polls were very badly conducted, and that it's a mistake to take them seriously. The first part of the Hausmann/Rigobon analysis is based partly on exit-poll results. But as everyone knows, if you start with junk data, you end up with junk results. JIJO.

You have to work through the logic of Hausmann and Rigobon's argument to see off this challenge. What their paper argues is empathically not that the exit poll results do not match the election results, but rather that the error given by the exit polls is positively correlated with the error given by the Nov. 2003 signature collection drive. As these two estimators are independent of one another - their sources of error are unrelated - the only way to account for positive correlation in their errors is fraud.

What the hell does all of that mean?

Well, start with the Nov. 2003 Signature count. In some sense, the signatures are predictors of the number of SI votes in a subsequent election. If, for instance, you collected 1000 signatures in a signing center in November, you would expect a similar number of Si votes in that voting center this year. The correlation will not be perfect, but it's something. You would not expect to get 200 SI votes in such a center, and neither would you expect 2000.

Then take the exit-polls. Again, you would expect them to be correlated - imperfectly - with the ultimate election results. There will be error, as there always is with any estimator, but the poll gives you an idea of voting intentions.

Now, what Hausmann and Rigobon have done is take these two estimators and analyze them together. And what they find is quite interesting: in some voting centers, the two estimators work quite well. But in other centers, the estimators are wrong - they predict more SI votes than CNE eventually reported. What's more, in a given center, when one estimator is wrong, the other one also tends to get that center wrong - and both in the same direction, of overestimating the number of SI votes.

Or, in their words, "in those places where the signatures are proportionally more wrong in predicting more Si votes than those obtained, the exit polls do the same."

So, say CNE reports only 800 SI votes in our hypothetical voting center that had 1000 signatures back in November. Well, ok, that might be possible - maybe 200 voters there decided to become chavistas. But imagine that the Sumate or PJ exit poll also estimates 1000 votes for that center - or 950, or 1100. Well, then you have a strange situation: it's not immediately clear why both estimators would make the same mistake in the same direction. "Because both measures are independent," they explain "what they have in common is the fraud. "

Of course, if this had happened in just one or two voting centers, you could put it down as a fluke. But Hausmann and Rigobon find it happening systematically across a whole swathe of voting centers.

They peg the chances of this pattern stemming from a random coincidence at less than 1%.

Now, what this means is that if the Sumate and PJ exit polls are wrong, they're wrong in a really funny and strange way. They're more wrong for some voting centers than for others. You could say that Sumate and PJ just happened to have some competent exit pollsters and some incompetent ones, so it makes sense that some got the results more wrong than others. But what you can't explain with that argument is why those Incompetent Pollsters ended up getting sent specifically to the voting centers where the number of Si votes also happened to be lower than one would expect from an analysis of the November 2003 signature patterns.

I can't see any easy way for CNE to explain this situation - other than asserting a wildly implausible conspiracy stretching all the way back to November 2003 where the opposition would seek to selectively manipulate both the number of signatures and the exit poll results for some voting centers, but not others, so that many months later two fancy US-based professors could keep the reasonable doubt hypothesis going. Now THAT'S UFO stuff!

So I can't see how JIJO applies to this analysis: what's at stake here is neither whether the Nov. 2003 signature collection was clean and valid nor whether the exit polls were well conducted. The mystery in need of explanation is why both of these estimators would be more wrong in some voting centers than in others, why both should coincidentally be more wrong in the same set of voting centers, and why they should both be wrong in the same direction - of undercounting the number of SI votes.

Hausmann and Rigobon can think of one neat, parsimonious little hypothesis to explain all of these questions: fraud.

If you can come up with a better one, do let us know...

Note: As statistics is not my forte, it's perfectly possible I've screwed something up in this post. Don't be shy to let me know if I did, so I can correct it:

Join a moderated debate on this post.

September 7, 2004

Hausmann and Rigobon: The incomprehensible, the evident and the damning

Hausmann and Rigobon's paper on the evidence of fraud in the Recall Referendum makes three basic arguments. Instant reaction.

1.The Incomprehensible.
Hausmann and Rigobon claim that a sophisticated statistical method comparing exit poll data and Nov. 2003 signature totals with CNE results generates evidence of fraud.

They set out to "develop a statistical technique to identify whether there are signs of fraud in the data." This is how they describe the method they used:

"We depart from all previous work on the subject which was based on finding patterns in the voting numbers. Instead, we look for two independent variables that are imperfect correlates of the intention of voters. Fraud is nothing other than a deviation between the voters' intention and the actual count. Since each variable used is correlated with the intention, but not with the fraud we can develop a test as to whether fraud is present. In other words, each of our two independent measures of the intention to vote predicts the actual number of votes imperfectly. If there is no fraud, the errors these two measures generate would not be correlated, as they each would make mistakes for different reasons. However, if there is fraud, the variables would make larger mistakes where the fraud was bigger and hence the errors would be positively correlated. The paper shows that these errors are highly correlated and that the probability that this is pure chance is again less than 1 percent."

The statistical methods employed to come to this conclusion are sophisticated beyond my ability to analyze. I imagine it would be possible to argue that Sumate and Primero Justicia's exit polls were so flawed, that what we have here is a classic case of JIJO: Junk In, Junk Out. But since I couldn't understand the analysis, I can't really judge it - I have to take Hausmann and Rigobon at their word.

2-The Evident
The analysis discards the Si-cap hypothesis.

This had always seemed to me as a weak argument, partly because the evidence was ambiguous, but mostly because it made no sense to me that CNE would plan out a big, complex, sophisticated fraud operation and then institute a mechanism that would leave such an obvious footprint in the data. I never thought it added up, and neither do Hausmann and Rigobon.

3-The Damning
The study provides statistical evidence that the sample chosen by CNE for the cold-audit was demonstrably not random.

This is the most eye-catching conclusion to the study, and of special concern to me, since I'd consistently held the view that the cold-audit was the strongest card in CNE's hand. If the audit sample was not randomly selected, and the non-randomness fits in neatly with the rest of the fraud hypothesis, then it's the first solid evidence of fraud I've seen yet.

Let me quote again from Hausmann and Rigabon's summary of their argument:

"First of all, the Carter audit, conducted on August 18, was based on a sample of about 1 percent of the ballot boxes. According to the Carter Center, the audit went flawlessly, except for the fact that the Head of the Electoral Council was not willing to use the random number generator suggested by the Carter Center but use instead its own program run on its own computer. At the time, this seemed odd, given that the objective of the audit was to dispel doubts.

"The paper finds that the sample used for the audit was not randomly chosen. In that sample, the relationship between the votes obtained by the opposition on August 15 and the signatures gathered requesting the Referendum in November 2003 was 10 percent higher than in the rest of the boxes. We calculate the probability of this taking place by pure chance at less than 1 percent. In fact, they create 1000 samples of non-audited centers to prove this.

"This result opens the possibility that the fraud was committed only in a subset of the 4580 automated centers, say 3000, and that the audit was successful because it directed the search to the 1580 unaltered centers. That is why it was so important not to use the Carter Center number generator. If this was the case, Carter could never have figured it out."

If this is right - and I have no reason to doubt it, at this point - this would dismantle the main reason to think there was no fraud: namely, that there was no way for CNE to tamper with the paper-ballots after the sample-selection but before international observers got there. There was no way to do it, as I'd thought...but there was also no need to do it: they already knew that the boxes audited would come from "clean" tables!

I've always taken the view here that I don't know if there was fraud or not, cuz I'm not omniscient. What's hard, as a commited antichavista, is to try to come to a judgment on this matter based on evidence rather than ideology or wishful thinking. The Haussman and Rigobon paper is significant precisely because it gives us the first bit of evidence that something was not right with the cold-audit. Having taken the cold-audit as the primary reason to think the election was fair, the paper obviously requires that I go back and reconsider.

One last consideration: if Hausmann and Rigobon are right, then there was no reason for CNE to go back and alter the paper ballots. If CNE did not tamper with the paper ballots, then it follows logically that even now a recount of paper ballots could detect fraud. For many reasons (the security codes printed on each paper ballot by the smartmatic machines, the fact that thermal paper fades over time, the daunting logistics involved, etc.) it would be very hard for CNE to go back now and tamper with the ballots. So even at this late date, it strikes me that the truth is certainly out there: sitting in those ballot boxes, slowly decaying into unreadability.

As a final note: as always, I'm eager to hear a reasoned explanation of how and why it is that Hausmann and Rigobon's analysis is wrong. I am much less eager, however, to hear ad hominem attacks against them, or blithe dismissals of anyone who would spend time inquiring into the fraud hypothesis.

Join a moderated debate on this post.

Puro cuarto bate

Ricardo Hausmann and Roberto Rigobon must be the two most respected and accomplished Venezuelan academics out there. You don't get teaching posts at Harvard and MIT for tirar piedras. Yesterday, I got my hands on their study of the fraud hypothesis and, as you'd expect, it makes the most comprehensive and serious case yet for the fraud hypothesis.

I'm not through with the report, and I don't want to comment at length until I've read it all, but one thing I can tell you for sure: there's no dismissing it as "UFO stuff." There are serious charges in here, and they demand serious answers from CNE and CC/OAS. In particular, their assertion that there is statistical evidence that the sample taken by CNE for the cold-audit was not random raises very troubling questions for the authorities.

I recommend reading it carefully.

You can download the paper (pdf format, 1.1 mb) by Hausmann and Rigobon here.

Join a moderated debate on this post.

September 6, 2004

Alemo, serious planning, and where the hell do you get $60 billion?

One of the most interesting articles I got to write when I lived in Venezuela grew from a meeting I had with five privileged white men who had held positions of some prestige under the old regime. They harshly condemned Chavez's performance in office and called for policy to take a radical new direction. They threw their hands up in frustration as they considered the government's folly, shook their heads gravely as they described their exclusion from the spheres of power, and generally despaired over their powerlessness to influence policy in the slightest.

The group's name is Alemo. It was founded in the early 1980s as a think-tank on Venezuela's urban housing crisis. Staffed by professional architects and urban planners - many are professors at the Universidad Central de Venezuela - the group had spent two decades in in-depth study of the urban planning challenge posed by Venezuela's mushrooming shantytowns, or barrios. With a bureaucrat's fastidiousness for detail, they had documented the scale of the problem, thought through the policy alternatives available to the state to respond to it, costed the various possibilities, and published the lot. In their estimation, the rock-bottom, lowest-imaginable price-tag for bringing all of Venezuela's urban dwellers to a basic standard of safety and comfort would come to...wait for it...$60 billion.

The price-tag is astronomical, ludicrous. It's over 60% of GDP, over twice the government's global yearly spending. Even if, as Alemo urges, the cost is spread over 10 years, a $6 billion/year price tag would require quadrupling the country's housing budget - well in excess of the state's ability to pay.

It's hardly a surprise, since Alemo's study points to a desastrous housing shortage and the need for a major project of infrastructure building to redress it. Just to give you a sense of how dire the housing crisis is, consider this: one-third of the country's housing stock consists of ranchos - shanties. However, fully half of all Venezuelans live in ranchos. Thousands of ranchos need to be bulldozed and replaced - they sit on geologically unstable ground, and could give way at the next rainstorm - while hundreds of thousands require investment to bring them up to minimal standards of safety and crowding. Moreover, hundreds of barrios need major investments to bring in basic urban amenities - from electricity and sanitation to schools, infermeries, playing fields and roads good enough for buses to use.

How did Alemo come to this $60 billion figure? The process was long and involved, but the innitial question was simple enough: what would it take to build a Venezuela where everyone who lives in a city has access to basic urban services, nobody lives at very high-risk areas for flooding or mudslides, nobody lives in extremely crowded or dangerously deteriorated housing, and the housing stock grows quickly enough to absorb the new families looking for places of their own as they hit their 20s?

To keep costs more or less reasonable, Alemo applied deep cost-cutting measures in their calculation, including a radical plan for the state to construct "proto-houses" (with all services, but minimal construction area) that residents would subsequently expand and complete, as well as financing mechanisms that would split costs between the state and beneficiary families. So the $6 billion/year tag is a rock-bottom figure, the very least the state could spend and still hope to meet its goals. Spend much less than that, and the problem deepens rather than receding.

"Normally," I remember them telling me, "you'd need to build at least 100,000 new low-income housing units a year - not to reduce current levels of crowding, but just to keep up with population growth - just to stay even. To start to make some headway against the trully appalling level of crowding in some barrios, you'd be talking about 120,000 houses a year, at least. Since the start of the Chavez era, we've never seen more than 60,000/year. So, in effect, we're going backwards: every house that you come up short from that 100,000 target means another family forced to choose between striking out on their own by squatting on an empty plot of land and building a rancho with their hands, or staying put and living in increasingly intolerable crowding with their relatives."

Serious Planning
Of course, this is just one area of concern: expert NGOs could (and have) come up with similar analyses for hospitals, schools, social security, the fire-fighters, the police, essentially any part of Venezuela's huge, overbloated but underfunded state. As a policy-wonk myself, it warms my heart to see the people of Alemo taking a long, hard, careful, uncompromising look at a policy problem like this and then propose specific, costed solutions. There is something reassuring about the mixture of genuine social concern and hard-headed planning in their work - their conviction that the more urgent a human problem is, the more down-to-earth and meticulous should be the planning for a solution. Alemo's proposals represent a hard-bitten marriage of idealism and pragmatism that eschews magical solutions and urges the state to tackle these matters with eyes wide open. It's a message, an attitude, that I can get 100% behind.

Now, how has the Chavez administration responded to Alemo's calls? The answer goes a long ways towards explaining my despair about the next two (or eight, or fourteen) years. First, the government purged Alemo members from Conavi, Fondur, Inavi, and every other state institute dealing with urban housing - putting the state's housing bureaucracy in the hands of doctrinaire chavistas that would neither produce independent estimates or raise troubling questions. Then, it set out to confuse the issue, making oversized claims for its decidedly undersized achievements.

At no point since 1999 has Venezuela come anywhere near to building the 100,000 low income housing units per year that it would take to keep up with population growth, let alone make headway into the housing crisis. Yet the government, conscious that very few people know this, continues to tout its house-building totals - just 25,000 units last year - as major revolutionary triumphs! Stop and think about what this means: last year alone, 75,000 poor Venezuelan families were forced to either build themselves an illegal shanty or stay on in impossibly cramped quarters...and the revolutionary people's government brags about this as a success!

(For next year, the government and the banks have reached an agreement where the government will pitch in $1.4 billion to build 50,000 low-income homes - watch for the headlines next year about the stunning, 100% growth in revolutionary housebuilding!)

What's the purpose of this detour into the minutiae of the Venezuelan low-income housing crisis?

Chavistas (and, more relevantly, most of the philochavistas who comment on my writing) start from the assumption that the only reason anyone opposes Chavez is class self-interest. This is undoubtedly true for some antichavistas. For many others, however, what's most irksome about Chavez is the magical strain in his government's thinking, its blanket rejection of any kind of independent advice, criticism, or debate. This dogmatism, this deep suspition about the motives of critics, locks the government into stances that not only cannot solve the problems at hand, but, even worse, close down the spaces for genuine debate about those problems.

The results is a politics of social inclusion that remains, largely, confined to the rhetorical realm - Chavez's discourse certainly makes his constituents feel included - while deepening the country's social problems. Point out that Chavez's housing policy will force 50,000 families to squat and build shanties next year and you're condemned as a counter-revolutionary element, an escualido fifth column that needs to be purged from the state. If you know the truth and you have a state job, you learn to keep quiet in order to protect yourself. Debate on a matter as seemingly apolitical as housing policy becomes deeply politicized, and those who criticize the government's policy are dismissed as wreckers or coup-plotters long before their criticisms have been seriously considered.

Put differently, in chavismo politics always takes precedence over policy. Uncomfortable facts are swept under the carpet and those who seek to bring attention to them are portrayed as traitors. So long as they're infused with reverence for the leader, pleasing fictions are always preferable to uncomfortable facts.

So long as criticism is seen as apostasy, so long as critics' voices are ignored as a matter of principle, so long as the worst of motives are automatically abscribed to all who dissent, the government will continue to make policy inside an ideological bubble where loyalty counts for far more than serious planning. Instead of hard-knuckled social policy development, instead of costed estimates and long-term projects, instead of serious plans aimed at grasping and then solving the underlying problem, we'll continue to get what we've been getting: ad hoc measures aimed at short-term political advantaged and divorced from any kind of serious analysis of what needs to be done in the long term.

Or, to say it in a single word, populism.

Where to the hell do you get $60 billion?
What Alemo's study makes clear, first of all, is that real solutions to the country's most pressing social problems will require public spending far in excess of the state's present capacity to pay. Venezuela's government is just too small and underfunded to make a serious stab at solving the housing crisis. And what's true of housing is true of any number of other social problems.

The solution, then, is clear: the public sector will need to grow very considerably to put itself in a position to face up to these expenditures. But after six years of rampant statism, heavy borrowing, and increased taxes, the chavista state has run up against the limits of its ability to extract resources from the economy. The housing crisis cannot be met by expanding the state's share of GDP - because the problem, precisely, is that GDP is too small at the moment to cover the country's basic necessities.

So, taken broadly enough, the lesson behind the Alemo proposal is that only rapid GDP growth, coupled with serious planning on how to use the extra resources that GDP growth makes available to the state, have any possibility of making real headway against the housing shortfall. On its current revenues, the state just cannot afford the expenditure needed, and given its current state, the economy cannot bankroll a state larger than the one we have now. Only a substantitally bigger economy can bankroll a substantially bigger state, and only a substantially bigger state can bankroll the investment needed to face up to the housing crisis.

This is an important point, because regime supporters tend to equate any mention of the need for rapid GDP growth with coded neoliberal speak for a policy of social abandonment. The truth, in fact, is quite the opposite: it's economic stagnation that has put the solution to the country's social problems beyond the reach of the state.

The problem, again, is that the cultural and ideological barriers that prevent chavismo from accepting an analysis like Alemo's also prevent it from accepting the kinds of policy reforms it would take to launch the economy into sustained rapid growth. The same lack of serious, techhnical planning, the same blanket mistrust of opponents, the same disdain for capitalist development, and the same belief in magical solutions have led to a brand of economic management that can produce, at best, ongoing stagnation.

Meanwhile, the revolution continues to build less than half of the homes it would need to build to stay even. Every ten minutes another Venezuelan family is forced to squat and build themselves a rancho just to find a place to live. Facing up to this situation is too difficult, too painful for chavistas. It requires too broad a reconsideration of el comandante's style of leadership. It's far easier, far more comforting, to propagandize about the revolution's great strides in house-building than to have an honest discussion about what's going wrong and how we might fix it.

For all its faults, the opposition is full of organizations like Alemo that are full of people eager to hold such serious debates - and not just on housing, on just about every topic of national relevance. The government, meanwhile, is forced by its ideology to run away from such debates - overstate its accomplishments, slur its opponents and obscure the nature of the challenges it faces. Personally, I can't for the life of me figure out what's "progressive" about this style of governance.

Join a moderated debate on this post.