Friday, May 28, 2010

What Do You Mean By "Best"?

Plurality voting is terrible and should be replaced, but what's the best voting system to replace it with? This isn't a new revelation, or a new question; for instance Thomas Jefferson considered the problem. But academic inquiries to it had been in a lull since Kenneth Arrow's Nobel-prize-winning work in 1950 showed that, given certain assumptions, there was no perfect system. Social-decision scientists everywhere were crushed.

What little debate that continued about the subject focused around various "voting system criteria". Arrow's work had shown that a group of five certain "clearly necessary" criteria were mutually exclusive, but perhaps by breaking certain ones in a minimally-damaging way an almost-perfect voting system could be found. The problem was, no one could agree which criteria were most important; each practitioner could always come up with some worst-case scenario in which their opponents latest new proposal clearly gave a horrible result (usually involving a candidate named "Hitler" winning the election, just to make the point clear.) And so the debate degenerated to what situations were more likely to come up or led to more damaging results: the terrible one I concocted for your new voting system, or the terrible one you concocted for mine. But all these arguments lacked one important piece: evidence.

To make good estimates of how often various worst-case scenarios happen and how bad they are, it would take at least hundreds of elections, each with a minimum of a few hundred participants, multiplied by each of dozens of systems that had been developed, in order to get a clear picture. But even then, what do you measure? When your experiment is to ask people "what's the best ice-cream favor," how do you measure whether the voting system was right without knowing the right answer ahead of time? And how would you determine the right answer ahead of time, without asking people to vote on it?

The problem is that economic utility can't be measured directly. Combined with the in-feasibility of performing enough test-elections, it's enough to make almost anyone throw up their hands in frustration.

But here's a clever idea: what if we replace real people with little bits of computer code? Instead of futilely trying to measure each participants utility, we can just assign them randomly from a statistical distribution. We'll have each little bit of code "vote" using every one of the electoral methods we've developed, but also calculate what the maximum possible utility could be from each election, and see how much we miss by. And we'll do it a few hundred times and take the average. Running the whole simulation should take maybe a long weekend. (If only Arrow had had access to a modern desktop computer!) What would we find? Let's ask Professor Warren D. Smith, who ran this simulation over the 1999/2000 New Year's holiday.

If the data from this simulation is to be believed, using approval voting, or score voting (listed here as range voting), could improve the results of our elections by the same proportion as voting at all is an improvement over choosing our leaders at random. That's an astounding result!

Of course, there are still critics: most of them just repeat their favorite criteria argument (usually later-no-harm or majority, since score and approval fail them) ignoring that this data already accounts for any downsides from those short-comings. A few smug folks point out that you can't measure utility; but we already know that, that's why we used a simulation. Some attacked the statistical distribution of utility (now we're getting to something meaty!), so a series of better distributions, based on their suggestions, were used: the results were virtually the same. Then they argued that voters are a poor judge of their own utility; so the experiment was rerun with a "voter uncertainty" parameter. Even with a 50% error factor, score and approval still top the list.

The most bizarre argument is that score and approval can't be the best voting systems, because they aren't voting systems at all. You see, one of Arrow's assumptions was that a voting system would convert a set of all voter's "ranked-order preferences" into a societal order of ranked preferences. But score and approval don't used ranked-order preferences; perhaps, if Arrow hadn't used this overly-restrictive requirement, it wouldn't have taken 50 years to find these results.

Not only is this an astounding result, it seems to be a fairly unassailable result. The "best" voting system is score voting, and approval is almost as good (but easier to implement).

Saturday, May 22, 2010

Hawaii 01: Another Example

The state of Hawaii has provided us yet another example of the need to reform our electoral system. Voting in the special election to fill HI-01, left open by Neil Abercrombie's (D) retirement, finished up today, and the results are out:

  • (R) DJOU, Charles 67,274 39.5%
  • (D) HANABUSA, Colleen 52,445 30.8%
  • (D) CASE, Ed 47,012 27.6%

Quick math will show that Hanabusa and Case, both Democrats, combined for 58.4% of the vote. (The other three Democrats, other four Republicans, and four third-party and independent candidates combined received about 1.5%.) It would be a hard argument to make that a majority of either of the Hanabusa or Case voters would prefer that Djou be their representative, and yet, for the next 8 months, he will be.

Some people will claim that this is mostly Abercrombie's fault: he knew that Hawaii election law would lead to this free-for-all election, and he should have stayed on the job until January. Some will blame Case: he should have known he was running behind Hanabusa, and therefore should have stepped aside. Some will claim that this shows Hawaii should update its election laws to allow for primaries, or at least for party-commission selection of candidates.

A better answer would be approval voting (or score voting). This wouldn't have required a gubernatorial hopeful to half-heartedly continue on as a congressmen. And it wouldn't have forced a legitimately viable candidate (27.6% is a pretty respectable percentage) to bow out prematurely. Nor would it require another expensive and time consuming set of elections, or required the choice to be made by an undemocratic party boss. Instead, voters whose concern is that a Democrat, any Democrat, wins could have approved of both Hanabusa and Case, and as long as about 9% or so of the electorate had done that (or about 1/6th of the voters who voted for any Democrat), a Democrat would have won.

Monday, May 10, 2010

Utah Republican Primaries

Ignoring, for a moment, the larger political implications of stalwart Republican Senator Robert Bennett failing to get on the Republican primary ballot for his re-election, I find it very interesting that Utah's Republicans have taken the idea of a primary to the next logical level, by having a pre-primary decision that makes the actual primary a top-two runoff; which then, of course, feeds straight into the two-party-dominated general election. It's the only rational response when you assume single-winner plurality elections.

Someone should tell them about approval voting.

Friday, May 7, 2010

Lies My Blogger Told Me; Or:The Republicans Were Never A Third Party

I've read this on a several small blogs and in many comments over the last few months: this misguided belief that the Republican party was, at one time, a third party; like the Constitutionist or the Libertarians or the Greens or the Socialist of today.

It's not true. The Republicans were never anything like any of the third parties we have now, because the Republicans were never on the outside looking in at the two-party system. Instead, one of the existing two major parties—the Whigs—collapsed due to an internal schism over the issue of slavery. Then, one of the factions met up and re-named themselves the Republicans. But the people involved were the same people, the same politicians! And a lot of them went straight from being elected Whigs to being elected Republicans!

(Although some of them, fed up with the squabbling, left a few years before the dramatic collapse, and had to be coaxed back later.)

That's completely different then any group of out-of-power citizens trying to build an organization up from the ground floor to challenge the two near-indomitable incumbents. Remember, it is a two-party system. The only road to success for a third party (other than a fundamental voting system change such as approval voting or score voting) is for one of the two major parties to collapse; which has only ever happened in American history because of internal disagreements, never because of an assault from the outside.

So good luck.

Thursday, May 6, 2010

Thought Experiments: On Top-Two Open Primaries

If you follow voting issues, or if you happen to live in California, you may be aware of Proposition 14, an initiative which seeks to replace party-based primaries in the state with the so-called "top-two open primary" (TTOP); top-two because only the top two vote-getters get to appear on the ballot for the general election, and open because its open to candidates and voters from all parties.

It's a terrible idea. Let me show you why, by means of a few thought-experiments:

#1: Party Hack

First, let's presume I'm an ultra-partisan party hack for one of the two major parites; all I care about is whether or not a member of my party wins the general election, and that a member of the other major party doesn't. What changes for me under TTOP?

Before TTOP, the only reason I cared about primaries at all was I wanted to candidate with the best chance to win the general election to come out of the party primary; typically, I tend to back someone who's a moderate, rather than an extreme, member of my party, so that they'll get some cross-over support in the general, and pick up the moderates and independents; but not someone who's crazy moderate, such that the "base" (read: extreme wing of the party) stay home on election day... which means I can actually back a much more moderate candidate than their grumblings would attest to (they'll come around; after all, they wouldn't want a member of the other party to win, would they?)

What's different for me with TTOP? To keep things simple, let's assume that the other party only has a single candidate in the TTOP. Well, I have a small problem: if my moderate is too moderate, then the base will abandon him for the more-extreme option. And then, fearing that the extreme candidate in my party and the moderate member of my party will be the top-two, my cross-over support starts to slip away. So I stand I good chance to see my moderate go down, and the general to be between my extremist and the other party's guy; and that's a contest that I worry I might lose.

On the other hand, maybe I pick up a lot of true moderates. As long as the primary voters really don't care in a fight between my extremist and the other party's guy, then they have nothing to lose in voter for my moderate. But there's two problems with that. One, they really have to not care which of the those two wins, and most moderates lean one way or the other. Two, actual moderate (or independents or what have you) aren't used to voting in the formerly party-only primaries, so I'll need to step up my get-out-the-vote game, while focusing it on an entirely new electorate. That'll cost me a pretty penny.

These two forces pull in opposite directions with respect to how "moderate" the final election winner will be. But which force is stronger? Which is more likely to come up? A lot of ink is being sacrificed over that debate, but the best analytical guess comes from The Center For Government Studies [PDF], who conclude that, averaged over time, there will only be minimal improvement. Sometimes you'll end up with a more moderate eventual winner, but sometimes you'll get the crazy. By "on average" though, it means we're adding a lot of variance, which is to say, randomness. But we don't want randomness, we want accountability, and transparency, and responsiveness. Randomness doesn't help anyone. Still, I have to play the odds, and no matter what, I back my moderate.

But what if the other party chooses to also run two candidates against my two candidates? Then things get a bit more complicated, because now there's the chance that neither of my guys makes it to election day. Now I need to do extensive polling, and advertise it widely, because I've got to make sure that, no matter which of our two guys we choose to back, we've all got to back him, because in this case, a split vote could mean we're out with no chance of recovery.

My instincts point me clearly towards a simple conclusion: it would be really nice if we could have a party primary before the open primary. Except that's, for one, illegal now, and two, kind of pointing out how ridiculously stupid this law is in the first place. And the problem only gets worse for me if there are more members of my party running; as long as there's more than one candidate not of my party, I need to do everything I can to limit the number of members of my party that run, and the one tool I have for this, a party primary, is gone.

If I'm a party hack, I don't really like TTOP.

Separately, let's consider the case where there's three candidates in the TTOP: my guy, the other guy, and a third-party candidate. I still need to do a lot of polling and advertising, because if the third-party guy is doing well, and is doing well at my party's expense, then it's similar to the case where the other party was running two candidates; I might be out before the general. This is the dream scenario that a lot of third-party backers are envisioning for themselves now, where they unseat a major party candidate and go into the final vote. But how likely is that? Thankfully, we already know the answer to that, because TTOP is functionally the same as another voting system we've looked at: instant runoff voting. And we know that, in many situations involving three candidates, IRV leads to spoiled elections, and spoiled elections lead to a two-party dominated system. Which means we get the "lesser of two evils" voters, and things keep working for us just as before.

#2: A Moderate Voter

On the flip side, the problems are equal, but opposite. I can try to vote for third-parties, but I still probably see a spoiler before I see a winner, which the party hacks will be betting on. So I'll try to vote for more moderate members of major parties; which is fine, unless both parties are running a moderate, in which case I have to do a lot of homework to make sure the moderate vote doesn't get split such that neither moderate goes to the general and we're stuck with a vote between two insane extremists.

Sound and Fury

So in the aggregate, what we find is an effectively identical average effectiveness (perhaps a smidge better, according to simulations), but one that's surrounded by a lot more random variance, and with a whole lot more money spent on polling and advertising; which is, again, the same conclusion as the Center For Government Studies came to. And really, do we want to enact a change whose most significant effect will be to make our elected representatives to be more dependent on their corporate financiers?

All in all, TTOP isn't worth the trouble. So if you're Californian, remember to vote NO on Proposition 14.