I’m skeptical of prediction markets because of this paradox (or at least perceived paradox):
For markets to work, there needs to be enough people participating. But there isn’t much incentive to participate, because we expect the probability to reflect the actual probability of the outcome, and therefore there is no money to be made. In the stock market, there is always incentive to participate, even when the stock is a true reflection of the company value, because the value of the company grows (on average). It is this cooperation of incentives that makes the stock market work, and it doesn’t exist in prediction markets. You are only incentivized to participate if you think the market is wrong. This will either be an extremely small number of (mistaken; therefore eliminating all incentive) people or the market will just not be a reflection of the true probability. I just don’t see the market effects kicking in.
This is not meant as condescending. It is a possible explanation of the deviation we see in prediction markets. They are quite good at prediction the sports playoffs. A good quarterback or array of pitchers on a given team is not opaque. Prediction markets are a fun analog for perceived understanding.
Many years ago I was doing work in the control and monitoring markets for nuclear power. An intelligent colleague back in the day shared a perspective I have never forgotten and believe it applies. He explained way back then why nuclear generation COULD NEVER SUCCEED broadly. It had NOTHING to do with the approach, it had to do with the degree of broad understanding. The creators and managers of finance and books are stuck in a region of MODEST understanding. It is likely they achieved their lot in life without much of an understanding of mathematics beyond the basics. Supply and demand curves are important but are dumbed down representations of reality. A person can probably be a "star" or "creator" and not have a firm grasp of even calculus. Being able to turn a phrase to effect opinion is a different sort of gift.
My colleague observed that ANYTHING we do societally that is based upon even modest complexity will always be speculative, never understood, easily manipulable, and hence feared. It's broad success will be undermined unless broad education were to change. This is why power generated by fission is doomed with the broader public. People just don't really know what a neutron is. We need Michael Lewis to explain it or some writer of a Newsletter. As for conductivity, while people have some grasp of what temperature and pressure are, I would surmise that <<1% of the population knows what conductivity even means!!! Sometimes when I share the old sop "a pint's a pound the world around" I am amazed that people don't even have a general sense of mass and volume. To expect them to translate their Meta/Twitter feelings (mostly based on who they follow) into meaningful predictions or sensibility about something they are in the dark about can only end badly. Their predictions are hence based on nothing more than the "wisdom of the crowd" -- any one they self-select perhaps by experience or taste in clothing. The closest finance has gotten to madness in the markets was flash trading and derivatives. Only Michael Lewis managed the trick or explaining their underlying basis. Our modern world is finally advancing in more realms exponentially. I fear the average person is not firmly comfortable with logarithms.
All of this is a longhand explanation of why imagining early clicking in a predictions / futures market on ANYTHING OF MODEST SCIENTIFIC COMPLEXITY is like betting green on the roulette wheel or who will win the coin flip at the SuperBowl. It is fun, it is harmless, it is NOT predictive. The more complex the concept, the more settling time required before behavior is sensible to even pay attention to.
When you’re investing in the stock market with a dividend-based strategy the dividends provide a tether to the real world every month/quarter. Of course, most retail investors these days don’t invest for dividends, but on the prediction markets, no one invests for dividends. On the really long term markets (eg “will we have a certain archaeological discovery by 2030?”) you won’t get a payout for years unless you’re optimizing for the hype, not the outcome.
I wonder what dividends would look like for prediction markets?
There seems to be a theoretical model of prediction markets, in which the prediction market is as liquid and deep as the US stockmarket. But the one we have now is illiquid and shallow. To the extent that the stockmarket effectively aggregates information about a particular stock and accurately prices it, it does so because of liquidity and depth. (And this is why thinly traded meme stocks like Gamestop can be so volatile.) But when all we have for prediction markets is a theoretical construct of a liquid and deep market, its predictive power is for nought.
Hi Erik, thanks for this thoughtful critique! I work on Manifold, but that just gives me a front seat to all the ways prediction markets can fall short.
I do agree that our headline LK99 market was consistently overoptimistic -- and I put my money where my mouth was, earning the largest overall profit (about ~$2k worth). And I agree this was mostly due to the influence of virality; loosely speaking, dumb money (new traffic flowing in from Twitter) outweighed smart money (experienced Manifold traders) for a week or so. This kind of dynamic isn't new to prediction markets; eg during the 2020 primaries, Andrew Yang markets soared on PredictIt because of his unusually online fanbase.
I think the comparisons to Gamestop and crypto do overstate the case somewhat, though. Gamestop was notable specifically because people do generally view stock market prices as an accurate indication of the value of a company, and the r/wsb trader influx led to an aberration; if LK99 is notable in the same way, that reflects well on prediction markets overall! And it's true that on crypto, there's a long tail of scams, just like on Manifold there's a long tail of markets with ~0 traders -- or for that matter Twitter with a long tail of low-quality posts. We set out to do prediction markets differently on Manifold, where anyone could create a market as easily as anyone could create a tweet on Twitter. The low barrier to entry means that we do have lots of low-quality markets, but also unexpected gems: QuantumObserver, the creator of the LK99 market, was previously unknown on our site, but turned out to be a thoughtful and informed moderator.
So where does that leave us on the utility of prediction markets? For starters, I would be thrilled if more commentators like yourself got on board with prediction markets just as a useful index for tracking online opinion. Markets have the edge over polls in some contexts: eg they allow people to express the strength of their opinion in the form of the amount they choose to bet (there were 2x as many YES vs NO traders on LK99, but the price stayed well below 40%, so individual NO traders were investing ~3x as much on average). But they also have some weaknesses compared to polls; eg they're subject to groupthink because of the visibility of the market price.
There's one more benefit to moving more discourse onto prediction markets, though: because they ultimately resolve on a specific criteria, markets build up a quantifiable track record around commentator accuracy, helping us see whose opinions should be weighed more heavily, over time. Regarding stock markets, Warren Buffett famously stated: "In the short-run, the market is a voting machine – reflecting a voter-registration test that requires only money, not intelligence or emotional stability – but in the long-run, the market is a weighing machine." For a couple weeks, the LK99 market reflected the beliefs of excited traders; now it serves to identify who was most able to update on information in a rapidly-shifting media environment. As someone whose beliefs were validated, I think that's really cool ;)
There are four kinds of people who win in speculative markets:
- those willing to take advantage of others
- those who make bets on people, not assets
- those who get lucky
- those who don't treat them as speculative markets at all (example: buying and holding stocks for decades)
Almost everyone who "plays fair and square, by the rules of the game" - almost everyone who comes into the market to speculate - loses.
Said differently: speculative markets are a mechanism for transferring wealth legally from those who use them "as intended" to those who don't. And they always will be.
I would only roughly categorize prediction markets as speculative markets... but to the extent that they are, and are not simple bets... the rule above kicks in
There can't be a market that doesn't become speculative and virality-based to some degree. Because there is no homo economicus - humans are not rational actors. Markets will always be made of human emotion and motive more than fundamental value.
Markets in general have this issue that they're weighing machines of current information and future expectations, and in the absence of information of hope and hype. I've written before on how the problem with prediction markets today is illiquidity and market structure, which causes these swings to be pronounced. In this instance I also think a lesson is that we need to move towards better reactions, more so than predictions since they're thought to be predictive but can't be since it's a scientific question and not a psychology one, as here from this week's essay: https://www.strangeloopcanon.com/p/the-case-against-prediction
I'm glad this piece mentioned the "weird gray zone" prediction markets are in legally. Polymarket is barred from serving U.S. persons under a settlement with the CFTC (and tbf is difficult to use anyway because it uses crypto). PredictIt and Kalshi operate in the U.S. under different straitjackets. This situation surely accounts at least in part for these markets' thin liquidity, which in turn limits their predictive power.
At a minimum, prediction markets could be viewed as a more honest version of polling, because the participants are putting money on the line. So the bettors are more likely to express what they really believe (even if if they have no idea what they are talking about), rather than what they think others want to hear. That's something.
Would deeper and more liquid prediction markets incentivize participation by those with real expertise? Until the regulations change, we can only, uh, speculate.... :p
Fischer Black beleived the markets were efficient, but he defined efficient as "the traded price is within a factor of 2 of the fundamental price 90% of the time", which is a very generous in my eyes lol
So, score another point for a handpicked roster of Superforecasters as the least bad option? Just throwing my hat in here that most people who see that tradeoff intuitively think prediction markets are superior to anything else currently on display. Just because it seems so intuitively outlandish that a handpicked crowd of 300 Spartans can outwit the rest of the Internet. But the point still stands.
There's a part of the argument I'm just not understanding, and that is the following:
All those people betting high on LK99 really did lose their money!
It's not "more profitable to go with hype", hype loses and truth wins, that's the point of the whole thing.
Yes along the way we got a bad prediction, but it wasn't because the alignment with incentives was somehow off. Seems likely to me that, as you suggest and then reject, prediction markets are just too low-volume and play money isn't a good enough incentive.
As a criticism of prediction markets, this post doesn't really hit for me.
We shouldn't expect prediction markets to keep virality (or whatever) from spiking prices; we should expect them to quickly correct to some equilibrium. Looking at the chart you compared to Google trends results, that looks to me to be exactly what happened.
A stronger criticism (made above but less emphatically) is that the equilibrium reached was, in some sense, "too high". The response of a prediction market enthusiast to this criticism (for this particular case) would be that market participants were using play money rather than real money. For prediction markets to have any chance of working, participants need to have skin in the game.
Another way to think about it: If I'm an expert on room-temperature superconductors, I might not bother logging onto Manifold and participating in the LQ-99 market. But if I can turn $1000 into $20,000 by betting against the virality, I'm much more likely to.
Bottom line, though - LK99 is not a superconductor and no amount of wanting it to be will make it so, so no amount of "market force" thrown at it will change that fact. So I guess I will never understand why measuring hype is being used as a measurement of truth.
Prediction markets aren't analogous to the stock market. Unlike with the stock market, you (the expert predictor betting against the market) can stay solvent longer than the market can stay irrational - you just wait it out till the resolution date.
And the result of a particular market has a true platonic value - it will resolve to some outcome - independent of what the market thinks. This isn't true of a stock market - the value of a particular stock is fundamentally dependent on the underlying company's ability to raise capital, which is directly dependent on the level of market hype for that stock.
So many respectable scientific posters shamefully chased this down without ever hesitating. The whole episode reflected the lack of restraint overpowering good judgment when it came to joining the feeding frenzy on this story.
I’m skeptical of prediction markets because of this paradox (or at least perceived paradox):
For markets to work, there needs to be enough people participating. But there isn’t much incentive to participate, because we expect the probability to reflect the actual probability of the outcome, and therefore there is no money to be made. In the stock market, there is always incentive to participate, even when the stock is a true reflection of the company value, because the value of the company grows (on average). It is this cooperation of incentives that makes the stock market work, and it doesn’t exist in prediction markets. You are only incentivized to participate if you think the market is wrong. This will either be an extremely small number of (mistaken; therefore eliminating all incentive) people or the market will just not be a reflection of the true probability. I just don’t see the market effects kicking in.
This is not meant as condescending. It is a possible explanation of the deviation we see in prediction markets. They are quite good at prediction the sports playoffs. A good quarterback or array of pitchers on a given team is not opaque. Prediction markets are a fun analog for perceived understanding.
Many years ago I was doing work in the control and monitoring markets for nuclear power. An intelligent colleague back in the day shared a perspective I have never forgotten and believe it applies. He explained way back then why nuclear generation COULD NEVER SUCCEED broadly. It had NOTHING to do with the approach, it had to do with the degree of broad understanding. The creators and managers of finance and books are stuck in a region of MODEST understanding. It is likely they achieved their lot in life without much of an understanding of mathematics beyond the basics. Supply and demand curves are important but are dumbed down representations of reality. A person can probably be a "star" or "creator" and not have a firm grasp of even calculus. Being able to turn a phrase to effect opinion is a different sort of gift.
My colleague observed that ANYTHING we do societally that is based upon even modest complexity will always be speculative, never understood, easily manipulable, and hence feared. It's broad success will be undermined unless broad education were to change. This is why power generated by fission is doomed with the broader public. People just don't really know what a neutron is. We need Michael Lewis to explain it or some writer of a Newsletter. As for conductivity, while people have some grasp of what temperature and pressure are, I would surmise that <<1% of the population knows what conductivity even means!!! Sometimes when I share the old sop "a pint's a pound the world around" I am amazed that people don't even have a general sense of mass and volume. To expect them to translate their Meta/Twitter feelings (mostly based on who they follow) into meaningful predictions or sensibility about something they are in the dark about can only end badly. Their predictions are hence based on nothing more than the "wisdom of the crowd" -- any one they self-select perhaps by experience or taste in clothing. The closest finance has gotten to madness in the markets was flash trading and derivatives. Only Michael Lewis managed the trick or explaining their underlying basis. Our modern world is finally advancing in more realms exponentially. I fear the average person is not firmly comfortable with logarithms.
All of this is a longhand explanation of why imagining early clicking in a predictions / futures market on ANYTHING OF MODEST SCIENTIFIC COMPLEXITY is like betting green on the roulette wheel or who will win the coin flip at the SuperBowl. It is fun, it is harmless, it is NOT predictive. The more complex the concept, the more settling time required before behavior is sensible to even pay attention to.
When you’re investing in the stock market with a dividend-based strategy the dividends provide a tether to the real world every month/quarter. Of course, most retail investors these days don’t invest for dividends, but on the prediction markets, no one invests for dividends. On the really long term markets (eg “will we have a certain archaeological discovery by 2030?”) you won’t get a payout for years unless you’re optimizing for the hype, not the outcome.
I wonder what dividends would look like for prediction markets?
There seems to be a theoretical model of prediction markets, in which the prediction market is as liquid and deep as the US stockmarket. But the one we have now is illiquid and shallow. To the extent that the stockmarket effectively aggregates information about a particular stock and accurately prices it, it does so because of liquidity and depth. (And this is why thinly traded meme stocks like Gamestop can be so volatile.) But when all we have for prediction markets is a theoretical construct of a liquid and deep market, its predictive power is for nought.
Hi Erik, thanks for this thoughtful critique! I work on Manifold, but that just gives me a front seat to all the ways prediction markets can fall short.
I do agree that our headline LK99 market was consistently overoptimistic -- and I put my money where my mouth was, earning the largest overall profit (about ~$2k worth). And I agree this was mostly due to the influence of virality; loosely speaking, dumb money (new traffic flowing in from Twitter) outweighed smart money (experienced Manifold traders) for a week or so. This kind of dynamic isn't new to prediction markets; eg during the 2020 primaries, Andrew Yang markets soared on PredictIt because of his unusually online fanbase.
I think the comparisons to Gamestop and crypto do overstate the case somewhat, though. Gamestop was notable specifically because people do generally view stock market prices as an accurate indication of the value of a company, and the r/wsb trader influx led to an aberration; if LK99 is notable in the same way, that reflects well on prediction markets overall! And it's true that on crypto, there's a long tail of scams, just like on Manifold there's a long tail of markets with ~0 traders -- or for that matter Twitter with a long tail of low-quality posts. We set out to do prediction markets differently on Manifold, where anyone could create a market as easily as anyone could create a tweet on Twitter. The low barrier to entry means that we do have lots of low-quality markets, but also unexpected gems: QuantumObserver, the creator of the LK99 market, was previously unknown on our site, but turned out to be a thoughtful and informed moderator.
So where does that leave us on the utility of prediction markets? For starters, I would be thrilled if more commentators like yourself got on board with prediction markets just as a useful index for tracking online opinion. Markets have the edge over polls in some contexts: eg they allow people to express the strength of their opinion in the form of the amount they choose to bet (there were 2x as many YES vs NO traders on LK99, but the price stayed well below 40%, so individual NO traders were investing ~3x as much on average). But they also have some weaknesses compared to polls; eg they're subject to groupthink because of the visibility of the market price.
There's one more benefit to moving more discourse onto prediction markets, though: because they ultimately resolve on a specific criteria, markets build up a quantifiable track record around commentator accuracy, helping us see whose opinions should be weighed more heavily, over time. Regarding stock markets, Warren Buffett famously stated: "In the short-run, the market is a voting machine – reflecting a voter-registration test that requires only money, not intelligence or emotional stability – but in the long-run, the market is a weighing machine." For a couple weeks, the LK99 market reflected the beliefs of excited traders; now it serves to identify who was most able to update on information in a rapidly-shifting media environment. As someone whose beliefs were validated, I think that's really cool ;)
There are four kinds of people who win in speculative markets:
- those willing to take advantage of others
- those who make bets on people, not assets
- those who get lucky
- those who don't treat them as speculative markets at all (example: buying and holding stocks for decades)
Almost everyone who "plays fair and square, by the rules of the game" - almost everyone who comes into the market to speculate - loses.
Said differently: speculative markets are a mechanism for transferring wealth legally from those who use them "as intended" to those who don't. And they always will be.
I would only roughly categorize prediction markets as speculative markets... but to the extent that they are, and are not simple bets... the rule above kicks in
There can't be a market that doesn't become speculative and virality-based to some degree. Because there is no homo economicus - humans are not rational actors. Markets will always be made of human emotion and motive more than fundamental value.
Great post as usual.
Markets in general have this issue that they're weighing machines of current information and future expectations, and in the absence of information of hope and hype. I've written before on how the problem with prediction markets today is illiquidity and market structure, which causes these swings to be pronounced. In this instance I also think a lesson is that we need to move towards better reactions, more so than predictions since they're thought to be predictive but can't be since it's a scientific question and not a psychology one, as here from this week's essay: https://www.strangeloopcanon.com/p/the-case-against-prediction
I'm glad this piece mentioned the "weird gray zone" prediction markets are in legally. Polymarket is barred from serving U.S. persons under a settlement with the CFTC (and tbf is difficult to use anyway because it uses crypto). PredictIt and Kalshi operate in the U.S. under different straitjackets. This situation surely accounts at least in part for these markets' thin liquidity, which in turn limits their predictive power.
At a minimum, prediction markets could be viewed as a more honest version of polling, because the participants are putting money on the line. So the bettors are more likely to express what they really believe (even if if they have no idea what they are talking about), rather than what they think others want to hear. That's something.
Would deeper and more liquid prediction markets incentivize participation by those with real expertise? Until the regulations change, we can only, uh, speculate.... :p
Fischer Black beleived the markets were efficient, but he defined efficient as "the traded price is within a factor of 2 of the fundamental price 90% of the time", which is a very generous in my eyes lol
So, score another point for a handpicked roster of Superforecasters as the least bad option? Just throwing my hat in here that most people who see that tradeoff intuitively think prediction markets are superior to anything else currently on display. Just because it seems so intuitively outlandish that a handpicked crowd of 300 Spartans can outwit the rest of the Internet. But the point still stands.
There's a part of the argument I'm just not understanding, and that is the following:
All those people betting high on LK99 really did lose their money!
It's not "more profitable to go with hype", hype loses and truth wins, that's the point of the whole thing.
Yes along the way we got a bad prediction, but it wasn't because the alignment with incentives was somehow off. Seems likely to me that, as you suggest and then reject, prediction markets are just too low-volume and play money isn't a good enough incentive.
As a criticism of prediction markets, this post doesn't really hit for me.
We shouldn't expect prediction markets to keep virality (or whatever) from spiking prices; we should expect them to quickly correct to some equilibrium. Looking at the chart you compared to Google trends results, that looks to me to be exactly what happened.
A stronger criticism (made above but less emphatically) is that the equilibrium reached was, in some sense, "too high". The response of a prediction market enthusiast to this criticism (for this particular case) would be that market participants were using play money rather than real money. For prediction markets to have any chance of working, participants need to have skin in the game.
Another way to think about it: If I'm an expert on room-temperature superconductors, I might not bother logging onto Manifold and participating in the LQ-99 market. But if I can turn $1000 into $20,000 by betting against the virality, I'm much more likely to.
R.R.E.A.M. -- Reflexivity Rules Everything Around Me.
Bottom line, though - LK99 is not a superconductor and no amount of wanting it to be will make it so, so no amount of "market force" thrown at it will change that fact. So I guess I will never understand why measuring hype is being used as a measurement of truth.
Prediction markets aren't analogous to the stock market. Unlike with the stock market, you (the expert predictor betting against the market) can stay solvent longer than the market can stay irrational - you just wait it out till the resolution date.
And the result of a particular market has a true platonic value - it will resolve to some outcome - independent of what the market thinks. This isn't true of a stock market - the value of a particular stock is fundamentally dependent on the underlying company's ability to raise capital, which is directly dependent on the level of market hype for that stock.
So many respectable scientific posters shamefully chased this down without ever hesitating. The whole episode reflected the lack of restraint overpowering good judgment when it came to joining the feeding frenzy on this story.