Since its inception nearly five years ago, Twitter has grown from the brainchild of three colleagues at a middling Silicon Valley start-up to a social networking giant.
Along the way, the micro-blogging site has revolutionized the way news is reported by bringing its more than 200 million registered users everything from the latest rants by Charlie Sheen to first-person dispatches describing the carnage resulting from the March earthquake and tsunami that leveled northern Japan. It even played a critical role in the ouster of Hosni Mubarak as Egypt's president in February.
And Twitter, which lets users send 140-character text messages to followers, has been used to gauge public sentiment and predict everything from election results to box office receipts. Now it's threatening to change the game on Wall Street. This spring, U.K.-based hedge fund start-up Derwent Capital will become the first asset manager officially to deploy a trading strategy based on Twitter.
Place Your Bets
The firm is wagering nearly $40 million on an algorithm, developed by computer scientists at Indiana University and the University of Manchester, that's designed to predict if the stock market will rise or fall. By tracking a wide range of tweets every day, Indiana University professor Johan Bollen claims the algorithm can accurately detect where the market is heading three or four days in advance with nearly 88 percent accuracy.
"Our algorithm scans these very large Twitter feeds on a daily basis and then analyzes their content to get a sense of the general public's mood state along six different dimensions," explains Bollen, who was hired as a Derwent consultant after publicizing the research last fall. "It then produces a time series that indicates fluctuations in the public mood on a day-to-day basis. We use that to make predictions as to where the market is headed."
Bollen says that when he and his partners - Indiana professor Huina Mao and Xiao-Jun Zeng of Manchester - set out to launch their research, they were simply trying to measure the public's mood and see what kind of data stood out; they weren't trying to figure out a way to win on Wall Street with Twitter. "We started to correlate this with a bunch of different social indicators, and the Dow Jones actually popped out," he says. "It's not like we had a little black box in our office looking to predict the market."
The research caught the eye of Derwent Capital, and now many on Wall Street will be watching the performance of the hedge fund's algorithm closely. However, the strategy has drawn skepticism from industry participants who argue that it could be ineffective in a real-time environment - and even dangerous, as it could be highly susceptible to manipulation from rivals.
The Twitter Speed Bump
"You could use it to get a good sentiment reading globally on particular topics, whether it be revolutions or how people feel about the economy," says John Bates, the founder and chief technology officer of Progress Software. "The problem is that by the time you've got that information, it's kind of a trailing indicator rather than a leading indicator. It's trails real time, which is why I was skeptical about it being a predictor."
The algorithm could be successful if it were used to track consumer confidence or to gauge consumer reaction to new products and predict sales, Bates acknowledges. But the four-day prediction model could doom Derwent's best-laid plans, he contends. "Unpredictable events occur all the time," Bates notes. "There's been many funds that claim they can predict what's going to happen in the next four days or the next week that have failed and gone out of business, particularly in the volatile conditions of 2008 and 2009."
Bollen counters that the research was conducted during one of the most volatile market periods in U.S. history - fall 2008. Yet it still managed to be accurate in a market rocked by the Lehman Brothers collapse, the global financial crisis and the federal government's bailout of the financial sector.
"We still achieved prediction accuracies during a tremendously volatile period in the markets, so we're confident this is pretty resistant to volatility," Bollen insists. "Now, how resistant, we just don't know. It stands to reason that if the market becomes very volatile, the interactions with public mood could actually be quite complex. But so far we haven't seen that kind of sensitivity."
As for the notion the algorithm can't function effectively in real time, Bollen says that Derwent won't be using Twitter for high-frequency algorithms, which have come to dominate Wall Street. However, with a powerful enough computer network, it is theoretically possible, he adds.