By Lauren Silva Laughlin; graphics Nicolas Rapp
FORTUNE -- Once upon a time, an ordinary investor -- call him Joe -- would take some of his retirement savings and put it into a giant brand-name mutual fund that advertised in the Sunday paper. The fund would take that money and buy shares in brand-name American companies -- "large caps," the jargon went. Joe would then dutifully put some of his nest egg in a bond fund -- and, maybe, on a flier, invest a small chunk in a tech fund that held pieces of the top 100 names on the Nasdaq. (It was 1999 -- everybody was doing it.)
Individual stocks would rise and fall with abandon on any given day. But the broad market would generally trade in a narrow band: It was rare for the Dow (INDU) or the S&P 500 (SPX) stock indexes to move 3% in any direction in a session. When they did, the news was big enough to merit a banner headline: BLACK MONDAY or ASIAN CONTAGION, for instance. The action was mostly on the Big Board, the New York Stock Exchange, though Nasdaq was sizzling and clamoring for share. The white-shoe firms on Wall Street dominated the action; hedge funds and day traders hovered on the edges.
What a difference a millennium makes!
Today every aspect of this picture has changed in some significant way -- from which financial instruments investors bet on to where they trade them to how fast those trades are executed. (For more, see Carol Loomis's "The ICE Man Cometh.") The first exchange-traded fund came into being in 1993, says the Investment Company Institute. Twenty years later ETFs have drawn $1.5 trillion of your money (though that's still only about one-tenth the size of the mutual fund industry). Just as striking is where investor dollars are going. Today we're putting a robust 17% of our holdings in funds and ETFs that wager on foreign stocks, almost double the share from a decade ago. Even the roster of companies selling and managing mutual funds has changed: Of the 25 largest fund complexes in 1995, 10 are no longer on the list.
Wall Street firms are still titans, of course, minus a few once-storied names (Lehman Brothers, Bear Stearns). But today's market movers are more apt to be bots than bankers. Automated trading systems, following preset algorithms, now trade nearly half the share volume on U.S. exchanges, jostling equities by the microsecond and often sending volatile stocks into a roller-coaster ride. Hedge funds, according to research firm HFR, have grown fivefold in net assets since 1999, to $2.5 trillion, and routinely move markets with aggressive bets.
The story comes through strikingly in the data -- and so, for this year's Investor's Guide, we've told it that way. Here, a graphic look at the stunning change in how we trade.
Enter the ETF
Before 2002, exchange-traded funds were barely a blip on investor radars -- investments in ETFs totaled less than $100 billion, compared with the $7 trillion then held in mutual funds, according to the ICI. The notion of buying and selling baskets of stocks as if they were a single equity caught on, however, and in a big way. The ETF industry has grown more than 10-fold in the past decade, including some $800 billion in net inflows since 2007. At the end of September 2013, investors could choose from nearly 1,300 ETFs, some 15 times the number at the end of 2000.
At the same time U.S. equity mutual funds have fallen well out of favor. Since January 2007, more than $600 billion has flowed out of these plain-vanilla funds. Meanwhile, investors have poured a mammoth $1 trillion into bond funds. And mom-and-pop investors have increasingly looked overseas too. Net cash flows to foreign mutual funds and ETFs have increased by more than $550 billion in the past six years.
Rise of the borg
Over the past few decades, as exchanges have transformed from high-decibel human slam-dances to whisper-quiet electronic billboards, the time it takes to do a single trade has collapsed to nearly zero. And as execution time has fallen from minutes to microseconds, so too have prices come down, enabling traders to jump in and out of stocks both faster and more cheaply. "There has always been a race in financial markets," says James Angel, an associate professor of finance at Georgetown University. He points to Nathan Rothschild's information network in the 1800s, which was set up to receive information about the Battle of Waterloo faster than anyone else, allowing him to profit from the battle's outcome. "It's no different now," Angel says.
With information traveling at nearly the speed of light, there is a physical limit to how fast any trade order can fly over a fiber-optic cable. That fact has forced traders to compete in a new realm: distance from the trading platform. The latest investing arms race, then, is to get an arm's length from an exchange server. Enter a new term in the argot: "Co-location," or renting space for one's trading computers in or near the server room of an exchange. Meanwhile a slew of regulatory and technological changes have wound up giving faster trading firms an edge. One regulation, known informally as Reg NMS, implemented in 2007, helped force the main exchanges to trade more electronically and as a result opened the competition to scores of smaller "cottage" exchanges. Now more than half the trades done in U.S. stocks are put through sources outside the NYSE and Nasdaq.
With more exchanges, computers can race to several places to ping in their trades ahead of others. The aim is to squeak out hundredths of a penny in millionths of a second. And while anyone can buy a machine to do lightning-fast trades and then rent space to nestle it by the exchange servers, a select few trading firms -- such as KCG (formerly Knight Capital) and Citadel -- have managed to develop software that gives them an edge in beating out other trades, says Eric Hunsader, a software developer at Nanex, a research firm in Winnetka, Ill. "That's the secret sauce," he says.
How investors have fared
Throw into the mix Borglike computers trading automatically according to the whims of preset algorithms, and you have a recipe for volatility. Particularly in the years following the financial crisis, turbulence has been a mainstay of 21st-century global financial markets. Between 2008 and 2011 there were 76 days in which the S&P 500 stock index swung over 3% in either direction, more than in the previous three decades combined, according to S&P Capital IQ. On 15 of those days, the market moved more than 8%, an occurrence that seldom happened before the millennium. Those who have sensed some stability in the market in recent months aren't imagining things: Since 2012 there has not been one 3% intraday swing in the S&P. But experts say the structural forces that ushered in the era of volatility -- algorithmic trading and HFT, for example -- haven't gone away. So when the next correction comes, it, too, is likely to come in feverish ups and downs.
With such dizzying changes, you might think investors would have passed out along the way. But even with the shock to the system brought on by the mortgage meltdown and the great panic that followed, most have done well. In aggregate, U.S. mutual funds -- including equity, fixed-income, and other funds -- have edged out the S&P 500 stock index, returning 6.6% yearly on average since 2002, compared with 6% for the S&P, according to Morningstar. Meanwhile, 401(k) balances continue to climb; accounts administered by Vanguard, for instance, hit a record average balance at the end of October, reaching $98,826.
"The toolkit that the industry has given investors is one that seems to work," says Don Phillips, head of investment research at Morningstar. "It used to be that you have huge amounts of money in company stock. Now you end up with a reasonable slug of equities but have a relatively diversified and balanced portfolio." It helps that people who sign up for 401(k)s have rolling contributions. The structure doesn't allow people to make quick decisions and therefore encourages good behavior, say many experts. "We are almost wired to do the wrong thing as an investor," Phillips says. "When the market is up, you want to put in -- and to take money out when the market is down." In a 401(k) you automatically contribute when the market is down -- this helped a lot during the financial crisis. "That's actually where investors bought securities that would appreciate, and it was ones that they'd be least likely to buy had they not been signed up," he says.
That's not to say it hasn't been painful at times. It surely has. Many investors, including those who had to pull money out for retirement in 2008, were hit brutally hard. But on the whole, those who stayed in have found there's one thing about the market that hasn't changed: "Buy and forget" is still a better formula than "Follow the herd."
This story is from the December 23, 2013 issue of Fortune.
If you wait until 70 to collect and live to your mid-eighties, you win big. But who says you'll make it that far?
FORTUNE -- Year-end festivities are approaching, with Hanukkah lights nearing their peak, Christmas lights going up, and the New Year's celebration almost upon us. So what better time than the season of light is there to talk about ... mortality?
No, I'm not raising this gloomy topic just to be MOREAllan Sloan, senior editor-at-large - Dec 4, 2013 5:00 AM ET
PIMCO's Bill Gross and the other big bond boys are experiencing some performance issues this year.
By Daryl Jones, Hedgeye
FORTUNE -- For those of you who missed it in high-school history class, Sir Ernest Shackleton is the best-known figure in the "Heroic Age of Antarctic Exploration," which began in 1911 after Roald Amundsen successfully reached the South Pole. Shackleton was a maverick. He wanted to one-up Amundsen. So he set MORENov 26, 2013 10:04 AM ET
The U.S. is transparent but overpriced. What's an individual investor to do?
By Geoff Colvin, senior editor-at-large
FORTUNE -- In a global investment bazaar, where's the best place to invest right now?
Depends on who you are, as three recent responses to that question make clear.
I asked David Rubenstein, co-founder and co-CEO of the giant Carlyle Group (CG) private equity firm (assets under management: $180 billion), where he's looking to buy companies MORENov 26, 2013 8:00 AM ET
The market is up, but Wall Street has less to be excited about now.
FORTUNE -- The stock market probably isn't in a bubble, but there's a bigger risk you need to consider: That buying now will mean much lower returns over the next few years.
The valuation on the S&P 500 is still reasonable enough – a P/E of 16.6, based on trailing earnings, which is only slightly higher than average.
That's MOREStephen Gandel, senior editor - Nov 19, 2013 5:00 AM ET
In the Fed's mirror, a stock market bubble is appearing farther away than it may actually be.
Fortune -- At Thursday's confirmation hearing for Janet Yellen, the nominee to be the next Federal Reserve chairman was asked whether she thought there was a bubble in the stock market.
Yellen gave a pretty clear answer: No.
"Stock prices have risen pretty robustly," Yellen said. "But I think that if you look at traditional valuation MOREStephen Gandel, senior editor - Nov 15, 2013 12:04 PM ET
Manager of world's largest hedge fund says Fed has made the market a risky place to invest.
FORTUNE -- Investors need to get ready for disappointment.
Ray Dalio, who manages the world's largest hedge fund, believes stocks will only return 4% over the next decade. But that might not be bad compared to other assets. Dalio says bond investors will do worse.
The Bridgewater Associates head made the downbeat investment projections on Tuesday MOREStephen Gandel, senior editor - Nov 12, 2013 11:50 AM ET
The market isn't reacting to certain economic news as we might expect, but that doesn't mean it isn't rational.
By Mohamed A. El-Erian
FORTUNE -- So, good news was interpreted as bad news by the markets on Thursday while, on the very next day, good news was indeed good news? Are markets really that fickle? Are convictions really that shallow?
It is tempting to respond yes based on the view that, over MORENov 11, 2013 9:02 AM ET
Twitter's investment bankers are telling their clients one thing while Main Street hears a different story.
FORTUNE -- Once again, Wall Street is telling its top-paying clients one thing, and the rest of us are getting a different story. This time it's the Twitter (TWTR) IPO.
According to the Wall Street Journal, analysts who work for Goldman Sachs (GS) and other banks on the IPO, which raised $1.8 billion, have been privately MOREStephen Gandel, senior editor - Nov 7, 2013 5:00 AM ET
The king of limited media has an unlimited budget for research.
FORTUNE -- One of the long-time knocks on Twitter is that it doesn't do a very good job of innovating. Users came up with hashtags. TweetDeck was created by another company that Twitter had to buy. Twitter is reportedly considering shutting down a music site that it launched earlier this year. Go to Twitter.com and you will basically see the MOREStephen Gandel, senior editor - Oct 25, 2013 5:00 AM ET
|November jobs report: Unemployment falls to 7%|
|Five key numbers behind the jobs recovery|
|Where should you put your money now?|
|2 million Facebook, Gmail and Twitter passwords stolen in massive hack|
|Sick days: A luxury many hourly workers don't have|