Back in June 2011, we wrote a white paper titled “Phantom Indexes”  which revealed that major market indexes including the Dow Jones Industrial Average, S&P 500, NASDAQ 100 and Russell 2000 were all being calculated intraday from only a subset of trades:

“In a post Reg NMS world, fragmentation amongst market centers has reduced the amount of trades that occur on the primary exchanges. The primary market alone is no longer a complete enough source of data when calculating an index value since it represents only about one in four trades. Index suppliers must adjust their methodology to accurately reflect all trades intraday in a timely manner.”

At the time, we figured that the index providers would adjust their calculations once we went public with this information.  We couldn’t imagine that they were intentionally calculating index values with only 25% of the trades available.  To make sure the investing public knew about this issue, we gave an exclusive interview to Bloomberg television where we highlighted our findings. We also spoke about the issue at industry conferences and brought it up with our meetings with regulators.  Unfortunately, the index providers refused to change their calculations and to this day are calculating the indexes with only primary market data.  

However, on October 15th, we finally received some good news on this issue – S&P Dow Jones Indices issued a consultation paper where they addressed the phantom indexes:

 S&P Dow Jones’ U.S. Equity Indices

S&P Dow Jones Indices (S&P DJI) is conducting a consultation with members of the investment community on the calculation of real time indices, as well as procedures for index calculations during market disruptions. The consultation relates to equity indices listed and traded in the U.S.

 Calculation of Real-Time Indices

Currently, real-time S&P Dow Jones’ indices, including the S&P 500 and the Dow Jones Industrial Average, use prices from the primary exchange for each stock. A stock listed on the NYSE is priced with NYSE prices; a stock listed on NASDAQ is priced with NASDAQ prices. The Consolidated tape includes prices from both primary exchanges and other trading venues.

1. Should the Consolidated tape be used rather than prices from the primary exchange for real-time indices?

If real-time calculation is changed to the Consolidated tape, closing prices will continue to use the closing price on the primary exchange for each stock. 

We think the answer to question #1 should certainly be the consolidated tape.  Indexes should be calculated based on every trade involving a component that crosses the consolidated tape, which includes trades from non-primary exchanges such as BATS, DirectEdge and NYSE Arca as well as off exchange venues.  We intend to respond to S&P Dow Jones Indices with our opinion.  If you feel the same way, please send a quick email to index_services@spdji.com  and let them know your opinion.  

We’re not sure why S&P Dow Jones decided to issue this consultation paper now, but suspect that ETF Flash Crash of August 24^th was the stimulus for them to finally act.  On that extremely volatile day when the NYSE invoked Rule 48 and delayed the opening of numerous stocks, indexes were not being calculated correctly and investors were making decisions on incorrect information.