Vanguard founder John Bogle took the stage Tuesday at the CFA Institute annual conference and took aim at the foes of indexing and cheered the soon-to-be-implemented fiduciary rule.

Speaking to a crowd of thousands at the jam-packed Pennsylvania Convention Center, Bogle predicted that 10-year equity returns could be as low as 4 percent annually, based on the current price-to-earnings compression in the stock market.

"That could be an impending disaster" for pension funds with long-term liabilities that are assuming much higher returns in the future, he said. "That's why low fees matter" today and going forward.

In his speech, Bogle noted that, since 2007, equity index mutual funds have enjoyed capital inflows of $1.8 trillion, while their actively managed counterparts have been hit with $800 billion of capital outflows.

However, "index-fund management is heavily concentrated among three giant, trillion-dollar money managers that together hold some 20 percent of U.S. stocks," Bogle said. "The impact of all this concentration remains to be seen."

Those three firms are Blackrock, State Street Global Advisors, and Vanguard, with a total of about 21 percent of the entire indexing market.

"That concentration is concerning to me," Bogle told the audience of chartered financial analysts, many of whom had traveled from as far as Africa, Asia, and Europe for the convention. He speculated that concentration could even invite scrutiny from regulators.

After his remarks, Bogle took questions through moderator Ted Aronson, cofounder of Philadelphia-based AJO, a quantitative-investment firm.

Bogle said that, when founding Vanguard, "I didn't intend to create a colossus," but that he should have realized investors would take advantage of a great deal. Vanguard currently manages more than $4 trillion of investor assets, a number he said he never imagined the firm would amass.

Nonetheless, he said, "passive investing is here to stay," overseeing $5.6 trillion in assets, vs. $10.5 trillion in actively managed funds.

"Will indexing ever get to 50 percent of the market? Maybe, but that could take 10 to 15 years. And we'll never get to 75 percent," he said.

As for exchange-traded funds, Bogle said the super-leveraged versions, particularly those investing in high yield or using up to four times leverage — or borrowed money — are "asinine."

Separately, Bogle said he hoped the Department of Labor's conflict-of-interest, or "fiduciary," rule would be implemented. The new rule is set to go into effect in a few weeks after a two-month delay.

On Monday, U.S. Labor Secretary Alexander Acosta wrote a Wall Street Journal op-ed piece stating that he had found "no principled legal basis" for further delay. As a result, provisions that close loopholes in the definition of fiduciary investment advice will go into effect June 9, as will standards requiring those operating with conflicts of interest to act in customers' best interests, charge reasonable fees, and avoid misleading statements.

The Consumer Federation of America's director of investor protection, Barbara Roper, said:  "We applaud Secretary Acosta for recognizing that respect for the rule of law demands that implementation of the rule be allowed to go forward without further delay. Unfortunately, his comments on the substance of the rule – developed without any meaningful consultation with rule supporters – suggest that Secretary Acosta has prejudged the outcome of the reconsideration and may plan to gut core provisions of the rule that are essential to its effectiveness."

"Secretary Acosta states in his op-ed that, 'This administration presumes that Americans can be trusted to decide for themselves what is best for them.' If that's what he believes, Secretary Acosta should give the fiduciary rule his unstinting support," Roper said.

"Americans have already clearly indicated that they believe all financial professionals should have to act in their customers' best interests," she added. "Across party lines, they have voiced strong support for the department's fiduciary rule."