Have you noticed the share-price performance of the banks lately? They went nowhere for years then fell sharply this year. Lloyds Banking Group (LSE: LLOY) almost touched the 54p I estimated it might back in November when the shares were 73p.

UK-focused big banks such as Barclays (LSE: BARC) and Lloyds have carried more downside risk than upside potential for a while, and still do — even with Lloyds shares at 63p or so and Barclays at around 164p.

Popular firms with cyclical challenges

I’m amazed that Lloyds and Barclays are as expensive as they are now, and more amazed that private investors seem attracted. Sure, if we compare the Banks’ valuations to firms in other sectors, their price-to-earnings (P/E) seem low.

Lloyds’ forward P/E rating is just over eight for 2016 and Barclays’ is 6.3 or so. However, valuing banks is tricky. They’re cyclical to the core, which makes them dodgy buy-and-forget investments. Share prices in the sector rise and fall with profits and cash flow, in line with the macroeconomic cycle. As we move through a cycle big banks tend to suffer from gradual P/E-compression in anticipation of the next peak-earnings event. That leads to valuation indicators working back-to-front, making banks seem like good value at precisely the wrong time in the macro-cycle. Buying a bank when the P/E rating is low and the dividend yield high can be disastrous because such conditions can presage the next cyclical plunge.

Looking at Lloyds’ record of earnings reveals a plateau after rapid recovery since the effects of the worldwide credit crunch. 2015’s earnings were just 3% up and City analysts following the firm expect earnings to slip 8% during 2016. What if this plateau proves to be a peak in earnings for the cycle? If so, downside risk is enormous and upside potential almost non-existent. It’s not pretty when bank earnings collapse on a cyclical downleg, and neither Lloyds’ 5.9% forward dividend yield nor Barclays’ 5% forward yield will help investors when profits and share prices collapse together.

Escalating regulation

Britain regulates its banks through the Bank of England’s Prudential Regulation Authority (PRA). Last decade’s credit crunch, and its fallout that almost toppled the UK’s financial system, shook the political establishment from its torpor. In 2012, the PRA replaced the previous, failed, regulatory regime that hadn’t seen the financial crisis coming.

The organisation aims to monitor firms to ensure they can’t take Britain to the brink of the financial abyss again. It won’t want to be seen slacking, and that’s causing the banks to rethink the way they do business.

I think the PRA will come down on the banks with the full weight of its powers in the coming years. Whether it’s over levels of capital reserves or other areas, the effect is likely to be the big banks finding it harder to earn money by taking risks as they did in the past. And in an apparent effort to dilute the influence of the big banks, the PRA has a secondary objective to facilitate competition in Britain’s banking market. That makes it easier for smaller challenger banks to compete here, just as the big banks are focusing more on their home markets.

Triple whammy

Looking forward, escalating regulation, rising competition and valuation-compression seem set to stymie the total-return performance of Lloyds , Barclays and other big banks until after we see another cyclical bottom. Recent share-price weakness doesn’t count as a cyclical bottom. We could easily see their share prices collapse by 50% or more. Against that risk, the upside potential for banks such as Lloyds and Barclays looks limited to me.

That's why I'm avoiding the banks and looking instead at a package of investment research compiled by our top investment analysts at The Motley Fool.

Our top Fools believe that focusing on high-quality businesses, with an investment time horizon measured in years, can produce market-beating returns. You can judge the strength of these stock market opportunities for yourself by downloading this free report right now.

The paper is called The Fool's Five Shares To Retire On, and it's currently free, and without any obligation whatsoever. To find out about these five shares, which we believe could form the basis for a solid portfolio, click here.

Kevin Godbold has no position in any shares mentioned. The Motley Fool UK has recommended Barclays. We Fools don't all hold the same opinions, but we all believe that considering a diverse range of insights makes us better investors.