More than any of its large rivals, Lloyds (LSE: LLOY) has moved since the Financial Crisis to reorient itself as a domestic-focused retail bank without sprawling global branches or a large investment bank. While share price performance may not reflect it, this about-face has paid off as capital buffers, performance metrics and risk levels are all in better shape than large competitors’. But the rapid expansion of small challenger banks, while currently no more than a nuisance, has the potential to force down margins and take market share.

Slow GDP growth combined with Lloyds’ sheer size (it now has roughly 20% of the UK mortgage market) means it will be difficult for Lloyds to dramatically improve its top line going forward. So, any gains the challenger banks make will likely be at the expense of Lloyds and other high street giants.

This long-term problem as well as the short term issues of continued PPI claims and high operating costs are enough to give me pause on Lloyds. Although analysts are expecting dividend yields to reach 6.5% this year, shares trading at 1.04 times book value with little possibility of dramatic growth make me believe share prices will likely continue to stagnate.

Like a Virgin

Challenger bank Virgin Money (LSE: VM) seems to have no problem growing fast and analysts expect this to continue for some time. Controlling 3.4% of domestic mortgages and 2.5% of the credit card market gives Virgin lots of room for high organic growth.

This high growth potential and proven ability to keep costs low give Virgin quite a leg up on staid competitors such as Lloyds. Return on equity (RoE) in 2015 was 10.9%, which is below Lloyds’ adjusted number, but a massive improvement on the 7.4% put up in 2014 and 2.3% in 2013 after the Northern Rock purchase. Management expects to crank RoE up into the mid-teens by 2017, which combined with significant revenue growth prospects leaves me quite bullish on Virgin Money.

Metro’s march

Metro Bank (LSE: MTRO) may have only gone public in March, but it has already snapped up significant numbers of depositors who are attracted by its customer-friendly approach to banking. Signing up an additional 62,000 customers in the first quarter of 2016 alone helps explain why deposits rose 15% quarter-on-quarter and revenue 11%. This additional revenue helped shrink losses for the period to £7.9m.

If growth continues at this pace, management expects to break even this year and finally turn a profit in 2017, which isn’t bad for a bank started in 2010. However, it’s not all roses for Metro Bank as the company appears to be behind schedule on its proposed branch-opening schedule. With 41 current branches and £5.9bn in deposits, it’s well below its 2020 target of 200 branches and £50bn in deposits. But, given the company’s torrid pace of growth, successful history of the founder’s American bank and large market to disrupt, Metro Bank is certainly higher on my watch list than Lloyds.

Of course, the low-risk nature of domestic retail banking also means each of these companies isn't very diversified and will suffer greatly from any downturn in the economy. If you find defensive companies with global diversification are a more attractive proposition, I recommend reading the Motley Fool's latest free report, Five Shares To Retire On.

These five consumer giants have rewarded shareholders over decades with high dividends, steady returns and peace of mind that the bottom wouldn't fall out on share prices.

To read your free, no obligation copy of this report, simply follow this link.

Ian Pierce has no position in any shares mentioned. The Motley Fool UK has no position in any of the shares mentioned. We Fools don't all hold the same opinions, but we all believe that considering a diverse range of insights makes us better investors.