Canadian investors, for the most part, hold the Big Five banks in very high regard, and with good reason—the banks have been on a major growth trajectory for nearly 20 years. In fact, over the past 10 years the S&P TSX Bank Index has had annual returns of 8.67% versus the S&P TSX Composite Index, which returned 4.32% annually.

With the banks all having strong yields and a history of dividend growth, investors would have done extremely well by simply holding a portfolio of bank stocks over the past 10-20 years (and many have). Despite this, it is very important that investors do not buy bank stocks based on past returns. Banks should continue to do well going forward, but at the present moment, there are several major headwinds facing Canadian banks.

Most of these headwinds are universal to all of the banks, although every bank is affected by every risk to different degrees. Investors should be very aware of these before buying.

1. The Canadian credit cycle

A credit cycle refers to a period where there debt levels rise as interest rates are low, funds are easy to borrow, and assets that serve as collateral (like homes) are typically rising. This is followed by a period where debt levels fall (known as de-leveraging) as interest rates rise, lending tightens, and asset values fall.

Canada is currently in the late stages of a massive private sector credit cycle as Canada’s debt-to household income recently set a new record of 165%. To make matters worse, the pace has been accelerating with mortgage debt growth now growing at the fastest pace since 2012.

This leads the developed world and is close to the levels seen in the U.S. and U.K. (165% and 167%) before the previous recession. As de-leveraging occurs in Canada, Canadian banks will need to charge more against their earnings (known as provision for credit losses) to protect themselves against loans that are impaired and may not be paid in full. Rising provisions will serve as a headwind for earnings.

2. Overvalued real estate

The CMHC (Canadian Mortgage and Housing Corporation) recently released a report that revealed nine out of 15 cities it monitors are showing overvaluation, and seven out of 15 are showing overbuilding.

These signs are especially evident in Toronto and Vancouver, where housing prices grew 15% and 22% year over year as of March 2016. The CMHC cited “strong evidence of problematic conditions” in Toronto, and it is inevitable—especially as interest rates begin to rise—that Canadian real estate will cool off.

This ties into the above credit cycle concerns, and Canadian banks (especially those with most exposure to Toronto and Vancouver) could see stress on their loan portfolios as well as a major continued slowdown in loans (especially mortgage growth) as consumers borrow less and repay more.

Currently, Royal Bank of Canada (TSX:RY)(NYSE:RY) leads the Big Five in terms of exposure to mortgages and HELOCs in B.C. and Ontario (about 21% of total loans), with Toronto-Dominion Bank (TSX:TD)(NYSE:TD) having much less exposure at only 14%.

It is important to note that real estate affects both the top line (as consumers borrow less due to high debt levels and high prices) and the bottom line (as banks need to put aside more earnings to cover losses).

3. Financial technology

The final threat that affects each bank equally is the threat of financial technology, or FinTech. This refers to the fact that many new firms (both large) and small, are in the process of using innovative digital and customer-focused solutions to take market share in traditional areas of banking such as lending, payments, and wealth management.

A recent report by McKinsey stated that between 20% and 60% of bank profits could be at risk as FinTech firms look to take market share. Canadian banks are investing heavily in technology and partnering with technology firms to deal with this threat, but the increased competition and required investment will undoubtedly be a headwind going forward.

Want more top dividend stocks?

These three top stocks have delivered dividends for shareholders for decades (and even centuries). Check out our special FREE report: "3 Dividend Stocks to Buy and Hold Forever". Click here now to get the full story.

NEW! This Stock Could Be Like Buying Amazon In 1997

For only the 5th time in over 14 years, Motley Fool co-founder David Gardner just issued a Buy Recommendation on this recent Canadian IPO.

Stock Advisor Canada’s Chief Investment Adviser, Iain Butler, also recommended this company back in March – and it’s already up a whopping 57%!

Enter your email address below to claim your copy of this brand new report, “Breakthrough IPO Receives Rare Endorsement.”

Fool contributor Adam Mancini has no position in any stocks mentioned.