There is quite possibly no company more admired in Canada than Toronto Dominion Bank (TSX:TD)(NYSE:TD). The bank has been on an incredible run over the past 10 years, returning over 10% per year to shareholders – well ahead of its peer group average.

TD famously avoided the U.S. subprime crisis despite an aggressive expansion into the United States. The bank consistently ranks at the top of customer satisfaction surveys. Outgoing CEO Ed Clark was recently named Canadian Business CEO of the year. So what’s not to like?

1. Growth comes in the wrong place

Of all the banks, TD is the one with the biggest focus on the United States; most people don’t realize that TD actually has more branches in the U.S. than in Canada. But the bank’s Canadian operations make nearly three times as much income. Return on equity, which most recently was above 40% in Canada, remains under 10% in the United States.

Yet with Canada saturated, it is in the United States where TD will be expanding the most. But the United States is far more competitive than the Canadian market, meaning it will be a constant struggle to generate sufficient returns. And with returns hard to come by, TD’s growth won’t create much value for shareholders.

2. Worries about Canada

The headlines are constant: Canada’s real estate market is overheated, and consumer debt levels are too high. And while those claims are up for debate, there is no denying that Canada is due for a slowdown at the very least. And that will impact loan growth for all of Canada’s banks, including TD.

As the bank’s name implies, TD is heavily concentrated in Ontario, which accounts for 56% of the bank’s loans outstanding in Canada. This comes with additional risk. Not only are there worries about Toronto’s condo market, but the rest of the province is struggling as manufacturers continue to move to other jurisdictions.

3. The most expensive

At 14 times earnings, TD is the most expensive of Canada’s big five banks. This is due to the bank’s excellent reputation, as well as hopes about growth in the United States. TD’s shares have increased 23% in the last year alone.

Foolish bottom line

A common saying among successful investors is taken from Warren Buffett: “Be greedy when others are fearful and be fearful when others are greedy.” And with TD’s popularity seemingly at an all-time high, this is a great time to apply that principle.