Conventional vs Portfolio Lenders, and Recast vs Refinance

Who this page is for

You are looking at homes in the Bay Area and trying to make sense of your financing, and the advice you are getting does not quite fit your situation. Maybe you are self-employed, paid from abroad, new to the country, or buying above the usual loan size, and a bank has already told you no. This page is for you, and the goal is to show you that a no from one lender is often just a sign you are talking to the wrong category of lender.

Two categories of lender

Most buyers borrow from a conventional lender, also called a conforming lender. Conforming means the lender fits you into the standard boxes set by Fannie Mae and Freddie Mac, the two large agencies that buy mortgages from lenders. The lender writes your loan to those rules, then typically sells the loan to one of the agencies within a few months and recovers its capital, which it lends out again. Because the lender does not plan to hold your loan for thirty years, its main job is to confirm that you pass the standard boxes.

A portfolio lender works differently. It keeps the loan in-house, on its own balance sheet (its own books), and does not sell it to the agencies. Because the portfolio lender lives with the risk for the life of the loan, it sets its own rules and looks harder at each borrower. That extra scrutiny is the tradeoff for the flexibility a portfolio lender can offer, which is the part that matters for buyers who do not fit the standard boxes.

Why the cheapest advertised rate can mislead

The lowest rate you see advertised almost always comes from a conventional lender, and there is a structural reason for it. Because that lender intends to sell your loan, it can sometimes be more forgiving on an edge case, as long as the file still passes the Fannie and Freddie boxes. A portfolio lender that plans to hold the loan does the opposite: it scrutinizes each borrower harder, because it carries the risk itself. The cheapest quote is only cheap if you actually qualify under that lender's rules.

One thing has been true of every lender since the 2008 financial crisis: income has to be verified the same way, no matter how long you have banked somewhere or how much you keep on deposit. A large balance sitting in your own account at the same bank buys you no underwriting accommodation (no easing of the approval rules). The lender still has to document where your income comes from and that it is likely to continue. Knowing that going in saves a lot of frustration later.

Who often needs a portfolio or non-QM lender

Some buyers do better with a portfolio lender or a non-QM lender. Non-QM means the loan does not meet the federal Qualified Mortgage rules, the safe-harbor standards (the borrower-protection rules a loan can satisfy to be treated as lower-risk) that conforming loans are built around. Non-QM loans are funded by specialty lenders rather than the Fannie and Freddie pool, which is what makes them more flexible. A few common profiles tend to land here.

Self-employed buyers who run their income through their business and minimize their taxable income often look weak on paper even when they are not. A bank-statement loan can help: instead of two years of tax returns, the lender underwrites roughly a year of bank deposits to estimate real cash flow. Buyers paid from foreign income, where the money and the employer sit outside the US, frequently need a lender set up to document that. Jumbo buyers need one too: a jumbo loan is simply a mortgage larger than the conforming limit, the ceiling above which Fannie and Freddie will not buy the loan (the limit changes and varies by county, so confirm the current figure with your lender). And recent immigrants without a long W-2 history (the standard US wage-and-tax statement an employer issues each year) or an established US credit record often fit a portfolio program better than a conforming one. The tradeoff across all of these is consistent: more flexibility, usually in exchange for a higher rate, a larger down payment, or both.

If one bank said no

A no from one bank is one of the most misread moments in the whole process. Many newcomers and self-employed buyers who get turned down were never unqualified. They were simply the wrong profile for that one lender's product shelf, and a single bank can only offer what is on it. That is a matching problem, not a verdict on you.

The fix is to match your profile to the right category of lender, which is often easier through someone with access to many loan programs than through one bank selling its own products. This matters especially if you are buying in a second language, are paid in ways a standard W-2 worker is not, or are early in your US financial history. The loan that fits you may well exist; it just may not exist at the first place you asked.

Recast vs refinance

These two get confused, and they do very different things. A recast is when you pay a lump sum toward your loan principal (the balance you still owe) and the servicer (the company that collects your monthly payment) re-amortizes the loan, meaning it recalculates your monthly payment downward over the remaining term. Your interest rate does not change. There is usually a small one-time fee, and typically no new appraisal and no new credit pull. A refinance is the opposite in spirit: it replaces your old loan with an entirely new one, with a new rate, a new appraisal, a new credit pull, and a fresh set of closing costs.

When does each one win? A recast is the move when you already have a low locked-in rate you want to keep and you have come into cash, from a bonus, a sale, or equity in another property, and simply want a lower monthly payment. A refinance is the move when the goal is a different rate, a cash-out (replacing your loan with a larger one and taking the difference in cash), or a change in the loan's terms. In a higher-rate stretch, many owners with a low existing rate prefer a recast precisely because it lowers the payment without surrendering the rate they already have. Lenders vary on whether they allow recasts and on the minimum lump sum, so confirm the rules with your servicer.

The practical close

The point of all of this is simple: the right loan is a matching problem, not a contest for the lowest billboard rate, and the answer depends on your specific income, history, and goals. If a bank has told you no, that is usually the start of the conversation, not the end of it.

If you want an introduction to a mortgage professional who can run your specific profile, message me. Every situation is different, and the right next step is to look at yours directly rather than guess from a general rule.

Lily is a REALTOR, not a lender, so any loan specifics, rates, limits, and program rules should be confirmed with a licensed mortgage professional.

FAQ