Families that have created or inherited considerable wealth are incredibly blessed, but such blessings can come with a wide range of challenges and responsibilities. Overseeing the often-complex lifestyle of the current generation involves can involve responsibility for homes, boats, planes, large support staffs, curated art and other substantial fixed assets, not to mention stewardship of ongoing corporate or other entities. Additionally, these families are seeking to preserve both their wealth and their beliefs and principles to benefit future generations and, in many cases, fund broader community and philanthropic aims.
These “family affairs” can become enormously complicated, and many multigenerational families have created family offices to manage these complex financial and investment matters. Recent years have brought some changes to the rules governing such offices, and how they are taxed, leading many families to reconsider (or consider for the first time) questions about whether to start a family office, or whether to restructure an office already in place.
If your family is thinking about these questions, this white paper may help. In it, we provide some background on recent relevant changes in tax law, and we offer a framework that your family can use to evaluate any restructuring decision in a comprehensive manner.
BACKGROUND
Decisions about whether to start a family office, and how to structure such offices, are multilayered. The thought process should be driven primarily by the goals and desired outcomes of the family: What do they want to achieve? What skills and competencies are most important given their priorities? Can an existing firm provide what they need, or will a dedicated team do a better job? Do they have the resources to support a high-quality operation?
Of course, it can be expensive to run a family office. Expenses typically include salaries and other costs for staff, fees for legal/accounting services, direct and indirect investment expenses, technology and infrastructure, and a variety of other requirements specific to a family’s needs.
Historically, tax treatment—specifically, the ability to deduct various expenses of a family office for tax purposes—was often the critical factor that determined whether a captive family office made financial sense. Recent developments in tax law have raised the stakes for families weighing family office structure decisions and helped to clarify the income tax landscape for those already operating a family office.
- Prior to 2017, families could deduct investment management fees (typically the largest cost for a family office) from their income taxes, but the 2017 Tax Act eliminated that deduction along with all miscellaneous itemized deductions. However, deductions for trade and business activities are still available to these families.
- Two recent court cases (between the Internal Revenue Service (IRS) and the Lender and Hellman families, respectively) have provided a template for structuring a family office so that it may qualify as a trade or business in the eyes of the IRS.
Because the tax savings from deducting family office expenses can be meaningful, many family offices are now tempted to restructure in a manner that seems to have been blessed by the courts. To be clear, one of the primary motivators for families in recent years—the provision that eliminated deduction of investment management fees in the 2017 Tax Act—is due to sunset in 2025. Should that come to pass, the financial case for restructuring one’s family office will become less compelling for many families.
However, tax treatment of expenses is only one part of a complete analysis needed to make the right family office choices for a given client. In this article, we seek to provide some background on the recent tax law developments, and then walk through a three-step framework that families can use to think through their options for structuring a family office—focused on the financial impact and feasibility of a new structure, and the family’s appetite for that new structure. Ultimately, each family and its trusted advisors need to make its own determination. No two families are the same, and even with a clearer tax environment, two families may examine the same quantitative scenario and make very different decisions about it.
RELEVANT LEGAL ISSUES—A BRIEF REVIEW
The 2017 Tax Act. Prior to 2017, investment management expenses were deductible (with some limits) as miscellaneous itemized deductions. Investment expenses usually represent a large proportion of family office costs, and their deductibility generally mitigated the net costs of running such an office by a meaningful amount.
The 2017 Tax Act eliminated miscellaneous itemized deductions entirely, and as a result, families are no longer able to deduct investment management fees from their taxable income. This immediately made the deductibility of business expenses an area of focus for families. If a family can structure its family office so that it qualifies as a trade or business, the costs of that office would still be deductible. (Such a step would also benefit families that could not deduct investment fees before 2017, because of the Alternative Minimum Tax (AMT).)
As might be expected, when families began seeking to deduct family office costs as business expenses, they ran into objections from the IRS, and matters began escalating to the U.S. Tax Court for resolution. Two recent cases involving the Lender and Hellmann families (whose wealth stems from the well-known Lender’s Bagels and Hellmann’s grocery brands, respectively) are now serving as early precedent.
The Lender Case: The Lender family office has been in operation for more than 25 years. After the family sought to deduct the costs of the office as business expenses, its case vs. the IRS ended up in court. In its decision (Lender Management v. Commissioner, TC Memo2017-246), the U.S. Tax Court concluded that the costs are in fact deductible, because the family office was:
- Structured like a professional advisory firm. The family office is a separate entity, a fact reinforced by the ownership structure of two investment LLCs that held private equity and hedge fund investments, respectively. The family office was an advisor, not a parent, of these LLCs. Its ownership interest in the LLCs was less than 10%, while family members individually owned more than 90% of the LLCs. (Note: LLC structures such as these are generally available to qualified purchasers or accredited investors only.)
- Staffed like a professional advisory firm. Keith Lender, the CIO, holds an MBA and works 50 hours per week for the family office, alongside a non-family member who is the CFO. Other staff include an office manager and several part-time employees. The office manages outside vendors and holds formal meetings with various members of the Lender family—in short, it is run like a real business.
- Compensated like a professional advisory firm. The family office receives payment for investment advice, as well as a share of the carried interest or “profits interest” of the two LLCs (contingent on the performance of the underlying investments held in the LLCs).
Notably, the Lender case did not rule on several matters pertaining to the family office’s status as a business—for example, whether it might be subject to additional business taxes, subject to mark-to-market rules, or if the carried interest it collects from the LLCs might be recharacterized as ordinary income vs. long-term capital gain. It also did not resolve potential estate tax implications (i.e., if trusts that benefit a younger generation are allowed to be owners of the management entity, it could allow for additional wealth transfer).
The Hellmann Case: A group of Hellmann family members own and operate GF Family Management LLC. They took the position that the Lender decision cleared the way for their family to deduct its family office expenses as well. The IRS objected.
There are four Hellmann family members related to the court case; they all reside in Atlanta, have a good relationship and the family office made investment decisions for the group as a whole. The Lender family members are geographically dispersed, in some cases do not get along or do not know each other at all, and the family office makes decisions for each family member separately.
There are four Hellmann family members related to the court case; they all reside in Atlanta, have a good relationship and the family office made investment decisions for the group as a whole. The Lender family members are geographically dispersed, in some cases do not get along or do not know each other at all, and the family office makes decisions for each family member separately.
The Hellman family members owned 99% of the relevant investment partnerships, and each of the four held a 25% interest in the family office. In contrast, the Lender family office owned a meaningful portion of the LLCs it managed, and most of the family members who invested in the LLCs did not have an ownership stake in the family office.
From these two cases, a new understanding is beginning to emerge about the criteria that the Tax Court may use in the future to determine whether a family office qualifies as a trade or business.
A FRAMEWORK FOR EVALUATING FAMILY OFFICE OPTIONS
Given these tax considerations, many family offices will seek to structure themselves in a manner that qualifies as a trade or business under IRS rules. Again, we want to remind our clients: Many family offices have pursued a new structure because the 2017 Tax Act eliminated deductions for investment management fees. That provision of the code is due to sunset at the end of 2025. If families are once again allowed to deduct that expense, it may change their financial calculus as they consider potential changes to their family office.
Furthermore, restructuring one’s family offices may create new problems while solving old ones. If their family office qualifies as a business, it may also be required to register with the SEC as a registered investment advisor (RIA), as outlined in the Investment Advisors Act of 1940 (the “Advisors Act”). SEC registration creates an entirely different set of costs, burdens and risks. So, the theoretical ideal for families is to thread the needle, so to speak, and structure the family office so the IRS recognizes it as a business and the SEC recognizes it as exempt from registration.
Establishing or restructuring a family office to fit the Lender template is much more challenging in reality than on paper. These structural decisions generally have impacts that extend far beyond the tax benefits being pursued.
Families who are considering their options may follow this three-step process to help ensure a more comprehensive evaluation:
Step 1: Evaluate current expenses (Financial benefit of new structure)
Families should quantify and evaluate their current costs, including staff, retained legal/accounting services, direct and indirect investment expenses, technology, infrastructure and any others. They should further review their current effective tax rate, and then look at how the effective rate could change under a Lender-like structure. State taxes are important to examine as well—for example, many states tax realized capital gains no differently than ordinary income.
This first step should help a number of families make a quick determination. Every family is different, but when aggregate expenses are less than $1million, often it may not make financial sense to form or restructure a family office—and for such families, the option of outsourcing may now be more attractive than it was a few years ago. Additionally, families that already have operating businesses may find that the incremental deductions are not enough to justify a change.
Step 2: Evaluate family’s fact pattern (Feasibility of new structure)
If the tax savings under a Lender-like structure would be compelling, the next step is to determine if such a structure is practical or even feasible. There are several hurdles to clear.
First, does the family’s situation serve as a logical basis for the structure? A “Lender-like” model makes sense for a family of multiple branches containing multiple generations, with independent advice for different family members, and outside investors in the family office. However, for a family of two generations within the same branch, served collaboratively, it would be difficult to defi ne the family office as an actual business in the eyes of the IRS—not to mention or to justify the many added costs of an intricate family office structure.
There are additional hurdles to qualification as a “trade or business.” For example, it is not always feasible to compensate staff based on a profits interest in the family office. Further, the ideal structure may come at a cost. Many families look to create a C-Corp, for example, but in such an instance it is desirable to keep the business at breakeven to avoid the potential for double taxation of profits that a C-corp creates (i.e., first at the corporate level, and again at the shareholder level once distributed). Finally, the structure may require new or different management skills, and it is important for families to know whether that talent is available at reasonable cost.
Step 3 – Evaluate likelihood of success (Durability of new structure)
Finally, if a Lender-like structure appears both financially attractive and achievable in practical terms, the family members still need to take an honest look at its likely impact on their day-to-day lives and determine whether they would be comfortable adhering to a new set of rules over time. A new family office structure, as mentioned, will likely require new costs and may require new staff and leadership to operate properly, and family members may not be unified in their desire to take on these additional burdens. Managing and calculating the profits interest in a family office is complicated and may very well create tension among family members. Any changes in how the family members meet and interact with the family office and each other may require adjustments as well.
No two families are identical, and every family will respond differently to a potential change in how its investment assets are managed, and to any downstream adjustments that are required to adapt to a new management structure. In all cases, a complete understanding of the anticipated changes is critical to the long-term success of the office.
CONCLUSION
Families should keep in mind that much of the law discussed in this article is fairly new, and precedent is still emerging. The Lender case, for example, represents the ruling of one Tax Court judge; it may serve to guide future decisions but that is not guaranteed. The risk of being wrong—or even of being right but having to pay the legal expense of proving it in court—is a meaningful one.
The issues involved with operating a family office have always been complex, and the recent changes in tax law have only added to that complexity. We expect it to be challenging for families to find the right solution that satisfies the demands of the IRS, the SEC and their own family, and the topic is worthy of extended and comprehensive consideration by families and their advisors. While maximizing the financial efficiency of the family office (by capturing allowable tax deductions) is important, it has become more challenging to achieve. For families with family office expenses over $1 million and a business that serves multiple family branches (and even some outside investors), an initial review through our suggested three-step process with their advisors is the prudent first step. The financial considerations are just one part of a complex analysis to determine whether a family should form a family office or how best to structure it. 
1Investment management expenses are sometimes referred to as “Sec. 212 expenses,” and business expenses as “Sec. 162 expenses,” based on the relevant sections of the Internal Revenue Code that cover the tax treatment of these expense categories.
The views expressed are those of Brown Advisory as of the date referenced and are subject to change at any time based on market or other conditions. These views are not intended to be and should not be relied upon as investment advice and are not intended to be a forecast of future events or a guarantee of future results. Past performance is not a guarantee of future performance and you may not get back the amount invested.
Any business or tax discussion contained in this communication is not intended as a thorough, in-depth analysis of specific issues. The information provided in this material is not intended to be and should not be considered to be a recommendation or suggestion to engage in or refrain from a particular course of action or to make or hold a particular investment or pursue a particular investment strategy, including whether or not to buy, sell, or hold any of the securities mentioned. It should not be assumed that investments in such securities have been or will be profitable. To the extent specific securities are mentioned, they have been selected by the author on an objective basis to illustrate views expressed in the commentary and do not represent all of the securities purchased, sold or recommended for advisory clients. The information contained herein has been prepared from sources believed reliable but is not guaranteed by us as to its timeliness or accuracy, and is not a complete summary or statement of all available data. This piece is intended solely for our clients and prospective clients, is for informational purposes only, and is not individually tailored for or directed to any particular client or prospective client.