United States: Ballard: The State Tax Implications Are Clear

It has been said that "rules of procedure exist for the sake of substantive law and to implement substantive rights, not as an end in and of themselves. "Procedure should always be indeed the ‘handmaid of justice.’"¹ In Ballard, the United States Supreme Court reminded the United States Tax Court – and, hopefully, all administrative and judicial forums that hear tax matters – that a fair, transparent procedure is critical to sound tax administration and determination of substantive rights.² The state tax implications are clear – secret reversals of internal administrative law judge decisions are improper and should cease. Better yet, systems that utilize internal hearing officers should be replaced with independent tax dispute resolution systems.

Justice Ginsburg authored the Court’s majority opinion, joined by six other justices. The Court held that the Tax Court’s rules do not authorize the Tax Court’s concealment of the report issued by a Special Trial Judge ("STJ") and the collaborative process that results in the issuance of a single decision of the Tax Court Judge which, invariably states that it "agrees with and adopts" the opinion of the STJ. The Court found that "[i]n comparison to the nearly universal practice of transparency in forums in which one official conducts the trial (and thus sees and hears the witnesses), and another official subsequently renders the final decision, the Tax Court’s practice is anomalous."³

Ballard involved the assertion of fraud by the IRS against three individuals – Claude Ballard, Robert Lisle, and Burton Kanter – in connection with a purported kick-back scheme allegedley orchestrated by Kanter, a tax attorney and law school professor, with Ballard and Lisle, who had a long-standing relationship with Prudential Life Insurance Company of America to "sell influence" with regard to its real estate department.⁴

After a lengthy trial where 55 witnesses were called and over 6,000 pages of transcript were taken over a five-week period, the STJ assigned to hear the case had allegedly issued a report concluding that fraud had not been established by the IRS.⁵ However, that report was never issued. Instead, the Tax Court judge assigned to render the decision concluded that fraud had been established, yet stated that "the Tax Court Judge agrees with and adopts the opinion of the Special Trial Judge."⁶

Although the STJ expressed his doubts about the IRS’s establishment of fraud from the bench,⁷ had it not been for conversations that counsel for one of the taxpayers had with two Tax Court judges who "were so offended by the unorthodox ‘fact-finding’ process used in this case that they revealed this off-the-record reversal to a lawyer who was, in turn, willing to make a record that neither the Tax Court nor the IRS has challenged,"⁸ the taxpayers might not have known about the apparent about-face. Counsel for Kanter and Investment Research Associates filed a declaration in the Tax Court stating, inter alia, that he had been told that "substantial sections of the opinion were not written by [the STJ], and that those sections containing findings related to the credibility of witnesses and findings related to fraud were wholly contrary to the findings made by [the STJ] in his report. The changes to [the STJ’s] findings relating to credibility and fraud were made by [the Tax Court judge]."⁹ The declarant also stated that he was told by a Tax Court judge that "the changed sections of the report were written in such a way so that wherever there was the ‘slightest issue of doubt, it went against’ the Petitioners."¹⁰ The taxpayers moved to gain access to the STJ’s "reports, draft opinions or similar documents, prepared and delivered to the [Tax] Court pursuant to Rule 183(b)."¹¹ That motion was denied. A second motion was then filed by the taxpayers requesting that the STJ’s report be placed under seal and be included in the record on appeal.¹² That motion was also denied. Another motion was then filed asking that the Tax Court reconsider its earlier denial of access to the report or requesting that another trial be held. That motion was denied as well, with a statement that the Tax Court decision adopted "the findings of fact and opinion" of the STJ.¹³

As a result of the rule which requires that appeals from Tax Court decisions be taken to the circuit court in which the taxpayer resides,¹⁴ Ballard’s appeal was heard by the Eleventh Circuit, Lisle’s by the Fifth Circuit, and Kanter’s by the Seventh Circuit. Ballard filed a writ of mandamus in the Eleventh Circuit seeking that the Tax Court be required to turn over the STJ report; the writ was denied. The Eleventh Circuit did not find anything amiss with the failure to include a copy of the STJ’s report to the parties and proceeded to uphold the Tax Court’s decision based on its deferential standard of review concluding that "we are not left with ‘a definite and firm conviction that a mistake has been committed.’"¹⁵

The next of the cases to be decided by the circuit courts was with respect to the now deceased Kanter, who the Seventh Circuit pointed out "participated in countless extremely large and lucrative business ventures, reported a negative adjusted gross income each year on his federal tax return and paid no federal taxes" and had, not surprisingly, been audited by the IRS "virtually, if not literally, every year since Richard Nixon was President."¹⁶ The Seventh Circuit concluded, as had the Eleventh Circuit, that the fact that the Tax Court’s statement that the report it adopted was the STJ’s rendered moot Kanter’s arguments surrounding the failure to provide the STJ’s report.

Seventh Circuit Judge Cudahy issued a strongly-worded dissenting opinion – cited approvingly numerous times by the Supreme Court – concluding that it was wrong for the Tax Court to withhold the STJ report from the parties. While Judge Cudahy agreed with the majority’s opinion that an STJ’s report was not required to be reviewed under a clearly erroneous standard, that Rule 183 was not violated if a quasi-collaborative effort takes place resulting in a revised STJ report, and that Rule 183 did not require that the STJ’s report be produced, he concluded that the current Tax Court procedure violated due process.¹⁷

The Fifth Circuit heard Lisle’s case and reversed the Tax Court’s holding with respect to fraud since it concluded that the IRS did not prove its case by clear and convincing evidence.¹⁸ With respect to the failure to provide the STJ’s report to the parties, the Fifth Circuit adopted the reasoning of the Seventh and Eleventh Circuits and ruled that such failure did not violate Lisle’s due process rights.¹⁹

When created in 1924 as the Board of Tax Appeals, the Tax Court was an agency in the Government’s executive branch.²⁰ In 1969 Congress formed the current Tax Court as a legislative tribunal under Article I of the Constitution.²¹ Although the distinctions between Article I tribunals and Article III courts are not often evident, but often disputed, the Constitution requires Article I tribunals to be inferior to the Supreme Court, and its judges are not required to be given life tenures and salary protection as are Article III judges.²² The Tax Court has 19 judges who are appointed by the President for 15-year terms.²³

STJs are specially appointed by the Chief Judge of the Tax Court to hear cases.²⁴ The Congressional grant of authority to appoint STJs has been upheld by the Court.²⁵ They earn 90 percent of a Tax Court judge’s salary and "serve at the pleasure of the Tax Court [and] lack[] the independence enjoyed by regular Tax Court judges and the prerogative to publish dissenting views."²⁶

Tax Court Rule 183 directs that, where an STJ hears the case but the Tax Court renders the decision, the STJ "shall submit a report, including findings of fact and opinion, to the Chief Judge, and the Chief Judge will assign the case to a Judge . . . of the Court."²⁷ The Tax Court judge to whom the case is assigned is required to give "[d]ue regard . . . to the circumstance that the Special Trial Judge had the opportunity to evaluate the credibility of witnesses [and that the STJ’s findings of fact] shall be presumed to be correct."²⁸

Prior to 1983, the STJ’s report was required to be served on the parties, who could file exceptions to the report. In 1983, Tax Court Rule 183(c) was amended with respect to the review of STJ reports. Since 1983, Tax Court practice has been to have the STJ and Tax Court judge "collaborate" on the decision. And, since 1983, all decisions "agree and adopt" the STJ report; no decision has been issued that modified or reversed an STJ report.

While the Court did not directly address the deference standard to be applied to STJ reports, its analysis suggests that substantial deference by the Tax Court is critical, particularly with respect to the STJ’s credibility findings. The majority’s opinion spoke of "deference" to the STJ report as being required under Rule 183, citing Stone ²⁹ for the proposition that STJ findings carry "‘special weight insofar as those findings are determined by the opportunity to hear and observe the witnesses,’" suggesting that Stone might shed some light with respect to the level of deference and its standard of review.³⁰ The dissenting opinion merely stated that the Tax Court had "complied with whatever degree of deference is required by Rule 183(c)."³¹

In Stone, the D.C. Circuit addressed the deference to be given to STJ determinations by the Tax Court under the predecessor to Tax Court Rule 183(c), Rule 182(d), which contained the identical "due regard" language of Rule 183(c). In that case, the issue was whether the corporation owned by the taxpayers or rather the taxpayers themselves were taxable on the income received by the corporation. The STJ found the taxpayers’ testimony to be credible and rejected the Commissioner’s position that the income was taxable to the taxpayers. The Tax Court rejected the STJ’s credibility findings on the basis of the record and held that the taxpayers were taxable on the income. After noting that "the issues of scope of review often seem to have ‘no more substance at the core than a seedless grape,’" the D.C. Circuit found that the Rule 182(d) language required a "relatively high level of deference," in the court’s view, by a "clearly erroneous standard," it found the Tax Court’s rejection of the STJ’s findings to have been "clearly erroneous."³² The IRS, however, has viewed Stone as erroneous on the basis that it failed to defer to the Tax Court’s interpretation of its own rule.³³

Stone addressed, but did not resolve, the thorny "multi-layered deference" issue, that is, what an appellate court should do when faced with differing decisions between the fact-finder and the intermediate reviewing body. It noted, however, that the majority view was for the court to review the fact- finder’s decision, basically "drop[ping] the intermediate forum out."³⁴

Interestingly, even though the taxpayers in Stone had pled guilty to criminal charges, the STJ found them to be credible witnesses. In reviewing the Tax Court’s reversal of the STJ decision, the D.C. Circuit concluded that credibility (or its absence) was at the heart of the differing decisions, and it undertook to determine whether objective evidence in the record materially contradicted the taxpayers’ testimony. Pyramiding the clearly erroneous standard at each level, it found that since the STJ decision was not clearly erroneous, the Tax Court was clearly erroneous in reversing the STJ’s decision.

In Kanter, the Seventh Circuit rejected the Stone court’s clear error requirement, stating that the adoption of such a threshold "would all but abdicate the Tax Court’s original decisionmaking authority."³⁵ However, Stone provides a proper balance between review of legal issues by the Tax Court, for which de novo review exists, and factual findings (particularly credibility findings), which are to be given substantial deference unless objective proof exists to the contrary. Under Stone, the Tax Court’s original decisionmaking authority remains potent, without denuding the STJ decision of those elements of review over which it has a unique perspective.

Judge Cudahy in his dissent in Kanter emphasized that "clearly erroneous review involves deference to the conclusions of the fact finder – the Tax Court in the present case. This is a deference that the Supreme Court has attributed, in the case of credibility, to the fact finder’s firsthand observations of the witnesses in question. . . . Thus, it is integral to the standard of clear error review [which Judge Cudahy notes is "virtually indistinguishable from the substantial evidence standard"] that there be deference to the credibility findings of the official who has actually heard the witnesses."³⁶

It is difficult to mesh Judge Cudahy’s rejection of the requirement that the Tax Court review STJ findings on a clearly erroneous basis with continuing to recognize the primacy of the role of fact-finder. Further, as Judge Cudahy recognized, based on the Supreme Court’s analysis in Anderson v. City of Bessemer City, N.C.,³⁷ "[i]f we are to give ‘even greater deference’ to the findings of a judge who has heard the witness whose credibility is at stake, we must inevitably give less deference to the judge who subsequently reverses those findings,"³⁸ giving further support to the conclusion that credibility findings are to be reviewed under a clearly erroneous standard.

In Ballard, the Court’s rejection of the Tax Court’s "idiosyncratic procedure" appeared motivated by the "outcome-influencing conclusions regarding . . . credibility"³⁹ that were made by the Tax Court. Additionally, the Court thought "curious" the IRS’s support of the Tax Court’s "concealment" of the STJ’s report.⁴⁰ However, since it is generally taxpayers that have the burden of proof, credibility is more likely to be of paramount importance to taxpayers than to the IRS; in actuality it is therefore not curious at all. Perhaps that is why Justice Ginsburg never received a response from the Commissioner’s counsel to her question, "aren’t there situations where it might be that the special trial judge would call a credibility question in the Government’s favor and then the Government loses the case before the Tax Court judge and might like to know, before it goes to the court of appeals, how solid the credibility findings were?"⁴¹

Credibility (or its absence) can make or break a taxpayer’s case and as the Court, referring to Judge Cudahy’s dissenting opinion in Kanter, noted "the Tax Court’s current practice allows it ‘very easily [to] reverse findings (credibility and otherwise) of [special trial judges] in a manner that is detrimental to the Commissioner as well as to’ taxpayers."⁴² While there may be instances where reversal of such findings might be warranted, departure from such findings (regardless of whether for the taxpayer or the Commissioner) should be the exception and should not occur without a reasoned explanation of the basis for the departure. Since, as the Court has recognized, "only the trial judge can be aware of the variations of demeanor and tone of voice that bear so heavily on the listener’s understanding of and belief in what is said,"⁴³ if an explicit finding of credibility is made, due process concerns should dictate that rejections of credibility findings not be buried, particularly when another court will be reviewing the decision reversing credibility findings and will be affording such decision deference.

The Court found that this "‘extraordinary unanimity’" due to "‘a collegial deliberative process’"⁴⁴ was not authorized under the Tax Court rules. Having decided the case on the basis of statutory interpretation, the Court ducked the due process issue, but suggested in dicta that if the Tax Court rules were amended to provide for the practice it rejected, it would be "subject to appellate review for consistency with the relevant federal statutes and due process."⁴⁵

In Kanter, the Seventh Circuit had concluded that "secreting" the STJ’s report and the Tax Court’s "purportedly quasi-collaborative process would not offend our notions of fundamental fairness, nor would due process require the inclusion of the report in the appellate record to preserve the fairness of our review."⁴⁶ However, Judge Cudahy dissented and would have found a due process violation because of the absence of the STJ report in the record, because such absence precludes meaningful appellate review.⁴⁷ While Judge Cudahy would not have found a due process violation merely because of a lack of a formal degree of deference or if an issue of credibility was reversed without having actually heard the witness, he still noted that "[t]here certainly is some cause for concern that a finding of credibility on a ‘cold record’ as voluminous as the one before us here increases the chance of an erroneous determination."⁴⁸

In reaching his conclusion, Judge Cudahy focused upon two Supreme Court cases, Raddatz, a criminal case, and Universal Camera, a civil case, finding them to be at opposite "end[s] of the due process spectrum.⁴⁹

Raddatz involved a suppression hearing which the district court referred to a magistrate. Based on his credibility determinations, the magistrate found that the respondent had made voluntary inculpatory statements, and he recommended that respondent’s motion to suppress be denied. The district court accepted the magistrate’s recommendation without independently hearing any testimony, and respondent was tried and convicted. The Seventh Circuit concluded that the district court’s failure to personally hear the witness testimony violated respondent’s due process rights. The Supreme Court, focusing on the fact that the statute provided that the district court make a "de novo determination," not have a "de novo hearing," did not require (but did allow) the district judge to rehear the witnesses’ testimony to comport with due process requirements. Both Justice Blackmun’s and Justice Powell’s concurring opinions noted that due process concerns might be implicated, however, had the district court reversed the magistrate on credibility determinations, which Justice Powell found would raise "‘serious questions.’"⁵⁰ The dissenting justices would have found a due process violation in the procedure since "critical issues of credibility can be resolved only by personally hearing live testimony."⁵¹

On the administrative end of the due process spectrum is Universal Camera. In Universal Camera the National Labor Relations Board ("NLRB") reversed the examiner’s findings. The Second Circuit determined that it was bound to follow the NLRB and could not consider the examiner’s report. The Court reversed and found that "the courts must now assume more responsibility for the reasonableness and fairness of [NLRB] decisions than some courts have shown in the past,"⁵² and that courts "accord the findings of the trial examiner the relevance that they reasonably command in answering . . . whether the evidence supporting the [NLRB’s] order is substantial."⁵³

Interestingly, the IRS argued that a non-Article III forum does not need to afford as high a degree of procedural due process protection as that which must be given by an Article III court, and the mere ability of the Tax Court to "receive further evidence" provides sufficient protection under Raddatz.⁵⁴ However, given the considerable overlap between Article III courts, Article I
tribunals, and administrative agencies, the criminal versus civil differentiation may be a more relevant inquiry. Further, as Ballard and numerous other Court decisions suggest, even if substantial deference to credibility findings were not constitutionally compelled, substantial deference to such findings is the only right result.⁵⁵

While it may be expedient for tax tribunals to dismiss the applicability of Ballard as a procedural aberration particular to the Tax Court, the decision highlights how easy it is for tax tribunals to push the procedural envelope to the detriment of taxpayers’ due process rights. The Court understood that the role of the STJ was analogous to that of an administrative law judge.⁵⁶ Time and again, uncontroverted testimony explicitly found by triers of fact to be credible is dismissed without explanation or as "self-serving." However, as the D.C. Circuit Court aptly acknowledged in Stone, a person’s testimony "is almost by definition self-serving."⁵⁷ Therefore, the oft-heard lament from tax officials that a taxpayer’s testimony (or that of its witnesses) is self-serving may be a truism, but adds nothing to the determination as to whether or not the testimony is credible.

Decisions are often drafted with an eye toward supporting the desired holding instead of providing a balanced, accurate recitation of the facts. The Court understood the "well-founded, commonly accepted understanding: The officer who hears witnesses and sifts through evidence in the first instance will have a comprehensive view of the case that cannot be conveyed full strength by a paper record."⁵⁸ A trial judge’s decision – like the STJ’s report – is not an "‘in-house draft to be worked over collaboratively’"⁵⁹ or, worse, ignored by a reviewing tribunal as a mere suggestion that can be rejected without ample, objective bases. However, "judicial independence in the context of due process is not a principle to be taken lightly, and its absence has consequences."⁶⁰

Further, it would be wise for tribunals reviewing trial judges’ decisions to remember that trial judges’ main role is to determine facts and "the trial on the merits should be ‘the main event’ . . . rather than a ‘tryout on the road,’" that is, having established the facts before a trial judge, unless objective evidence is contrary to a witness’s testimony or such testimony is so fundamentally incredible as not to be believed, the party should not be required to again persuade the reviewer; "deference to the trier of fact – is the rule, not the exception."⁶¹

Additionally, when a court is faced with appellate review of a decision that reversed the trial judge’s factual findings, its scrutiny of such reversal should be heightened, since the "evidence supporting a conclusion may be less than substantial when an impartial, experienced examiner who has observed the witnesses and lived with the case has drawn conclusions different from the [reviewing tribunal’s] than when he has reached the same conclusion."⁶² The right to appeal is no substitute for the "meaningful" and "effective" appellate review required to comport with due process.⁶³ If the appellate review gives little scrutiny to reversals of credibility findings by those who have not had the opportunity to hear the witnesses, one is hard-pressed to conclude that appellate review has met fundamental notions of fairness, which is one of the hallmarks of due process.

Finally, as Judge Cudahy appropriately recognized, even if no actual impropriety occurs, transparency in the process would protect the Tax Court from the "perception that an STJ’s ‘findings’ are arbitrarily malleable at the discretion of the Tax Court."⁶⁴ That recognition should govern all tax administration processes. Tax administration systems work best when all parties – particularly the taxpayers – believe that the system operates fairly, openly, and objectively and they are not merely going through a futile exercise toward a foregone conclusion. Replacing internal tax review systems with independent tax review systems goes a long way in the right direction. It is evident from the transcript of the oral argument in Ballard, as well as the tone and holding of the Court’s decision, that the Court wholeheartedly agreed that the appearance of impropriety is troubling even if no actual impropriety exists. Having individuals employed by the same agency that issues the assessments review the assessments does not pass the Court’s smell test. Ballard suggests that tax administration systems that do not toe the line will – as they should – be called to task.

Because of the generality of this update, the information provided herein may not be applicable in all situations and should not be acted upon without specific legal advice based on particular situations.

© Morrison & Foerster LLP. All rights reserved