High-risk and intermediate-high–risk results from the ThyroSeq v2 and v3 thyroid genomic classifier are associated with neoplasia: Independent performance assessment at an academic institution

Abstract

Background

The ThyroSeq panel tests for genetic alterations to risk-stratify cytologically indeterminate nodules. The authors assessed the test performance of the tests, including the latest version (v3), at an academic center.

Methods

Results from ThyroSeq testing (v2 and v3) were reviewed over 2 years, and patient demographics, cytology diagnoses, results of ThyroSeq testing, and histopathologic diagnoses on resection (if available) were collected.

Results

One hundred eighty-five nodules were tested from 178 patients, including 94 nodules tested with v2 and 91 nodules tested with v3. Overall, 28 of 185 nodules (15%) yielded a high-risk or intermediate-high–risk mutation (HRM). Of the patients with these nodules, 19 of 25 (76%) had neoplastic nodules, and 11 of 25 (44%) had a malignancy or a noninvasive follicular thyroid neoplasm with papillary-like nuclear features (NIFTP). Only 16 of 147 nodules (11%) that were negative or had low-risk genetic alterations underwent resection, with 1 false-negative result (a papillary thyroid carcinoma tested with v2). No false-negative results were identified with v3. Two nodules had TP53 mutations identified, both of which were benign on resection. Nodules with HRM that were tested with v2 and v3 had a positive predictive value (PPV) for malignancy of 57% and 39%, respectively, and a PPV for neoplasm of 86% and 72%, respectively. The negative predictive values for v2 and v3 were 92% and 100%, respectively.

Conclusions

The PPV of an HRM result on ThyroSeq v3 was low for malignancy or NIFTP, and the PPV for neoplasm was higher. RAS-type mutations were the most commonly identified in both benign and malignant nodules. Thyroseq v3 had a lower PPV for both malignancy/NIFTP and neoplasm than v2 but did not produce any false-negative results.

Introduction

Thyroid nodules are common, with a reported prevalence based on detection by physical examination ranging from 5% to 7% and from 20% to 76% by the routine use of high-resolution ultrasound.^{1, 2} The detection rate of thyroid nodules by ultrasound corresponds to published autopsy findings (range, 50%-65% prevalence).³ Fine-needle aspiration (FNA) biopsy is a critical method for the initial evaluation of patients with thyroid nodules because it is minimally invasive and has a high sensitivity (range, 65%-98%).⁴ Thyroid cytology is reported in a standardized way using the Bethesda System for Reporting Thyroid Cytopathology (TBS).⁵ Most thyroid nodules that undergo FNA are diagnosed definitively as either benign or malignant. A subset of thyroid nodules (range, 15%-30%) subjected to FNA have indeterminate cytology, which encompasses the TBS categories of atypia of undetermined significance/follicular lesion of undetermined significance (AUS/FLUS), and follicular neoplasm/suspicious for follicular neoplasm (FN/SFN). The majority of these indeterminate nodules (range, 65%-85%) are benign on final histologic evaluation.^{6, 7} Indeterminate category nodules not only lead to potentially unnecessary surgeries but also destine patients to additional and costly medical surveillance.

Several molecular test platforms have been developed to help risk-stratify cytologically indeterminate thyroid nodules.⁶ The objective of such tests is to reduce the need for surgery in these nodules.⁸ The ThyroSeq panel, developed by the team at the University of Pittsburgh Medical Center (UPMC), tests for genetic alterations associated with thyroid neoplasia and has undergone several iterations over the years.^9-11 The initial limited panel (BRAF, RAS, RET/PTC, and PAX8/PPAR) was replaced with the ThyroSeq v2 panel, which was a next-generation sequencing-based assay that queried 56 genes for point mutations, fusions, and abnormal gene expression. The v2 test reported the presence or absence of these alterations and gave a risk of cancer associated with alterations, if present. The v2 report formatting underwent slight modifications; whereas the initial v2 reports were formatted only to relate the presence or absence of genes under the Result heading, later formatting of the v2 reports included a field labeling test results as either negative or positive, a field for probability of cancer, and a field for potential management, suggesting observation or surgery. The latest version of the test is ThyroSeq v3, which assesses 112 genes using next-generation sequencing technology to identify point mutations, gene fusions, copy number alterations, and gene expression alterations. All genetic alterations included in v2 are included in v3.¹² ThyroSeq v3 carries forward the later report formatting for v2 and is reported as either positive or negative, then given a probability of cancer or NIFTP (noninvasive follicular thyroid neoplasm with papillary-like nuclear features), and suggested potential management. The positive or negative result is based on a genomic classifier, taking into account the alterations found. Low-risk mutations are sometimes seen reported as currently negative, with a note in the interpretation that clonal expansion and progression may occur. Specific genetic alterations are reported, and a detailed interpretation is provided summarizing the existing data on these alterations, notably taking into consideration the entity of NIFTP. Published performance analyses from the UPMC group indicate that ThyroSeq v3 has both high sensitivity (94%) and high specificity (82%) for thyroid FNAs classified as AUS/FLUS or FN/SFN.^{12, 13} There have been some independent evaluations of ThyroSeq v2 and the 7-gene panel^14-19; however, ThyroSeq v3 performance data to date are limited to those from the ThyroSeq investigators.^{12, 13} For the current study, we assessed the test performance of ThyroSeq v2 and v3 at an academic center. Because ThyroSeq v3 aims to improve on the performance of ThyroSeq v2, we reviewed ThyroSeq v2 and v3 data from our institution for direct comparison between the 2 versions within a single institution.

Materials and Methods

After obtaining Institutional Review Board approval, we reviewed results from ThyroSeq v2 and v3 testing at our institution over a 2-year period and collected the corresponding patient demographics, cytology diagnoses, and histopathologic diagnoses (if resected). At our institution, molecular testing is generally performed on nodules with repeat-indeterminate cytology (TBS category III or IV). Rarely, nodules with cytology showing TBS diagnoses other than III or IV were tested, generally because of clinical concern or a prior indeterminate cytology diagnosis. Only Bethesda III or IV nodules were included in the main study. The choice of molecular testing platform is at the discretion of the clinician. Biopsies at our institution are performed by radiologists, pathologists, and endocrinologists. All cytology diagnoses were rendered by 1 of 8 cytology board-certified cytopathologists using TBS. We assessed test performance using positive predictive values (PPVs) and negative predictive values (NPVs) for ThyroSeq v2 and v3 by assessing PPVs and NPVs either for malignancy or NIFTP or for neoplasm. NIFTP was grouped together with the malignant nodules because the current understanding of this entity is that it should be treated surgically and for a better comparison with most published data on ThyroSeq performance, which also consider NIFTP together with malignancies for the purposes of calculating performance.^{13, 16, 20}

Results

One hundred eighty-five nodules were tested from 178 patients (Table 1). The demographics of patients tested with v2 versus v3 were similar. The majority of patients (80%) were women, and the mean age was 58 years. Ninety-four nodules were tested with v2 (Tables 2 and 3) and 91 were tested with v3 (Tables 4 and 5). For nodules tested with v3, all nodules reported as having an intermediate-low, intermediate, intermediate-high, or high probability of cancer or NIFTP were reported as having a positive test result. All nodules that were reported as having no detected mutations or low-risk mutations were reported as having a negative v3 result.

Table 1. Patient Demographics and Cytologic Diagnoses in the ThyroSeq v2 and v3 Cohorts

Patient Characteristics, n = 178	No. of Patients (%)
	ThyroSeq v2 Cohort	ThyroSeq v3 Cohort	Total
Sex
Women	72 (80)	71 (81)	143 (80)
Men	18 (20)	17 (19)	35 (20)
Age: Mean [range], y	58.14 [9-87]	58.98 [14-85]	58.56 [9-87]
Bethesda category of nodules tested, n = 185
AUS/FLUS	86 (91)	76 (84)	162 (88)
SFN/FN	8 (9)	15 (16)	23 (12)

• Abbreviations: AUS, atypia of undetermined significance; FLUS, follicular lesion of undetermined significance; FN, follicular neoplasm; SFN, suspicious for follicular neoplasm.

Table 2. ThyroSeq v2 Results by Mutation and Histopathology Categories, n = 94

ThyroSeq v2 Result	No. Not Resected	No. Resected		Total No.
		Benign	Malignant/NIFTP
No mutations detected	62	6	1	69
Nonhigh-risk mutation	10	5	0	15
Intermediate-risk mutation	0	1	0	1
High-risk mutation	2	3	4	9

• Abbreviation: NIFTP, noninvasive follicular thyroid neoplasm with papillary-like nuclear features.

Table 3. Molecular Abnormalities Detected by ThyroSeq v2 and Associated Histopathologic Diagnoses

Mutation Category	Molecular Alteration	Prevalence in Mutation-Positive Samples, No. (%)	Histopathologic Diagnosis, No. (%)		Diagnosis at the Time of Surgery
			Cancer/NIFTP	Benign
High-risk group	TERT (and NRAS), n = 2; TP53, n = 1	3 (12)	1 (50)	1 (50)	FVPTC, n = 1; BNH, n = 1
RAS-like group	THADA fusions, n = 3; NRAS, n = 2; HRAS, n = 2; KRAS, n = 4; EIF1AX, n = 6; BRAF K601E, n = 1	18 (72)	3 (30)	7 (70)	FA, n = 3; HA, n = 1; BNH, n = 3; FVPTC, n = 1; NIFTP, n = 2
Copy number alteration group	Copy number abnormality, n = 1	1 (4)	0 (0)	1 (100)	BNH, n = 1
Other group, not high risk	NIS (SLC5A5), n = 2; TSHR, n = 1	3 (12)	0 (0)	0 (0)	None resected

• Abbreviations: BNH, benign nodular hyperplasia; FA, follicular adenoma; FVPTC, follicular variant papillary thyroid carcinoma; HA, Hurthle cell adenoma; NIFTP, noninvasive follicular thyroid neoplasm with papillary-like nuclear features.

Table 4. ThyroSeq v3 Results by Mutation and Histopathology Categories

Thyroseq v3 Test Result	ThyroSeq v3 Reported Probability of Cancer or NIFTP	No. Not Resected	No. Resected		Total No.
			Benign	Malignant/NIFTP
	Inadequate	0	0	1	1
Negative/currently negative	No mutations detected	46	1	0	47
	Nonhigh-risk mutation	13	2	0	15
Positive	Intermediate-low–risk mutation	0	1	0	1
	Intermediate-risk mutation	3	3	2	8
	Intermediate-high–risk mutation	0	9	4	13
	High-risk mutation	1	2	3	6

• Abbreviation: NIFTP, noninvasive follicular thyroid neoplasm with papillary-like nuclear features.

Table 5. Molecular Abnormalities Detected by ThyroSeq v3 and Associated Histopathologic Diagnoses

Mutation Category	Molecular Alteration	Prevalence in Test-Positive Samples, No. (%)	Histopathologic Diagnosis, No. (%)		Diagnosis at the Time of Surgery
			Cancer/NIFTP	Benign
High-risk group	TERT (and NRAS), n = 1; TP53 (and TSHR), n = 1	2 (5)	1 (50)	1 (50)	FVPTC, n = 1; FA, n = 1
BRAF-like group	BRAF/non-V600E (G469A), n = 1; NTRK3 fusion, n = 1	2 (5)	1 (100)	0 (0)	NIFTP, n = 1
RAS-like group	NRAS, n = 6; HRAS, n = 5; KRAS, n = 3; GNAS, n = 1; THADA-IGF2BP3, n = 1; PTEN, n = 1; EIF1AX, n = 4; DICER, n = 1; PAX8-GLIS3 fusion, n = 1	23 (53)	4 (27)	11 (73)	BNH, n = 3; FA, n = 8; FVPTC, n = 1; FTC, n = 2; NIFTP, n = 1
ALK group	ALK, n = 1	1 (2)	1 (100)	0 (0)	PTC, n = 1
Copy number alteration group	Copy number abnormality, n = 9	9 (21)	1 (17)	5 (83)	BNH, n = 2; FA, n = 3; FTC, n = 1
Gene expression alteration group	Low-level gene expression alterations, n = 1	1 (2)	0 (0)	0 (0)	None resected
Other group, not high risk	TSHR, n = 5	5 (12)	0 (0)	0 (0)	None resected

• Abbreviations: BNH, benign nodular hyperplasia; FA, follicular adenoma; FTC, follicular thyroid carcinoma; FVPTC, follicular variant papillary thyroid carcinoma; NIFTP, noninvasive follicular thyroid neoplasm with papillary-like nuclear features; PTC, papillary thyroid carcinoma.

Overall, 28 of 185 nodules (15%) yielded a high-risk or intermediate-high–risk mutation. All of these nodules were reported as positive on either the v2 or the v3 test if a binary classification was provided. The majority of these nodules (n = 25; 89%) were resected. Of these patients, 19 of 25 (76%) had neoplastic nodules, and 11 of 25 (44%) had a malignancy or NIFTP on resection. For nodules that were tested only with v2, on resection, nodules with a high-risk mutation had a malignancy/NIFTP rate of 57% (95% CI, 31%-80%) and a neoplasm rate of 86% (95% CI, 48%-97%). For nodules that were tested only with v3, on resection, nodules with a high-risk or intermediate-high–risk mutation (all of which were reported as test-positive) had a malignancy/NIFTP rate of 39% (95% CI, 33%-46%) and a neoplasm rate of 72% (95% CI, 63%-80%) (Table 6).^{9, 16-18, 21}

Table 6. Comparison of ThyroSeq v2 Performance in the Current Study and Published Data

Study	PPV, %	NPV, %	Sensitivity, %	Specificity, %	False-Negative Rate, %	Comments
Current study, n = 94	57	92	80	79	8	Bethesda III-IV nodules; PPV/NPV for cancer/NIFTP
Marcadis 2019,^16 n = 273	59	86	85	62	NR	Bethesda III/IV nodules; PPV/NPV for cancer/NIFTP
Taye 2018,^17 n = 156	27	91	89	43	4	Bethesda III/IV nodules; PPV/NPV for cancer/NIFTP
Valderrabano 2017,^18 n = 190	42	91	70	77	NR	Bethesda III/IV nodules
Nikiforov 2014,^9 n = 143	83	96	90	93	NR	Bethesda IV nodules only
Nikiforov 2015,^21 n = 465	77	97	91	92	NR	Bethesda III nodules only

• Abbreviations: NIFTP, noninvasive follicular thyroid neoplasm with papillary-like nuclear features; NPV, negative predictive value; NR, not reported; PPV, positive predictive value.

Only 16 of 147 (11%) of nodules that were mutation-negative or only had low-risk genetic alterations underwent resection, yielding a resection rate of 14% for v2-negative nodules and 6% for v3-negative/low-risk nodules. All but 1 of these resected nodules was benign. The 1 false-negative case was a papillary thyroid carcinoma (PTC) tested with v2.

There were 9 nodules with alterations reported as intermediate-risk/moderate-risk, of which 7 had RAS mutations. There was also 1 nodule with PAX8-GLIS3 fusion which was not resected and 1 nodule with a TP53 mutation. Of these nodules that were resected, 3 of 7 (43%) were follicular patterned carcinomas; these included 2 follicular variants of PTC and 1 minimally invasive follicular carcinoma (FTC). The nodule with the TP53 mutation was reported as moderate risk (range, 40%-50%) on v2 and, on resection, was identified as a benign hyperplastic nodule. The NPV based on the surgically resected nodules was 92% for v2-tested nodules and 100% for v3-tested nodules (Table 6). One nodule was inadequate for molecular analysis and, on resection, was identified as PTC with a prominent cystic component.

Sensitivity for malignancy/NIFTP was 80% and 100% for ThyroSeq v2 and v3, respectively. Specificity, when calculations were limited to surgical data, was 79% and 22% for ThyroSeq v2 and v3, respectively. Because most nodules that were negative for high-risk mutations were not resected, specificity also was calculated assuming that all nonresected mutation-negative nodules were true-negative, yielding specificity of 97% and 85% for ThyroSeq v2 and v3, respectively.

Six nodules that were not TBS III/IV were tested and were not included in the tables or study data calculations. A TBS category I nodule was tested using ThyroSeq v2. The smears showed rare follicular groups, with evaluation precluded by obscuring gel. The sample was sufficient for molecular analysis by ThyroSeq and showed no mutations. Four category II nodules were tested using ThyroSeq v3, some of which were tested because of a prior indeterminate cytology diagnosis. Three of those nodules were negative for mutations, and 1 (nonhigh risk) nonsplice site EIF1AX mutation was detected. None of these nodules were resected. One nodule with TBS V cytology was tested with v3 and found to have an intermediate-risk NRAS mutation, which was identified as a follicular variant of PTC on resection.

Discussion

Molecular testing has expanded our workup of cytologically indeterminate nodules. In our current study, as in prior studies, the most common group of mutations found in these nodules (72% of mutations identified with v2 and 53% of those identified with v3) were RAS-like mutations, including HRAS, KRAS, and NRAS.¹³ These mutations are believed to be involved in follicular neogenesis and are found in both benign and malignant follicular-patterned lesions,^{14, 16, 19} which limits their prognostic significance when found preoperatively, at least when identified in isolation. The majority of isolated RAS-like mutations in our study (18 of 25; 72%) were associated with benign disease at resection. When found in conjunction with another mutation (such as TERT), RAS mutations have been associated with malignancies and potentially more aggressive behavior.²² Both cases with coexisting TERT and NRAS mutations in our series were found in patients who had infiltrative follicular variants of PTC, 1 of which showed vascular invasion.

Interestingly, none of the nodules in this study were found to have the BRAF V600E mutation, which is in keeping with our institutional experience. This may be because of differing thresholds for cytologic categorization or perhaps is reflective of patient demographics at a referral center, where many patients are referred to us after receiving an outside indeterminate cytology diagnosis elsewhere.

The binary characterization of positive or negative on these tests, in our experience, did not add additional value to the risk categorization given for a specific identified genetic alteration—v3 positive results ranged from intermediate-low–risk to high-risk. It has been argued that the main value of a preoperative test such as ThyroSeq would be in negative results; ie, that a strong NPV has value to help patients avoid surgery. Although this is certainly true, one of the strengths of the ThyroSeq test is in reporting individual gene alterations found, providing the ability to give more nuanced information to guide choice of treatment in patients who test positive as well—a group that expands in proportion as additional genetic alterations are added to the panel. For this reason, we chose to assess the PPV of a high-risk or intermediate-high–risk on v3 separately from all positive tests. The PPV of a high-risk or intermediate-high–risk mutation result on ThyroSeq v3 was low for malignancy or NIFTP in our cohort, at only 39% (95% CI, 33%-46%). This is lower than the 66% PPV reported in the validation study by Steward et al (Table 7).¹³ It is also lower than the PPV in our v2 series, which was 57% (95% CI, 31%-80%) (Table 6). However, there was a wide confidence interval for v2, leading to considerable overlap in the 95% CI for these 2 tests. PPV for neoplasm was higher for both versions of the test, at 86% (95% CI, 48%-97%) for v2 and 72% (95% CI, 63%-80%) for v3, again with overlap in the confidence interval. Many of the mutations characterized by the tests as intermediate-high–risk were RAS mutations or copy number alterations, which can be seen in both benign and malignant nodules and may explain the low PPV for malignancy as well as the higher PPV for neoplasm. Interestingly, we had 2 cases of TP53 mutations, both of which were benign on surgical excision. One case was classified as moderate risk by v2 and was identified as a benign hyperplastic nodule. In the report on that case, it was noted that isolated TP53 mutations were found in 3 benign follicular adenomas in the UPMC validation, leading to classifying this case as moderate risk. The other case was found in conjunction with a TSHR mutation and was identified as a Hurthle cell adenoma on resection. In that case, a papillary microcarcinoma was found elsewhere in the lobe, which raises the possibility that cells from the PTC may have contaminated the ThyroSeq sample.

Table 7. Comparison of ThyroSeq v3 Performance in the Current Study and Published Data

Study	PPV, %	NPV, %	Sensitivity, %	Specificity, %	False-Negative Rate, %	Comments
Current study, n = 91	39	100	100	22^a	0	Bethesda III-IV; PPV/NPV for cancer/NIFTP
Steward 2018,^13 n = 247	66	97	94	82	3	Bethesda III/IV; PPV/NPV for cancer/NIFTP

• Abbreviations: NIFTP, noninvasive follicular thyroid neoplasm with papillary-like nuclear features; NPV, negative predictive value; NR, not reported; PPV, positive predictive value.
• ^a Only 3 nodules that were mutation-negative or carried nonhigh-risk mutations by v3 were resected, limiting specificity in this category. If all nonresected, negative nodules were assumed to be true-negatives, then specificity would be 85%.

The NPV was high in our study for both v2 and v3, with only 1 false-negative test (a nodule tested with v2). The patient with this nodule had bilateral thyroid nodules, and the smaller 1 was sampled because of ultrasonographic characteristics. The nodule was 1.5 cm in greatest dimension and showed Bethesda IV cytology with follicular architecture but unconvincing nuclear features of PTC. On resection, the patient had bilateral foci of PTC. It is possible that sampling variability may have played a role in leading to the negative ThyroSeq test result in this nodule. It is important to note that the true NPV is elusive because most (89%) of the nodules that were negative for mutations or carried a nonhigh-risk mutation were not resected, particularly for the patients who were tested with v3. Tables 6 and 7 show ThyroSeq v2 and v3 performance in the current study compared with published data from other institutions, including UPMC.^{9, 13, 16-19, 21}

Limitations of our study are that it is retrospective in nature and that there may be bias because it was a surgical series. Nodules that go to surgery may have more worrisome imaging and clinical features than those that do not, which has the potential to enrich the study for malignancies. These limitations are shared by most other studies of real-world molecular test performance in thyroid nodules. The low number of surgically resected nodules with negative results, particularly for v3, is a limitation in assessing NPV and specificity in our study. Only 3 nodules that were mutation-negative or carried nonhigh-risk mutations (test-negative) by v3 were resected, leading to a specificity of 22%. If all these test-negative nodules were assumed to be true-negatives, then the specificity would be 85%.

Because of the relative novelty of this test, the follow-up was short, limiting our ability to assess long-term outcomes, which is particularly of interest for those patients who did not receive resections. This limitation is another that is shared by studies of molecular tests in general. Test platforms and designs, appropriately, are rapidly evolving and changing to reflect medical knowledge and technological advances. Unfortunately, once enough time has elapsed to collect even a few years of follow-up on tested patients, the test has changed, which somewhat limits our ability to accurately assess long-term outcomes. Presumably, as more molecular alterations are included with each iteration of the gene panel, the NPV should increase. We had only 1 false-negative with v2 and none with v3. On the basis of our current results, it may be argued that good long-term outcomes for patients who were test-negative on earlier test versions can be extrapolated to patients who are tested with a broader panel. Because fewer and fewer test-negative patients undergo resection, we will need to continue to monitor these outcomes.

Despite the limitations, independent assessment of real-world performance of molecular tests is important to inform the management of patients with thyroid nodules. The ThyroSeq panels have become an important tool to help risk-stratify patients. Understanding test performance and limitations is key to successfully applying this test. A negative test result on both the v2 and v3 panels is relatively reassuring in our study and the existing literature. Nodules with isolated RAS-type mutations still are essentially indeterminate because both benign and malignant nodules may harbor these mutations. For the purposes of selecting the extent of surgical excision, the portion of the report specifying the identified mutation is more useful than the binary positive or negative result; for instance, the finding of a high-risk mutation such as BRAF V600E or TERT may suggest a more aggressive surgical approach (if clinically appropriate) versus the finding of RAS-type mutations, which may be compatible with a more conservative approach (if clinically high-risk features are absent).

Interestingly, in our cases, TP53 mutation was not invariably associated with carcinoma. TP53 mutations are rare, so further study is needed. Additional independent studies assessing the real-world performance of these tests and sustained follow-up of mutation-negative patients will continue to aid in guiding the management of those who have indeterminate thyroid nodules.

Funding Support

No specific funding was disclosed.

Conflict of Interest Disclosures

The authors made no disclosures.

Author Contributions

Rachel Jug: Conceptualization, data curation, investigation, methodology, validation, visualization, writing–original draft, and writing–review and editing. Wen-Chi Foo: Conceptualization, methodology, resources, and writing–review and editing. Claudia Jones: Conceptualization, methodology, resources, and writing–review and editing. Sara Ahmadi: Conceptualization, methodology, resources, and writing–review and editing. Xiaoyin “Sara” Jiang: Conceptualization, data curation, investigation, methodology, project administration, supervision, validation, visualization, and writing–review and editing.