Natural products have continued to offer tremendous opportunities for drug development, as they have long been used in traditional medicinal systems. SHP2 has served as an anticancer target. To identify novel SHP2 inhibitors with potential anticancer activity, we screened a library containing 658 natural products. Polyphyllin D was found to selectively inhibit SHP2 over SHP1, whereas two other identified compounds (echinocystic acid and oleanolic acid) demonstrated dual SHP1 and SHP2 inhibition. In a cell-based assay, polyphyllin D exhibited cytotoxicity in Jurkat cells, an acute lymphoma leukemia cell line, whereas the other two compounds were ineffective. Polyphyllin D also decreased the level of phosphorylated extracellular signal-regulated kinase (p-ERK), a proliferation marker in Jurkat cells. Furthermore, knockdown of protein tyrosine phosphatase (PTP)N6 (SHP1) or PTPN11 (SHP2) decreased p-ERK levels. However, concurrent knockdown of PTPN6 and PTPN11 in Jurkat cells recovered p-ERK levels. These results demonstrated that polyphyllin D has potential anticancer activity, which can be attributed to its selective inhibition of SHP2 over SHP1.