Src homology-2-containing protein tyrosine phosphatase 2 (SHP2), the first oncoprotein identified in the protein phosphatase family, has emerged as a promising anticancer target in recent years. Here, we report the discovery of a novel series of pyrazine N-oxide derivatives as potent SHP2 allosteric inhibitors and the identification of compound C5 as a highly potent and selective SHP2 allosteric inhibitor (SHP2WT IC50 = 0.023 μM, SHP2E76K IC50 = 0.119 μM). At the cellular level, C5 exerted significant antiproliferative effect on KYSE-520 and MV-411 cells (KYSE-520 IC50 = 6.97 μM, MV-411 IC50 = 0.67 μM) and induced apoptosis of MV-411 cells by downregulating the SHP2-mediated ERK cell signaling. Molecular dynamics simulations revealed that C5 could form stable hydrogen bond interactions, cation-π interactions and water bridges with key residues Glu110, Arg111, Phe113, Gly115 and Thr253, thereby effectively binding to the tunnel allosteric site of SHP2. Notably, the pyrazine N-oxide scaffold could additionally form a strong and stable hydrogen bond with Arg111. Collectively, this work uncovers a novel and potent scaffold as well as presents compound C5 as a promising lead for the development of new chemotypes of SHP2 allosteric inhibitors.