Farnesyl pyrophosphate synthase (FPPS) is known to participate in a variety of disease-related cell signaling pathway and bisphosphonates (BPs) are served as FPPS inhibitors. However, the high polarity of BPs often induces a series of side effects, limiting their applications. In the present study, novel non-BP FPPS inhibitors were discovered by in silico screening and experimental validation. From the structure-based virtual screening (SBVS) strategy combining molecular docking, pharmacophore and binding affinity prediction, 10 hits with novel scaffolds were filtered. The inhibition activity of hits against FPPS was identified and 7 hits showed comparable or higher inhibition activity than Zoledronate. The hit VS-4 with higher lipophilicity (XlogP = 1.81) and binding affinity (KD = 14.3 ± 2.63 μM) to FPPS was selected for further study on cancer cells with different FPPS expression level. Experimental results revealed that VS-4 could better target the FPPS high-expressing colon LoVo and HCT116 cancer cell lines with IC50 of 51.772 ± 0.473 and 43.553 ± 1.027 μM, respectively, whereas the IC50 value against FPPS low expressing MDA-MB-231 cells was >100 μM. The mechanism of VS-4 against colon cancer cells was investigated by flow cytometry and the results indicated that VS-4 induced cell apoptosis by increasing the intracellular reactive oxygen species (ROS) level. Taken together, the SBVS strategy could be used to discover promising non-BP FPPS inhibitors and the lead compound VS-4 might shed a light on designing more potent inhibitors as novel anticancer drugs.