Homeodomain interacting protein kinase 2 (HIPK2) has emerged as a promising target for the discovery of anti-renal fibrosis drugs. Herein, to develop specific pharmacologic inhibitors of HIPK2, we designed and synthesized a series of compounds containing benzimidazole and pyrimidine scaffolds via fragment-based drug design strategy. Kinase assay was applied to evaluate the inhibitory activity of target compounds against HIPKs enzyme. The molecular docking study suggest the contribution of tyrosine residues beside the active sites of HIPK1-3 to the selectivity of active compounds. Compound 15q displayed good selectivity and potent inhibitory activity against HIPK2 compared to other two subtype enzymes. 15q could downregulate phosphorylated p53, the direct substrate of HIPK2, and decrease the fibrosis-related downstream of HIPK2, such as p-Smad3 and α-SMA in NRK-49F cells. 15q showed no effect on the cell apoptosis in fibrotic or cancer cell lines, suggesting little cancer risk of 15q. Notably, 15q displayed encouraging in vivo anti-fibrotic effects in the unilateral ureteral obstruction mouse model, which could be used as a potential lead for structural optimization and candidate for the development of selective HIPK2 inhibitors.