The development of tailings in concrete technology is not only conducive to the realization of the goal of reducing carbon emissions, but also conducive to the inhibition the occurrence of shortages of sand and gravel supplies. In this study, graphite tailings were used to replace sand in the range of 0~100%, and the mechanical mechanism of graphite tailings concrete was examined through compressive and flexural tests. The mechanical experimental results were evaluated and verified based on concrete macroscopic failure appearance, mesoscopic failure appearance, and physical characteristics of graphite tailings. The results revealed that the concrete strength increases first and then decreases with the increase of the graphite tailings content. Compared to GT00 (GT00 is a specimen with a graphite tailings content of 0%, and so on), GT10~GT60 exhibited better mechanical properties, of which 30% was recommended as the optimal replacement rate. The mechanical properties of GT10 and GT20 had an upward trend, and GT30 had low spalling, with aggregate fragmentation found on the fracture surface. GT30 showed the best resistance to bending and deformation. The mechanical properties of GT40~GT60 had a downward trend. When the graphite tailings content was higher than 70%, the interface defects of the aggregate matrix increased, thus making it easier for cracks to propagate along the interface. Furthermore, the mechanism of graphite tailings replacing sand verified the test results from different perspectives, which provides new analysis ideas for other tailings in environment-friendly concrete.