Gallic acid (GA) is known to possess diverse biological activities, including anticancer. Histone deacetylase (HDACs) are controlled by tumor suppressor gene transcription and are overexpressed in various tumors, resulting in tumor development, progression and poor prognosis. This study aims to demonstrate the effect of GA on inhibition of prostate cancer (PCa) progression by modulating the expression of HDAC1 and 2 in PCa cells. To prove our research rationale, we used diverse experimental methods. GA decreased the cell viability of only PCa cell lines and not normal cells (contrary to another HDAC inhibitor, suberoylanilide hydroxamic acid) and also inhibited colony and tumor spheroid formation. Exposure to GA decreased the mitochondrial membrane potential (ΔΨm), increased the number of apoptotic cells and induced DNA fragmentation. Western blot analysis revealed down-regulated expression of HDAC1 and 2, leading to up-regulation of acetyl-p53 expression at the protein level, subsequent to down-regulating the expression of cell-cycle-related genes, i.e., proliferating cell nuclear antigen (PCNA), Cyclin D1 and E1, up-regulating the expression of cell cycle arrest gene p21 and regulating the expression of apoptosis intrinsic pathway-related genes, such as Bax, Bcl-2, cleaved Caspase-3 and poly (ADP-ribose) polymerase 1 in both PCa cell lines. Furthermore, oral administration of GA for 8 weeks on PC-3 cells-derived tumor xenograft mice model decreases the tumor size, damages the tumor structure and down-regulates the expression of HDAC1 and 2 and PCNA in tumor mass, as confirmed by histological analysis. These results indicated that GA may hinder the PCa progression by inhibiting HDAC1 and 2 expression, thereby demonstrating the potential of GA to be used as HDACs inhibitor and anti-PCa therapeutics.