Prostate cancer is the second most commonly diagnosed cancer, and prostate cancer is the second most common cause of cancer death in United States men after lung cancer. Many therapies are used to treat prostate cancer, and chemotherapy is one of the most relevant treatments. However, chemotherapy has many side effects, and repeated administration of chemotherapeutic agents leads to acquired resistance. Thus, new drugs with few side effects are needed. We investigated the molecular mechanism of action of JI017 in human prostate cancer cells. We identified an endoplasmic reticulum (ER) stress pathway that depended on the reactive oxygen species (ROS) pathway and played a crucial role in JI017-induced apoptosis. We measured cell viability by the MTS assay to determine the effect of JI017. Analysis of apoptosis, mitochondrial dysfunction, and cell cycle features was performed by flow cytometry. We used western blot and RT-PCR to measure the levels of the proteins of the unfolded protein response (UPR) pathway and apoptosis markers. Immunoprecipitation assay and transfection were used to determine the expression levels of proteins interacting with the pathways influenced by JI017 in prostate cancer cells. The anticancer effects induced by JI017 were evaluated. JI017 induced cell death that regulated apoptotic molecules and caused cell cycle arrest that inhibited the proliferation of cancer cells. Moreover, JI017 generated ROS. Accumulation of ROS caused ER stress through the PERK–eIF2α–CHOP and IRE1α-CHOP pathways. Furthermore, persistent activation of the UPR pathway induced by JI017 treatment triggered mitochondrial dysfunction, including dissipation of mitochondrial membrane potential, which activated intrinsic apoptotic pathway in human prostate cancer cells. The data indicated that N-acetyl-L-cysteine diminished apoptosis. We demonstrated that JI017 induced ER stress and cell death. Anticancer properties of JI017 in prostate cancer cells and in a human prostate cancer model involved ROS-mediated ER stress. Thus, JI017 treatment provides a new strategy for chemotherapy of prostate cancer.