Colorectal cancer (CRC) initiation and growth is often attributed to stem cells, yet little is known about the regulation of these cells. We show here that a subpopulation of Prox1-transcription-factor-expressing cells have stem cell activity in intestinal adenomas, but not in the normal intestine. Using in vivo models and 3D ex vivo organoid cultures of mouse adenomas and human CRC, we found that Prox1 deletion reduced the number of stem cells and cell proliferation and decreased intestinal tumor growth via induction of annexin A1 and reduction of the actin-binding protein filamin A, which has been implicated as a prognostic marker in CRC. Loss of Prox1 also decreased autophagy and the survival of hypoxic tumor cells in tumor transplants. Thus, Prox1 is essential for the expansion of the stem cell pool in intestinal adenomas and CRC without being critical for the normal functions of the gut.