Prebiotic supplements are used to promote gastrointestinal health by stimulating beneficial bacteria. The aim of this study was to compare the potential prebiotic effects of fructans with increasing degrees of polymerization, namely fructooligosaccharides (FOS) and inulins with a low and high polymerization degree (LPDI and HPDI, respectively), using an ex vivo fermentation system to simulate the colonic environment. The system was inoculated with pooled feces from three healthy donors with the same baseline enterotype. Changes in microbiota composition were measured by 16S metagenomic sequencing after 2, 7, and 14 days of fermentation, and acid production was measured throughout the experiment. Alpha-diversity decreased upon inoculation of the ex vivo fermentation under all treatments. Composition changed significantly across both treatments and time (ANOSIM p < 0.005 for both factors). HPDI and LPDI seemed to be similar to each other regarding composition and acidification activity, but different from the control and FOS. FOS differed from the control in terms of composition but not acidification. HDPI restored alpha-diversity on day 14 as compared to the control (Bonferroni p < 0.05). In conclusion, the prebiotic activity of fructans appears to depend on the degree of polymerization, with LPDI and especially HPDI having a greater effect than FOS.