During exit from mitosis in Xenopus laevis egg extracts, the AAA+ ATPase Cdc48/p97 (also known as VCP in vertebrates) and its adapter Ufd1-Npl4 remove the kinase Aurora B from chromatin to allow nucleus formation. Here, we show that in HeLa cells Ufd1-Npl4 already antagonizes Aurora B on chromosomes during earlier mitotic stages and that this is crucial for proper chromosome segregation. Depletion of Ufd1-Npl4 by small interfering RNA (siRNA) caused chromosome alignment and anaphase defects resulting in missegregated chromosomes and multi-lobed nuclei. Ufd1-Npl4 depletion also led to increased levels of Aurora B on prometaphase and metaphase chromosomes. This increase was associated with higher Aurora B activity, as evidenced by the partial resistance of CENP-A phosphorylation to the Aurora B inhibitor hesperadin. Furthermore, low concentrations of hesperadin partially rescued chromosome alignment in Ufd1-depleted cells, whereas, conversely, Ufd1-depletion partially restored congression in the presence of hesperadin. These data establish Cdc48/p97-Ufd1-Npl4 as a crucial negative regulator of Aurora B early in mitosis of human somatic cells and suggest that the activity of Aurora B on chromosomes needs to be restrained to ensure faithful chromosome segregation.