Simon Cowell on "The X Factor" shakeup

(CBS) Simon Cowell is talking about the major shakeup that happened last week on his reality talent series "The X Factor."

Judges Paula Abdul, Nicole Scherzinger and host Steve Jones are not returning for the show's next season. Cowell told "Extra" that he and Fox executives began discussing changes "at the beginning of the year - and we were told it had to happen straight away."

Pictures: Simon Cowell
Pictures: "The X Factor"

Of Abdul, Cowell revealed, "I would've liked to have kept her. She was very gracious and I said that to her. She understands it's business; it's never personal."

He added that he didn't think Scherzinger was angry about her ouster: "I think she's disappointed."

That leaves Cowell and L.A. Reid as the only remaining members of the original "X Factor" team. Keeping Reid was important, Cowell told "Extra," because, "There's not a record executive out there who is as good as him for the job."

When asked why certain things didn't work out, Cowell replied, "It's only because I opened my big mouth and said we were going to get 20 million [viewers]. If I hadn't said that, everybody would be saying the show was a huge success."

As for who will fill the three vacant spots, the judge shot down rumors that Beyonce was under consideration. He added that he would "love" to have Mariah Carey, but "you can't expect someone who's just had two kids to make the kind of commitment you would need on this show."

He also said that Madonna would be "great" but would also be "expensive," while Katy Perry "would be fun - and she's feisty."

Cowell's full interview airs on "Extra" on Wednesday.

Tell us: Who should replace the departing "X Factor" judges?