Well, it looks like without Simon...there is no show!! This just announced, the US version of The X Factor is canceled!! According to their Facebook page he is headed back to the UK version of the show saying: 

“I’ve had a fantastic time over the last 12 years, both on The X Factor and American Idol. And apart from being lucky enough to find some amazing talent on the shows, I have always had an incredible welcome from the American public (most of the time!),” Cowell said in a statement. “Last year, for a number of reasons, I had to make a decision to return to the U.K. version of The X Factor in 2014. So for now, I’m back to the U.K. … America, I’ll see you soon!” That's too bad.. I kinda liked that show!