The game was to mark the end of a daylong "Shoulder to Shoulder" conference in Oslo that encouraged religious dialogue between the two faiths.
Church of Norway spokesman Olav Fykse Tveit said differences began when the imams refused to play against a mixed-gender team of priests because it would have gone against their religious beliefs in avoiding close physical contact with women.
"We found that out two days before, but because we thought it would be a nice conclusion of the conference we didn't want to call it off, so we decided to stage an all-mens team game instead," he said.
But when the church decided to drop its female players, the priest team captain walked out in protest.
Just hours before Saturday's scheduled game, the church released a statement saying it had called it off because it was sending the wrong signal.
"We realize now that it will be wrong to have a priest team without women... The reactions we have had today shows us that this is being interpreted as a gender-political issue. This is why we cannot go through with the soccer match."
Fykse Tveit said that, despite the differences on the field, "one very good lesson we've learned from this is that when we cooperate, we also enter each others boundaries, and that's a positive thing."