Abu Dujana, the group's military commander, and Zarkasih, who acted briefly as JI's caretaker leader, were found guilty of conspiracy to commit terrorist attacks, harboring fugitives and stockpiling illegal arms.
The rulings were handed down in separate, lengthy trials at the South Jakarta District Court.
Jemaah Islamiyah and its allies are accused of carrying out the 2002 bombings on Indonesia's resort island of Bali, a 2003 attack on the J.W. Marriott Hotel in Jakarta, a 2004 attack on the Australian Embassy in Jakarta, and 2005 triple suicide bombings on restaurants in Bali.
Many of the more than 240 killed in the attacks were foreign tourists.
Neither Dujana nor Zarkasih - both of whom faced possible death sentences - were charged in connection with those blasts.
Dujana's conviction was over recent attacks on Christians on the eastern island of Sulawesi, which was plagued by religious violence from 1999 to 2001. He has condemned al Qaeda-style bombings, arguing they were counterproductive to Jemaah Islamiyah's reported aim of establishing Islamic law across the region.
Presiding Judge Wahjono, who like many Indonesians uses one name, sentenced Dujana to 15 years in prison, saying his recent public condemnations of terrorism had been taken into account. He also said he was convinced Dujana could play a role in helping reform other jailed terrorists.
Asked if he would appeal the ruling, Dujana, 37, said, "I'll think about it."
Judge Eddy Risdianto said Zarkasih, 45, was given a reduced sentence because he only served as a two-month caretaker leader of Jemaah Islamiyah in 2005, not the emir as had been alleged. The judge also cited his good behavior in prison.
The two judges also labeled Jemaah Islamiyah a terrorist group, a move that could pave the way for the government to ban the group, something it has previously said would be difficult because it was not a "formal organization."
Even without a ban on the network, the government's crackdown has met with huge success, resulting in hundreds of arrests in recent years, thanks partly to forensic and technical help from foreign governments.
Jemaah Islamiyah was formed in the early 1990s as an offshoot of another militant network stretching back decades. Its core leadership fought or trained in Afghanistan and some came under the influence of al Qaeda.
A regional crackdown following the Bali attacks netted hundreds of members and sympathizers, severely weakening the group. Former members and analysts say the hard-core faction that carried out the bombings no longer operates under its command.