The discovery announced Monday sheds a rare light on the city the Romans built after destroying the second Jewish Temple in 70 A.D. and expelling the Jews from Jerusalem following their revolt.
Ofer Sion, the director of the excavation in the Jewish quarter of Jerusalem's Old City, said the site helps prove that the Roman city was larger than previously thought.
"It is very important because in all the excavations in the Jewish quarter (we have) never found a building from the 2nd and 3rd century," he said Monday.
History-rich Jerusalem is one of the most excavated cities in the world, and archaeologists are routinely deployed to examine sites ahead of any planned construction projects.
The archaeologists found steps leading to the pool's white mosaic floor and hundreds of terra cotta roof tiles stamped with the name of the Roman unit - the famed Tenth Legion - that built the pool. Sion suggested the site was part of a larger complex where thousands of soldiers once bathed.
After the city was sacked and the Jewish kingdom overthrown, the Romans founded the city of Aelia Capitolina as capital of the new province of Syria-Palestina.
Later, when the Jordanians ruled in Jerusalem's Old City from 1948-1967, a sewing factory was built on the same site, Sion said.
When the archaeologists complete their dig, the Jerusalem city council will continue their plans to build a mikveh, a Jewish ritual bath, Israeli authorities said. Some religious Jews routinely use the bath for ritual purification.
Israel's Antiquities Authority said that the remains of the ancient Roman bathhouse would be integrated into the design of the new ritual bath.
Archaeology and politics often go hand in hand in Jerusalem, and especially in areas surrounding the Old City and its many layers of history that are enmeshed in a web of competing claims.
Rafi Greenberg, an archaeologist from Tel Aviv University, said Israeli authorities often focus on findings from Jerusalem's Jewish past at the expense of historical remants from other eras.
"Every modern building in the Old City is built above an ancient building," Greenberg said. "The question is if religious authorities are tolerant enough to accept alternative histories."