Nearly all foreigners age 16 or over, including longtime residents, will be scanned. The only exceptions are diplomats, government guests and permanent residents such as Koreans who have lived in Japan for generations.
Tokyo has staunchly backed the U.S.-led attacks on Iraq and Afghanistan, raising fears Japan could be targeted by terrorists.
Officials said the new security measures, while inconvenient for visitors, were necessary.
"There are people who change their names, use wrongly obtained passports, and pretend to be other people," said Toshihiro Higaki, an immigration official at Narita International Airport near Tokyo. "The measure also works as a deterrent."
The fingerprints and photos will be checked for matches on terrorist watch lists and files on foreigners with criminal records in Japan. People matching the data will be denied entry and deported.
Japan is the second country after the United States to implement such a system, said Immigration Bureau official Takumi Sato.
He said there had been no reports of trouble since the checks began Tuesday morning.
Critics, however, said the measures discriminate against foreigners and violate their privacy.
About seventy people gathered in front of the Justice Ministry on Tuesday for a rally protesting the measures.
"I don't like the government having personal information of mine," said Rebecca Miller, an Australian student living in Japan since July. "I don't think they got any rights on my bodily information."
On Monday, a group of nearly 70 civic groups from around the world delivered a letter of protest to Justice Minister Kunio Hatoyama.
"We believe that your plans ... are a gross and disproportionate infringement upon civil liberties, copying the most ineffective, costly and risky practices on border management from around the world," the letter said.
Immigration officials say the bureau plans to store the data for "a long time," without saying how long. It is unclear how many people will be affected; Japan had 8.11 million foreign entries in 2006.
Concerns about extremists coming into Japan spiked when reports emerged in May 2004 that Lionel Dumont, a French citizen with suspected links to al Qaeda and a history of violent crime, repeatedly entered the country on a fake passport.
Dumont, who was later sentenced to 30 years in prison in France, was reportedly trying to set up a terror cell when he lived undisturbed in Japan in 2002 and 2003.
Last month, Justice Minister Hatoyama came under fire over his assertion that a friend of his had an acquaintance who was a member of the al Qaeda terrorist group.