Chuganji was pronounced dead from natural causes Sunday evening, said his 65-year-old nephew, Tadao Haji.
Bedridden in recent years, Chuganji had been living with his 72-year-old daughter Kyoko in the city of Ogori, about 550 miles southwest of Tokyo.
He had just finished drinking some apple juice when his family noticed he wasn't looking well, Haji said.
"As always, he had been thanking everyone for taking such good care of him and for cooking his meals," Haji said of Chuganji's last day.
Chuganji was born March 23, 1889 in the farming town of Chikushino on Japan's southernmost main island of Kyushu. He worked as a silkworm breeder and adviser after graduating from technical school in the early 1900s.
He liked to eat beef and pork with his meals of rice and miso soup. He would drink milk everyday but didn't consume alcohol.
Kyushu is also home to the world's oldest person, a 116-year-old woman named Kamato Hongo.
There are an estimated 20,000 Japanese over the age of 100, and women make up about 80 percent of the total.
Japan's life expectancy is the longest in the world for both sexes — 85.23 years for women and 78.32 for men in 2002. The country's traditional fish-based, lowfat diet may be the secret to the long lives, researchers say.
Japan's oldest man is now Kameni Nakamura, who turns 109 on Friday, said Hideki Matsumoto, of the health ministry. Nakamura lives at a retirement home in Okinawa, which is famous for its high concentration of old people.
Japanese officials said that, with Chuganji's death, they did not know who the oldest living man in the world was.