The 78-foot sub was found Wednesday by two research craft on routine training dives about three miles from Pearl Harbor, in an area described as a "military junkyard" about 1,200 feet below the surface.
"To actually come across it was a sobering moment, realizing that was the shot that started the Pacific war," said Terry Kerby, chief pilot of the Pisces IV deep-diving submersible.
Although the surprise Japanese aerial attack is most widely associated with Pearl Harbor, the U.S. military has long asserted that an underwater skirmish occurred first.
The military says it inflicted the first casualties when the USS Ward sank the approaching Japanese sub at 6:45 a.m. on Dec. 7, 1941, about an hour before the aerial attack.
Historians have had no proof of the sinking until Wednesday's discovery, researchers said.
Although it has been known the submarine was somewhere outside the harbor, previous expeditions — including a November 2000 National Geographic expedition headed by the team that found the wreckage of the Titanic — failed to locate it.
"The thing is quite difficult to find because of all the massive amounts of junk out in the area, and we were simply fortunate because we've run our test and training dives through here and know where a lot of the junk is," said John Wiltshire, associate director of the Hawaii Undersea Research Laboratory, which found the sub.
"Local boys make good sometimes," he said.
Underwater video of the sub shows the hull rusted and slightly leaning to its port side with the periscope up. Two Japanese crewmen are believed still inside.
The sub led four other Japanese midget submarines to Pearl Harbor before the Sunday morning attack by Japanese planes that lasted two hours and left 21 U.S. ships heavily damaged, 323 aircraft damaged or destroyed, 2,390 people dead and 1,178 wounded.
Wiltshire said the crew is certain that the sub was sunk by the Ward because of a bullet hole in the conning tower and because it still has both torpedoes. Three of the subs have been previously accounted for. One is still missing, but it had fired both its weapons.
Until the submarine was found, historian Daniel Martinez said, eyewitness accounts were unconfirmed. Martinez, a historian for the USS Arizona Memorial, has interviewed the crew who fired the first shot, and a pilot who saw the submarine sink.
"What they saw and what they felt was their recollection, now the proof has been found," he said.
Immediately following the sinking of the midget sub, the USS Ward sent the message: "We have attacked, fired upon and dropped depth charges operating in defensive sea area."
But the military base and ships were not immediately placed on alert, which would have prepared the United States for the ensuing attacks.
"They notified the Navy headquarters (in Washington) and they needed confirmation before they would act," Martinez said. "Remember, there was nothing to tell them that it was a Japanese submarine. It could've been anybody and it could've been friendly."
Yoshiaki Yoshimi, a historian at Tokyo's Chuo University, said the finding was not likely to change historians' interpretation of the Pearl Harbor attacks. It was already well known that Japan had sent subs to scout the harbor before the attack.
"The finding is an important piece of historical evidence because it backs up Japan's attempt to conduct a surprise attack," he said.