Christopher Savoie said the ordeal of more than two weeks in a Japanese jail was bad enough. But coming home without his children -- Isaac, now 9, and Rebeccab 6 -- was excruciating.
"There are no words for it, you know. There just aren't any words for it," he said.
Savoie, who's from Nashville, made international news when he picked up his kids as they walked to school with their mother.
Savoie Speaks Out on Husband's Release
Japan Frees U.S. Father in Custody Case
American Dad in Japanese Jail "Frightened"
He says his ex-wife, Noriko Savoie, had abducted the children to Japan in August -- and Tennessee courts gave him full custody. But Japan doesn't honor foreign custody agreements, so Chris took matters in his own hands.
Savoie told Williams the physical act of taking his children from their mother wasn't aggressive, saying, "Oh, no, hugging your kids and putting them in a car, I hardly think that is a violent act."
Savoie added he didn't push or hit his ex-wife when he took the children.
Finally reunited with them, Savoie raced to the nearby U.S. consulate. Savoie said the consulate knew they were coming, because he called ahead.
However, Savoie's plans for returning his children were thwarted by his ex-wife, who had alerted local police. They were waiting outside the consulate.
With Rebecca in his arms and Isaac trailing behind, Savoie said he tried to race past a police barricade, to get the children to U.S. soil. Savoie said he ran to the door with police in riot gear running after him with shields and batons.
"It felt like a movie, actually," Savoie said. "It was very unreal for me."
Japanese police arrested him and, for 17 days, held him in jail, repeatedly interrogating him, while they decided whether to indict him on kidnapping charges.
Savoie said, "Everything that you're not supposed to do to a defendant, especially pre-indictment, they did -- and a whole lot more."
While imprisioned, Savoie said he argued that he had a right to his children.
"They didn't disagree with me on that," he said. "They just said I'm not allowed to see them."
Then, last week prosecutors let Savoie go, with the stipulation that he leave the country and his kids.
"Basically, I'm not allowed to see them. I'm not allowed to call them," Savoie told Williams. "I'm not even allowed to send them birthday presents."
While Japanese authorities say he can pursue custody of his children through Japanese courts, Savoie knows the odds are against him. He said he just hopes the memory of the incident will let his children know he tried.
"They're going to find out who their dad is, what he's all about, and that he loves them," he said. "And if loving my kids so much that I really want to be with them is a crime, then, well, I'm guilty. I'm guilty of loving my kids."
Williams added on "The Early Show" that Savoie also said that, when he got to the consulate gates, one official reached out to take his daughter. But for reasons he doesn't understand, the gates never opened.