Abby Wambach's penalty kick early in the second half went wide after Colombia goalkeeper Catalina Perez - a backup herself - was ejected for a foul on Morgan. Stefany Castano, who replaced Perez in goal, got a hand on Morgan's shot five minutes later, but couldn't stop the goal to put the United States up 1-0.
Carli Lloyd also scored for the second-ranked Americans, who will face No. 16 China on Friday in Ottawa. The United States is seeking its third World Cup title, but first since 1999.
The Americans have not allowed a goal in 333 minutes.
Colombia has never won soccer's premier tournament, but the No. 28 Las Cafeteras pulled off one of the biggest upsets in any World Cup in the group stage when they defeated third-ranked France 2-0.
Morgan and Wambach started up top for the United States, which used the same starting lineup as it did in the group-stage finale against Nigeria - a first since Jill Ellis became coach.
It was Morgan's second straight start after working her way back from a bone bruise in her left knee. Morgan came in as a sub in the first two matches of the tournament.
Perez, a 20-year-old junior at Miami, started because regular goalkeeper Sandra Sepulveda was serving a suspension for yellow-card accumulation. Sepulveda had six saves in Colombia's win over France. Castano had started in Colombia's World Cup opener, a 1-1 draw with Mexico.
The teams played to a goalless first half, with the United States unable to finish several good chances.
Wambach was ruled offside for her attempt at a rebound goal in the fourth minute. Morgan later had a chance, but her shot bounced in front of Perez, who tipped it up and over the crossbar. Perez made three saves in the first half.
The United States was hurt in the 17th minute when Lauren Holiday got a yellow card, her second of the World Cup. She'll have to sit out the quarterfinal, and it happened again in the 41st minute when Megan Rapinoe got her second yellow.
Perez was sent off at the start of the second half after sliding into Morgan, who was charging toward goal. After Castano took over, Wambach fooled her on the right side but the penalty kick sailed well left of the post.
After Morgan's goal in the 53rd minute, Lloyd scored on a penalty kick in the 66th, Lloyd's first goal of the tournament.
The U.S. had won each of the previous two meetings. When they met nearly three years ago in the London Olympics, Colombia striker Lady Andrade sucker-punched U.S. star Abby Wambach in the eye, drawing a two-match suspension.
In the days before the match in Edmonton, some of Colombia's players said they felt they'd been disrespected by the Americans ever since.
"Because of something that happened three years ago, they've said things that have not been taken by us in the best way," midfielder Yoreli Rincon said. Andrade told reporters she thought the Americans had "belittled" the Colombians.
Colombia, the third-place finisher in Group F behind France and England, was making its second World Cup appearance; it finished in 14th in 2011 in Germany. Colombia had never won a match in the sport's premier tournament until the upset of France.
The second-ranked Americans finished on top of the so-called Group of Death, with victories over Australia and Nigeria and a 0-0 tie with Sweden.
China, the Americans' next opponent, has played in six World Cups, but missed out four years ago. The Steel Roses have never won a title, but they made the final in 1999, losing memorably to the United States on penalty kicks at the Rose Bowl.