Jacob Trouba had a goal and an assist, while Ryan Strome and Adam Fox each had two assists to help the Rangers win their second straight.
Igor Shesterkin stopped 31 shots for his second straight win, although he left the game at 13:15 of the third after a collision with Vladislav Gavrikov. He was replaced by Alexandar Georgiev, who stopped both shots he faced.
"After the first period, I thought everybody played to their role," Rangers coach Gerard Gallant said. "They did a great job. We maintained pressure in the offensive zone. If we're going to be successful team, that's how we have to play in the offensive zone."
Jakub Voracek, Eric Robinson and Alexandre Texier scored for Columbus, which has lost two straight at home. Elvis Merzilikins had 36 saves.
With the score tied 1-1 after the first period, Robinson put the Blue Jackets ahead at 6:56 of the second with his second of the season.
The lead didn't last long as the Rangers scored three times in a 63-second span to take the lead.
Kreider redirected Trouba's pass from the goal line past Merzilikins to tie it at 7:39, Panarin's second goal of the game off a laser shot between the circles at 8:15 put the Rangers ahead, and Trouba had a wraparound 27 seconds later to make it 4-2.
It was the quickest three goals by the Rangers in a regular-season game since Nov. 16, 1993, and the fastest trio of scores against the Blue Jackets, breaking the previous mark of 1:15 set by Tampa Bay in 2019.
"For one minute everyone went to sleep," Blue Jackets coach Brad Larsen said. "That cost us the game. Then we were just chasing. We lost a lot of wind in our sails. It got pretty ugly there for a while."
Kreider made it 5-2 with his second of the night and 11th of the season just 52 seconds into the third to make it 5-2. It was his league-leading seventh on the power play, and gave him seven goals in the last six games.
"I think it was the end result of us doing the right things," Krieder said. "There are going to be stretches where pucks aren't going in but we're getting looks and they can come in come in bunches like that, especially with some of the skill that we have on this group."
Texier scored with 6:45 left for the final margin.
Voracek's power-play goal 8:59 into the game got Columbus on the board first with his first since returning to Columbus in an offseason trade with Philadelphia.
Panarin tied it with 2:31 left in the opening period, beating a sprawling Merzlikins on an open stick side off perfect tic-tac-toe passing from Fox and Strome.
Rangers: Host New Jersey on Sunday night.
Blue Jackets: Host Detroit on Monday night.
(© Copyright 2021 The Associated Press. All Rights Reserved. This material may not be published, broadcast, rewritten or redistributed.)
for more features.