Papa John's John Schnatter loses his office space in corporate HQ

Papa John's founder John Schnatter has lost his office in the pizza chain's corporate headquarters in Louisville, Kentucky.

A special committee of Papa John's board of directors has "approved and directed the company to terminate a sublease agreement granting Mr. Schnatter the right to use certain office space at the company's corporate headquarters," the company said in a statement released late Friday. 

The committee of independent directors with the fewest business and personal ties to the onetime chairman was formed to "evaluate and take action with respect to all of the company's relationships and arrangements" with Schnatter, according to the statement.

Papa John's has already asked Schnatter to stop talking with the media about the company, the statement said, and has formally ended the "Founder Agreement" it had that defined Schnatter's role as the image and spokesman for the brand.

Schnatter stepped down as the company's chairman on July 11. He remains Papa John's largest shareholder, holding about 30 percent of the company's stock. Shares of Papa John's were down nearly 5 percent Monday morning, trading at about $50.90 a share. 

The break with the company he founded is the latest in a series of setbacks for Schnatter after he admitted last week to using a racial slur during a business call. Schnatter apologized for his words, but has also said that his statement was taken out of context.

He told CBS affiliate WLKY that, after the training, the company conducting it tried to blackmail Papa John's and then "ran to Forbes" with his comments after payment was not forthcoming. Forbes was the first to report the racial slur on July 11.

At least five sports teams have dropped partnerships with Papa John's, and two universities have distanced themselves from the company or Schnatter in the wake of his comments, as has the mayor of his hometown of Jeffersonville, Indiana.

CBS News' Irina Ivanova contributed to this report.