Sarah Doukas, head of the Storm modeling agency, said Moss was considering leaving Britain after the furor spawned by photos in the Daily Mirror tabloid that appeared to show her snorting cocaine.
"She loves England. But I'm sure she's feeling concerned about living here again," Doukas was quoted as saying by The Times newspaper.
Doukas said Moss, 31, was "gutted, absolutely devastated" by the tabloid's story.
"I read it over the phone to her and the avalanche started almost immediately," Doukas was quoted as saying.
Moss, 31, lost contracts with H&M, Burberry and Chanel in the wake of the drug claims.
H&M initially said it would proceed with the campaign, but company spokeswoman Liz Asarnoj said "after evaluating the situation, we have decided that a campaign with Kate Moss is not consistent with H&M's clear disassociation with drugs."
Chanel, the French fashion house, said it wouldn't renew Moss' contract when it expires next month. "The contract was always going to end in October," said Julie Le Blevec, spokeswoman for Chanel perfumes.
Moss has been the face of Chanel's Coco Mademoiselle perfume since 2002.
Burberry also said it was canceling a planned campaign involving Moss.
Doukas, who helped launch Moss's career after she spotted her in New York's John F. Kennedy International Airport when Moss was 14, said the model was continuing with her career and would soon sign a contract to represent a luxury perfume brand.
"She loves modeling and she needs to be busy," Doukas said.
The Daily Mirror tabloid printed images from a video which it said showed the model doing five lines of cocaine in 40 minutes at a late-night music recording session, preparing them with a credit card and snorting the drug through a five-pound note.
The tabloid said the video was made at a West London recording studio last week during "a Mirror undercover investigation," but gave no further details.
The newspaper said Moss had taken a large amount of cocaine out of her handbag. Her boyfriend Pete Doherty, the musician whose alleged drug problems and brushes with the law have made headlines for months, was also present, the paper reported.
The Mirror said Moss had shouted obscenities when one of its reporters asked her for comment on the allegations outside a restaurant in New York, where she was attending the city's Fashion Week shows.
Moss won an apology and an undisclosed settlement from the Sunday Mirror in July over a story it ran in January alleging that she had collapsed into a cocaine-induced coma in Barcelona in June 2001.