The couple was chased by photographers, cameramen and reporters, forcing them to turn back after a 20-minute ride Sunday that took them past stores in downtown Pune, in western India.
Several traffic signals slowed their short rickshaw ride. At every red light, security men traveling in a rickshaw behind jumped out and surrounded the three-wheeler to prevent the media from snapping pictures of Pitt and Jolie.
They then returned to the Le Meridian Hotel, where they have been holed up since arriving Thursday.
A British photographer was violently grabbed by the throat by the celebrity couple's bodyguard on Saturday after he tried to snap a picture of the pair, CBS News reports. The incident was caught on tape by the AP (see video, right).
"I am a guest at this hotel, I have every right to be here … I was outside anyway, doing my job along with everybody else," the photographer, Sam Relph, told the AP. Relph said he was considering taking legal action.
Two other Indian journalists said they were threatened by the same bodyguard, Sky News reports.
Security for the star couple has been tight since their arrival in Pune by private jet with their children Maddox, 5, Zahara, 1, and 4-month-old Shiloh Nouvel Jolie-Pitt.
The couple reportedly is scheduled on Monday to start shooting scenes for "A Mighty Heart," in which Jolie plays Pearl's widow, Mariane.
The movie is based on an adaptation of Mariane Pearl's book, "A Mighty Heart: The Brave Life and Death of My Husband Danny Pearl," and will be co-produced by Plan B, a production company founded by Pitt and his ex-wife, actress Jennifer Aniston.