That would be a departure from the policy of the Bush administration, which targeted medical marijuana dispensaries in California even if they complied with that state's law.
"The policy is to go after those people who violate both federal and state law," Holder said in a question-and-answer session with reporters at the Justice Department.
Medical marijuana advocates in California welcomed the news, but said they still worried about the pending cases of those already in court on drug charges.
California law permits the sale of marijuana for medical purposes, though it still is against federal law.
Holder did not spell out exactly who no longer would face the prospect of raids by the Drug Enforcement Administration. But he was quick to add that law enforcement officers will target anyone who tries to "use medical marijuana laws as a shield" for other illegal activity.
"Given the limited resources that we have, our focus will be on people, organizations that are growing, cultivating substantial amounts of marijuana and doing so in a way that's inconsistent with federal and state law," the attorney general said.
Advocates and government officials had been waiting since President Obama was sworn into office for a clear signal on what the new president's drug policy would be toward medical marijuana. As a candidate, he repeatedly promised a change in policy in situations in which state laws allow the use of medical marijuana.
Yet shortly after Mr. Obama took office, DEA agents raided four dispensaries in Los Angeles, prompting confusion about the government's plans.
Thirteen states have laws permitting medicinal use of marijuana. California is unique among them for the presence of dispensaries, which are businesses that sell marijuana and even advertise their services. Legal under California law, such dispensaries are still illegal under federal law.
Kris Hermes, a spokesman for national medical marijuana advocacy group Americans for Safe Access, said he welcomed Holder's perspective.
"It signals a new direction and a more reasonable and sensible direction on medical marijuana policy," he said.
Still, Hermes said his Oakland-based organization was concerned about the fate of more than two dozen California medical marijuana cases currently pending in federal court.
"There remains a big question as to what the federal government's position is on those cases," Hermes said. He pointed specifically to the case of Charles Lynch, who was federally convicted for running a medical marijuana dispensary collective in San Luis Obispo County last year.
Hermes said Lynch could face decades in prison when he is sentenced Monday even though his clinic had been compliant with state law.
According to the government's sentencing recommendation for Lynch, which says the five-year mandatory minimum prison term is an appropriate one, Lynch had violated California state law because his "operation was rife with activities having more to do with business and casual drug distribution than anything medical."
U.S. attorney's office spokesman Thom Mrozek declined to comment on what would happen to the outstanding marijuana cases in the Los Angeles area.
The 13 states that permit medical use of marijuana are Alaska, California, Colorado, Hawaii, Maine, Michigan, Montana, Nevada, New Mexico, Oregon, Rhode Island, Vermont and Washington.