Democratic Gov. Jon Corzine supports the legislation and could sign it before leaving office next week, making it law.
The bill allows patients with ailments such as cancer, AIDS and multiple sclerosis to buy up to 2 ounces of marijuana a month at state-monitored dispensaries.
Assemblyman Reed Gusciora, a Princeton Democrat, was a co-sponsor of the bill and pushed for it for years.
He said medical marijuana can alleviate suffering and there's no evidence it increases overall drug use.
"I don't think we should make criminals out of our very sick and terminally ill," he said.
Incoming Republican Gov. Chris Christie, a former federal prosecutor, said he supported the concept of the bill but remained concerned that a loophole could lead to abuses.
A compromise bill was worked out after some other lawmakers expressed similar concerns. For example, a provision allowing patients to grow marijuana was removed.
Driving while high would continue to be against the law.
The other states that permit medical use of marijuana are Alaska, California, Colorado, Hawaii, Maine, Michigan, Montana, Nevada, New Mexico, Oregon, Rhode Island, Vermont and Washington.
New Jersey's legislation authorizes the Department of Health to issue to patients with "debilitating medical conditions" registry ID cards that allow them to use marijuana.
Patients with specified diseases such as cancer and glaucoma must also demonstrate severe or chronic pain, nausea, seizures, muscle spasms or wasting syndrome to qualify.
Patients with registry cards would be immune from arrest or prosecution for the medical use of marijuana.
Gusciora said the legislation, titled the Compassionate Use Medical Marijuana Act, would be the nation's strictest such law.
Lawmakers said they wanted to make sure New Jersey didn't duplicate California's liberal marijuana laws, which they said border on decriminalization of the drug.
"We have learned from the mistakes of other states," Senate co-sponsor Nick Scutari, a Democrat from Linden, said after the vote.
The Senate vote was 25-13; the Senate previously approved a less restrictive version. The earlier Assembly vote was 48-14.
Advocates and patients, who had waited for hours for the final vote, cheered the outcome.
Nancy Fedder, a 62-year-old multiple sclerosis sufferer who lives in Hillsborough, gets around in a scooter and said she has used marijuana for years to reduce pain.
"I'm in heaven," she said after the Senate vote. "It means I am no longer a criminal in the state of New Jersey."
Roseanne Scotti, director of Drug Policy Alliance New Jersey, an organization that says it's dedicated to making the state a leader in drug policy reform, thanked lawmakers for voting their consciences.
"We are absolutely thrilled," she said. "This really was a triumph of compassion."