Greenbaum was born in Malden, Massachusetts. He was raised in a traditional Jewish household and attended Hebrew school. His initial interest in music was sparked by southern blues music and the folk music that was popular in the late 1950s and early 1960s. He performed with various bands in high school and studied music at Boston University for two years. In college he performed at local coffeehouses but eventually dropped out and moved to Los Angeles in 1965. He currently resides in Santa Rosa, California.
Greenbaum is best known for his song "Spirit in the Sky". The song, with its combination of 'heavy' guitar, hand-clapping, and spiritual lyrics, was released by Warner Bros. Records in 1969. It sold two million copies in 1969 and 1970, and received a gold disc from the R.I.A.A.. It has subsequently been used in many films, advertisements, and television shows.Although "Spirit in the Sky" has a clear Christian theme, Greenbaum was and remains an observant Jew. Greenbaum says he was inspired to write the song after watching country singers Dolly Parton and Porter Wagoner singing a religious song on television. In an interview Greenbaum stated that western movies were the real inspiration for "Spirit In The Sky":
Norman Greenbaum: If you ask me what I based “Spirit In The Sky” on ... what did we grow up watching? Westerns! These mean and nasty varmints get shot and they wanted to die with their boots on. So to me that was spiritual, they wanted to die with their boots on.
Ray Shasho: So that was the trigger that got you to write the song?
Norman Greenbaum: Yes. The song itself was simple, when you’re writing a song you keep it simple of course. It wasn’t like a Christian song of praise it was just a simple song. I had to use Christianity because I had to use something. But more important it wasn’t the Jesus part, it was the spirit in the sky. Funny enough… I wanted to die with my boots on.
Though Greenbaum is generally regarded as a one-hit wonder; several of his records placed prominently in the charts. In 1966, under the name Dr. West's Medicine Show and Junk Band, he recorded the novelty hit "The Eggplant That Ate Chicago".