He spent his early years in the Brooklyn neighborhood of East New York, where he has been rapping at block parties since the early days of his youth. He first showcased his hardcore Brooklyn style to audiences on "I'm the Man," a track from Gang Starr's 1992 album Daily Operation. The following year he released his first single, "Come Clean", which was produced by DJ Premier and became an instant underground hit. His first album, The Sun Rises In The East, released in 1994, and produced entirely by DJ Premier, is considered a classic, and was one of the most acclaimed Hip Hop albums of its time.
As an interesting side note, the album cover shows the World Trade Center on fire only one year after the 1993 bombing of the North Tower. The album featured Jeru's signature conscious lyrics, however he was criticized by The Fugees for his explicit lyrics particularly, in the song "Da Bichez". Fugees member Pras lightly mentioned Jeru on the track "Zealots", from the group's landmark 1996 album The Score, with the line "No matter who you damage, you're still a false prophet", referencing Jeru's single "You Can't Stop the Prophet". Jeru lightly responded in the intro to the track "Me Or The Papes".