Riding the Low are fronted by actor Paddy Considine, who was inspired to start a band after seeing Guided by Voices play live. "Robert Pollard's songwriting is just incredible. The immediacy of it just blew me away. His presence on stage... Being a fan felt like being in a gang. I wanted my own gang". 

The name Riding The Low came from a biography on Lee Marvin that Paddy read before the idea of having a band was even conceived. "There was a section in the book where it described the way Lee felt after he’d completed a movie. He used to see a psychiatrist who advised him that after movies he should fill his time doing the things he enjoys to take his mind off things and settle him back into normal life. The psychiatrist called this period 'Riding the Low'."

'Riding the Low' initially came about as a hobbyist outlet for Paddy's musical interest and ability; writing Pavement inspired songs on acoustic guitar on his own, before developing them with The Leisure Society's Nick Hemming.

Upon discovering Robert Pollard's music, Considine's approach and vision for where the music needed to be shifted. Paddy was introduced to composer Chris Baldwin by school friend and ex-bandmate Rich Eaton, "I’d heard [Chris'] home recordings and thought he was a real talent, and somebody who could help me get the songs down." Baldwin brought in fellow guitarist Dan Baker to work with Eaton and drummer Dan Thompson to create the backline for Paddy's songs. Later, drummer Justin Chambers replaced Dan Thompson with a style and dynamic more suited to the developing ensemble. More recently, Justin has also begun composing music for the band.