James grew up in a fiddling family in Drimnagh, Dublin and bucked tradition by taking up the button accordion at the tender age of six. He and his brother Seán (who would later play the fiddle with the Chieftains) were among the founders of the famed Castle Céilí Band in the early 1960s. Later in that decade, James found himself much in demand during the “ballad boom,” often playing multiple gigs in one night. He moved to New York in 1968 and spent years partnering singer Jesse Owens and touring with the Canadian band Ryan’s Fancy. He then settled down in Queens and worked a steady day job but continued to issue new solo recordings, to play in some of the city’s pubs, in concert with Mick Moloney or singer Robbie O’Connell and at festivals.