Flaherty began his coaching career at his alma mater, Delone Catholic High School in McSherrystown, Pennsylvania from 1978-80. Flaherty moved to the collegiate ranks, coaching the offensive line at his alma mater, East Stroudsburg University, from 1980-82.
He then joined the Penn State coaching staff from 1982-1993 as an assistant under offensive line coach Dick Anderson. There he helped the Nittany Lions win the 1982 National Championship with a Sugar Bowl victory over Georgia. When Anderson left to become head coach at Rutgers prior to the 1984 season, Flaherty joined him on the Scarlet Knights' staff, where he remained through the 1991 season.
Flaherty spent the 1992 season coaching the defensive line at East Carolina. From 1993-98 he was on the staff at Wake Forest, where he coached the offensive line, tight ends and special teams. In 1999, he coached tight ends and special teams and was in charge of recruiting at the University of Iowa.
Flaherty joined the Giants in 2004.