Sparma received a $32,000 bonus to sign with the Detroit Tigers in 1963. He played six seasons with Detroit, from 1964-1969. In 1965, Sparma was in his first full season with the Tigers, when he was assigned to be the starting pitcher against the Yankees on "Mickey Mantle Day" in New York. When Mantle came to bat for the first time in the game, Sparma walked off the mound, approached Mantle, and said: "You know, I've never had a chance to meet you in person, and I've always admired you." Sparma and Mantle shook hands, and Sparma went back to the mound and struck Mantle out. Mantle turned to Detroit's catcher Bill Freehan and said: "They have a day for me and your manager's got to put some hard-throwing kid out there. Couldn't he have put in some soft-tossing left-hander for me to hit off of, so I could look like a hero in front of all those people?" (Bill Freehan, "Behind the Mask" (1970), pp. 7–8)
In 1968, Sparma lost his spot in the rotation after a run-in with manager Mayo Smith. Sparma was pulled from a game and told Detroit sportswriter Joe Falls that Smith "had no respect for him; didn't like him, didn't trust him, etc., etc." When Falls asked Smith for a response, Smith said he "didn't want to get into a spitting contest with a skunk." (Joe Falls, Joe Falls 50 Years of Sports Writing (Sports Publishing 1997), p. 159.) Smith refused to start Sparma for several weeks afterward. When Sparma finally got another start on September 17, he pitched a one-run, complete game win against the Yankees to clinch the Tigers' first pennant since 1945. The Sparma-Smith feud continued, and Sparma pitched only 1/3 of an inning in the 1968 World Series, giving up two earned runs for a 54.00 ERA in postseason play.
During the 1969 season, Sparma roomed with Freehan, as Freehan kept a diary that was published in 1970. In the book, Freehan wrote that Sparma and Smith were usually not even on speaking terms as Sparma remained in Smith's doghouse.
Freehan also wrote memorably of Sparma's talent for ESP: "My roomie's got an ESP thing. Every now and then, he'll wake up on a morning he's supposed to pitch and say, 'I've got a good feeling. I know we're going to win.' Joe doesn't get this feeling too often, but every time he does -- every single time -- we win." (Freehan, "Behind the Mask," p. 104) Sparma had that "good feeling" on June 13, 1969, before he shut out the Royals, 6-0. On another occasion, Sparma told teammate Dick McAuliffe of a premonition: "Mac, I know you're going to have a good day today. Something just tells me you're going to go three for five." McAuliffe went 3-for-4 in his first 4 at-bats and struck out in his 5th at bat. When he came back to the dugout, McAuliffe blamed Sparma for jinxing him on the 5th at bat. In his book, Freehan wrote that McAuliffe was "actually angry" at Sparma. "He could've at least thanked Joe for the three hits." (Freehan, "Behind the Mask," p. 106)
Toward the end of the 1969 season, Sparma got into a fight in the bullpen with Fred Lasher. On December 3, 1969, Sparma was traded to the Montreal Expos. Sparma pitched only 27 innings for the Expos in 1970, compiling a record of 0-4. Sparma pitched his last major league game on May 12, 1970.
In 183 career games, he had a 52-52 record with a 3.94 ERA. Sparma died in Columbus, Ohio at age 44 in 1986, after suffering a heart attack and undergoing heart bypass surgery.