Graham attended Wheaton North High School in Wheaton, Illinois, where he was awarded the National High School Quarterback of the Year honor by The National Quarterback Club in 1986. Recruited by Notre Dame, he won his first career start for the Irish as a freshman against Boston College in 1987. However, the option-run offense implemented by coach Lou Holtz was an imperfect fit for the drop-back passer, and after his sophomore year he transferred to Ohio State, where he was the starter for the 1991 campaign in which the Buckeyes finished 8-4.
Graham began his NFL career with the Giants in 1992, after the team drafted him in the eighth round of that year's NFL Draft. Due to injuries to starters Phil Simms and Jeff Hostetler, Graham was forced into the starting lineup and started three games before he too was injured. He stayed with the team for two more years, only getting one more start, before leaving to play with the Detroit Lions in 1995. After not seeing any action, he signed with the Arizona Cardinals in 1996, where he eventually became the team's starting quarterback. In two separate stints as Cardinals quarterback, Graham threw for a combined 3,032 yards, 16 touchdowns, and 12 interceptions.
Graham returned to the Giants in 1998 to serve as backup to Danny Kanell, who had led the team to the NFC East title the previous year. He moved into the starting lineup in Week 12 after Kanell had led the Giants to a 3 - 7 record in the first ten games. Graham led the Giants to two wins in his first three starts, then secured a major upset over the then-undefeated and eventual Super Bowl champion Denver Broncos in Week 15 by throwing a late touchdown pass to Amani Toomer to clinch the victory. Graham finished the season 5-1 as the team's starter, and won his final four starts.
Graham started the 1999 season as the starter ahead of offseason acquisition Kerry Collins. He held a winning record in his nine starts, finishing 5-4, but struggled most of the year. After back-to-back ineffective performances he was benched during a Week 11 game against the Washington Redskins in favor of Collins. He did not see another snap as a Giant. Graham was released during the off-season (February 10, 2000) due to his lack of consistency and to create room under the salary cap.
In February 2000 Graham was signed by the Pittsburgh Steelers on a $5.1 million deal over three years to replace Mike Tomczak, who had been released in the off-season, as the backup for Kordell Stewart. In the wake of Stewart, who was having to cope with drastic changes to the offensive team members (including Tomczak, who had served as his mentor), having a mediocre 1999 season and a competitive first training camp, Steelers coach Bill Cowher (who, according to media rumors, was nearly fired by the Steelers organization for giving Stewart too many chances) assigned Graham as the starting quarterback for the start of the 2000 season. Despite solid efforts, Graham performed less than impressive, beginning the season 0-3 due to Pittsburgh's defensive weaknesses. In the third game of the season vs. the Tennessee Titans, Graham threw for 254 yards in one of the best offensive showings for the Steelers in over two seasons before injuring his hip late in the game and being replaced by Stewart, who went on to lead the team to two straight victories as Graham recovered. This built a quarterback controversy among fans and media, as many of the Steeler faithful supported Graham ahead of Stewart despite Stewart's recent successes with the team (having led the team to the playoffs twice in his first two full years as starter). However, Graham struggled mightily upon his return to the Steelers and eventually lost whatever playing time he had earned. After platooning with Graham Stewart returned to the starting position and Graham was cut at the end of the year.
In 2001, he was signed by the Redskins. After initial starter Jeff George was cut, Graham became the team's second stringer for much of the season. In 2002, Graham was picked in the expansion draft by the Houston Texans to be their third-string quarterback. He never played again.