Sampson was already 6 ft 7 in (Script error m) tall by the ninth grade, reaching 7-foot-1 in high school in Harrisonburg, Virginia. He averaged nearly 30 points, 19 rebounds, and 7 blocked shots as a senior (after averaging 14 points and 11 rebounds as a sophomore, and 19 points and 17 rebounds as a junior), at Harrisonburg High, leading the team to state AA basketball championships in 1978 and 1979. His senior year he lost the high school player of the year award to another talented center, Sam Bowie. However, he did get a form of revenge against Bowie, outplaying him in the Capital Classic, getting 23 points and 21 rebounds with 4 blocks in a game styled "Battle of the Giants".
Sampson was arguably the most heavily recruited college basketball prospect of his generation and appeared on the cover of Sports Illustrated six times in a span of less than four years (December 17, 1979; December 1, 1980; March 30, 1981; November 29, 1982; December 20, 1982; and October 31, 1983).
Playing center for the University of Virginia, he led the Cavaliers to an NIT title in 1980, an NCAA Final Four appearance in 1981 and an NCAA Elite Eight appearance in 1983. He earned three Naismith Awards as the National Player of the Year, only the second athlete to do so (Bill Walton was the first), and a pair of Wooden Awards. Sampson considered leaving Virginia after his junior year and declaring for the 1982 NBA draft. The San Diego Clippers and Los Angeles Lakers would flip a coin to determine who would draft first overall, but the deadline for Sampson to make himself available came before the scheduled coin flip. Rather than risk playing for the Clippers (who ended up losing the toss), Sampson stayed in school.
The Rockets managed only a 29–53 record in 1983–84, which qualified them to pick first in the 1984 NBA draft. Houston selected fellow center Hakeem Olajuwon out of the University of Houston. Many observers criticized the Rockets' choice, believing the two 7-footers (known as the Twin Towers) would not be effective playing together, while others thought the combination could be overpowering. Sampson, playing a new style of power forward, had new expectations placed upon him. At the time, Dallas Mavericks Coach Dick Motta said, "That front line, when history is written, when they've grown up, might be the best ever assembled on one team. Ever." Houston guard John Lucas said of Sampson's move to forward, "He'll revolutionize the game."
In 1984–85 the Rockets improved by 19 games to 48–34 and made the playoffs for the first time in three seasons. Sampson had his best individual campaign, averaging 22.1 points and 10.4 rebounds and earning a berth on the All-NBA Second Team. He and Olajuwon both played in the 1985 NBA All-Star Game, and Sampson, after scoring 24 points and grabbing 10 rebounds, earned the game's MVP Award. On March 5, 1985, in a loss against the Denver Nuggets, Sampson recorded 30 points, 15 rebounds, 8 assists and 5 steals and was the first player in NBA history to record at least 30 points, 15 rebounds, 5 assists and 5 steals since the league started recording steals.
The next season Houston won the Midwest Division with a 51–31 record. In the playoffs, the Rockets swept the Sacramento Kings, but faced a stiffer challenge against Alex English and the Denver Nuggets in the Conference Semi-Finals, eventually winning the series 4–2, with the sixth and deciding game going to double overtime. Against the defending champion Lakers in the Conference Finals, the Rockets were ready to knock off their rivals who had the best of them during the season. The Rockets lost game 1, but the Rockets fought back, winning four straight to take the series four games to one. In Game 5 of that series, Sampson provided one of the most memorable moments in NBA Playoff history. With the score tied at 112, Olajuwon having earlier been ejected, and with only one second remaining on the clock, Sampson took an inbounds pass and launched a twisting turnaround jumper that sailed through the hoop at the buzzer, giving the Rockets a 114-112 victory and a shocking series upset.
In the NBA Finals the Rockets faced the Boston Celtics. Boston sportswriters were not happy about not getting revenge against the Lakers who had beaten the Celtics in the Finals the year before, but the matchup was interesting with the young front court challenging the old guard of the Celtics. During the season at the Boston Garden, the Rockets were playing the Celtics well until Sampson suffered a jarring fall on his back. At the start of the Finals, Sampson quickly found himself in foul trouble early in Game 1 as Boston easily went up 2-0 going back to Houston. The Rockets won a close Game 3 under the leadership of Sampson. Game 4 also went down to the wire with the Celtics pulling it out on late Larry Bird 3-pointer heroics and untimely turnovers by Rockets guard Mitch Wiggins. In a similarly close Game 5 in Houston, (under the 2–3–2 format) Sampson succumbed to taunting by Boston's much smaller 6-foot-1 backup guard Jerry Sichting resulting in Sampson taking a swing and earning an ejection from the game. Strangely, this fired up the Rockets, who won Game 5 by 15 points without Ralph thanks to the inspired play of Olajuwon, Jim Petersen, and Robert "Bobby Jo" Reid. Game 6 went back to Boston with Sampson finding himself again in foul trouble and of little effect against the older and wiser Celtic front court of Bird, McHale and Parish. After the series, Boston coach KC Jones called the Rockets, "the new monsters on the block" with the future looking very bright for the Rockets. During the six-game championship series loss against the Celtics, Sampson averaged 14.8 points on .438 shooting, 9.5 rebounds and 3.3 assists per game.
Injured halfway into the 1987–88 season, Sampson fell out of favor with Rockets Coach Bill Fitch and was traded, along with guard Steve Harris, to the Golden State Warriors for Eric "Sleepy" Floyd and Joe Barry Carroll. Sampson's knee and back troubles worsened, and he never played a full season in the remaining years of his NBA career. He averaged 6.4 points and 5.0 rebounds with Golden State in 1988–89 and was traded to the Sacramento Kings for Jim Petersen. Sampson's injury issues continued in Sacramento as he totaled just 51 games in two seasons, averaging 4.2 and 3.0 points, respectively, in 1989–90 and 1990–91. Released by the Kings, Sampson played a 10-game stint with the Washington Bullets in 1991–92 before being waived. He played 441 games in 10 NBA seasons, slightly more than half the 820 scheduled.
In 2005, Sampson pleaded guilty to owing more than $300K in back child support for two children from different mothers in the Northern Virginia area. In 2006, he was sentenced to two months in prison for mail fraud associated with the purchase of an SUV.
On November 22, 2011, Sampson was inducted into the National Collegiate Basketball Hall of Fame. In February 2012, Sampson was honored by Houston Rockets and fans as a member of the Decade Team of the 80s. On April 2, 2012, Sampson was named a member of the Naismith Memorial Basketball Hall of Fame's induction class of 2012.
In October 2012, Sampson joined the Phoenix Suns' player development staff. In June 2013, Sampson announced that he would not return as an assistant head coach.