Whitfield started for the New York Giants at left tackle during the 2006 season filling in after starter Luke Petitgout went down with injury. His time as a starter was marked by two episodes of him losing his cool and drawing costly personal fouls, twice on headbutts in key spots against the Dallas Cowboys and later in a week 16 loss against the New Orleans Saints. These actions lead to him receiving the nickname "Head-butt Bob" from many Giant fans as well as local columnists. Due to these actions and general inconsistent play, Whitfield was benched for the final game of the season against the Washington Redskins and the subsequent playoff game against the Philadelphia Eagles.
On February 12, 2007, he announced his retirement on Sirius NFL radio.
Whitfield also serves as a guest analyst for the UK's NFL Coverage on Sky Sports as well as contributing on local Atlanta television and radio.
On November 11, 2011, Whitfield was inducted into Stanford University's Hall of Fame and in an interview, Whitfield admitted he was considering a return to his alma mater to complete his degree. Whitfield returned to Stanford in 2012 and received a degree in economics during a commencement ceremony held Father's Day with his kids present.