During his time at Miami Northwestern Senior High School, Baker was an All-American Sprinter and participated in the 4 × 100 metres relay on the second leg.
Baker signed with Georgia on national signing day, despite speculation that he would sign with Texas based on a tweet he sent out the night before. As a true freshman at Georgia in 2015, Baker played in 11 games. During the 2016 season, Baker was injured in a game against Vanderbilt. He recovered, but was later injured against Georgia Tech. Following the 2017 season, Baker was named to the Second Team All-SEC. Baker considered declaring for the 2018 NFL draft, but he ultimately elected to return to school for his senior year.
Baker was selected by the New York Giants in the first round of the 2019 NFL Draft with the 30th overall pick where the Giants traded up with the Seattle Seahawks to acquire the pick giving up their 37th, 132nd and 142nd picks. He was the first cornerback to be selected in that draft.