Pollard attended Albert G. Lane Manual Training High School in Chicago, also known as "Lane Tech," where he played football, baseball, and ran track. He then went to Brown University, majoring in chemistry. Pollard played halfback on the Brown football team, which went to the 1916 Rose Bowl. He was the first black football player at Brown. He became the first black running back to be named to Walter Camp's All-America team.
Pollard, along with all nine of the black players in the NFL at the time, were removed from the league at the end of the 1926 season, never to return again. He spent some time organizing all-black barnstorming teams, including the Chicago Black Hawks in 1928 and the Harlem Brown Bombers in the 1930s.
Pollard coached Lincoln University (Pennsylvania)'s football team during the 1918 to 1920 seasons  and served as athletic director of the school's World War I era Students' Army Training Corps. During 1918–1919, he led the team to a victorious season defeating Howard University's Bisons 13–0 in the annual Thanksgiving classic as well as Hampton (7–0) on November 9, 1918 and teams of military recruits at Camp Dix (19–0) on November 2, 1918  and Camp Upton (41–0). By the fall of 1920, he had begun to play for Akron, missing key Lincoln losses to Hampton (0–14) and Howard (0–42), much to the consternation of the alumni and administration.Paul Robeson was enlisted by Lincoln's alumni to coach the Thanksgiving 1920 game against Howard.
Pollard later criticized Lincoln's administration, saying they had hampered his ability to coach and had refused to provide adequate travel accommodations for the team. "Prior to the Hampton game, the team was compelled to go to Hampton by boat, sleeping on the decks and under portholes," he told a reporter. "No cabins were provided, nor were they given a place to sleep after reaching Hampton. They lost the game through lack of rest." He also blamed the school for not providing the proper equipment. "I, myself, bought and paid $200 out of my pocket for football shoes for the team." He missed the 1920 Howard game, he said, because his Lincoln salary was so low that he was compelled to augment it with pay from Akron.
In the 1930s, Pollard founded his own professional football team, the Brown Bombers. The Depression ended the Brown Bombers’ run in 1938, and Pollard went on to other ventures, including a talent agency, tax consulting and film and music production. He produced Rockin' the Blues in 1956, which included such performers as Connie Carroll, The Harptones, The Five Miller Sisters, Pearl Woods, Linda Hopkins, Elyce Roberts, The Hurricanes, and The Wanderers. Pollard also published the New York Independent News from 1935 to 1942, purportedly the first black-owned tabloid in New York City.