Grosser attended Harvard College. Although his academic field of concentration was mathematics, he was already an active painter in his student years. His first one-man exhibition was mounted in the University's Sever Hall in 1923, the year of his graduation. A Sheldon Fellowship awarded by Harvard provided for two years travel in France and Italy, and in 1925 his first professional one-man exhibition was presented by the Grace Horne Gallery, Boston. Grosser also worked at the Art Students League in New York.
His landscapes and still lifes reflected his long periods living a working abroad, in Europe, the Near East, North Africa, and South America; in the United States he passed extended periods of time in New England, the Southwest, and California. As a portraitist his distinguished sitters included Alfred North Whitehead, Mary Garden, and Christopher Isherwood.
A writer as well as painter Grosser was the author of several books expressing his views on painting, Painting In Public, 1948; The Painter's Eye, 1951; The Critic's Eye, 1962; and Painter's Progress, 1971. From 1961 to 1967 he was art critic of The Nation. He wrote the scenarios for the operas by Virgil Thomson and Gertrude Stein, Four Saints in Three Acts, 1934, and The Mother of Us All, 1947, and worked with Thompson again in 1985 on 18 Portraits, a collection of Grosser's lithographs and the composer's musical images.
In 1970 Grosser was a visiting professor of art at the University of Ife in Nigeria, where a one-man exhibition of his work was also presented that year. From 1979 he was represented by the Fischbach Gallery, New York.
