Swartwout graduated from Yale in 1891 and then entered the office of McKim, Mead and White. In 1904 he married Isabelle Geraldine Devenport of Cambridge, England.
Swartwout was active at the American Institute of Architects, where he prefected the code of competitions, worked for the American Battle Monuments Commission and served as vice chairman of the National Commission of Fine Arts. He also published a number of articles on architecture.
Swartwout's buildings include: the Missouri State Capitol and Department of Highways building at Jefferson City; the Mary Baker Eddy Memorial erected in Mount Auburn Cemetery, Cambridge, MA; the Somerset County Court House at Somerville, NJ; an addition to the Museum of Fine Arts at Yale University; the Elks National Memorial Headquarters building, Chicago; the Municipal Auditorium at Macon, Georgia; the Yale Club, New York City; the U.S. Post Office and Court House, Denver, Colorado; and the George Washington Memorial Building, Washington, DC.