Born to wealthy parents, de Forest grew up in New York City, spending his summers on the family estate at Cold Spring Harbor, New York. At fourteen he made his first trip to Europe, and four years later he went abroad again, spending the winter in Rome. There he took lessons from Hermann Corrodi and Frederic E. Church. Church's wife, Isabel, was related to de Forest. Church took an interest in the young artist, continuing to offer advice and criticism when they returned to New York. With the addition of a brief period of study under James Hart in New York in 1870, this was the extent of de Forest's formal artistic education.
In 1872 de Forest began contributing landscapes to the Academy's annual exhibitions. Three years later, he once more went abroad with his family, this time traveling to Egypt, Palestine, Syria, and Greece. His first exposure to Eastern cultures was a formative experience. From 1877 the paintings he contributed to Academy annuals were mainly Orientalist in subject. De Forest also developed a particular interest in Middle Eastern architecture. Returning to New York in 1878, he remodeled his father's house in an Eastern style, and he began working in interior decoration with Samuel Colman and Louis Comfort Tiffany. As this work expanded, de Forest found less time to paint, a fact that may have postponed his election to the Academy for a decade. He married Meta Kemble in late 1880, and the two left immediately for India. There de Forest created his greatest decorative enterprise. He established traditional wood-carving workshops in Ahmadabad that he directed for more than twenty-five years. Breaking a partnership with Tiffany, he opened a New York showroom in 1882 and also began publishing books on Indian architecture.
Over the next two decades, de Forest traveled in Europe and India, vacationed in Maine and Vermont, managed his successful business, and painted when he could. By 1902 he had begun summering in California, where the desert landscape and trips to the Grand Canyon and Mexico inspired him to paint more frequently. Selling his business to Tiffany Studios in 1907, de Forest occupied himself with studying the landscape on both American coasts. After a final trip to India, Korea, Japan, and China in 1913-14, he built a home in Santa Barbara and gradually broke his ties with the East Coast. In his later years, de Forest wrote a great deal about art theory, museum education policies, and human development. As he noted in a biographical questionnaire circulated by the Academy, he hoped "to solve the problem of man's real place in the world and his education to fit him for his place in harmony with all creation instead of the discord we see increasing everywhere today."
Despite the fact that he was not always a full-time painter, de Forest took an interest in Academy affairs while he lived in New York. He served as treasurer from 1899 to 1907 and was one of a group of six members who acquired Samuel F. B. Morse's miniature self-portrait (n.d.) for the Academy collection in 1900.