Mrs. Pomeroy had her former home, a lovely Federal-style house, removed from the site in 1869, shortly after her husband’s death, and commissioned Lambert and Bunnell to build a French-roofed house, all the rage in Civil War-era America.