Is horsehair plaster on chimney safe?

I was planning to install a woodstove into the old chimney and ripped off the panelling around it and discovered that the chimney was previously coated with what appears to be horsehair plaster (no lath boards -just a thick coat of plaster). The chimney would be lined with a new steel insulated flu and the contractor didn't seem concerned, but I am wondering if the heat could cause the plaster to combust or give off fumes. If so, is there a way to remove the plaster without ruining the masonry and mortar? Thank you.