Luis Rubiales has refused to resign after kissing forward Hermoso on the lips following Spain's Women's World Cup final win over England in Sydney.
The Spanish government started legal proceedings seeking to suspend the 46-year-old, while Fifa has also launched disciplinary proceedings.
Luis Rubiales had been widely expected to resign at an extraordinary general assembly called by the Royal Spanish Football Federation (RFEF), but instead said "I don't deserve this manhunt".
He added: "It was a spontaneous kiss. Mutual, euphoric and consensual. That's the key. A consensual 'peck' is enough to get me out of here?"
But a statement released by players' union Futpro included quotes from Pachuca's Hermoso, which read: "I want to clarify, that at no time did I consent to the kiss. I don't tolerate that my word is questioned, much less that words are invented that I haven't said."
The statement was signed by a host of players, including all 23 members of the squad which just won the World Cup.
It read: "After everything that happened during the delivery of medals of the Women's World Cup, we want to state that all the players who sign this letter will not return to a call for the national team if the current leaders continue."
Spain's next game is against Sweden in the Nations League on 22 September.
Borja Iglesias, who plays for Real Betis, said earlier on Friday he would not play for the men's national team again while Rubiales is in charge.
The Spanish government will ask Luis Rubiales to explain himself to a Spanish court as soon as possible, secretary of sport Victor Francos said earlier on Friday.
If the administrative court deems he violated the professional sports code, he could then be suspended.
Meanwhile, Fifa will look at whether his actions constitute violations of Article 13 in its disciplinary code, concerning offensive behaviour and fair play.