Does it matter what the survivor was wearing when the attack happened?

No. Sexual assault has nothing to do with what you were wearing during the attack. The only person responsible for a sexual assault is the assailant. You always have to right to choose with whom, when, and if you have sex. Sex without consent is rape, regardless of what someone may wear leading up to the attack.