There is great variability in whether foreign sounds in loanwords are adapted, such that segments show cross-word and cross-situational variation in adaptation. Previous research proposed that word frequency, speakers' level of bilingualism and neighborhoods' level of bilingualism can explain such variability. We test for the effect of these factors and propose two additional factors: interlocutors' level of bilingualism and the prestige of the donor language in the loanword's domain. Analyzing elicited productions of loanwords from Spanish into Mexicano in a village where Spanish and Mexicano enjoy prestige in complementary domains, we show that interlocutors' bilingualism and prestige influence the rate of sound adaptation. Additionally, we find that speakers accommodate to their interlocutors, regardless of the interlocutors' level of bilingualism. As retention of foreign sounds can lead to sound change, these results show that social factors can influence changes in a language's sound system.