Background and objectives: The burden of the COVID-19 pandemic remains high among migrant populations, especially asylum seekers (AS). Moreover, the social impact of COVID-19 public health measures disproportionally affected AS. In this context, we aimed to better understand the pandemic experiences and sources of worries of AS, as well as the associated social and structural factors. Methods: We conducted a cross-sectional mixed-method study among AS in the canton of Vaud, Switzerland, which combined and integrated the results of the following quantitative and qualitative methods. First, participants answered questions in an online or paper survey about their worries, sleep problems and fear of dying. The associations with their socio-demographic characteristics were investigated using multivariable logistic regressions. Second, we conducted in person semi-structured interviews with participants who had not completed the survey. We performed an inductive thematic analysis focusing on how social and living conditions affected their experiences and worries. Results: 203 participants answered the surveys and 13 took part in interviews. AS in community centres experienced more sleep disorders related to the COVID-19 pandemic than those living in private apartments. (aOR 2.01, p=0.045). Similarly, those with lower education had greater fear for their life due to the COVID-19 pandemic compared to AS with higher education (aOR 2.31, p=0.015). In addition, interviews showed that having to share living spaces was an important source of worries for AS. Protective measures were perceived to increase social isolation, with negative effects on their pandemic experiences. Conclusions: Our study results suggest that a better understanding of AS experiences and worries could help decision-makers adapt public health policies for this population. These include avoiding high-density facilities, encouraging the transfer of AS from community centres to private facilities, and tailoring health recommendations to the language, cultural and health literacy barriers of AS.