Plant-based alternatives (PBAs) are increasingly becoming part of diets. Here, we investigate the environmental, nutritional, and economic implications of replacing animal-source foods (ASFs) with PBAs or whole foods (WFs) in the Swedish diet. Utilising two functional units (mass and energy), we model vegan, vegetarian, and flexitarian scenarios, each based on PBAs or WFs. Our results demonstrate that PBA-rich diets substantially reduce greenhouse gas emissions (30–52%), land use (20–45%), and freshwater use (14–27%), with the vegan diet showing the highest reduction potential. We observe comparable environmental benefits when ASFs are replaced with WFs, underscoring the need to reduce ASF consumption. PBA scenarios meet most Nordic Nutrition Recommendations, except for vitamin B12, vitamin D and selenium, while enhancing iron, magnesium, folate, and fibre supply and decreasing saturated fat. Daily food expenditure slightly increases in the PBA scenarios (3–5%) and decreases in the WF scenarios (4–17%), with PBA diets being 10–20% more expensive than WF diets. Here we show, that replacing ASFs with PBAs can reduce the environmental impact of current Swedish diets while meeting most nutritional recommendations, but slightly increases food expenditure. We recommend prioritising ASF reduction and diversifying WFs and healthier PBAs to accommodate diverse consumer preferences during dietary transitions. The authors found that replacing animal source foods with plant-based alternatives would lead to substantial reductions in environmental impacts, while meeting most nutrition recommendations and being cost-competitive with the current average Swedish diet.