Alternating flashing LEDs with a just a single mechanical switch can be done.
Affiliate link ad.
Affiliate link ad.Nice looking supply.
The circuit as shown in the diagram will work well with a 5V supply (Vcc). Higher voltages will likely need more than 220Ω from Vcc to the switch if current/power limits of any component is exceeded.
As an Amazon Associate I earn from qualifying purchases.