You know the saying, "It's not the heat, it's the humidity". I tend to use a variation of that for winter bicycling in this cold part of the world where I cycle. It's not the cold, it's the wind.
The more I bicycle in the winter, the more I believe that. Only in the past year or so have I got my layering figured out for just about anything the weather can throw at me. That means I don't have to think about it each morning as I dress and I pretty much nail it everyday. It's such a fine line between overdressed and underdressed. If overdressed you will sweat too much and get cold from being wet. If underdressed, you just get cold and can't warm up.
People new to winter riding will look at the air temperature and go from there when dressing for the weather.
Once the temperature drops below freezing I will pay as much, or even more, attention to the wind speed and direction. The clothes you wear insulate and hold the heat in. But the rate of heat loss is determined by the wind speed. The higher the wind, the faster your heat is moved away from your body and the quicker you start feeling cold.
Here's an example from Monday. I dressed exactly the same for both my morning commute and my afternoon commute. The morning air temp was 2F/-17C degrees. The afternoon air temp was 17F/-8C degrees. Even though the afternoon air temp was 15 degrees higher, I was able to stay much warmer on the morning commute. Why? The air was calm in the morning, while in the afternoon I had a 18-20 mph sustained headwind. The calm morning air allowed my insulating layers to work. Later in the day, the wind sucked all the heat my body was producing right out of the insulating layers.
It takes time and experimentation to figure out what works best for you. But a mix of insulating base layers and windblocking outer layers can help you keep the proper amount of heat next to your body.
When dressing for outdoor activities in frigid temperatures always pay attention to the wind speed and dress appropriately.