I would say the the season, biome, proximity to mountain ranges, proximity to water, the elevation, and the local plant life.

The question is a bit unclear. What factors? For what purpose?

When I use weather in tabletop RPGs I typically create a markov chain. Each type of weather has a table of weathers it can become, and that table often includes itself. I then roll on table for the current weather every game day, and that decides what the weather changes to.

Simple example: Sunny: 1-3: sunny 4: rain 5: heat wave 6: overcast

Rain: 1-2: rain 3-4: overcast 5-6: sunny

Overcast: 1-2: overcast 3-5: rain 6: sunny

Heat wave: 1-2: heat wave 3-5: sunny 6: rain

Huh doing it without a map would be tricky. I guess maybe you could decide based on the rough latitude, but I can’t think of anything else that doesn’t need a map.

But also remember that this is your world, you can just pick whatever climate you want and justify it later.