The Future Simple tense is used to talk about future events. The auxiliary verb 'will' is used.
In this case, we need to use 'will be'.
Answer: It will be windy in Madrid tomorrow.