I've just noticed a clicking sound, it is usually when the car is idling and nothing else is on. It is very troubling to me, can't figure out what could be causing it. Any ideas?

 

Also notice that a belt is slipping, worse in cold weather.

Should I be worrying about these things?

 

yes.

clicking may be a blend door actuator in the ventilation system, try changing floor/defrost/vent/off/etc see if it quits... hope its somethinf else, but... Ive never had my dash apart, but know some cars can be a nightmare to change these things

on the belt squealling,if its a V6 , take a look at the crankshaft pulley and observe the belt is riding in the grooves properly- the 4.0 V6 has been known to occasionally have failures in the harmonic balancer- big name for a counterweighted hub with rubber c\glued to a outer damper ring- on the 4.0, the outer ring is the pulley- when they fail, the pulley can walk off the assembly, and belt can hop completely off and ride on the now exposed rubber ring- saw pics of a couple doing this.
My wifes V6 had a squeaky belt when cold, I checked, pulley was fine, but belt sounded bad...just put a new belt on and it quit- belt looked fine, but swapping it quieted it down... noisy belt should be checked sooner rather than later- my sons expedition was making some squeaking noise, swapped the belt and here almost 4" of the vee ribbing was broken away in one spot- was running on the aramid fiber backing only! his belt hopping around was probably beating the heck out of alternator bearings and idler...