Ujjain is one of the underrated religious city of India. Few historical references for this city are listed below -
For Devotees -
1) Sinhasth (Kumbh) mela at every 12 years, last held in April-2016, mainly to take a bath / dip in river Shipra (most famous ghats are - Ram ghat / Narsingh ghat). And, the reason for sinhasth to be organizated in Ujjain as one of the four place is, during sagar manthan, drops of nectar had falllen in Ujjain too.
2)...
more...

Mahakaleshwar Temple, which is one of the 12 jyotirlinga & only one facing towards south. Morning Bhasma aarti is treat to watch, thought you may need to book in advance and online booking option is available for that. Booking has to be done atleast a day in advance, can be done through temple counter, may need to spend hours in queue. So, online booking would be the best option. Rudrasagar pond adds beauty to this temple. Nearby temples (all walkable) to visit are Bade ganesh mandir, Harsidhi temple, Santohi mata mandir, Char dham temple, Ram mandir, Gopal mandir. Distance from Ujjain railway station is 1 KM.
3) Its an place where Lord Krishna has got his education from guru sandipani, the ashram is located in peaceful area, around 3 KM from Ujjain railway station. There are 3 must to visit temple nearby (within 500 meter), which are -
(i) Mangalnath temple (famous for bhat puja, one who has manglik dosh in kundli), situated on the bank of river shipra. Ghats are beautiful too.
(ii) Kalbhairav / vikrant bhairav temple (Most curios temple among tourist as lord Bhairav drink madira/wine)
(iii) Sidhanath temple (Temple famous for pind daan rituals), situated on the bank of river shipra.
4) Pirmatsyendranath samadhi sthal - Another place on the bank of river shipra, one of the greatest saint who has achieved maximum level of control over the Indriya. Nearby temples to visit are Gadh-kalika mata temple (enjoy bite of tamarind / plum), Bharthari caves (caves are must watch), Rinmukteshwar mahadev. Distance from Ujjain railway station is 3 KM.
5) Jantar mantar (Onservatory) - Its one the four observatory in India, other 2 which are opration now are in Delhi and Jaipur. It was constructed in 17th century. Distance from Ujjain railway station is 1.5 KM
6) One might have read the Vikram baital story, idol of King Vikramaditya with 32 putli can be seen just behind Harsiddhi temple.
For Foodies - You should not miss to taste Ice cream / Kulfi from Famous shop (Gopal mandir / Mahakaleshwar) & Poha Jalebi at railway station. Try some namkeen from Bafna shop (sati gate, Gopal mandir area), Kachori (Dhaba road, Gopal mandir area)
For Bikers - Enjoy riding to -
(i) Chintman ganesh temple , 8 KM from city center towards Badnagar,
(ii) Gambhir dam, 15 km from city center towards Badnagar, during rainy season.
(iii) Hamu khedi temple 8 KM from city center towards Dewas
(iv) KD (Kaliadeh) palace - 10 KM from city towards Agar road, during rainy season. On the bank of river Shipra

Bus rides are available at cost of Rs 70 to visit all temples in Ujjain in period of 5 hrs.
Nearest Ariport - Indore (45 KM)