We present a large-scale anatomically constrained spiking neuron model of the lateral geniculate nucleus (LGN), which operates solely with retinal input, relay-cells and inter-neurons. We show that inter-neuron inhibition and sparse connectivity between LGN cells could be key factors for explaining a number of observed classical and extra-classical response properties in LGN of monkey and cat. Among them are: (i) Weak orientation tuning, (ii) Contrast invariance of spatial frequency tuning in the absence of cortical feedback, (iii) Extra-classical surround suppression, and (iv) Orientation tuning of extra-classical surround suppression. The model also makes two surprising predictions: (a) A possible pinwheel-like spatial organization of orientation preference in the parvo layers of monkey LGN, much like what is seen V1, and (b) A stimulus-induced trend (bias) in the orientation and phase preference of surround suppression, originating from the stimulus discontinuity between center and surround gratings rather than from specific circuitry.