Can I prune my fig tree eventhough new growth has begun?

My 20 year old fig tree has gotten too large (15-20 ft) , and too tall. I need to cut it bak to about 8 ft tall and remove a large limb. I failed to do so this winter. Now there is new spring growth, and formed figs. Can I cut the tree back radically, and not hurt the tree for next year?