I would say to start out moderately and see how you feel with training and where your weight falls naturally.
First take out the milk and see where your weight goes. If training keeps going well and you keep making improvements it may be a case where you can stick to the weight you are and continue making gains.
You don't want to have to really work to get down in weight. Cutting hard or too much will definitely have a negative effect on your training long term. It is best to try and see where your weight sits without having to force feed or restrict. Cutting down 11kg will be a pretty big cut.
Try taking out the milk and going back to your regular eating habits and see where the weight goes and then make an adjustment from there.