The temp is 35 °C idle and 75 °C after 1-2 hours under consistent load (OCCT / Mafia 3 / Battlefield 1). I have never seen temperatures over 83 °C on the hottest core.
I don't think it's worth to upgrade from a Skylake @ 4,6 GHz because it has the same IPC, therefore you would only have a 5-6% increase of performance.
By the way my settings are all Auto BUT:

AVX Offset: 2
Ai overclock profile: XMP
Core ratio limit: 49
CPU Core / cache voltage: Adaptive
Offset mode sign: +
Additional turbo mode CPU core voltage: 1.25
DRAM frequency: 2666
DRAM voltage: 1.20
CPU VCCIO Voltage: 1.0
CPU System agent voltage: 1.0

Basically I followed the ROG guide at this link and then "retouched" the voltages step-by-step in order to lower the temperatures and test the stability at each adjustment:
https://rog.asus.com/articles/guides/the-kaby-lake-overclocking-guide/

Yes, you're probably right about the upgrade. With my settings below, the CPU barely gets to 60C under load. My VCCIO and System Agent Voltage are set on AUTO but it shows at a much higher voltage than 1.0V. Are yours set to AUTO or did you enter the figures manually? Not sure how low I could go on these since I'm running the memory at 3466MHz (16-18-18-38 1.35V 3200 DIMS)

BIOS version 1302
Ai overclock profile: XMP
Core ratio limit: 46
CPU Core / cache voltage: Adaptive
Offset mode sign: +
Additional turbo mode CPU core voltage: 1.24
DRAM frequency: 3466
DRAM voltage: 1.35
CPU VCCIO Voltage: 1.224 [AUTO]
CPU System agent voltage: 1.280 [AUTO]

No, I adjusted the idle voltage, VCCIO and VCCSA all to 1,0 V manually. When set to Auto, they were at 1,1 - 1,15 V (don't remember well).
The reason why your VCCIO and SA are so high is the very high frequency of your RAM. You could try to lower them at steps of 0,05 V but I don't know if it will be stable then with the RAM at 3466 MHz. By the way if your CPU is stable and cool (60 °C under load is a very good temp), I would rather try to push it at 4800 MHz with Vcore 1,28 - 1,30. You should also set an AVX offset of 1 or 2 because under AVX loads the CPU gets very hot.
In the end it is a matter of trading between RAM frequency (and consequent VCCIO and VCCSA voltages) and CPU frequency.
Maybe the additional thermal headroom obtained by lowering the voltages connected with the RAM controller will give you the opportunity to push the CPU further (better solution for gaming scenarios if you look at the comparison between different frequencies of DDR4 - negligible benefits over 2400 MHz with the core architecture!)

Not sure where this AVX setup is. I cannot find it anywhere. Is it possible because of the BIOS version? I am planning to upgrade with the next BIOS release. What does the AVX offset do anyway?

It sets a negative offset for AVX loads. For example with an AVX offset = 2 if your multiplier is set to 46, when the CPU is under AVX load the frequency will be 4400 MHz (44x100)

Thanks. And where is this AVX offset settings?

In the "Extreme tweaker section". You can read this guide which is comprehensive and with BIOS menu screenshots.
https://rog.asus.com/articles/guides/the-kaby-lake-overclocking-guide/
I believe you need minimum BIOS 2202, better if you update to 3401.