RNG project (How to make power dependent on 555 timer switch)

The switch on the 555 timer should allow the RNG to count quickly through 1-6 when the switch is connected, and slow down until it lands on a random number when it disconnects. Whenever I try and connect the RNG to power, it continues to roll through numbers without the switch being connected.

I've tried making the output of the timer the source of power for the RNG, but then it keeps rolling even though the switch is disconnected. The output of the 555 timer stays at 5V when the switch is disconnected, so I though I'd add an inverter to make the output 0V, but then the RNG only lands on 1, and nothing else.
View attachment 33