Whatever output you made to toggle the wall, duplicate it and add a 5s delay.
You can choose to ignore below - this is another possible solution.
You can use the func_brush entity. Give them all a "texture" render mode of 0. Then instead of just toggling the faces you can manually hide or show them.
When you want to show the die faces, send the "alpha" command of 255.
When you want to hide the die faces, send the "alpha" command of 0.
Then after each button press, you randomly show one, but then hide all after 5s delay:
On press > func_wall_toggle_1 > alpha > 0 > Delay: 5s
On press > func_wall_toggle_2 > alpha > 0 > Delay: 5s
On press > func_wall_toggle_3 > alpha > 0 > Delay: 5s
On press > func_wall_toggle_4 > alpha > 0 > Delay: 5s
On press > func_wall_toggle_5 > alpha > 0 > Delay: 5s
On press > func_wall_toggle_6 > alpha > 0 > Delay: 5s