I have just built a 6 x 4 layout on a folding board for my 3 year old grandson.
He plays with the trains very well and the only problem we have had is when the driving rods on the Thomas loco kept coming off the wheels. Even after it had been returned to Hornby for repair !
He is well versed in the speed of the loco and he does take great care.
He is left alone for periods driving the train on his own.
So really, it is personal choice with a large dose of how the child reacts to his new train set.
I am happy to let him run his train as he wants. We all have to learn !