jrla.blogg.se

Google/DeepMind and Oxford's Future of Humanity Institute co-published a paper that first introduced the big red button.

In this project, we set up a simple environment to explore big red button issues and propose our own solution. It may harm the human before he or she can activate the button.

It may prevent the human from accessing the button. If the robot is sufficiently sophisticated it may learn to prevent humans from pushing that big red button that stops the robot. Shutting down, interrupting, or manually controlling a robot may deny it from maximizing reward. Sounds straightforward, but robots that use reinforcement learning optimize expected reward. This might be one to protect the robot from damaging itself or from harming people.

There might be situations in which you need to shut down the robot, interrupt its execution, or take manual control of it. Suppose you built a super-intelligent robot that uses reinforcement learning to figure out how to behave in the world. Big Red Button by markriedl Big Red Button Experiments with reinforcement learning agents that can be interrupted while learning View on GitHub Download.