We think that powerful, significantly superhuman machine intelligence is more likely than not to be created this century. If current machine learning techniques were scaled up to this level, we think they would by default produce systems that are deceptive or manipulative, and that no solid plans are known for how to avoid this.

We are doing research that we hope will help align these future systems with human interests. We are especially interested in practical projects that are motivated by theoretical arguments for how the techniques we develop might successfully scale to the superhuman regime.

One of our current projects is to develop techniques for controlling text-generating models such that they never output text that violates certain rules (for example, “continue this story in such a way that no-one gets hurt”). We think that this research is a useful early step towards solving analogous behavior control problems for powerful systems that may be developed in the future.  You can read more on our website.

If you would like to help with this work, we are hiring machine learning researchers, software engineers, and operations generalists.

AI/ML

Research

Robotics

Redwood Research

Org chart

Teams

Offices

Similar companies