We humans put a lot of stock in our intelligence and self-awareness: it’s taken as a measure of our superiority over other species, that we (alone?) are able to reason about our current circumstances, build tools to change our environment as we see fit, and think about thinking itself. We are self-aware, “conscious”, we say – ignoring the fuzziness of the concept – and it is this particular cerebral quirk that separates us from (or elevates us above) the animals.
Imagine, then, a “created intelligence”: self-aware software that demonstrates at least some of the characteristics that we associate with human(-level) consciousness and intelligence.
Wait But Why has an awesome series on the Artificial Intelligence Revolution, which you should definitely read if you haven’t. Or re-read if you have – I know I will.