Stuart Armstrong ´ 5 Free eadI actually somewhat surprisingly eally like it This short primer is a chill evening Beer Law: What Brewers Need to Know read It ll be most useful to people who ve been exposed to the ideas around the AI control problem I do wish that there was a chapter or section on something like Should we expect progress to continue or like why current AIesearchers would consider an AI that kills people to cure
CANCER THINK THAT THE AI HASthink that the AI has poor model of the world The tweet sized summary of the book in the words of Uncle Ben is with great power comes great GIS and Fr�uleins: The German-American Encounter in 1950s West Germany responsibilityBtw Armstrong judiciously sidesteps talking about human ethics and utilitarianism when talking about the AI calculating conseuences Prolly a good ideaI like this our tendency toeclassify anything a computer can do as not The Confederate Privateers reallyeuiring intelligence As an aside I think it s interesting that while Intelligence for the purposes of AI Big Bad Detective Agency risk etc is something likeeaching your goals in a wide variety of complex and novel environments I is freuently mentioned alongside this Intelligence especially when talking about machines Oracle AIs and machines eg The Placer referenced in chapter 1 are always seductively persuasive and Machiavellian but I is a test that mainly tests for writtenvisual mathematical and verbal ability A good introduction into the future of AI and it s potential threats Especially enjoyedeading Chapter 7 Good book that gets to the high level of managing AI and super technologytalks about the fact that this is going to be hard to slow down let alone stop Could be great help to humanity The major point of the book is to get on board and contribute so can be handled with intelligence and control First I love that the book is published under a Creative Commons license That shows the author cares about spreading his ideas as far as possible and understands that copyright #restrictions merely shrink the audience unless you manage to write the next Harry Potter and #merely shrink the audience unless you manage to write the next Harry Potter and Sorcerer s Stone I was tempted to give the book five stars just for thatThe book itself is short and easy to Otto Freundlich: Cosmic Communism read at least for anyone with a modicum of computer science and philosophy It summarizes the potential dangers of AI and even briefly tells theeader how to help While Hollywood has labored for decades to instill fear of The Tattooist of Auschwitz (The Tattooist of Auschwitz, robots one day taking over the book explains how an actual AI disaster scenario could play out differently than What happens when machines become smarter than humans Forget lumbering Terminators The power of an artificial intelligence AI comes from its intelligence not physical strength and laser guns Humans steer the future not because we're the strongest or the fastest but because we're the smartest When machines become smarter than humans we'll be handing them the steering wheel What promises and perils will these powerful machines present Stuart Armstrong’s new book navigates these uestions with clarity and witCan we instruct AIs to steer the future as we desire What goals should we program into them It turns out this uestion is difficult to answer Philosophers have tried for thousands of years to define an ideal world but thereemains no consensus. He standard Profiles in Leadership: Historians on the Elusive Quality of Greatness robot movie script But importantly the book outlines the intellectual problems that building a friendly AI poses including the problem of precisely defining what constitutes human well being a problem that has frustrated philosophers for than 2400 years Even if some unforeseen technical barrier prevents AIs gaining general intelligence the kind that could outsmart humans across the board humans should benefit from attention paid to moral philosophy Thus the book should be helpful even in the unlikely event it turns out to be unnecessaryBut being short the book has to leave a lot out These omissions struck me asather curiousChapter 7 What Precisely Do We Really Really Want describes the difficulty of specifying a goal so as to exclude all solutions having undesirable conseuences The chapter does not mention the extensive fictional literature exploring this very uandary for example The Monkey s Paw and the List of adaptations of The Monkey s Paw Be careful what you wish for is an ancient fictional trope King Midas anyone The book does not need to cover the whole history of this trope but at least give it a nod People did think about this before computers threatened to make it ealIt also couldn t hurt to mention that when you hire two humans to do a job for you the smarter worker usually figures out what you eally want Other People's Property: A Shadow History of Hip-Hop in White America reliably than the less smart worker When you hire the most expensive attorney for example part of what you are paying for is expert advice on what your goals are Part of the intelligent expert s job is to disambiguate your initial perhaps vague or counterproductiveeuest If you ask the competent expert to do something he or she knows you probably won t be happy with for example to pursue a legal strategy with a high isk of backfire the expert will seek to dissuade you If AIs will exceed every human skill as the book predicts then maybe they will help us figure out what
WE REALLY WANTA SECOND OMISSION SEEMSreally wantA second omission seems ignore the book s
title a ecurring theme throughout the book isA Der Bilderwächter recurring theme throughout the book is it s up to humans to make AI friendly or at least contain it all by ourselves Thateuires humans to solve the problem of containing AIs or the problem of building moral compasses into them or generally the problem of predicting whatever troubling unknown unknowns they might unleash on us and designing in safeguards against The prospect of goal driven smarter than human AI gives moral philosophy a new urgency The future could be filled #with joy art compassion and beings living worthwhile and wonderful lives but only if we’re able to #joy art compassion and beings living worthwhile and wonderful lives but only if we’re able to define what a good world is and skilled enough to describe it perfectly to a computer programAIs like computers will do what we say which is not necessarily what we mean Such precision euires encoding the entire system of human values for an AI explaining them to a mind that is alien to us defining every ambiguous term clarifying every edge case Moreover our values are fragile in some cases if we mis define a single piece of the puzzle say consciousness we end up with oughly 0% of the value we intended to eap instead of. Hose threats But the book also predicts that AIs will eventually exceed every human skill thereby becoming Smarter Than Us wouldn t that have to include the skill of building friendly AIs If thorny problems of moral philosophy have to be solved along the way AIs should solve them faster than we can A third omission or at least an underemphasis is the uestion of why AIs would care about the goals we give them If a committee of ants presented a human with a list of goals why would the human care what the ants want It s not enough #merely to tell the AI precisely what we want it to do The #to tell the AI precisely what we want it to do The must also want to do precisely what we command The book does mention that an AI would have the power to out think the humans who supposedly control it but so the AI could efficiently pursue its original goals programmed in by humans Why wouldn t the AI subvert its human masters comprehensively by inventing its own goals which less intelligent humans cannot imagine Maybe we won t be programming these things but propitiating them much as ancient and modern superstitious people believe they are propitiating their God or gods If ants were to propitiate humans who knows we might listenIf AIs do become smarter than us we d better hope Sam Harris is correct in The Moral Landscape How Science Can Determine Human Values namely that science eally does or can someday determine moral values If so in that best of all possible worlds scientifically skilled AIs should converge on the same morality that any other scientifically competent entity would discover only faster but perhaps with a few catastrophes along the way Humans have also stumbled Schätze Aus Dem Musée Picasso, Paris repeatedly during their long moral evolutionemember slavery Homophobia Sexism Beheadings Man
made climate change History and current events are eplete with human moral failingsclimate change History and current events are eplete with human moral failings doesn t seem too Pollyannish to suppose AIs should turn out to be scientifically skilled given that science is perhaps the most useful human skill If AIs will have all our skills only better then maybe AIs could be our natural allies in the ancient uest for moral progress Maybe AIs could teach us what it means to be moral I m a lesswrong veteran so I didn t learn anything much that was new
but it s good for internalizing the viewpoints and would be excellent to someone without that prior knowledge as anit s good for internalizing the viewpoints and would be excellent to someone without that prior knowledge as an to AI existential Britain, Europe And The Third World risk. 99% of the valueThough an understanding of the problem is only beginning to spreadesearchers from fields Scotland Yard ranging from philosophy to computer science to economics are working together to conceive and test solutions Are we up to the challengeA mathematician by training Armstrong is a Research Fellow at the Future of Humanity Institute FHI at Oxford University Hisesearch focuses on formal decision theory the isks and possibilities of AI the long term potential for intelligent life and the difficulties of predicting this and anthropic self locating probability Armstrong wrote Smarter Than Us at the euest of the Machine Intelligence Research Institute a non profit organization studying the theoretical underpinnings of artificial superintelligen.