« December 2019 | Main | February 2020 »

Tuesday, January 7, 2020

UNUM 3.1: Updated to Unicode 12.1.0, UTF-8 Support Added

Version 3.1 of UNUM is now available for downloading. Version 3.1 incorporates the Unicode 12.1.0 standard, released on May 7th, 2019. Since the Unicode 11.0.0 standard supported by UNUM 3.0, a total of 555 new characters have been added, for a total of 137,929 characters. Unicode 12.0.0 added support for 4 new scripts (for a total of 150) and 61 new emoji characters. Unicode 12.1.0 added the single character U+32FF, the Japanese character for the Reiwa era. (In addition to the standard Unicode characters, UNUM also supports an additional 65 ASCII control characters, which are not assigned graphic code points in the Unicode database.)

This is an incremental update to Unicode. There are no structural changes in how characters are defined in the databases, and other than the presence of the new characters, the operation of UNUM is unchanged. There have been no changes to the HTML named character reference standard since the release of UNUM version 2.2 in September 2017, so UNUM 3.1 is identical in this regard.

UNUM 3.1 adds support for the UTF-8 encoding of Unicode, and allows specification of characters as UTF-8 encoded byte streams expressed as numbers, for example: 

    $ unum utf8=0xE298A2
       Octal  Decimal      Hex        HTML    Character   Unicode
      023042     9762   0x2622     ☢    "☢"         RADIOACTIVE SIGN
A new --utf8 option displays the UTF-8 encoding of characters as a hexadecimal byte stream: 
  $ unum --utf8 h=sum
     Octal  Decimal      Hex        HTML       UTF-8      Character   Unicode
    021021     8721   0x2211 ∑,∑    0xE28891      "∑"         N-ARY SUMMATION
UNUM Documentation and Download Page

Posted at 15:53 Permalink

Reading List: The Simulation Hypothesis

Virk, Rizwan. The Simulation Hypothesis. Cambridge, MA: Bayview Books, 2019. ISBN 978-0-9830569-0-4.
Before electronic computers had actually been built, Alan Turing mathematically proved a fundamental and profound property of them which has been exploited in innumerable ways as they developed and became central to many of our technologies and social interactions. A computer of sufficient complexity, which is, in fact, not very complex at all, can simulate any other computer or, in fact, any deterministic physical process whatsoever, as long as it is understood sufficiently to model in computer code and the system being modelled does not exceed the capacity of the computer—or the patience of the person running the simulation. Indeed, some of the first applications of computers were in modelling physical processes such as the flight of ballistic projectiles and the hydrodynamics of explosions. Today, computer modelling and simulation have become integral to the design process for everything from high-performance aircraft to toys, and many commonplace objects in the modern world could not have been designed without the aid of computer modelling. It certainly changed my life.

Almost as soon as there were computers, programmers realised that their ability to simulate, well…anything made them formidable engines for playing games. Computer gaming was originally mostly a furtive and disreputable activity, perpetrated by gnome-like programmers on the graveyard shift while the computer was idle, having finished the “serious” work paid for by unimaginative customers (who actually rose before the crack of noon!). But as the microelectronics revolution slashed the size and price of computers to something individuals could afford for their own use (or, according to the computer Puritans of the previous generations, abuse), computer gaming came into its own. Some modern computer games have production and promotion budgets larger than Hollywood movies, and their characters and story lines have entered the popular culture. As computer power has grown exponentially, games have progressed from tic-tac-toe, through text-based adventures, simple icon character video games, to realistic three dimensional simulated worlds in which the players explore a huge world, interact with other human players and non-player characters (endowed with their own rudimentary artificial intelligence) within the game, and in some games and simulated worlds, have the ability to extend the simulation by building their own objects with which others can interact. If your last experience with computer games was the Colossal Cave Adventure or Pac-Man, try a modern game or virtual world—you may be amazed.

Computer simulations on affordable hardware are already beginning to approach the limits of human visual resolution, perception of smooth motion, and audio bandwidth and localisation, and some procedurally-generated game worlds are larger than a human can explore in a million lifetimes. Computer power is forecast to continue to grow exponentially for the foreseeable future and, in the Roaring Twenties, permit solving a number of problems through “brute force”—simply throwing computing power and massive data storage capacity at them without any deeper fundamental understanding of the problem. Progress in the last decade in areas such as speech recognition, autonomous vehicles, and games such as Go are precursors to what will be possible in the next.

This raises the question of how far it can go—can computer simulations actually approach the complexity of the real world, with characters within the simulation experiencing lives as rich and complex as our own and, perhaps, not even suspect they're living in a simulation? And then, we must inevitably speculate whether we are living in a simulation, created by beings at an outer level (perhaps themselves many levels deep in a tree of simulations which may not even have a top level). There are many reasons to suspect that we are living in a simulation; for many years I have said it's “more likely than not”, and others, ranging from Stephen Hawking to Elon Musk and Scott Adams, have shared my suspicion. The argument is very simple.

First of all, will we eventually build computers sufficiently powerful to provide an authentic simulated world to conscious beings living within it? There is no reason to doubt that we will: no law of physics prevents us from increasing the power of our computers by at least a factor of a trillion from those of today, and the lesson of technological progress has been that technologies usually converge upon their physical limits and new markets emerge as they do, using their capabilities and funding further development. Continued growth in computing power at the rate of the last fifty years should begin to make such simulations possible some time between 2030 and the end of this century.

So, when we have the computing power, will we use it to build these simulations? Of course we will! We have been building simulations to observe their behaviour and interact with them, for ludic and other purposes, ever since the first primitive computers were built. The market for games has only grown as they have become more complex and realistic. Imagine what if will be like when anybody can create a whole society—a whole universe—then let it run to see what happens, or enter it and experience it first-hand. History will become an experimental science. What would have happened if the Roman empire had discovered the electromagnetic telegraph? Let's see!—and while we're at it, run a thousand simulations with slightly different initial conditions and compare them.

Finally, if we can create these simulations which are so realistic the characters within them perceive them as their real world, why should we dare such non-Copernican arrogance as to assume we're at the top level and not ourselves within a simulation? I believe we shouldn't, and to me the argument that clinches it is what I call the “branching factor”. Just as we will eventually, indeed, I'd say, inevitably, create simulations as rich as our own world, so will the beings within them create their own. Certainly, once we can, we'll create many, many simulations: as many or more as there are running copies of present-day video games, and the beings in those simulations will as well. But if each simulation creates its own simulations in a number (the branching factor) even a tiny bit larger than one, there will be exponentially more observers in these layers on layers of simulations than at the top level. And, consequently, as non-privileged observers according to the Copernican Principle, it is not just more likely than not, but overwhelmingly probable that we are living in a simulation.

The author of this book, founder of Play Labs @ MIT, a start-up accelerator which works in conjunction with the MIT Game Lab, and producer of a number of video games, has come to the same conclusion, and presents the case for the simulation hypothesis from three perspectives: computer science, physics, and the unexplained (mysticism, esoteric traditions, and those enduring phenomena and little details which don't make any sense when viewed from the conventional perspective but may seem perfectly reasonable once we accept we're characters in somebody else's simulation).

Computer Science. The development of computer games is sketched from their origins to today's three-dimensional photorealistic multiplayer environments into the future, where virtual reality mediated by goggles, gloves, and crude haptic interfaces will give way to direct neural interfaces to the brain. This may seem icky and implausible, but so were pierced lips, eyebrows, and tongues when I was growing up, and now I see them everywhere, without the benefit of directly jacking in to a world larger, more flexible, and more interesting than the dingy one we inhabit. This is sketched in eleven steps, the last of which is the Simulation Point, where we achieve the ability to create simulations which “are virtually indistinguishable from a base physical reality.” He describes, “The Great Simulation is a video game that is so real because it is based upon incredibly sophisticated models and rendering techniques that are beamed directly into the mind of the players, and the actions of artificially generated consciousness are indistinguishable from real players.” He identifies nine technical hurdles which must be overcome in order to arrive at the Simulation Point. Some, such as simulating a sufficiently large world and number of players, are challenging but straightforward scaling up of things we're already doing, which will become possible as computer power increases. Others, such as rendering completely realistic objects and incorporating physical sensations, exist in crude form today but will require major improvements we don't yet know how to build, while technologies such as interacting directly with the human brain and mind and endowing non-player characters within the simulation with consciousness and human-level intelligence have yet to be invented.

Physics. There are a number of aspects of the physical universe, most revealed as we have observed at very small and very large scales, and at speeds and time intervals far removed from those with which we and our ancestors evolved, that appear counterintuitive if not bizarre to our expectations from everyday life. We can express them precisely in our equations of quantum mechanics, special and general relativity, electrodynamics, and the standard models of particle physics and cosmology, and make predictions which accurately describe our observations, but when we try to understand what is really going on or why it works that way, it often seems puzzling and sometimes downright weird.

But as the author points out, when you view these aspects of the physical universe through the eyes of a computer game designer or builder of computer models of complex physical systems, they look oddly familiar. Here is how I expressed it thirteen years ago in my 2006 review of Leonard Susskind's The Cosmic Landscape:

What would we expect to see if we inhabited a simulation? Well, there would probably be a discrete time step and granularity in position fixed by the time and position resolution of the simulation—check, and check: the Planck time and distance appear to behave this way in our universe. There would probably be an absolute speed limit to constrain the extent we could directly explore and impose a locality constraint on propagating updates throughout the simulation—check: speed of light. There would be a limit on the extent of the universe we could observe—check: the Hubble radius is an absolute horizon we cannot penetrate, and the last scattering surface of the cosmic background radiation limits electromagnetic observation to a still smaller radius. There would be a limit on the accuracy of physical measurements due to the finite precision of the computation in the simulation—check: Heisenberg uncertainty principle—and, as in games, randomness would be used as a fudge when precision limits were hit—check: quantum mechanics.

Indeed, these curious physical phenomena begin to look precisely like the kinds of optimisations game and simulation designers employ to cope with the limited computer power at their disposal. The author notes, “Quantum Indeterminacy, a fundamental principle of the material world, sounds remarkably similar to optimizations made in the world of computer graphics and video games, which are rendered on individual machines (computers or mobile phones) but which have conscious players controlling and observing the action.”

One of the key tricks in complex video games is “conditional rendering”: you don't generate the images or worry about the physics of objects which the player can't see from their current location. This is remarkably like quantum mechanics, where the act of observation reduces the state vector to a discrete measurement and collapses its complex extent in space and time into a known value. In video games, you only need to evaluate when somebody's looking. Quantum mechanics is largely encapsulated in the tweet by Aatish Bhatia, “Don't look: waves. Look: particles.” It seems our universe works the same way. Curious, isn't it?

Similarly, games and simulations exploit discreteness and locality to reduce the amount of computation they must perform. The world is approximated by a grid, and actions in one place can only affect neighbours and propagate at a limited speed. This is precisely what we see in field theories and relativity, where actions are local and no influence can propagate faster than the speed of light.

The unexplained. Many esoteric and mystic traditions, especially those of the East such as Hinduism and Buddhism, describe the world as something like a dream, in which we act and our actions affect our permanent identity in subsequent lives. Western traditions, including the Abrahamic religions, see life in this world as a temporary thing, where our acts will be judged by a God who is outside the world. These beliefs come naturally to humans, and while there is little or no evidence for them in conventional science, it is safe to say that far more people believe and have believed these things and have structured their lives accordingly than those who have adopted the strictly rationalistic viewpoint one might deduce from deterministic, reductionist science.

And yet, once again, in video games we see the emergence of a model which is entirely compatible with these ancient traditions. Characters live multiple lives, and their actions in the game cause changes in a state (“karma”) which is recorded outside the game and affects what they can do. They complete quests, which affect their karma and capabilities, and upon completing a quest, they may graduate (be reincarnated) into a new life (level), in which they retain their karma from previous lives. Just as players who exist outside the game can affect events and characters within it, various traditions describe actors outside the natural universe (hence “supernatural”) such as gods, angels, demons, and spirits of the departed, interacting with people within the universe and occasionally causing physical manifestations (miracles, apparitions, hauntings, UFOs, etc.). And perhaps the simulation hypothesis can even explain absence of evidence: the sky in a video game may contain a multitude of stars and galaxies, but that doesn't mean each is populated by its own video game universe filled with characters playing the same game. No, it's just scenery, there to be admired but with which you can't interact. Maybe that's why we've never detected signals from an alien civilisation: the stars are just procedurally generated scenery to make our telescopic views more interesting.

The author concludes with a summary of the evidence we may be living in a simulation and the objection of sceptics (such that a computer as large and complicated as the universe would be required to simulate a universe). He suggests experiments which might detect the granularity of the simulation and provide concrete evidence the universe is not the continuum most of science has assumed it to be. A final chapter presents speculations as to who might be running the simulation, what their motives might be for doing so, and the nature of beings within the simulation. I'm cautious of delusions of grandeur in making such guesses. I'll bet we're a science fair project, and I'll further bet that within a century we'll be creating a multitude of simulated universes for our own science fair projects.

Posted at 01:00 Permalink

Wednesday, January 1, 2020

Reading List: The City of Illusions

Wood, Fenton. The City of Illusions. Seattle: Amazon Digital Services, 2019. ASIN B082692JTX.
This is the fourth short novel/novella (148 pages) in the author's Yankee Republic series. I described the first, Pirates of the Electromagnetic Waves (May 2019), as “utterly charming”, and the second, Five Million Watts (June 2019), “enchanting”. The third, The Tower of the Bear (October 2019), takes Philo from the depths of the ocean to the Great Tree in the exotic West.

Here, the story continues as Philo reaches the Tree, meets its Guardian, “the largest, ugliest, and smelliest bear” he has ever seen, not to mention the most voluble and endowed with the wit of eternity, and explores the Tree, which holds gateways to other times and places, where Philo must confront a test which has defeated many heroes who have come this way before. Exploring the Tree, he learns of the distant past and future, of the Ancient Marauder and Viridios before the dawn of history, and of the War that changed the course of time.

Continuing his hero's quest, he ventures further westward along the Tyrant's Road into the desert of the Valley of Death. There he will learn the fate of the Tyrant and his enthralled followers and, if you haven't figured it out already, you will probably now understand where Philo's timeline diverged from our own. A hero must have a companion, and it is in the desert, after doing a good deed, that he meets his: a teddy bear, Made in Japan—but a very special teddy bear, as he will learn as the journey progresses.

Finally, he arrives at the Valley of the Angels, with pavement stretching to the horizon and cloaked in an acrid yellow mist that obscures visibility and irritates the eyes and throat. There he finds the legendary City of Illusions, where he is confronted by a series of diabolical abusement park attractions where his wit, courage, and Teddy's formidable powers will be tested to the utmost with death the price of failure. Victory can lead to the storied Bullet Train, the prize he needs to save radio station 2XG and possibly the world, and the next step in his quest.

As the fourth installment in what is projected to be one long story spanning five volumes, if you pick this up cold it will probably strike you as a bunch of disconnected adventures and puzzles each of which might as well be a stand-alone short-short story. As they unfold, only occasionally do you see a connection with the origins of the story or Philo's quest, although when they do appear (as in the linkage between the Library of Infinity and the Library of Ouroboros in The Tower of the Bear) they are a delight. It is only toward the end that you begin to see the threads converging toward what promises to be a stirring conclusion to a young adult classic enjoyable by all ages. I haven't read a work of science fiction so closely patterned on the hero's journey as described in Joseph Campbell's The Hero with a Thousand Faces since Rudy Rucker's 2004 novel Frek and the Elixir; this is not a criticism but a compliment—the eternal hero myth has always made for tales which not only entertain but endure.

This book is currently available only in a Kindle edition. The fifth and final volume of the Yankee Republic saga is scheduled to be published in the spring of 2020.

Posted at 20:59 Permalink