The Committee of Public Safety

I’m going to resurrect a dormant software project I started a year ago for creating a web-based strategic awareness application. It will be written in Python, Pylons or Django (I haven’t decided) and will exploit some interesting emerging technologies like MongoDB. Blogging may be lighter but I will post periodic design posts since that’s a good way to flesh things out.

Python now has the tools necessary to take over the world.

In Linus We Trust

Zenpundit links back to The Tactical Loop in Following up on Learning Organizations. I found this section interesting:

I will quibble with Adam [Elkus] in that “crowdsourcing” phenomena or open systems that acquire a “community” aspect with recognizably distinct cultural norms are usually dominated by a “natural aristocracy” among the membership – a cadre of “influencers” who make qualitative contributions frequently and disproportionately and have individual and collective ”auctoritas” to police the larger mass than contribute occasionally. There’s a “feedback loop” where certain members of “the crowd” add complex thinking for collective evaluation, modification and ratification. Linux is the original example as is Wikipedia, where senior and elusive ”Wikipedians” carry great weight compared to say, me, logging on and editing away.

The “natural aristocracy” Zen refers to is enforced in Linux, other open source software projects, and Wikipedia by a version control system (VCS).  The simplest VCS is the undo/redo functionality found in most application software. Undo/redo is linear: you either go x steps back or x steps forward. A slightly more complex VCSes like the one used by Wikipedia adds an additional wrinkle: while undo/redo is usually confined to a single user using a single application, Wikipedia gives multiple users the ability to undo and redo. This creates complications because just anybody can come along and make a change, sometimes improving but other times defacing or inserting personal idiocy. The Wikipedia community must assume the wiki’s burden and preserve the integrity of the wiki from faction, malice, trolls, spammers, and anonymous cowards. This involves ’senior and elusive ”Wikipedians” standing vigilant over the wiki, fending off attacks in often vicious edit wars.

BDFL. Refreshing.

In open source software projects, a VCS usually is under even tighter control. While Wikipedia is a libertarian paradise where any yahoo can make a contribution, open source software projects feature access control that restricts who can commit or revert changes to the VCS. Many projects are divided into a “natural aristocracy” of committers and users. Committers themselves can be subdivided into tiers of commit rights, with some developers only having rights to commit to some portions of the code and some developers having the right to revert other developer’s changes.

Some projects have a dictator who exercises final control over what code does or does not make it into the main software release (often called a trunk because other versions of the code branch off of it). This is probably true for most projects since most open source software projects only have one or two developers. Others have a dictator surrounded by key lieutenants who are delegated certain parts of the codebase. This is the case with Linus Torvalds and Python’s Benevolent Dictator For Life Guido van Rossum and their merry men. Others have an meritocratic oligarchy like the Apache Web Server core developers.

When a major disagreement breaks out between developers that comes to (virtual) blows, often the result will be a fork. Programmers, generally being anal-retentive types, love to quibble over arcane and obscure minutia. In this regard they are a lot like early Christians and prone to schism. A schism in an open source software project, called a fork, occurs when one faction of developers takes the source code from a particular project and begins developing it independently of the original project. They vote themselves off the island in order to maintain their vision for the software.

The saga of one core group is indicative of how this system can play out in practice. BSD was originally a fork of the One True AT&T UNIX that was developed and maintained by the University of California-Berkeley. AT&T, prevented from making money off of UNIX initially because of an antitrust agreement with the Justice Department, distributed the source code for UNIX to interested parties for the nominal cost of the media to ship it on. Berkeley was one such party and made their own variant of core UNIX. After Ma Bell was broken up, AT&T was allowed to enter the computer market so they decided to monetize UNIX. They sued Berkeley and forced them to stop developing BSD UNIX, the basis for Sun’s original versions of UNIX. This produced an effort to make a version of BSD that was no longer tainted by AT&T’s copyrighted code. Berkeley released a final version of BSD in 1995 and stopped developing the operating system.

Beware Lucifer, Son of the Morning Star

An earlier version of BSD developed into what eventually became the NetBSD project. The original developers of NetBSD were unsatisfied with the couple developing 386BSD and forked their code. NetBSD was devoted to making BSD run on every object in the universe, especially your toaster. However, the original developers had an argument and Lucifer was expelled from heaven.  The sign of his fall was the revoking of his commit rights to the NetBSD source. Lucifer, his evil designs thwarted, plotted his vengence. He picked over the NetBSD source code until he had his own Canadian version: OpenBSD. Lucifer’s vision was to allow every object in the universe to be a firewall with military grade security. The reasons for his madness are hidden. Pray they are never revealed. (A friend of mine had a brush with Lucifer when he tried to help an innocent OpenBSD newbie configure the operating system (OS). Lucifer howled with rage, cursed my friend, and expelled him from Hell.) In the meantime, FreeBSD was also born, dedicated to running BSD on your PC instead of your toaster.

This saga, much of it caused by rigidity some development oligarchies maintained over there VCS repositories, led to BSD falling by the wayside as the open source OS of the future. Instead Linux, with its more efficient benevolent despotism, became the open source operating system (unless you count MacOS X as a BSD, in which case BSD has 9% of the OS market while Linux has ~1%) while BSD forked and schismed.

Linus, however, is a tyrant that sits easily upon his throne. He popularized the adoption of the Distributed Version Control System (DVCS), a system which replaces the one central repository under the control of a dictator or oligarchy model favored by traditional VCSes like CVS and its successor Subversion with many distributed repositories under the control of many developers. Instead of changes being submitted to a few committers and then, if blessed, being added to the main trunk of the software, developers using the DVCS approach push and pull changes from their own local repositories to and from other developer’s repositories in a peer-to-peer relationship. Linus’s own repository of Linux is technically equal with every other Linux developer’s repository. He, as with every other Linux developer, can push and pull changes from other repositories as he wants. However, Linus, being Linus, is more equal than others developers. He still possess the sole right to “sprinkle holy penguin pee” on a Linux software release and declare it the official Linux release.

He uses his power wisely and intends to liberate the world from the domination of monolithic VCSes. The video that made me believe is a talk Linus gave at Google where he walks in, tells them that their VCS (Perforce) is lousy and that they should follow the DVCS model. Linus compellingly argues that the DVCS scales an open source software project better than a VCS. It allows more developers to contribute, reduces the more prominent developers tasks to picking the best code possible from multiple repositories instead of guarding their precious, precious core repository, and minimizes the chance that an outright fork has to occur in order to push minority changes into the codebase. His performance is a tour de force.

Letting a hundred flowers blossom and a hundred schools of thought contend.

I'll meet Perl 6 at the end of time.

Now for some good old fashioned naked Python bigotry…

Where to start? Python 3.o, the future of the Python programming language? Let’s start with Perl 6, the Duke Nukem Forever of programming languages:

Perl 6 has been in development since 2000. The Perl 6 project has never had a clear timeline, although various contributors have given estimates over the years. In early 2007 Jesse Vincent, the Perl 6 Project Manager said, “The Perl 6 project has no schedule … one doesn’t want to rush a largely volunteer effort to design and implement a worthy successor to Perl 5.”

Perl 6 has been in development nearly as long as Duke Nukem Forever. A generation has been raised up who know not Duke Nukem or Perl. Perl has been rotting in its infernal strangeness all the while. The Perl community is a large, dead mass of gnomish graybeards, shunned by more polite company, stuck in a 1989 fashion timewarp, the preferred language of those who found awk less too user friendly.

The task of Perl 6 should have been easy. Hook up a 1200 baud modem. Open a XModem terminal. Capture some line noise. Congratulations. You’ve created a more readable and more modern language than Perl 5. Instead the Perl 6 developers, as in someone’s phrase, “voted themselves off the island”. They pursued Perl as art, despite the fact that there is no art in Perl. In their pursuit of some false aesthetic, they wandered off into dark and inaccessible places.

Consider the task of the Python 3000 developers, far more daunting than Perl 6. They had to top perfection. Python is the greatest programming language ever created. They had to improve it.

Python 3.0 development started as early as 2000 but it differs from Perl 6 in one respect: it shipped, nine years after Perl 6 undertook its heroic development cycle. Python 3.0 final was released Wednesday. They eschewed radical changes to the language in favor of removing some minor warts and re-emphasizing Python’s strengths. They should be commended for demonstrating what an excellent software engineering process should look like. They put on a show.

There has been mixed reaction to its release. Some are esctatic:

Python 3.0 is not a release for today, it’s a release for the ages. Python will now be known as one of the best pieces of software in world history. Python 2 is great; Python 3 is masterful.

Some are positive with reservations:

Still, I don’t think that Python 3.0 is a bad thing. But that it’s displayed so prominently on the Python web site, without any kind of warning that it’s not going to work with 99% of the Python code out there, scares the hell out of me. People are going to download and install 3.0 by default, and nothing’s going to work. They’re going to complain, and many are going to simply walk away.

Some are cumulatively negative.

Some talk about monkeys.

The counter-reaction-reaction has begun.

My own reaction is positive but pragmatic.

I’m glad Python has a clear way forward, something to build towards. I downloaded Python 3.0 and installed it on my MacBook. I’m one of those nameless toilers who doesn’t hack on the core language, doesn’t contribute to the prominent projects, or comment upon the great shakings of the community. I just use the language on a daily basis to get software development done at a faster clip than a bondage and discipline language would allow. Python 3.0 is far from something I would use for production.

Our company will proceed with caution towards Python 3.0 (as most in the community have been suggesting for years). We will continue to use a 2.x release for the next 3-4 years. We’re running on 2.5 right now and at some point we’ll move to 2.6. We will move to 3.o when it makes sense and when software like Django, Pylons, and SQLAlchemy are available for it. However, Python 3.0 answers a fundamental question: Where is Python going? Now we know. If management was worried about where Python was going, they now have a Golden Path forward for a decade or more.

Nudge

I vaguely remembered a someone making a linkage between the Automatic and Reflective Systems (the terms used for Systems 1 and 2 respectively by Thaler and Sunstein in Nudge) and the OODA loop’s Orientation and Decision phases. It came from this post by Dan tdaxp. The post includes a diagram that  highlights that an OODA loop is, at minimum, three loops:

• Observe
• Orient
• Decide
• Action

or:

• Observe
• Orient
• Action

or:

• Observe
• Orient

• The Automatic System is heuristic and is optimized for Taleb’s Mediocristan, governed by Gauss. The Reflective System spends most of its time in Mediocristan but can see the faint outlines of Extremistan, the land of Pareto. Automatic is a hash table (Python dictionary) (in software engineering speak), a O(1) system, while Reflective is an array (Python list), a O(n) system. The Reflexive System is an iterative exploration while the Automatic System is a quick lookup.
• Consider URL dispatching, where you have to match a URL to the computer code that will handle it. You can either iterate through a list of URL => code mappings or you can map each part of a URL to a hash table (or object). One may take as many steps as their are URL => code mappings; one is just a quick lookup but tightly couples the URL to the underlying implementation, violating the principle of loose coupling. The solution is similar to the human mind: you have both. You resolve the URL iteratively on the first lookup and then cache it in a hash table.
• How does this play out in the meta-OODA loop:
  

  Multilevel adaption cycles

  A tactical OODA loop seems more Automatic than Reflective. Yet Culture and the invisible Instinctive OODA loop also seem Automatic. Perhaps it’s a bell curve, with the Automatic System falling at 0 on the X axis and as values increase they become more Reflective as the curve bulges upward. Culture and Instinct would fall on the upward slope under the Automatic System, Politics, Strategy, and Operations would reach towards Reflective System as the middle bulged, and Tactics would fall back towards the Automatic System on the downward slope.

• I sometimes think in a black swan that the OODA loop becomes Observe-Orient-Moralize-Stumble since all you can do with a black swan is observe it and weave fables about it afterwards. Fables with their morals are a sort of preemptive prediction, a nudge to choose one stumble over another. The Automatic System may simply be evolutionarily hardwired fables and the Reflexive System a diversity generator of new fables. They may not actually fight the black swan but they make you feel better afterwards.

• We may be entering the age of the nudge. Sunstein is a friend of the President-elect and libertarian paternalism may be in the cards for the policies of the new administration. David Brooks hints at such an outcome.

Cerf's Up

Python testing guru Grig Gheorghiu tracks Google’s march to global domination of the IT industry through its hiring of prominent figures in IT history:

• Ken Thompson == Unix
• Vint Cerf == TCP/IP
• Andrew Morton == #2 in Linux
• Guido van Rossum == Python
• Ben Collins-Sussman and Brian Fitzpatrick == subversion
• Bram Moolenaar == vim

I wouldn’t brag about vim though.

Just in time.

Just-in-Time thinking has afflicted America for the last 15 or so years. The belief that knowledge could be perfect enough, that markets were omniscient enough, that planning could be farsighted enough, that globalized supply chains were robust enough, that workforces were interchangeable enough, that alternative suppliers were available enough, that credit would be constant enough, has wreaked untold havoc. The on demand world has worked well enough while sprinting to lull Americans into the unconscious expectation of a frictionless existence. It’s when it attempts the grueling marathon that the system begins to break up. It is the beguiling into quiescence, the lulling into easy luxury that went before the fall. When the margin of safety is removed, the fall is hard. And, in the case of JIT, it was based on American incomprehension of one of the models of JIT: the Toyota Production System (TPS). Even worse, it’s incomprehension of ourselves since JIT was invented in America at Piggly Wiggly. Americans are unaware of themselves. Normally, it’s one of our great accomplishments.

Spear

Steven Spear and H. Kent Bowen in Decoding the DNA of the Toyota Production System, after extensive research, reveal a more subtle system. Many firms have tried to copy TPS, especially in Detroit, but no one’s ever captured the essence of it. Spear spent a year working for Toyota while he was a doctoral student. He observed the TPS perhaps more deeply than anyone before him.

Spear and Bowen boil TPS’s operating principles down to four rules:

1. All work shall be highly specified as to content, sequence, timing, and outcome.
2. Every customer-supplier connection must be direct, and there must be an unambiguous yes-or-no
   way to send requests and receive responses.
3. The pathway for every product and service must be simple and direct.
4. Any improvement must be made in accordance with the scientific method, under the guidance of a teacher, at the lowest possible level in the organization.

Margin of Safety

An example of rule 1 is illuminating:

Let’s look at how operators at a typical U.S. autoplant install the front passenger seat into a car. They are supposed to take four bolts from a cardboard box, carry them and a torque wrench to the car, tighten the four bolts, and enter a code into a computer to indicate that the work has been done without problems. Then they wait for the next car to arrive. New operators are usually trained by experienced workers, who teach by demonstrating what to do. A seasoned colleague might be available to help a new operator with any difficulties, such as failing to tighten a bolt enough or forgetting to enter the computer code.

This sounds straightforward, so what’s wrong with it? The problem is that those specifications actually allow (and even assume) considerable variation in the way employees do their work. Without anyone realizing it, there is plenty of scope for a new operator to put the seat into the vehicle differently than an experienced employee would. Some operators might put the front bolts in after the rear bolts; some might do it the other way around. Some operators might put each bolt in and then tighten them all; others might tighten as they go along. All this variation translates into poorer quality, lower productivity, and higher costs. More important, it hinders learning and improvement in the organization because the variations hide the link between how the work is done and the results.

At Toyota’s plants, because operators (new and old, junior and supervisory) follow a well-defined sequence of steps for a particular job, it is instantly clear when they deviate from the specifications. Consider how workers at Toyota’s Georgetown, Kentucky, plant install the right-front seat into a Camry. The work is designed as a sequence of seven tasks, all of which are expected to be completed in 55 seconds as the car moves at a fixed speed through a worker’s zone. If the production worker finds himself doing task 6 (installing the rear seat-bolts) before task 4 (installing the front seat-bolts), then the job is actually being done differently than it was designed to be done, indicating that something must be wrong. Similarly, if after 40 seconds the worker is still on task 4, which should have been completed after 31 seconds, then something, too, is amiss. To make problem detection even simpler, the length of the floor for each work area is marked in tenths. So if the worker is passing the sixth of the ten floor marks (that is, if he is 33 seconds into the cycle) and is still on task then he and his team leader know that he has fallen behind. Since the deviation is immediately apparent, worker and supervisor can move to correct the problem right away and then determine how to change the specifications or retrain the worker to prevent a recurrence.

It’s interesting that TPS introduces into a sort of rigidity into its Observe-Orient-Decide-Act loop. Because the injected rigidity makes breakages in the loop obvious and easier to observe and, following rule 4, correction of disruptions is handled at the lowest possible level between the task performer and a teacher, the rigidity makes the system curiously more adaptive than a looser system. It’s organic goo with a learning glass jaw. Process wise, this gives them a margin of safety since adaptibility is pushed to the edge of the network.

TPS manages to find a margin of safety in traditional ways as well:

…contrary to the impression that the concept of zero inventory is at the heart of the Toyota system, we’ve observed many cases in which Toyota actually built up its inventory of materials as a countermeasure. The ideal system would in fact have no need for inventory. But, in practice, certain circumstances may require it:

• Unpredictable downtime or yields. Sometimes a person or a machine is unable to respond on demand when a request is made because of an unexpected mechanical breakdown. For this reason, safety stock is held to protect the customer against random occurrences.The person responsible for ensuring the reliability of a machine or process owns that inventory and strives to reduce the frequency and length of downtimes so that the amount of the safety stock can be reduced.
• Time-consuming setups. Difficulties in switching a machine from processing one kind of product to another can prevent a supplier from responding immediately. Therefore,suppliers will produce the product in batch sizes greater than one and hold the excess as inventory so it can respond immediately to the customer. Of course, suppliers will continually try to reduce the changeover time to keep batch sizes and stores of inventory as small as possible.Here,the owners of both the problem and the countermeasure are the machine operator and the team leader, who are responsible for reducing changeover times and batch sizes.
• Volatility in the mix and volume of customer demand. In some cases, variations in customers’ needs are so large and unpredictable that it is impossible for a plant to adjust its production to them quickly enough. In those instances, buffer stock is kept at or near the shipping point as a countermeasure.The buffer stock also serves as a signal to production and sales managers that the person who works most directly with the customer must help that customer eliminate the underlying causes of any preventable swings in demand.In many cases, the same type of product is held in different types of inventory. Toyota does not pool its various kinds of inventory, even though doing so would reduce its inventory needs in the short term. That might sound paradoxical for a management system so popularly known to abhor waste. But the paradox can be resolved when we recognize that Toyota’s managers and workers are trying to match each countermeasure to each problem.There’s no link between the reason for keeping safety stock, process unreliability, and the reason for keeping buffer stock, fluctuations in customer demand. To pool the two would make it hard to distinguish between the separate activities and customer-supplier connections involved. The inventory would have many owners, and the reasons for its use would become ambiguous. Pooling the inventory thus muddles both the ownership and cause of the problems, making it difficult to introduce improvements.

Spear’s doctoral supervisor, Clayton Christensen, has taken Spear’s work and extended into other domains like semi-conductor manufacturing, hospital care, medical education, and aluminum processing among others. In this podcast, he extracts four principles of TBS which descend from Spear’s original four rules:

1. Never do anything until it is ready for use.
2. Produce items for one-to-one, not one-to-many pathways.
3. Use the exact same method consistently.
4. Avoid the frontier of design as much as possible.

Principle 1 finds a margin of safety in sparing effort, 2 and 3 in simplicity, and 4 in time.

The Dreaded Dane

The logic of gaining flexibility through seeming rigidity and simplicity is inherent in the Python programming language. The motto of Python is that there should be one—and preferably only one—obvious way to do it. This is in opposition to the Ruby programming language or the Perl programming language, whose philosophies are more there is more than one way to do it (TIMTOWTDI, usually pronounced “Tim Toady”). Simplicity on the programming language level allows Python to be more flexible on the application programming level. Python’s ability to fit in your head means the brain can be occupied with larger issues.  Rigid tactics lead to flexible operations. Other languages often need rigidity on the application level to compensate for the sloppiness on the programming language level, such as Ruby gets with the totalitarian Ruby on Rails web application programming framework. From my own experience I can only warn, once you have paid the Danegeld, you never get rid of the Dane.

Similarly, on the software engineering process level, selective injection of rigidity might work out. One obvious example is Test Driven Development:

Test-Driven Development (TDD) is a software development technique consisting of short iterations where new test cases covering the desired improvement or new functionality are written first, then the production code necessary to pass the tests is implemented, and finally the software is refactored to accommodate changes. The availability of tests before actual development ensures rapid feedback after any change. Practitioners emphasize that test-driven development is a method of designing software, not merely a method of testing.

Benevolent Dictator for Life

The rigidity inherent in writing automated tests first leads to a cycle where the specification written is a living thing, capable of being run over and over again. As functionality is written against the iron standard of the tests, defects are revealed and action can be taken to correct them. Rigidity leads to more purposeful adaptation. Small steps are taken iteratively, providing a margin of safety by shortening the software development OODA loop so the process enables faster reaction. All of these provide a true just in time margin of safety. And that provides some protection against black swans and wicked problems.

The Zen of Python

    Beautiful is better than ugly.
    Explicit is better than implicit.
    Simple is better than complex.
    Complex is better than complicated.
    Flat is better than nested.
    Sparse is better than dense.
    Readability counts.
    Special cases aren't special enough to break the rules.
    Although practicality beats purity.
    Errors should never pass silently.
    Unless explicitly silenced.
    In the face of ambiguity, refuse the temptation to guess.
    There should be one-- and preferably only one --obvious way to do it.
    Although that way may not be obvious at first unless you're Dutch.
    Now is better than never.
    Although never is often better than *right* now.
    If the implementation is hard to explain, it's a bad idea.
    If the implementation is easy to explain, it may be a good idea.
    Namespaces are one honking great idea -- let's do more of those!

zenpundit, a blog I subscribe to, had a post on quantum libraries. Quantum libraries are those books you return to time and time again and glean fresh insights every time. Here’s mine:

• Dune (Frank Herbert)
• The Dune Encyclopedia (Willis E McNelly)
• Snow Crash (Neal Stephenson)
• The Diamond Age (Neal Stephenson)
• The Lord of the Rings (J.R.R. Tolkien)
• The Complete Sherlock Holmes (Sir Arthur Conan Doyle)
• Dawn and Decadence (Jacques Barzun)
• Chaos (James Gleick)
• Guns, Germs, and Steel (Jared Diamond)
• The Black Swan (Nassim Nicholas Taleb)
• The Intelligent Investor (Benjamin Graham)
• The Cash Nexus (Niall Ferguson)
• The Pity of War (Niall Ferguson)
• Strange Victory (Ernest R. May)
• On War (Carl von Clausewitz)
• Masters of War: Classical Strategic Thought (Michael Handel)
• War in Human Civilization (Azar Gat)
• The Seven Military Classics of Ancient China (Ralph Sawyer)
• The Face of Battle (John Keegan)
• The Pursuit of Power (William H. McNeill)
• The Memoirs of Ulysses S. Grant
• The Rise of the Roman Empire (Polybius)
• History of the Peloponnesian War (Thucydides)
• How to Make War (James F. Dunnigan)
• Novus Ordo Seclorum (Forrest McDonald)
• Python in a Nutshell (Alex Martinelli)
• Code Complete (Steve O’Connell)