Virtual Reality promised to create engrossing environments to fool the senses and transport the participant to amazing places and magical experiences. There are a few efforts still moving in that direction, but have drifted away from the head tracking helmets and input gloves used in earlier efforts and more into an externalized (from the perspective of the participant) infrastructure: surrounding the subject with lots of screens and integrating with more natural input modes. High-speed 3D has been a nice addition too, but the cost and inherent limitations of the infrastructure are still a limiting factor.

Enter Augmented Reality, stage left. This has been an interesting evolution, and is still in its early stages of development as the mechanics and utility of features are worked out. It started, interestingly enough, as part of VR directly - incorporation of real details into virtual environments and the reverse. It's migrated away from the high-cost infrastructure and the gargoyle-like wearable computing that promised to make it portable into handsets - now ubiquitous technology with increasingly powerful computing performance. These are capable of motion-matching the surrounding environment and overlaying positional, geographic, or interactive elements onto a camera-captured realtime view, or in some creative instances literally projecting an interface onto something else and measuring interaction that way. One of the common themes of the camera-augmented view is the attempt to create a 3-dimensional feel: the view has essentially been downsampled to 2D, so stereoscopy isn't an option, and instead other standard 3D presentation mechanisms are used such as shading, occlusion, distance cues, etc., positionally represented based on the relative angle between the view and the virtual object. This is an interesting hack of modest utility, and when combined with viewer tracking can create some very convincing effects.

This morning I saw an unlikely (or at least unexpected) implementation of the concept on my way in to work that has started me thinking. A 3D rendered scene (very simple, a few gift-wrapped packages) was shown on a digital billboard (in 2D of course, though given the distance the stereoscopy is essentially uniformly columnated anyway), and had a virtual camera shift on the display which corresponded roughly (and I do mean roughly) with the anticipated perspective shift of a driver moving at average speed on the freeway. There were some aspects that made it less convincing, such as the vertical offset being mismatched (the camera was looking slightly down on the package, as opposed to the driver's lower perspective) and the horizontal shift speed being off, but the gimmick caught my attention enough to get me thinking about the possibilities - especially the integration of time/condition aware presentation for lighting and shading, and what could be done with very little work to make it far more immersive. I'm not even sure that their use of the effect was intentional or if they were just happening to use fancy graphics while I was coincidentally driving by in a way to perceive it as such. No matter what, it was nifty and I'm sure we can expect to see more of it.

Despite my rambling on there's some useful information to be found - hardly all inclusive, but it should stand on its own as an introduction to the level of forethought that's gone into an undertaking of this complexity.

The ~20 minute recording posted today - go check out Episode 46!

1. Temporary or short-term hibernation, characterized physiological activity.
2. The first mature persistence layer abstraction for PHP, provided by Yours Truly under the MIT license.

Link: torpor-php (via Google Project Hosting)

The name is a direct take off of Hibernate since it provides very similar functionality for Object Relational Mapping.  It is not, however, a port, emulation, reimplementation, or imitation of it or any other specific ORM/Persistence abstraction out there.  This is all new code from the ground up, with the project started in the last half of July 2009.

The premise is simple: having written a significant amount of database-centric code over the years, in a variety of environments and in several different development languages, I have come to understand what I feel are the basic features every good database abstraction should have in order to be genuinely useful.  I have very frequently been without those features, and spent a great deal of time re-implementing and refining them in each of the different environments in play.

By doing this so many times over I have realized that, for a well defined (preferably 3rd-normal) schema, the feature set can be deterministically derived.  In order to test that theory, and potentially save myself an inordinate amount of tedium and re-invention in the current employment-selected language, I have written Torpor (though not until after thorough examination of other potential offerings, which I found to be thoroughly lacking and usually tied to other larger frameworks which most certainly did not meet my needs).

There are a few features left to do before the code will be packaged for distribution under the 0.9β release, which should be available in the next 1-2 weeks.  This beta release will be for MySQL only, and while it's been coded with PHP 5.1.0 in mind I can't guarantee full compatibility with less than 5.2.8 (have not yet tested it with the target prior version).  SQLite and Oracle adapters won't be far behind, but Oracle probably won't be making use of the OCI bind variables yet - though this is definitely planned for a later release

Features in a nutshell:

1. Configuration is controlled via straightforward XML (produced in large part by a helper script).
2. Objects have rich accessor and mutator interfaces with just-in-time fetching and optionally selective publishing.
3. Objects are capable of acting as intelligent factories for other related objects (and sets of objects), allowing for simple and powerful code for navigating from one portion of a database to another (intrinsic deep-loading, even in data engines lacking explicit support for foreign key relationships).
4. Full support for associating related un-published objects, with optional (on by default) cascading publish of dependent objects (deep-saving) - useful for those tables whose primary keys are generated on publish.
5. Full support for multi-column keys, multiple unique keys (with or without multiple columns), and multiple reference relationships between similar object types (e.g., a User may act as both a Seller and a Customer on an Order and be accessed as each in turn).
6. Extension of provided primitives is supported and encouraged, with event-based call-back hooks and a hierarchical overload/override scheme that's easy to configure.
7. Custom SQL support for Load, Publish (optionally differentiated by insert vs. update), Delete, including stored procedures and pseudo-bind variables (SQL placeholders filled by order of occurrence and/or named variables).
8. Criteria based select, optionally using arbitrarily nested AND/OR collections and a rich set of criteria types and controls (including custom SQL) with support for automatic joins to related tables.
9. Intermediate read/write -through caching interface support (can be used in distributed network based cache settings, such as memcache, for nearly-free lateral scalability).
10. Fully extensible and customizable.

That's a pretty big nutshell, and it's not anywhere near complete.  There are just a couple gaps between this list and the current revision (no. svn:35 as of this writing) on both sides: a few features which need better testing and a little internal wiring, and some features provided but not listed.  Eventually the full user documentation will fill in the gaps, but so far this list and the actual available feature set have a > 90% overlap.

Implementation example snippet:

// Singleton context; support for instance-context
// and automatic config loading also available.
Torpor()->initialize( "myconfig.xml" );

$user = Torpor()->getUserById( 12345 );

$orderHistory = $user->getOrderSet();
foreach( $orderHistory as $order ){ /* ... */ }

$order = $user->newOrder();
$order->setSeller( Torpor()->getUserById( 54321 ) );
// OR $order->setSellerId( $user->getUserId() ), etc.
$order->setPurchaseDate( date() );

$product = Torpor()->newProduct();
$product->setName( 'Something Pink' );
$product->setUnitPrice( 49.95 );

$orderItem = $order->newOrderItem();
$orderItem->setProduct( $product );
// causes $product->publish(), $order->publish() cascade:
$orderItem->publish();

Code is available now via Google's project hosting: Torpor: Persistence Abstraction for PHP

Feel free to poke around, make contributions, revisions, remarks (snide or otherwise), and generally check it out.  The official beta release will be announced here with great fanfare and tremendous sighs of relief, followed shortly by the tension of worrying whether it's good enough and trying to polish user-docs.

Update (2009-08-28 09:16)

Torpor has been submitted as a potential topic of discussion for the Utah Open Source Conference 2009, October 8th-10th! This is a rather late submission, so your votes will count to help get this one on the board!

In theory this is excellent, and in practice it's proved rather useful especially where projects call for deviation of underlying structures which can be done in a relationally intact way by severing the link to the main repository. However, when it comes to actual permissions, the Merge table is not akin to a symlink which verifies the access rights to the underlying object: if you have access to the merge table, you have all of the corresponding access to the underlying tables regardless of whether the permissions have been granted. This means that UPDATE and DELETE operations intended to be constrained by the combination of structure and permissions are in fact freely available across the board: developers may perform these functions against the PSR (to re-use terminology).

Your mileage may definitely vary between "meh" and "dealbreaker", but it would be irresponsible of me not to disclose a minor leak in the tank.

In the case of internet applications this means an interface server (usually web), processing environment, and persistence layer (disk or database storage). Most simplistically this is just another subdirectory or virtual host directed to the developers' local copy of the operating code, where they get to make their changes without affecting or being affected by the work of others (shared processor and disk constraints aside, and out of scope for this discussion). The persistence layer, most commonly a database, has some additional constraints though: getting enough good sample data in place to be a good representation of real-world activity, both in terms of permutations and raw number of records, can be cost prohibitive either in documentation and setup (assembling the samples) or simply in the size of the data set.

Working with smaller data sets tends to help with the validation of structure, and doing it on scaled down hardware is often used as a way of simulating (poorly) the probable performance of proportionately larger data sets in a higher-class processing environment (and typically under greater load). There are some things that just can't be worked out on a reduced scale, however, especially related to index and query tuning.

A solution we've recently implemented is the MySQL Sandcastle - a more elaborate construct on top of the typical sandbox concepts that has some significant benefits for web app development.

We started with a rich staging environment: a copy of the production database, minimally sanitized (personally identifying and/or financially sensitive data obscured) and trimmed down only as appropriate (retaining 90 days worth of operational data from history tables) for the hosting environment. The sanitation makes it more like a bridge between test and literal stage, but gives us exactly what we need to work with regardless.

On top of that staging environment we're taking advantage of the MySQL Merge storage engine, which essentially marries multiple table definitions under a single abstracted table interface. This is similar to a unioned view, except that the structural checking is done during creation, the contents are fully malleable to all standard CRUD operations, and there's no need to spool data to temporary tables (which most databases do in some form or other for these kinds of operations) during complex queries. A default insert mode also tells the table which of the component tables to add records to (exclusively).

Per the illustration then, we have the Primary Staging Repository (PSR), plus 2 additional databases per sandbox (STAGE_[user] and STAGE_[user]_M) which round out the environment. The STAGE_[user] database replicates the structure of the PSR but starts out completely empty. STAGE_[user]_M shadows this structure, but swaps out the engine definition from MyISAM to Merge in order to combine user stage and PSR. In order to keep PSR clean for all developers to continue their work, each user is granted an un-privileged account with full access to their own sandbox databases and read-only to the large store (Merge table definitions must be created with full CRUD permissions to the underlying tables as well, so these are created by a privileged user prior to turning the reins over to the per-user accounts), then accesses the environment only through the ..._M instance.

Obviously this restricts activity: any attempt to update or remove records which exist only in PSR will likely produce errors, and at the very least will be ineffectual and quite probably anomalous. The most effective usage pattern is for unit tests and general developer activity to still recreate data they intend to directly modify, leaving the large store as a good general sample set for read activities (which accounts for ~60-80% of all development activity anyway). The benefits are pretty big: developers get cheap access to large swaths of regularly refreshed data without having to continually repopulate or propagate in their own environments, can work destructively without interference, and even test structural changes (simply excluding PSR from the Merge and redefining the table) to the database before finally recombining efforts in Stage (which does work destructively against the PSR) for integration testing before promoting to production.

There are some other disadvantages as well: it's possible for local insertions to the underlying tables to create keys in identical ranges which will appear to the client as duplicate primary keys (in violation of the very clear and appropriate designation of "Primary Key") in the merged data set. Setting new exclusive AUTO_INCREMENT ranges per table doesn't help yet either, due to a bug in the engine that treats it like a combinatory (multi-column) key definition, using MAX( PK ) + 1 instead of the defined range of the target table. Merge tables are also restricted to MyISAM table types, excluding the oft popular and appropriate InnoDB (and other) options. Treading carefully easily avoids these, but it's good to be aware of them.

A more complete workaround would be updateable views, using a combination of exclusive UNION (rather than implicit UNION ALL) selects and a series of triggers which not only manage insert activity, but replicate into the developer shadow sandbox the intended target record prior to modification in the case of insert. Establishing this kind of more extensive sandcastle pattern would be far more capable, should be scriptable without great effort, but still carry a small number of its own gotchas (most notably that un-altered trigger definitions on the underlying target tables with any dependence on external tables would be operating locally rather than through the same abstracted view, and insert triggers may fire at unexpected times). For now the limited Merge version is sufficient for our needs however, so we haven't gone so far as to even kick the tires on this possible approach.

I have a few more changes to make to the sandbox recreation script before it's ready for wider consumption, but it's not an especially arduous process for someone enterprising to reproduce on their own with little effort.

Enjoy!

Rather than put together an Intel hacintosh or risk the wrath of my wife by messing with her new Intel iMac (used for her wedding photography) as required to run the X Code IDE/SDK, I decided to use Jiggy's JavaScript based framework and IDE. While this surely introduces some functional limitations and performance overhead, it was also a very painless way to slap together a proof-of-concept to get a good idea of the problem space I was approaching.

Given that the device contains a set of accelerometers (used by the iPod/iPhone user interface to determine landscape and portrait layout), I thought it would lend itself well to a Persistence of Vision hack - which is to say, using a thin portion of the display, flash certain bits of it on and off as the device is shaken back and forth. The flashes correspond to certain spaces of the visual field during the swipe, and as a result of the mechanics of the human eye (which takes roughly 1/60 of a second to clear out the stimulus) appears to create many little spots floating in the air which can be used to write letters, display images, etc.

I first encountered the effect at a novelty store when I was about 9, when I saw a clock which consisted of a sturdy base and a spring-loaded wand which, when flicked, would flash it's LEDs as at waved back and forth and created the illusion of floating time. While several devices have picked up the gimmick since then, I've never actually bothered to own any of them. Now that the opportunity was right in front of me with a low barrier of entry, why not give it a shot?

First came the math. Working with my wife we were able to determine that a good hand-wave average was roughly 16", making a full cycle (back and forth) every 1/2 second or so (11 over 5 seconds). In order to achieve sufficient resolution to be able to display any letter of the alphabet discernibly, while still keeping the overall number of cells sufficiently low (so as to have low logical and processing requirements) we chunked this 16" section into 6 letters per wave, with each letter broken into 5x6 segments. 6 letters per wave is pretty low, but the idea was to have a word persist only through a single wave - as soon as the accelerometer detected a significant shift in direction it would load the next word, allowing sentences to be spelled out sequentially.

( 6 letters * 5 columns = 30 columns ) / 16" / 0.25 sec (single wave phase) = 120 columns / 1 second, or 120hz switching. In order for a line to pass through 1/2" of space during the course of a single wave, it would need to be on for only 0.008333... seconds. That's for a perfect line - I would of course be switching a region rather than a mathematical construct, so I would in theory need to divide that 120^th of a second by the width of the region in order to achieve the target resolution. I fudged this part, and for the sake of the proof-of-concept development decided to have the region correspond to the width of the column itself - 1/2" square, which means that if everything worked perfectly a total of 1" would be illuminated, a 1/2" of virtual overlap with itself (area of continual brightness) giving a linear fall-off to either side. So the letters would look a little fat and maybe just a touch blurry - but that's acceptable to start with.

However: most displays only refresh at a rate somewhere between 70-90hz. If I'm trying for 120hz it means that each cell would fall quickly out of sync, ending up only partially drawn or have some frames of display skipped altogether (compounded by the actual switching rate of the LCD). The second component of the problem is the brightness of the display itself - in order to sufficiently invoke that perceived persistence of stimulus, a certain intensity is required. If intensity is not available contrast may be substituted, but this means it will only ever work in a darkened environment.

Despite the math working against me I decided to give it a go. Setting the interior timer to 120hz is easy enough, but I suspect the time to navigate the DOM logic to the desired cells and switch their states runs slower than that anyway, plus whatever overhead the Jiggy framework required to assist in doing that. I set up the code to follow this plan, draw a column of cells and switch them over the course of fractional sections to the different mapped columns which comprised an entire letter table (in this case a letter is an array of 5 different numbers 0-63 which corresponds to a 6-bit binary display indicating which cells to turn on and off - makes for very tight book-keeping and rapid addressing, and binary logic is just fun anyway so why not?). My test consisted of letters A through E sequentially set up in memory in a per-wave array (all in a single wave) with events tacked to the accelerometer which would catch significant departures from one direction to the next and determine whether that array was being walked forward or backward, even capable of reversing mid-display.

And it almost worked. In the end I had to simplify the display to at most 2 letters, and it only works if 1) the room is dark and 2) one covers the persistent battery-level display at the top of the screen (otherwise it wrecks the PoV effect). Photographing this is especially difficult when one is both the photographer and demonstrator, compounded by doing it in a dark bathroom shot "through the mirror" without being able to effectively test the composition. In the end I got a few good shots and many not-so-good - shown here are 2 images (ISO 80 f/2.0 @ 1/25 sec, the display simplified to just the letter 'A') combined and slightly enhanced for contrast. This shows the difficulty of keeping in sync with the animation - something that with enough tuning of the accelerometer event code and a steadier hand would become easier, but which I don't plan to pursue due to the aforementioned global limitations. The predicted linear fall-off is visible, as are the very slight gaps between vertical blocks on the display.

Overall I'm very pleased with the exercise: it taught me more about a really fun toy, gave me a chance to play with some simple science, and allowed math to Save the Day. Everything worked almost perfectly the first time and I come out richer in knowledge for the effort. The fact that any of this at all pleases me just reinforces the fact that I'm in the right line of work - I'm pretty sure most of the world would find this pretty dull. Hurray for geeks!

The human brain is separated into two distinct hemispheres, split down the middle front-to-back. Anyone familiar with basic biology will have seen this on cross section diagrams, as well as the generalized labels of specialization for the two halves. While it's true that the functions of higher consciousness within the neocortical regions of the respective hemispheres differ, it's also important to note the operative similarities and redundancy in the design as well.

Each half is equally paired to sensory stimuli as provided by taste, smell, sight, touch, and sound, as well as input from the limbic and primitive sub-areas of the brain which process and filter reactions to those stimuli before insertion into consciousness. The redundancy is not perfect, however, in that the signals received are those generated by a fixed half of the body directly wired into the respective hemisphere. The simultaneity of the combined experiences prevent this from being a disorienting experience however (since anything we hear with one ear is generally heard with both, for example) and is reassembled into a single contiguous phenomenon, leveraging hemispheric disparities to be able to proved depth perception and rudimentary echolocation, etc.

The real divergence comes when one reaches the frontal lobes. These portions are responsible for housing operations dealing with conscious thought and complex reasoning, doing so with a high degree of specialization. The right tends more toward the spatial, intuitive, abstract, emotional and creative tasks, while the left provides language, critical and logical thought, and temporal referentiality (chronological awareness). These are not separations of "personalities" inherently conflicted with one another, nor a contest of dominance within which there is any "proper" way to think or behave. Personality is a construct expressed across both sets of functions, and what matters to the personality matters equally within them, in their own ways.

Hemispheric (Un)Cooperation

But they don't communicate very well with each other. Which is not to say that there is an inhibition or implicit miscommunication which will confound information as it crosses between the the two. There is also, to borrow hardware design terminology, no latency or bandwidth restrictions on the communications bus. This bus takes the form of the corpus callosum, a dense bundle of nervous (meaning "of nerves," not "anxious or excitable") fibers whose only duty is to provide a channel by which information and data may be exchanged.

The difficulty in exchanging that data is based on differing APIs: the right hemisphere has no mechanism to appreciate the number 5, although it can be familiar with the shape and sound of it, and spatial groupings which miraculously come to the same total when counted. Similarly, the left hemisphere cannot comprehend the color yellow other than as a word (non-pictorially), wavelength of light, or hexadecimal RGB representation (my favorite tends a little more toward the golden variety with a touch more red/less green, an #FFED00). How then is it possible for a word to evoke spacial concepts, or for creative concepts to be placed within the logical structures required by software development?

It's interesting to note here that complex structures are almost always based on abstract spatial relationships rather than strict logic, and it is the rigidity of rules in the medium in which they are represented which forces them to appear complex. Most accountants are left-handed, showing a clear preference for the abstract processing capabilities of the right hemisphere - meaning they're also cut from the more emotional and creative fabric of society, directly opposite of the bean-counter exactitude embodied by the more simple and stereotypical book-keep the word "accountant" usually invokes. Real rules of accounting are difficult for us right-hander folk to grasp at a glance because we make the mistake of attempting to process them solely in within the logical realm of the left hemisphere, rather than correctly mapping them into their true spatial connotations on the right.

How does one establish this ability to do that translation if the two halves don't speak the same neurological language?

Memory & Concurrency

The first way is through the parallelism of experience, mentioned briefly earlier, and then through concurrency of processing.

Memory is stored across the entire brain as coordinated by the hippocampus (found in the limbic system atop the brain stem). Even though there are very different types of memory we'll treat them under the generic class name here just to be easier on the science. An event/experience or concept is stored in the brain the same way it is initially represented, as a series of impulses in a variety of locations. The memory is committed to long term storage as this impulse pattern is reinforced by branching of the synapses, which is literal physical growth and extrusion on the dendrites of neurons. The formation or reinforcement of the connections created by this growth make it more probable that subsequent stimulation in similar areas will reproduce a similar pattern or portion thereof, like the path-of-least-resistance in any electrical circuit. Trigger cells, regular neurons which just happen to fill this particular role during the memory formation, are then specifically assigned to be the gateway to the reproduction of the pattern in its entirety. The number and strength of the connections to the trigger help determine the likelihood and ease with which it can be remembered. Thus all habit and practice in the brain changes its shape, and does consume massive amounts of resources from the bloodstream; it's OK to be tired if all you've done all day is think really hard.

Since the experience takes place in both hemispheres simultaneously, there is a temporal coordination: a time-index which can be used to reproduce coinciding patterns. Returning to the example of color, a child knows what ____ is long before it knows what "yellow" is. Putting the two together is a matter of hearing the word at the same time the color is physically present, in enough different contexts (different objects of the same color, different textures and materials, smells, etc.) that the color stands out as the sole common right-brain event which overlaps with all of the times the funny-looking big person has been making the same new sound with their face. Once this gels and is well practiced, remembering and interacting with the stored fact in a variety of partial contexts becomes easier because the whole pattern is subconsciously reiterated. Largely through the acquisition of language, the process of learning simultaneously with both halves of the brain becomes a simple habit.

Once enough patterns are stored in this fashion, dwelling on their recall while exploring new territory creates the opportunities for the synthesis of new associations based on the discoveries of either hemisphere, still effective across the whole because they've coincided temporally once again.

The Full Brain

How then to use this to the advantage of a software developer or any discipline requiring interaction with highly-complex systems? Making the most of the specialization of both halves must be done by engaging each of them fully in the task at hand through the channels in which they are most apt.

For me, I'm barely able to function when I crack open a new API regardless of whether or not I'm already familiar with the language. The features and structures required by the unfamiliar interfaces have to be walked through, step by step, module by module until I can grasp them categorically and generically. Even then it takes a fair amount of practice within that framework before it becomes comfortable and intuitive, which it manages to do all at once with enough of the right kind of exposure: it just clicks. It is then that an analog for the logical structures has been created, and I can leverage the spatial navigational strengths of the right hemisphere to make my way around the program seemingly blindfolded.

The spatial mapping happens piece by piece, understanding the code as its general "physical shape" (patterns of line lengths or syntax highlighting), its relative position within its containing file (amount of code above vs. below), and more nebulously it's "flavor." This is an intentional synaesthetic translation of logic into kinds of phantom sensations: how the code makes me feel regarding its elegance and efficiency, the degree to which it needs to reach out to other modules vs. self-containment/sufficiency, etc. These become aspects of spatiality, but not something that can be drawn well or represented in three-dimensions because it is abstract enough to be not so constrained. The process may sound exotic, but happens largely unconsciously, and most developers do it to some degree or another.

Once a portion is in place, others then build off of it by using it as a point of reference, and soon the entirety of the application has a feel to it as well as a virtual expansive layout. The whole of this "abstractification" is like defining a subroutine - a simple function which stands in for a body of logic and can be comprehended simply by identifying the name of that function in the future. This allows for a higher-level overview, essentially zooming out on the granularity of the code all the way to an architectural flow diagram. It's the reason those things are ever drawn in the first place, and these methods of whole-brain analysis are the key for reverse engineering or developing the visual/logical relationships which result in a usable, maintainable product without overloading the mind. Which means, for you non programmers out there, we have some secret tricks we do to make all the gobbledygook into something meaningful; participation in this field is not an automatic indicator of intelligence or even aptitude (just take a look over at TheDailyWTF, which celebrates "Curious Perversions in Information Technology").

The conclusion? Try bringing the unconscious analysis to the foreground and more intentionally leverage the kinesthetic aptitudes. There are several ways of doing this and even artificially jump-starting it (diagramming with a non-dominant hand, for instance). Perhaps in a future article I'll explore exercises and tests designed around that, but for now this should be enough food for thought to expand your universe just a little more. And if not, it was a free read.