20100623

Book Review: Priests and Programmers

I recently finished reading Priests and Programmers by J. Stephen Lansing, a book on irrigation practices and native religion in Bali.  To be honest, I found this book in the $1 used book bin at a textbook store outside UC Irvine, and I thought they were talking about computer programmers.  So I bought it expecting to find some Orwellian tale of a totalitarian government using religion and technology to blind it's populace.  Turns out they mean "programmers" as in "people who make up schedules," in this case, the priests setting the irrigation schedules.  Once I started reading it though, it was so interesting I kept going.

Apparently, the island of Bali (located in Indonesia) has an amazing irrigation system that has to be carefully maintained in order for anyone to grow any rice.  During the rainy season, there's water enough for everybody, but during the dry season, the water usage has to be regulated so that all of the farmers get enough water for their rice.  This gets slightly tricky since rice needs lots of water at the beginning of it's growth cycle, and very little at the end, so if you stagger the planting of the various farmers in the region, you can minimize the water shortages and maximize the rice.  The real trouble comes from the rice pests, which hop from field to field as long as somebody nearby is growing rice.  So to control the pest population you want to coordinate nearby fields so that they all lie fallow at the same time and the bugs and things die.  Striking the correct balance between synchronizing and staggering the fields is a complicated process that has to be tuned to the current conditions like rainfall and local weather.

In Bali the farmers attach mystical significance to the river and irrigation waters as being the property of the various gods and goddesses that inhabit the island.  As such, all of the water usage is coordinated by priests that inhabit temples built at the source of each canal, river, and lake.  These are called water temples, and serve to coordinate the water usage and planting cycles of all of the farms that belong to a particular watershed.  Lansing also argues that the religious rituals of the Bali people that involve the mingling of holy water from the various springs and lakes attached to the temple, symbolize the reliance of all these separate farmers and peoples on each other, and create a sense of group consciousness of the responsibility and social implications associated with water use.

Besides offering detailed accounts of the relevant religious ceremonies and an analysis of a computer model demonstrating the impact of water temple coordination with pest cycles, Lansing also tells the history of the Bali island and its water temples.  The Dutch arrived and eventually conquered the island in the 1850's or so, and tried to manage the local populace and the rice production.  However, they failed to notice the role of the water temples in coordinating the farming activities, because they tended to discount anything related to the local religions.  So the system kept functioning undisturbed until the "Green Revolution" hit Bali in the 1970's and 1980's.  Bali was now being run by Western-educated Indonesians, who didn't understand the roles of the water temples, and started disrupting the traditional farming practices by introducing new high-yield crops and farming technologies.  This quickly resulted in disaster as pest populations swarmed out of control, and devastated rice yields.  Farmers who resisted the transformation were written off as religious conservatives who were fighting change.  (Granted, it's not clear to me that the farmers or the priests completely understood their role in regulating the ecology of the rice fields, so I don't think you can completely write off the Western influence as being unduly dismissive of the traditional practices.)  At the time of the writing, the government of Bali was starting to study and understand the role the water temples play in managing rice production, and is beginning to allow them greater control in regulating the rice production.

All in all it was a good read.  There was a little bit of social-science style "Marxian Analysis" at the beginning and the end, but the rest of it was just anthropological study, history, and ecology.  I think it's definitely worth the few hours it takes to read it.  (It's not quite 200 pages altogether).

20100622

The future of mico-pricing schemes...

I just read The Man Who Could Unsnarl Manhattan Traffic on Wired, about one man's attempt to model New York city's traffic system and find ways to improve it.  He's released a giant spreadsheet outlining his findings, along with a plan to improve the efficiency by introducing a toll system to charge drivers entering high congestion areas.  The article itself was a fun read, but what really got my attention was the stuff at the end on micro-pricing schemes.  The idea is, you can put GPS technology in everyone's cars, and then charge people on a per-street basis for where and how much they drive or park.  From a theoretical modeling stand-point, this level of control always looks amazingly efficient, because you can optimize prices to more accurately reflect the cost of maintaining that stretch of road, and the cost to society of the delays you cause on other drivers during congestion.  You get to decide, at 4pm rush hour, whether or not it's worth $5 for you to drive home the quick way, and in return, society has a quick way, for the people who are willing to pay $5.

Another industry that could benefit from a serious micro-payment system is online newspapers.  Newspaper "pay-walls" repeatedly fail, and the newspapers blame the publics' unwillingness to pay for anything on the Internet because they feel entitled to it.  That's probably at least part of the problem, but I don't think that's the whole story.   I pay for things on the Internet all the time, but I'm not willing to shell out $50, or even $20/year for a subscription to some random newspaper.  I get most of my news from links on aggregator sites, things like Google News, Slashdot, and the few dozen blogs and RSS feeds I subscribe to on Google Reader.  Now partly that's increased competition driving the prices down, but mostly I just don't read enough articles from a particular source.  Now, over the course of a year, I probably read a few hundred New York Times articles, and they're usually very good.  I could even assign a value to the amount they enrich my life, and would easily be willing to pay $10-$50 (depending on my mood) for the privilege of doing so.  But each individual article contributes relatively little to the total, perhaps 10 cents for a good article.  So when facing a pay-wall to read any particular article, there's no way I'm going to shell out $10 in order to read it.  While it might rationally be in my best interest, I don't have enough information on-hand to feel justified making a commitment to read $10 worth of articles over the next year.  This changes when you're going to a particular site frequently enough that you know you'll get your money's worth.  I actually bought a one-year online subscription to Scientific American for $50, because I'm signed up for their RSS feed, and I read at least 20-30 articles a week off of their website, so I felt fairly confident that I could read $50 worth (in value to my life) of articles in a year.

As a small aside, I ended up canceling my subscription, because it didn't end up doing my any good.  What I ended up buying was a PDF version of their magazine (complete with the usual ads) every month.  I don't sit down and spend a few hours reading a magazine, I spend 10-30 minutes in a stretch reading a smattering of individual articles off of my RSS feeds.  The subscription wasn't at all integrated with their website, and when I would click an article off of my feed and hit a pay-wall, it would redirect me to a page letting me know how I could buy the PDF and read the article.  But I already had the PDF, it was just too much work for me to go dig it up, and figure out what page the article was on, and dodge the advertisements in order to read it.  So I would move on with my life to the next thing on my feed.  I ended up not doing anything with the PDFs, and still reading the same articles I was reading for free off of their website.  One day I finally found an article that looked cool enough for me to track down in the PDF, only to find that I had moved or deleted it (realizing that I never used it) and thought I could download it again. But, alas, your download link is only valid for a week or two, so they wanted me to pay them another $15.  That was the last straw for me.  I canceled my subscription and they were nice enough to refund me $30 of it.  Kudos for that at least. 


 Personally, I don't think that the business-end of the publishing companies and news agencies have completely caught up with the change in people's reading habits due to technology.  They're still thinking in yearly subscriptions, when people are now thinking in individual articles.  The conundrum they always face with these pay-walls, is that to make people realize how much value they are getting from The New York Times, you need to let them see enough content for free that they feel justified in making the commitment of a subscription, but not so much that they can just get it all for free.  What they really need is a micro-payment system that works on an article level.  If faced with a pay-wall that could take 5 cents out of my Paypal account (so that it's site independent), then buying the article becomes a legitimate decision that I feel qualified to make on the spot.  They tried to do this with online advertising, where the advertisers essentially pay my 5 cents for me, but between the bottom falling out on the online ad markets, and the increased competition from every paper in the world (as opposed to the smaller local papers you can actually buy physically at the store), they still seem to be failing.  I think moving to an article-based model would actually help them, but it requires a good micro-payment system to be in place.  Of course, it also requires the news agencies to step up their game, and compete with each other on the quality of their articles.  When you can just buy the headline articles, they don't get to force you to bundle all their filler with it.  I mean, honestly, have you ever read the "Cooking Tips" articles in the back, and if so, would you pay for them?  So while the news agencies might be making less money selling you "Pet Diet Tips", the actual efficiency of the system increases, because they stop writing articles nobody cares about.


But practically, (in my opinion), the problem with all of these micro-pricing schemes is that people have a hard time managing that many micro-decisions.  First, they have a hard time making that many decisions optimally.  It's one thing to decide on a given day whether or not you want to pay $5 to take the toll-road on your way to work and save an hour of commuting time.  It's another thing entirely to make block-by-block decisions while driving, about whether it's cheaper to turn left now, or go an extra block and then turn left, and to do so at every turn.  It's just too many decisions for a person to make to be worth the hassle of them caring.  Granted, the granularity of the pricing scheme can be adjusted, but the coarser you make the controls, the less efficiency gains you get compared to the theoretical ultra-fluid model.  Even if the number of decisions is reduced to a manageable level, the associated costs of each option have to be large enough that people can realistically assess them.  For instance, if taking a particular one-block detour from my normal route will save me 5 cents, but cost me a minute of time, it's kind of a toss-up to me whether it's worth the 5 cents and the mental strain of taking a detour, or the extra minute I save going that way.  If I were particularly motivated, and I travel the route every day, with predictable prices, I might justify a permanent detour in the name of saving myself $10/year or so and costing myself an hour of my time.  But I'd have to really care.  But if you increase the granularity a step further, and the trade-off was between 5 tenths of a cent, versus a few seconds of my time, I'm really unable to make the call, and for stakes that small, I really don't even care.  Again, on a larger scale, making a consistent decision for every trip I take over a year might add up to something meaningful, but it's really hard to tell from the collection of micro-decisions.  Let alone adjusting these fluidly to account for the fact that if I'm late for an important meeting, as opposed to on my way home from the park, my priorities may be drastically different.  At some point, the fine-grained types of theoretical control that these super-efficiency gains need break down when you require people to be making these sorts of decisions continually, and rationally.

But even counting the number of times people make irrational decisions, either because they can't tell, or they don't care, let's suppose that they can make rational decisions enough of the time, that you can get efficiency gains.  The next problem with a practical implementation of this is the perennial customer-service favorite: the Bill.  The cellphone companies are starting to get a taste of this with the consumer back-lash to what is essentially, a flat-rate micro-payment system for your voice and data traffic.  Discrete, simple choices lend themselves to customer satisfaction.  Assuming I get what I think I paid for, when I spend $50 and buy a chair, I did so because I knew the chair would be worth more to me than $50 and whatever else I could exchange for it.  At the end of the day, I'm happy with my chair, or at very least, I feel responsible for having made the decision.  I'm not surprised when the bill comes at the end of the month, because I remember making the call.  If I eat out at a fast food restaurant a few times a month, I may not remember the exact number of visits, or how much each trip cost when I get my credit card bill.  But, I at least feel confident that each time I went there, I made the decision that at the time, it was worth $5 to get my taco.  But if I visit 2,000 websites on my cellphone, and get a bill at the end of the month for $30, I may not feel so confident that I visited each of those websites will a full understanding of the consequences.  Partly this is due to lack of transparency.  If I could see the cost of a page before I went there, or at least track my progress (debt?) over time, I could get enough feedback to self-regulate my usage.  That would help me keep my overall use and overall cost in the same ballpark, but I'd still have a hard time looking at my bill at the end of the month, and deciding whether or not the 2,000 micro-decisions I made were each justified.  Now take out the flat-rate, like in the case of the road networks or newspaper articles.  Imagine that your data-usage fees varied by time of day, day of the week, and which websites you visit.  Instead of the one-shot decision of 3 cents per google search, now you're faced with scheduling issues where waiting an hour to do your google search might only cost you a penny, or you can do it in 15 minutes for 2 cents.  As the complexity of these decisions moves beyond the ability and motivation for people to make, your satisfaction and confidence in your bill decreases.  This is why the cellphone companies use block-subscriptions to sell their service.  People buy more than enough minutes or bandwidth to cover their needs, and can then make one decision that $30/month is worth their overall usage.  (It doesn't help that the cellphone companies are predatory, and quite happy to charge you obscene rates for going over your plan.)  But the efficiency of these networks suffers as a result.  With a subscription plan, everyone calls during peak hours, because they have no incentive not to, and the services get overloaded.  Things like free "night and weekend minutes" are a halfway measure to increase efficiency (and act as a great marketing ploy) by encouraging people to move nonessential calls to times when the network is free by offering monetary rewards (free minutes and therefore a reduced bill).  The trade-off then is between the unpopularity and the efficiency gains that are lost, or only partially achieved.  And that's assuming that there really were efficiency gain on average with an unpopular pricing scheme.  Otherwise, if people end up not caring or acting irrationally because it's beyond their abilities, you simply end up with capricious and opaque pricing schemes, on top of the normal inefficient usage patterns.  So now the micro-management system is useless, and unpopular.

That's why I think micro-payment systems imposed on people are doomed to fail.  But, they have a shot if people can express their preferences, and let a computer make the hundreds and thousands of micro-decisions for them while protecting their interests.  The most talked-about example of this is probably the so-called "Smart Grid".  Power companies are faced with incredibly volatile and complex markets.  Every time you flip on an appliance, a huge amount of power starts suddenly starts flowing into it, creating what's essentially an electrical shock wave that flows all the way back to the power plant.  If they don't compensate for all these jolts to the grid, the system collapses, and everybody loses power, and/or fries their toasters (both literally, and figuratively).  A number of years ago, someone documented that relatively large spikes would occur during television commercial breaks, as everyone left their televisions to go turn something on or off.  (I can't find a source for this right now, but if I do I'll put it in.  I'm fairly confident in the assertion however.)  The power companies also have to match total power output with the amount people are using, to avoid either wasting power, or causing black/brown outs.  So, as demand spikes during the day, they turn on increasingly expensive generators to make sure it all works.  But if everyone washed their clothes at 3 AM, when all the lights were off and the businesses were closed, it would be much cheaper for the power companies because the total load on the grid is greatly reduced.  But currently there's no way to communicate such a complex set of power pricing to consumers, and even if there were, there's no way to measure it so that the bills could be computed.  And after you solve those, you still have my earlier points on the failure of micro-pricing schemes.  What the designers of smart grids hope to do, is to put electronics into every appliance and home on the grid, to report and coordinate power usage and pricing schemes.  The dream is that your computer or heater could enter "low power mode" during peak usage times, or short bursts of strain on the grid, and your dishwashers and laundry machines could wait to turn on until rates were cheaper.  The power company would then return some of the savings to you in the form of cheaper power when you're using it at the non-peak times.  Assuming you could get all the electronics in place, I think there's hope for regulating continuous things like air conditioners.  Almost everyone I know regulates their thermostats and air conditioner usage to save money on the power.  If they could either get a read-out of the current, actual costs, or be able to set preferences on their computer, they could only do so more efficiently.  Imagine if you could tell your air-conditioner to not only turn off when you're not home, but try to keep your monthly bill under $20.  Perhaps while keeping the house under 85 degrees at all costs.  At that point its just a question of user-interface, to specify what you're willing to spend and tolerate.  I have less hope for sporadic things like dishwashers and laundry machines.  I guess I wouldn't mind my dishes being done by morning, but if I have to go switch my clothes into the dryer I want to know before hand when it'll be done.  But at any rate there's hope.  I would do my laundry at 3 AM if you paid me enough.

The salvation for micro-pricing systems of road-usage is the advent of trip-planning GPS navigators.  Whether you're planning your trip on Google Maps which advises you that you can save time and money if you wait an hour to leave, or informing your portable GPS device that you value your time at $15/hour, the computer can process all of the possible route information and optimize it for you.  Saving 5 tenths of a cent for a 3 second delay may not matter much to you at any given time, but saving $20 for an hour of time spent driving over the course of a year might be worth it.  At any rate, now you have the capabilities to make that call.  Run one of these in conjunction with the ability to look up your current account status, and you've got a shot at having a workable system.  Particularly if you can see options and prices you can choose between before each trip, or the pricing and timing differences on your bill at the end of the month had you have selected one of the "economy" or "speed" modes.  Granted this all really only matters in big cities and other high-congestion areas, but those are also probably the only areas where you would want to implement some kind of variable toll or taxing system.

Newspapers are harder.  I'm not sure about the per-article production costs a newspaper had, so it's hard for me to say whether or not they could get enough to compensate for the loss of subscriptions with mandated filler purchase, or how much they actually make off of the in-page advertising.  I honestly think a micro-payment system through Paypal could work if the prices are small enough.  But sometimes it's hard to tell if an article is worth $0.15 cents before you read it, and you still have the problem of getting your bill at the end of the month and not quite being able to account for where it all came from.  Computer technology is less able to help us here, because even with access to the full text, computers aren't yet good enough to tell us how good an article is (on average), and predict whether or not we'd like it.  I think there is hope down the road if AI technology continues to improve however.  I don't think it would take much to integrate a payment system into Google Reader.  Individual articles could have some kind of "paid content" icon, with a listed price next to it.  AI technology could process what kinds of articles I read, and allow me to rate them, and set price levels that I would be willing to pay for them to train it.  All the "Web 2.0" is doing this already so that they can data-mine peoples preferences for advertising.  Just take some of that technology and pump it into predicting price levels I am willing to pay for it.  Then, let me set maximum price thresholds or monthly bills, and view their progress, and let the AI automatically filter content that is either outside my price range, or costs more than expected interest in it.  Perhaps have a "premium" category of things I'd like to look at the headlines for, but be warned about the price before I click them.  Add a button to let me flag "this article was stupid/over-priced" and you've got a shot at a workable system.  Cable companies and Newspaper companies have always gone with subscription services to "networks" of content (be it channel selections or a series of newspapers), but I really think with the Internet proliferating the number of content/article providers, there's too many with too little control to make subscribing to a giant "news and content network" plausible.  The only way I think it could work is with a micro-payment model, perhaps coupled with an AI to help keep my bill within reasonable bounds.  The big problem is that it's so hard to quantify the trade-offs involved.  In my car, I can choose to spend the extra hour on the road, or pay $5, and time and money are both quantities that aggregate across multiple trips.  The amount of "life-enrichment" I get from reading an article is hard to compare to another article, let alone calculating the amount of money I saved by missing out on a few dozen "liferichment points".  But if you read a lot of articles and the prices involved are too small or large, I'm not sure a human could keep up with the necessary decisions.  So put me down for a "maybe" on the future of online news articles with micro-payments.

Finally, cellphones are a lost cause.  Choosing to delay a call from "prime-time" daylight, to "free night and weekend minutes" is about as complicated a choice as I think people are prepared to make.  For web surfing its even worse, because there's often no way to tell how much data is required for a given visit to a site, or surfing session.  The best I can offer here is lots of tracking and transparency (which so far the cellphone companies have resisted providing, whether out of avarice or laziness).  I think there's some hope for AI programs to track your usage patterns, and offer suggestions, but I have a hard time seeing large enough price incentives for people to care.  A trip to Youtube might pop up a message saying something to the effect of "Normally you download 5 MB on a visit to Youtube, which will cost you $2 right now.  If you wait an hour, it would only cost you $1."  Perhaps more useful would be "You usually talk to your sister for 3 hours, but if you call her after 6pm, it would be free."  But I suspect that most phone web-usage is too time sensitive enough to delay, and setting truly variable rates would be too chaotic to make meaningful future predictions.  The best I can offer if the networks get really desperate is a message when you start your browser saying "Current rates are 0.10 cents/ KB", and maybe someone doing causal browsing would decide it's not worth it right now, but I'm not sure that's really much of an improvement over the flat-rates they offer now.

So uh, I'm not sure if I had a point in all this, but the next time you see someone talking about micro-pricing schemes, be skeptical, and look for the computers!

20100607

Another victory over Amarok!

Playlist filtering works!!!!


I was bemoaning my inability to search, when I noticed that editing tags on the previously unhideable tracks made them start working normally.  But only some tags.  I can only assume, for some reason, they were never entered in the amarok database?  I dunno.  It was  a mix of flac and mp3 files that were in there.  Possibly things I've never edited by hand since the reincarnation of my amarok database.  But at any rate, giving them all a star rating makes them work now.

Now maybe I can finally finish tagging this mess...