With high and increasing crude oil prices this year, everybody who comments on economics is weighing in on what he/she thinks are the causes and what should be done—or more often, what should not be done—about the situation. A large majority of commentators make the threshold error of assuming real-world crude oil markets work like the idealized models in their economics textbooks. They don't. Here is an excellent summary of how the methods of pricing OPEC crude oils at the producer level have changed several times over 40 years and how it really works today. Here's my summary of the summary.

In the mid-20th Century, international oil ownership and production was very much in the control of the Seven Sisters and a few smaller international oil companies like Occidental. In a foreign province, they obtained the concessions from the government, made the investments, owned the oil, determined how fast to pump it, set the prices, and simply paid a royalty to the host governments. Prices were administered entirely by the oil companies; they "posted" the selling prices and, very occasionally, changed them in their own discretion.

Then in the 1970s the big oil producing nations in rapid succession partially or totally nationalized their oil concessions and took over price administration. During this period, which lasted until the mid-1980s, the host governments set the prices and constrained production as necessary to enforce price discipline within the producers' cartel. But then demand plummeted, non-OPEC producers became significant, and there was so much excess capacity that discipline among producing nations could not be maintained. Companies that bought directly from the producing States refused to pay the posted prices, and started buying in the spot market or from non-OPEC members. OPEC's administered pricing system collapsed.

Saudi Arabia fought back by adopting a netback pricing system, where the price of the crude oil would be calculated from the selling prices of refined products by deducting a negotiated refining margin and transportation costs. Prices crashed from $26 to $10, and that ended netback pricing.

This was followed by a system of long-term contracts between OPEC members and their first purchasers in which the price of each shipment was set by reference to published reports of spot prices for marker crudes such as West Texas Intermediate and/or dated Brent (North Sea) crude oil. These reports were recognized as problematic for several reasons. Because actual sales of wet barrels in the reference markets were fairly infrequent, it was possible and useful for players to manipulate prices on critical dates. The reported prices were third-party estimates of prices based on surveys that were vulnerable to misrepresentations by the persons interviewed. Marker crudes were quite different from typical Middle East crudes in both quality and location. Marker crude production rates declined as Middle East crude production increased, making the tail smaller and the dog bigger.

To replace the unsatisfactory system based on reported spot prices, producing nations adopted the current system, which is to sell on long-term contracts with terms that price each shipment by reference to the crude oil futures market(s). The change was made because the futures markets seemed more transparent, larger and more liquid, and less susceptible to manipulation or misrepresentation than reported spot pricing. Long term contracts with this type of pricing provision are now the norm.

Today there are three distinct—but interrelated—crude oil markets: the primary long-term, reference-price contract market between OPEC members and first purchasers, spot markets for actual delivery of wet barrels, and the commodities futures markets for paper barrels. Probably a substantial number of transactions and prices in all three markets are never publicly disclosed. Thus, actual crude oil markets are not at all like the ideal markets assumed in textbooks and by most commentators on economics. The critical question, to which I have seen no convincing answer is how, to what extent, and under which circumstances each of these three markets—and the players in them—affect the other two markets.

Coda on blogging

The foregoing summary and most of my understanding about what's going on came from the blogosphere. Mainstream media reports were limited by space, reportorial understanding, and a bias for reporting controversies instead of information. In wonderful contrast, relevant parts of the blogosphere have experts arguing with one another in depth. Sure, the blogosphere contains vast quantities of wasted and erroneous words, but I have found several sites run by real and careful experts who elicit some very informed and insightful comments. Now I'd give up my subscription to the Los Angeles Times before I would stop following my favorite blogs.

Most of my improved understanding of oil prices came directly or indirectly from Paul Krugman's blog, where he started posting on oil prices in April 2008 and followed up with at least 6 more. At first, PK thought he would just explain to everybody that there is no speculative bubble because there is no physical hoarding. Full stop. He illustrated this with diagrams from a textbook. He got a storm of criticism, at least 10-20% of which was obviously well-informed, about practical realities of crude oil and/or futures markets. Some of the critics and supporters were obviously professional economists and some were recognized industry experts like Phil Verleger. Apparently, PK was also communicating with his peers off-line, and he linked to other blogs even when he didn't agree with them. I believe PK's own understanding changed considerably during these 10 weeks of dialog. Mine certainly did. It was a fascinating, collaborative, civil, intellectual process in which anybody could join.

PK's blog enriched me in another unexpected way. The citation to the report I summarized above was provided by JD, comment #35 to this Krugman post. [UPDATE 2/8/2011: The JD comment is now #16; apparently some comments were purged.] Before that, I had made a targeted online search for an explanation like this but could not find one and had given up. (JD provided other useful links too, as did other commenters.) Before I started regularly visiting selected blogs, I had assumed they would be useless as a source of basic facts and resources. I have found that the right blog can be a good tool for finding relevant and reliable information.