10 Rules for Radicals
10 Rules for Radicals
10 Rules for Radicals
Thank you for your kind invitation to speak before you today on this occasion of the 19th International World Wide Web Conference. I would particularly like to thank Paul Jones, a man whose sites have hosted poets and presidents and who has given a home on the Internet to the people of Linux and the people of Tibet. Tim O'Reilly says you should put more into the ecosystem than you take out, and there is no better example of this than Paul Jones and his decades of public service to the Internet.
Before I turn to the subject of my talk, I feel I can give you a little context about how to judge these words by telling you about the first time I saw the World Wide Web in action. I was visiting Geneva in 1991 because I was interested in CERN's role as a hub for the growing net, using X.25 to gateway to the Soviet Union and Eastern Europe.
While I was visiting CERN, the head of networking, Brian Carpenter, said I should go see one of the researchers who was doing some interesting work. I went into a dark room and a young man sat behind his spiffy brand-new NeXT workstation and he showed me his research project, a derivative of SGML with a little bit of local area networking thrown in.
I politely watched Tim Berners-Lee give me a demo of his so-called World Wide Web, but I was skeptical. It looked nice, of course, but then anything looked nice on the NeXT workstation, a high-priced hunk of hardware created by a bunch of Apple refugees. Tim showed me how with a click you could pull up a file on another computer, but I wasn't sure this was something that would ever catch on, certainly not on a global level. I distinctly remember thinking to myself "this won't scale."
Having presented to you my credentials as a pundit, I would like to talk to you today about some bureaucracies I've had occasion to encounter, and some lessons I have learned about how citizens—citizens with no official portfolio or status—"mere" citizens if you will—how citizens can change the way government works.
I hope these tales are more than mere war stories, I hope to leave you today with some rules for radicals, 10 rules to apply to governing institutions as we attempt to change their behavior.
—Bureaucracy № 1—
We begin in ancient times, a time so long ago that the term broadband referred to ISDN lines which would operate at a massive 64,000 bits per second, the speed of a leased line—but magically switched on and off on-demand using the ISDN "intelligent network."
This time—the late 1980s and the early 1990s—was a time when the idea of a hyperlink was still considered the mad delusion of a wild-eyed prophet named Theodor Nelson, hence my skepticism about TimBL's research project.
In those days, there were two kinds of networks. There was the so-called Internet, and there were respectable networks. The respectable networks were being defined by international institutions such as the International Organization for Standardization and the International Telecommunication Union.
These institutions were based in Geneva and their work product was meant for grownups, grownups of sufficient means that the cost of a few thousand dollars to buy standards documents was considered not only eminently reasonable, but absolutely necessary to the functioning of our standards-making organizations. These grownups worked for telephone companies like AT&T and their PTT peers around the world, and for a few industrial concerns like IBM and Siemens.
Asking the International Organization for Standardization—asking ISO to give away technical standards would be as silly as asking the restaurant to give away the Entrecôte and the Beaujolais. In this world of many fine lunches and dinners, there was no free lunch.
In those days, I couldn't afford the Entrecôte and certainly not the Beaujolais. I was a hack, a hack in the traditional sense of the word making my living as a writer, a hack with a strong interest in the Internet, a network based on open standards, a network with no kings, a network built on the then-radical notion that standards should be based on decision by rough consensus and rule by working code.
While this loose-knit band of anarchists that were defining an Internet based on open standards were free to ignore most of the nonsense coming out of the "standards professionals" and their "Open Systems Interconnection" effort—there was one thing we couldn't ignore and that was the telephone network on which we built our unreliable, best-effort datagram service.
This telephone network was defined by the International Telecommunication Union, the ITU, in the 1988 edition of a document called the Blue Book, a 19,000 page compendium that contained the standards for things like how modems worked, how to compress audio, and the operation of Signaling System Number 7. Anybody defining Internet standards that interfaced effectively with the underlying telephone network needed the Blue Book.
But, I couldn't afford the Blue Book—it cost 2,500 Swiss Francs—and since I was making my living as a columnist for trade press publications such as Communications Week, since I couldn't write about what the Blue Book contained, I instead wrote a lot of columns about how I couldn't afford the Blue Book and why it should be free.
At the time, the Secretary-General of the ITU was a big Finn named Pekka Tarjanne, a job he got as a reward after a career in Finnish politics followed by 12 years as head of the Finnish PTT. Tarjanne had hired a lawyer named Anthony Rutkowski as his counsellor, and that was probably a mistake as Tony Rutkowski was in reality a double-agent, an Internet sympathizer who even used email.
In 1991, the ITU was not exactly a technically progressive organization. They had lots of rotary-dial telephones of course, because their founding treaty specified they got free phone calls from all the PTTs, a fact they were inordinately proud of and never ceased to point out.
But there was no email and only a single fax machine for the entire 17-story building, and this telefacsimile device was carefully secured in a deputy secretary-general's office and required a special form with many signatures before a document was considered fax-worthy.
Tony Rutkowski read my columns flaming about the ITU and he got me a meeting with Dr. Tarjanne. I flew to Geneva and soon found myself in the rather spectacular Secretary-General's suite on the top floor of the ITU tower.
After a few pleasantries about Finland—Reindeer, saunas—we got down to business. I stated my case: the Blue Book ought to be available for free on the Internet.
Dr. Tarjanne smiled the smile of a patient father and told me that in his ideal world, the Blue Book and indeed the Entrecôte and maybe even the Beaujolais would all be free, but this was of course impossible, as much as we both might share this dream of an ideal world.
You see, it really wasn't about the money, Tarjanne explained, there was a technical obstacle. The Blue Book was being produced on an ancient mainframe using an ancient program, a program so old that they had lost the source code and nobody was quite sure exactly how it worked.
They were developing a new typesetting system, but that was several years away, and for now they were stuck with only one output device, and that was their printing press.
Dr. Tarjanne was sure I could see, while he'd love to give me the source to the Blue Book, it wouldn't do me any good, even his own expert technical staff didn't know quite how this black box worked.
I suggested that perhaps we could try an experiment—and this is my first rule for radicals: call everything you do an experiment—and the experiment I proposed was that perhaps the ITU could furnish a set of tapes for the Blue Book and the Internet would try its hand at reverse-engineering the system. If we succeeded, we'd give the ITU back their Blue Book in some coherent ultra-modern format like TROFF.
Since Dr. Tarjanne knew the Internet had only a few users, none of them serious people of means who buy standards, he'd be able to shut up the critics by saying he had cooperated, but the Internet had not been up to the task.
He said we could have a set of tapes.
I went to Boulder, Colorado and enlisted the help of Mike Schwartz, a professor of computer science and the creator of the original search engine, netfind. After a few false starts, we managed to mount the tapes and read the raw data into a series of octal dumps hundreds of pages long, which we spread on the floor next to a printed copy of the Blue Book.
By comparing the octal dump to the final form, we were able to confirm that the Secretary-General was correct—this system was a total mess—but after a lot of head-scratching and a few surprises, we managed to turn their system into TROFF and then into NROFFed ASCII and PostScript, posted the tarballs on an FTP server, and sent a note to the IETF list.
The next day, the National Science Foundation called, complaining that the Blue Book release was using up all the bandwidth on the international backbones. The cross-Atlantic line was still just an E1, running at 2 megabits per second and costing the NSF $60,000 per month, and we were using all the bits.
I reassured the government program managers that this was a temporary phenomenon, and soon we had saturated our market with files spreading out to 500 hosts in 27 countries.
Then, word started to trickle back to Geneva that maybe the Internet was a bit bigger than previously thought, and soon after that a telefacsimile arrived from an ITU official, who explained that he had been instructed by the Secretary-General to convey to me a message, and that message was that our experiment was now over. The Secretary-General was insisting that we remove the Blue Book from our server. Oh, and while we were at it, remove all copies of the Blue Book from the rest of the Internet, within 20 days.
I conveyed my kind regards to the Secretary-General, and explained that while of course I would comply with respect to my own server, that vis-a-vis the rest of the world, there was not much that could be done as the proverbial cat had escaped from the proverbial bag.
This is thus my second rule for radicals, and that is when the authorities finally fire that starting gun—and do something like send you tapes—run as fast as you can, so when they get that queasy feeling in their stomach and have second thoughts, it is too late to stop.
Tarjanne the Finn was my first real bureaucrat, but the Blue Book underscored for me the importance of open standards, that if code is law then it must surely follow that law is code—and if that is the case, then the only way that makes sense to release this code has to be open source.
—Bureaucracy № 2—
For the rest of my story today, I turn from Geneva back to the United States.
In 1993, I had graduated from print to the wonderful world of multimedia, which meant mostly 8-bit GIF files. The blinky tag had not even been invented yet.
Most of us running network ops were using FTP, email, and perhaps gopher and Archie. With those tools, I was running an Internet radio station called Internet Talk Radio. The flagship program was Geek of the Week, which most people retrieved by launching an overnight FTP job and then—assuming the sound card was properly installed—listened to the sound file on their workstations.
Not everybody had FTP, and one listener used the MCI Mail FTP gateway, which broke the 30-megabyte sound files up into several hundred mail messages. When all the messages arrived, he reassembled them and curled up to his workstation for his episode of Geek of the Week.
We did a lot of "the future is here" Internet demos in those days, and after giving one in Congress, I was called aside by the staff of Congressman Edward Markey, and they showed me a letter from a Nader's Raider named Jamie Love, saying that the Securities and Exchange Commission database of public filings of corporations—known as EDGAR for the Electronic Data Gathering and Retrieval System—that this EDGAR database should be available on the Internet for free.
These EDGAR filings were used by stock brokers, economists, and analysts, and a $300 million/year industry had sprung up retailing these documents. When I was a doctoral student in economics, I learned that sometimes you could write to the corporation and ask them to send you their annual report by U.S. Mail, but often I ended up forking over $30 a document to some information retailer to read these filings electronically.
To feed this $300 million/year industry, the SEC had set up a $30 million deal with Mead Data Corporation. The theory was that these filings were indigestible raw data, so Mead would act as the information wholesaler and add "value" to the documents—and they would sell to information retailers, who would add even more "value" to the documents—and finally these documents would reach the information consumer, presumably professionals on Wall Street who knew how to read these highly technical filings full of, you know, numbers and stuff.
A very brave bureaucrat at the National Science Foundation, Dr. Steve Wolff, arranged for my nonprofit radio station to get a few hundred thousand dollars, enough so we could buy a feed of the EDGAR data. Eric Schmidt, then the CTO at Sun Microsystems, pitched in a box with four SPARC-2 processors. UUNET offered free transit, Cisco threw in a router, and MFS Datanet provided a 10 megabit fiber link to the Internet Exchange known as MAE East.
That 10-megabit link was fast enough that when the new Clinton administration took office, they asked to borrow it. Turns out the new tenants over at the White House were having trouble getting their routers cleared by the Secret Service and they wanted to do a "we know what the Internet" is event with the President, so ARPA helped us run an infra-red link from the roof of the National Press Building down to the White House lawn to get them hooked up.
About ninety days after the NSF grant came through, the server was up and running and raw EDGAR data was on the net. Remember rule 2, when the starter's pistol gets fired, run as fast as you can.
We ran the service for a year and a half, starting with FTP for tarballs, then Gopher for docs, then an HTTP server, then finally a WAIS database. By mid-1995, there were 50,000 people a day using the service. Some of those people were financial fat cats on Wall Street, but there were also students, journalists, government employees, senior citizen investment clubs, and others that were of insufficient means to afford $30 documents.
And this brings us to rule number 3, which is that eyeballs rule. Build up a user base, and you have much more leverage than if you're just blowing smoke.
Perhaps we should have used our "first-mover advantage" to—as they said in the .com days—"monetize the eyeballs." But, I didn't want to be the face of the SEC, I wanted the SEC to do their job, which was to make the EDGAR database available to the public, on the Internet, for free, and in an at least moderately clueful manner.
So I pulled the plug.
A sign appeared on the web server saying "this service will terminate in 60 days. Click here for more information."
When you clicked, you got a page with source code, usage stats, cost figures, and configurations to run the system, and a series of "click here" links if you felt that termination of the system would somehow inconvenience you.
The first "click here" link was "click here to send mail to Newt Gingrich," the hip young Speaker of the House. The next was "click here to send mail to Al Gore," the hip young Vice President. They both had email accounts and were very proud of them.
The third "click here" link was "click here to send mail to the Chairman of the SEC." Chairman Arthur Levitt, a grand old man of finance, didn't have an email address, so we created one for him. A couple of days later, the 17,000 messages he received were printed and delivered to the SEC front desk.
Coincidentally, the SEC had scheduled an EDGAR Industry Day meeting, which we weren't invited to, so we crashed it. After some theatrics, one of the Commissioners came up and asked some simple questions, like how much it would cost to run the service and who the users were.
The Commissioner evidently briefed the Chairman, because that evening Chairman Levitt called the Associated Press and the Wall Street Journal and said the SEC was going to offer this EDGAR database on the Internet. The filings weren't a product, they were the glue that make our financial markets work efficiently by requiring corporations to disclose information to the public.
The next day, the chief of staff called up and said that while he fully supported his Chairman, there was one hitch and that was that there was no way they could buy a computer in 60 days and besides, their Internet line had been installed but didn't seem to be working. Could we extend the deadline? I said the deadline was firm.
The chief of staff ended up signing a loaner agreement, we put some Sun boxes in the back of a station wagon, drove down to SEC headquarters and helped them configure their Cisco router and T1 line.
They were up and running by the deadline. The computer staff ended up tickled pink they were running the U.S. government's busiest web server and getting tons of fan mail from their adoring public.
Rule 4 is that when you achieve your objective, don't be afraid to turn on a dime and be nice. You can bang the table and be a total pain in the ass, but there comes a time to be helpful, courteous, and friendly.
—Bureaucracy № 3—
For the next bureaucracy, let's fast-forward this Wayback Machine to late 2006 and early 2007—when Google bought YouTube for $1.65 billion, when C-SPAN started allowing you to use their video of congress on your blog, the year Netflix started streaming videos—the year "you" were named Time Man of the Year. This is when video came to the Internet.
Well, not all the Internet.
Back in Washington, D.C. was an agency called the National Technical Information Service, NTIS, a government profit center tasked with, among other things, being the official retailer of videos from all across the government.
A look at the NTIS web site showed thousands of videos from 54 different federal agencies. There was all sorts of useful stuff—none of it viewable on the Internet—like training materials for volunteer firefighters from the U.S. Fire Academy.
But the prices! Ooh la la! Talk about champagne wishes and caviar dreams! An Ellis Island documentary—"Island of Hope, Island of Tears"—cost $55 for a 29-minute VHS tape.
"The Time of Apollo" from NASA? $50 for 28 minutes. I forked over $336, ordered some tapes, and posted them to YouTube and the Internet Archive. "John F. Kennedy: Years of," from the U.S. Information Agency. "Firefighter Safety and Survival" from the U.S. Fire Academy, and "Day of the Killer Tornados" from FEMA.
The nice thing about the U.S. government is pretty much anything they produce is called a "Work of the Government" and that means, at the federal level, it is public domain. There are a couple of exceptions and grey areas, but the basic rule is no copyright.
Simple enough: buy video from the government and upload it. Nobody can stop you. Simple that is, except for the cost.
But what if we spread the pain out? What if other people bought some of these tapes and donated them to the public domain? For $29.95 a month, I signed up for EBay's ProStores, one of those anybody-can-build-a-store ecommerce solutions, and built a front-end proxy on top of the NTIS store. The deal was we'd take your money, order the tape, upload it to the Internet Archive and YouTube, and you'd get a tax deduction.
In a fit of marketing, we festooned the site with slogans: "Be the last person to buy this fine video" and "Buy from us and you get nothing but everybody gets something."
And, my favorite: "Made by the government—buy in confidence knowing the source."
OK, so it was a little cheeky and perhaps even a bit silly. But, the whole business model was silly. With no intellectual property protection on the content—all of it works of the government, all already paid for by taxpayer dollars—if we had enough money we'd simply buy one copy of each video and we'd be done with it. They'd be out of business.
The store was snazzy, but there were more "lookie loos" than buyers. In fact, we got only one order for $106, and that order was actually a mistake—the guy thought we were going to send him a DVD.
One day I lost my patience and sent a rather intemperate fax to the director of the NTIS. A letter is probably not the appropriate characterization for this communique and maybe flame would be more accurate. I basically accused the entire of agency of falling down on the job.
So imagine my surprise the next day when my phone rang and the voice on the other end said "Mr. Malamud, this is Ellen Herbst. I'm director of the National Technical Information Service."
Oh-oh, I thought, here it comes.
Well, Ms. Herbst turned out to be perfectly reasonable. She wanted the video out there, but by law they were required to recover their costs and by the time you added up the people to run the service and factored in the almost nonexistent sales, well, it cost $70 to sell you a videotape.
If they had to recover their costs, what if we didn't cost them anything?
"Can you just loan us your videotapes?" I asked the Director. "You know, send us the tapes, we make a copy, we send them back to you? We can even pay the postage!"
A long pause. "Yes, I suppose we could do that."
And thus was born FedFlix, where government loans us tapes which we digitize and send back to them, with a DVD included for each videotape.
Rule 5 is pretty simple. Keep asking—keep rephrasing the question until they can say yes.
In November 2007, a couple months after that phone call—fast by government standards—we signed a joint venture agreement in which every month NTIS would send 20 tapes.
We ran that program for a year and put a couple hundred tapes on-line. At the end of the year, we renewed the agreement and upped the quantity to 100 tapes a month, and they started sending Betacam masters.
For about $10,000, we've built up a nice little studio that does professional-quality encoding of the Betacam masters, producing 8-megabit H.264 MP4 files We don't do anything fancy like color correcting or normalizing the sound, but for the content we have that just doesn't seem necessary.
FedFlix really isn't a funded project, it is something to fill in gaps in the day. Instead of writing to a Facebook wall, I choose to rip. I put an egg timer on top of the Betacam deck, and pop in a tape and set the timer. When it goes off, I put in another tape.
After a few weeks, there will be a nice collection of files, I put together a packing list with the metadata, then use a big old hairy RegEx to turn the packing list into a bunch of curl calls that use the Internet Archive S3 interface and Python scripts that use the YouTube API.
Perhaps the most subversive thing we do with this video is put the masters on our server for FTP and rsync. The hardest part of making a film or a news piece today is clearing the rights in that absurd thicket of copyright-obsessed stock footage libraries. With our multi-terabyte public domain stock footage library, you don't have to ask, and there are never any late charges in the public domain.
It sometimes amazes me what video gets popular. A World War II film called "Principles of Refrigeration" has received 78,000 views. Turns out there isn't any good HVAC material on the net. The biggest hit on YouTube is a bulldozer safety film called "Stay Calm and Stay in the Cab!" which has over a half-million views.
—Bureaucracy № 4—
One of the big challenges facing government is the deluge of paper, videotape, and other legacy formats. For agencies in the information business, such as the Library of Congress, the National Archives, and many others, the dual challenges of dealing with legacy formats and how to face a digital future have been overwhelming. In many cases, the agencies have turned to what they call public-private partnerships, so-called "no cost to the government" deals that have proven to be especially troublesome.
An example of such a no cost to the government deal was one cut by the Government Accountability Office, an arm of Congress, which has the definitive library of federal legislative history—folders for each public law that contain all the hearings, bills, and reports that led up to each statute.
GAO entered into a deal with Thomson West where the government shipped off all those federal legislative histories to the vendor, which scanned them and then sent the paper back.
Not that different than the FedFlix program, but with an important twist. Thomson West didn't send the GAO back digital copies of their data. Instead, Thomson West gave GAO a couple of logins for their staff to use the digitized material, but for everybody else, including government folks—including congressmen—everybody else has to pay to access the U.S. federal legislative histories.
The deal wasn't really no cost to the government since it took a huge amount of effort to pack these 60 million pages of paper up and send them to the vendor. The vendor got a sweetheart deal: an exclusive lock on a vitally important government database. The government got snookered.
For my next bureaucracy, I want to talk about one of those public-private partnerships, this one being a deal that the National Archives cut with Amazon. In December of 2009, I got a call from Congress asking if I could testify as part of the inaugural hearing for the new Archivist, David Ferriero.
As part of the research, I looked at the deal the Archives had cut with Amazon. This was part of Amazon's new DVD print-on-demand service, and what they had done was digitize about 1,800 government videos which they were making available for about $10 per DVD.
I've got nothing against Amazon selling DVDs, even DVDs of public domain video. But, if you went to the government site, there was only a 2-minute preview of each video, in a Microsoft proprietary format, and a 320x240 picture. Next to the 2-minute preview was a government statement saying you could buy this video from "our partner" Amazon.Com.
Rick Prelinger—creator of the Prelinger Library and the real pioneer in rescuing government video—had FOIA'd the contract behind this arrangement, and it looked like that while the National Archives got a DVD of their video back, they agreed not to post it on-line for 5 years. There was a weird arrangement where the government got some kind of royalty from Amazon, but the royalty was after they deducted "ingestion fees" for scanning the videos. The government was paying for the digitization, but wasn't allowed to use the material.
I asked the Chief of Staff of the National Archives how much these royalties they were getting were, and it turned out to be—in 2 years of operation of this partnership—a total of $3,273.66.
This seemed nuts. So, I forked over $251 and bought 20 DVDs from Amazon and posted them in all the usual places. Some great stuff, like footage of Richard Nixon in the White House, explaining why he was innocent of any wrongdoing.
Then, I wrote to Cory Doctorow at Boing Boing and he posted a note telling all the happy mutants that if they watched Richard Nixon on YouTube, they could help save the public domain because we were counting all the views to show members of Congress that people really care about this stuff.
"Watch Richard Nixon, help save the public domain."
The next day, I sent another $461 to Amazon and ordered another 28 videos, and that led to another Boing Boing post, "Watch the Bob Hope Christmas Special and Help Save the Public Domain."
By the time I testified before Congress on December 16, we were able to show more online views for these 48 videos than the total unit sales from the Amazon program over two years. The message was pretty clear: the Amazon deal had not brought the government any revenue and it had come at a substantial cost of public access.
We had our point as far as Congress was concerned, but when I went home I kept looking at those 1,800 videos and wondered if there was some way to liberate them without forking over $18,000 to Amazon. I was musing about this on Twitter and somebody at-replied back and asked if I had considered an Amazon wish list—the way you let other people buy stuff for you for your Bar Mitzvah, birthday, baby shower, or wedding?
Whoa, I thought, what a nice hack!
So, 153 of the most impressive titles went on an Amazon Wish List and Boing Boing issued a new post suggesting that if people had an extra $10.95, perhaps they could buy a Christmas gift for the public domain? (Tax deductible, no less!)
That list sold out in a matter of days, and the day before Christmas, my Amazon sent to me, 43 boxes of DVDs. I spent the holidays ripping the discs, finding metadata, and uploading files.
It was great. Footage of the Hindenburg blowing up, James Cagney narrating a cold war film called "the Wall," the Cambodian Royal Ballet, old CIA propaganda films, Disney war films, early space footage, and the Roswell Area 51 investigation.
When you criticize a government agency to their congressional oversight committee, you're probably going to get a response. So here is rule 6 for radicals, which is when you get the microphone, make sure you make your point clearly and succinctly.
Pretty soon, I got a call from the National Archives to discuss the "Amazon situation." When I said that this video was totally unavailable to the public, I had misspoke—anybody could go to the National Archives in College Park, Maryland and watch any of those 1,800 DVDs onsite. They'd also let you make a copy of a DVD, and they'd even furnish the blanks to make those copies—up to 6 copies per visit.
And, they had more than the 1,800 DVDs in question, they had over 3,000 DVDs onsite.
"You mean," I asked, "if I went out there often enough, I could copy all 3,000 of the DVDs and post them?"
"Absolutely. You bet, go for it."
Well, at 10 minutes per DVD, that's 30,000 minutes— 500 hours—more time than I could spend in College Park, but a perfect opportunity for crowd-sourcing and thus was born the International Amateur Scanning League.
I wrote to the National Archives chief of staff to give a courtesy heads-up that I was going to draft a bunch of volunteers to go out to College Park and systematically copy all their DVDs. Imagine my surprise when she wrote back and said David Ferriero thought this was such a great idea that he'd like to come to the initial meeting of volunteers and personally teach them how to rip DVDs.
Next thing we knew, we were in a meeting room at the Sunlight Foundation in the middle of a major blizzard, and the Archivist of the United States was teaching us how to rip video. We printed a bunch of red, white, and blue FedFlix return envelopes for people to send the DVDs as they finished, and created Public Domain Merit Badges for volunteers who reached certain milestones. If you copy 5 DVDs you get a John F. Kennedy Public Domain Merit Badge, at 25 discs you get the Bob Hope, and for 50 you get the Duke Ellington.
—Bureaucracy № 5—
Video is really just a hobby for me, something I do in my spare time. I run a 501(c)(3) nonprofit and we get our money in the form of grants from foundations such as the Omidyar Network and corporations such as Google and Justia. We also get contributions from private foundations such as the Elbaz, Kapor, and O'Reilly foundations.
Foundations aren't going to give you much money if your mission statement is "we upload government videotapes to YouTube."
My day job, as it were, the stuff we're paid to do in the form of grants and contributions, is to help change our legal system by making the law more freely available.
You'll remember that with government video, at the federal level, there is no copyright in works of government. This principle that there is no copyright is even more sacred for a protected core—the law. The principle that we're a nation of laws not a nation of men means that we write down the rules that citizens must obey. How can we be a nation of laws if those rules are not open source?
Despite this principle, access to legal materials in the United States is a $10 billion per year business. Often, government will erect barriers to access as a way of extracting rent from the public.
This is particularly true for a database run by the Administrative Office of the U.S. Courts, a database called PACER, which stands for Public Access to Court Electronic Records. PACER contains 500 million pages of the proceedings of the U.S. district courts, including the dockets, briefs, motions, and opinions of every U.S. federal case.
The courts charge 8 cents per page and require a valid credit card to access PACER. A prisoner or other citizen can petition a judge for free access. But, petitioning a federal judge isn't exactly a low barrier to entry.
This is a big business for the courts: they drag in $120 million a year in revenue. The courts even charge the executive branch of the federal government millions a year to access filings!
To poke a few fingers in the eyes of the Administrative Office, we put up a recycling site, which let people upload their PDF documents from PACER—where we'd recycle them into the public domain.
Since PACER is a half-billion-page database, it was really kind of a bluff, a vehicle for an FAQ that tried to expose the finances behind PACER. But one of the things in the FAQ caught the attention of a couple of volunteers.
You see, the Courts, under strong congressional pressure to do something about public access, had just launched a trial program, putting one terminal in each of 17 libraries around the country. In the FAQ for the PACER recycling site, I encouraged volunteers to join the so-called Thumb Drive Corps and download docs from the public access libraries and upload them to the PACER recycling site.
Aaron Swartz, whom many of you may know as the editor of the RSS spec and a prolific contributor to the Internet, called up and said he'd like to join the Thumb Drive Corps. I told him to be careful, knowing he was technically astute and inclined to script things pretty, um, aggressively. I warned him to make sure he didn't violate any of the guidelines the courts had set: if they said don't download too many docs, don't download too many docs.
A few weeks later I got email saying he had some data, could he maybe get an account to upload his docs directly? Sure, no problem, we let him SSH in, and data started to come in, and come in, and come in, and soon there were 760 gigabytes of PACER docs, about 20 million pages.
Aaron evidently had super-sized his Thumb Drive, but he's a bright guy, so we weren't totally surprised.
Then, the stream abruptly stopped and I got email from Aaron saying we needed to talk. Right away.
The Administrative Office had evidently finally looked at their usage logs after two months—and then abruptly cancelled the public access program overnight, saying a security breach had occurred. The Superintendent of Documents at the Government Printing Office gave a speech and said that not only had a security breach occurred, the FBI had been called in to investigate.
Aaron and I talked again, and after grilling him, I was still convinced we had done nothing wrong. There were no signs or appropriate use statements saying this was intended for casual use only. I'll grant you that 20 million pages had perhaps exceeded the expectations of the people running the pilot public access project, but surprising a bureaucrat isn't illegal.
From previous experience putting Court of Appeals decisions on-line, I was pretty sure this PACER data was going to be a mess. Rather than release the data on the net, I started an audit looking for privacy violations.
For the next two months, a series of scripts ran that looked for personal identifiers. Any files with a hit were manually examined. Many of them were false positives, such as government contract numbers.
But, there were also a whole bunch of files that did have problems, and for each of those I looked around for things the regex didn't catch and ended up finding even more Social Security numbers, and other illegal data like the names of minors and bank account numbers.
There was the obvious stuff, like the IRS suing a citizen and forgetting to redact their Social Security number on tax returns filed as evidence. Or, redacting the number by placing a black rectangle on top of the text or turning the color of the text to white.
There was also some really heart-wrenching stuff, like a list of 350 patients of a doctor who was being sued for malpractice. For each patient, the supporting document listed their home address, birth date, Social Security number, and a list of all their medical problems. Or the list of the members of labor unions involved in pension disputes, with their personal identifying information, home address, and earnings history.
After completing that analysis, we sent a formal audit over to the Administrative Office of the Courts with a carbon copy to the judge who chairs the Judicial Conference Rules Committee. In addition to a printed list, they got a DVD that let them compare the redacted to the un-redacted version of 2,000 offending documents.
You'd think this was pretty shocking evidence, but the Administrative Office of the Courts ignored the preliminary audit, then ignored the final audit, then continued to ignore us. Finally, over the Christmas holidays in 2008, letters went to the Chief Judges of 30 district courts.
On the top of those letters—in big red type—were the words "Third and Final Notice." The letters said we had sent a preliminary audit to the Administrative Office and a final audit to the Administrative Office, and of course, these letters said, it goes without saying that the Administrative Office had promptly notified the judges of these very serious problems, since of course they didn't want to be breaking the law.
Needless to say, the judges hadn't heard about this situation, and you have to swallow real hard before you send Chief Judges of U.S. District Courts letters saying "Third and Final Notice," but you know what ... judges are reasonable people. They got these letters, their clerks checked them out, and we started getting letters back saying in effect "you're right, thanks, we'll take steps."
As a result of those audits, the Senate sent a strongly worded letter to the Administrative Office asking them why they weren't obeying the law. The judge who chairs the Rules Committee wrote back to the Senate saying that while they had privacy rules in place, they were obviously not going far enough and they would change their rules. And, a few months later, they changed their rules.
Here is my seventh rule for radicals, which is to get standing. One can criticize government all one wants, and they'll often ignore you. But, if there is something clearly wrong and against the law and you can document that malfeasance and wrongdoing, they have to talk to you. If you have standing, you can insist.
—Bureaucracy № 6—
There is a related rule, and that is rule 8, which is to try to get the bureaucrats to threaten you. Remember how the law has a special place when it comes to copyright? While a state government might be able to assert copyright over some things, the Supreme Court has repeatedly ruled that nobody can copyright the law. This means no copyright on court opinions, but it also means no copyright on state statutes.
So, you can imagine our surprise when the Oregon Legislative Counsel, the lawyers for the legislature of Oregon, sent a takedown notice to Public.Resource.Org and to Justia, a company that has been instrumental in putting free law on-line. The state said that by making tarballs of the 2007 Oregon Revised Statutes available for anonymous FTP, we had violated their copyright.
Why would the Oregon legislature insist on copyright? Money! They sold a print edition and they made money on that print edition. We were threatening their revenue stream.
Now, to be totally fair, the policy in question had been put in place in the mid 1940s and nobody had ever questioned that policy. The takedown notice was the action of a bureaucrat just doing what they'd been doing for 70 years.
Once you have a takedown notice, particularly from a body as eminent as the lawyer for the Oregon legislature, you are in legal peril. You have a right to think they're going to sue you, because that's what the takedown notice says.
If you're in legal peril, you can go to a judge and ask for what's called declaratory relief, asking the court to rule on the issue. So, we hired a lawyer and put together a draft declaratory relief request and posted it on the net.
The thing about a state sending you a takedown notice for putting the law on the Internet is that this is not one of the subtle legal issues that you have to carefully explain to people. Everybody gets this—you can walk into any bar and explain what's going on and everybody will instantly get the issue and say "that's really stupid."
And that's my rule 9 for radicals—look for over-reaching, something that is clearly nuts. Not being able to publish state statutes certainly qualified on that count.
A few days after we threatened to sue, we got a notice saying the Oregon Legislature had scheduled hearings on this issue and would we be prepared to testify? Of course! If they were willing to talk, we certainly were.
We testified, as did some Oregon citizens. The lawyer for the legislature gave his testimony, and I was impressed by how well informed and willing to look at the issues everybody was.
After a lot of questions, the Legislative Counsel Committee, which was chaired by the President of the Senate and the Speaker of the House, voted unanimously to waive any assertions of copyright. It was democracy in action, and way quicker than a law suit.
To prove the point about why this was so important to do, a few months later, a second-year law student at Lewis & Clark took the statutes and created OregonLaws.Org, a dramatically better version of the Oregon statutes featuring a great UI, valid HTML, permaURLs, an iPhone app, tag clouds, a twitter feed, and loads of other bells and whistles.
When a state asserts copyright over legal materials, it is important to remember that while this is partly about democracy and justice, it is also about innovation. By requiring a license as a precondition to access primary legal materials, we create a barrier to innovation.
—Bureaucracy № 7—
I'd like to end this tale with a bureaucracy that is a bit amorphous, a little hard to visualize and thus an exceedingly difficult target and that bureaucracy is all the lawyers in the United States of America. When it comes to bureaucracies, the bar truly is the borg.
The principle that access to the law must be unfettered is a basic foundation of our system of justice. The U.S. constitution says that "equal protection under the law" may not be denied. Equal protection means that your basic rights cannot be arbitrarily denied because you are poor, are of a certain religion or race, or because somebody disagrees with your political views.
A poll tax, which preconditions access to the polls on access to money, is wrong because it denies equal protection under the law. I put it to you that just as a poll tax is wrong, preconditioning access to primary legal materials on having a credit card is just as wrong, and it violates our rights to equal protection under the law.
Turning primary legal materials from public property into private parcels violates more than equal protection, it makes due process under the law impossible, when rich lawyers can do more research than poor lawyers.
By poor lawyers, I mean public interest lawyers and solo practitioners. I also mean government lawyers in places like the Department of Justice, who, believe it or not, get memos telling them to please stop doing so much research because the department is over budget.
Going back to the 1824 decision in Wheaton v. Peters, one of the landmark cases of the great Marshall Court, the Supreme Court, has been clear, over and over again, that there is no copyright over primary legal materials, be they court opinions or administrative regulations or state statutes or OSHA regulations or even building codes drafted by third parties but duly enacted as the law of the land.
Despite this clear public policy, states and municipalities have erected a thicket of copyright restrictions, pay walls, and click-through contracts around the raw materials of our democracy.
How do you change something so basic, so fundamental as access to the law?
This year, people involved in the free law movement have been gathering together under the banner of the Law.Gov initiative, an effort to try and convince policy makers from water districts to the President and Chief Justice that access to primary legal materials matters.
Our strategy to get this basic principle—that access to primary legal materials must be unfettered and reliable, must be available in bulk, and cannot be subject to pay walls or copyright restrictions—has started with a national conversation, a series of working groups and workshops held in many of the top law schools in the country.
We are in the middle of this process right now, in fact have just completed our workshop at Duke Law School's Center for the Public Domain, a workshop that featured not only well known legal scholars, but had the active participation of the Archivist of the United States and Andrew McLaughlin, who is Deputy Chief Technology Officer in the Executive Office of the President.
We will be completing this process of workshops and working groups in June, and will issue a set of recommendations to government officials. The Ninth Circuit of the Court of Appeals has granted us time to brief the judges, the U.S. Senate has asked for a copy of the report, and I've been very impressed at the number of top administration officials, members of Congress, and Chief Judges in the Judicial Conference who have taken a keen interest in these proceedings.
—10 Rules for Radicals—
In this talk, I've tried to present to you some rules for radicals, some techniques that I use in my work, but techniques that perhaps might be useful in your own efforts to change how institutions function. We've covered nine of those rules so far. Let me recap.
Rule 1: Call everything an experiment.
Rule 2: When the starting gun goes off, run really fast. As a small player, the elephant can step on you, but you can outrun the elephant.
Rule 3: Eyeballs rule. If a million people use your service, and on the Internet you can do that, you've got a lot more credibility than if you're just issuing position papers and flaming the man.
Rule 4: When the time comes, be nice.
Rule 5: Keep asking until they say yes. Gordon Bell, the inventor of the VAX, once said that you should keep your vision, but modify your plan.
Rule 6: When you get the microphone, get to the point. Be clear about what you want.
Rule 7: Get standing. Have some skin in the game, some reason you're at the table.
Rule 8: Get them to threaten you.
Rule 9: Look for overreaching, things that are just blatantly, obviously wrong or silly.
And finally, rule 10, which is don't be afraid to fail. It took Thomas Edison 10,000 times before he got the lightbulb right, and when he was asked about those failures, he said "I have not failed, I've just found 10,000 ways that won't work."
Fail. Fail often. And don't forget, you can question authority.
Thank you very much.