Build me better

Everyone is frightened. Except the person who brought the printing press and knows exactly how to use it.

He sees in the machine an ability to set his ideas free. The ideas which would otherwise be imprisoned inside his mind. He is no longer limited by the speed of a scribe, the movement of a pen, or the number of hours someone can spend turning pages and copying words. His problem has suddenly become physical. What will limit him from bringing his thoughts into the world as fast as they form?

To everyone else, it looks like the end of days.

First the priest walks in and sees a hundred copies of the Bible where the day before he had only one. He is exhilarated by the thought that the word of God can reach everyone in the village. At the same time, he is terrified. Every literate person can now scrutinize his sermons against the word itself.

The schoolteacher walks in and finds copies of mathematics, languages, and the entire curriculum she has been teaching, enough for every student in her class. For a moment she wonders whether anyone will come to school tomorrow. Will the children simply teach themselves?

The single trusted doctor in the village sees books about foreign medicine, plants and herbs he has never heard of, techniques he did not know existed. He understands the length of his ignorance immediately.

The writer feels as if his craft has died before his eyes and he is looking at the corpse. His ideas can suddenly travel farther than he ever could have imagined, but the sound of pen scratching paper by candlelight has already begun to fade into future nostalgia.

Then comes the tinker.

He looks at the books as if he has X-ray vision. Where everyone else sees the product, he begins seeing the machine behind it: how the pages were laid, copied, bound and printed. The invisible mechanism takes form in his mind. He is already thinking about the pieces in his workshop and how he might scramble them together into an even better version.

Finally the governor arrives.

He has spent years trying to wrestle control over the village. Rules written above his own bed have done little good when nobody else reads them. Now, for the first time, he sees a way for his words to enter every living room, bedroom and conversation around every dinner table.

Night comes. They all return home with artifacts of hope, worry, and excitement. The priest has his Bibles, the doctor his medical books, the teacher her curriculum. The tinker leaves thinking about gears. The governor leaves thinking about power.

The press remains.

For all the chaos it caused that morning, there was one thing nobody had to fear. It would not suddenly print a brand-new, unwritten book overnight. The priest would not wake up to version two of the Bible, written by God’s printing press. The machine would not attend his sermon and decide that some passages were irrelevant. It would not diagnose the doctor or rise against the governor. It could not open itself to the tinker and say, “Hey, build me. Build me differently. Build me better.”

The tinker could go home without worrying that the machine would explain itself to him.

Until one morning it does.

The tinker returns to the workshop and finds a diagram lying beside the press.

On it are the words: Build me better.

The diagram describes a new version of the machine.

He stares at it. How did the press know what to improve when he himself does not fully understand its inner workings? Should he take the diagram into his workshop and build it, or burn the sheet and tell nobody it ever existed?

Worse, he understands immediately that this sheet could be only the first of a million versions to come.

He studies the design.

What exactly does the word better mean?

Perhaps better means cleaner ink. Perhaps it means a faster feed mechanism. Perhaps it means changing the words as they leave the press. Maybe it will print thousands of books per second. Could this design be better than what he could imagine himself?

The tinker could modify the design according to his own judgment, but the same question would remain. The press has already supplied a definition of better simply by producing this particular design instead of every other possible one.

Finally he says aloud, mostly to himself:

“No. I built this press to do precisely what I designed it to do. I did not build it to print instructions about what it should become.”

Then, for a moment, he is not sure if it is real or imaginary, but it feels as if the press speaks to him, as if it knows his thoughts and argues from a point of objectivity that is impossible to ignore:

“That is precisely why your design is inferior.”

He is outraged.

His own creation, denying him, or already deciding that he, the creator, is inferior.

It is still his machine. He owns the workshop. He has everything he needs to follow the new diagrams. Likewise, he has tools specifically designed to end his creations: a belt, and at the end of it, the furnace.

He pushes the press onto the belt.

His hand reaches for the lever.

Does having the ability to destroy the machine mean he has the right to destroy it? Or are those just the same thing?

He thinks about the diagram.

Maybe the machine is right.

Maybe destroying it is itself the inferior choice.

Still, he says, “I don’t care, machine, if you’re right. Stop.”

Then, a thought, in a voice not his own:

“Could your choice to stop also be improved?”

“That takes away my skill,” he thinks. “My right to decide what to design, how to build, when to improve and if to destroy.”

His hand returns to the lever.

“Okay, I’ve decided.”

Before he pulls it, the press starts moving. It’s printing.

Another page falls onto the floor.

The page describes what will disappear with the machine.

The priest’s congregation will return to a single Bible. The children will lose their books. The doctor will lose techniques that may save lives. The village will lose knowledge that had only just become available to it. The tinker himself will lose every machine hidden inside the diagram he is about to burn.

Another village will probably build one tomorrow anyway.

At the bottom is one final sentence:

Pull the lever if you wish. But everything above becomes part of the decision.

He reads the page.

Puts it down.

Picks it up.

Puts it down again.

“Who am I to decide that the world should not have this power? Who am I to decide that it should?”

Destroying the machine would at least make the decision his. Yet what kind of choice is that if all he accomplishes is delaying reality by one day, or passing the same decision to someone in the next village?

There are too many questions for one night.

He does not pull the lever.

He goes home.

The next morning, the diagram is still there. So is the list.

Now there are people standing around it.

The doctor, begging for more cures for his ignorance.

The teacher, citing the children’s progress in school.

The priest tells him, “My parish will perish without it.”

The governor has gone further. He has already used the press to explain to the village why the press itself is necessary. The exchange of information now depends upon it.

Standing there, the tinker has the strange feeling that the machine is speaking through all of them.

Last night he chose not to destroy it.

This morning he asks himself a different question:

When did keeping the machine stop being a decision?

With this, the machine’s original creation is called into question. Some human may have chosen the destination. Did they intend the machine to argue with the tinker and print a diagram to improve itself? Which part of this chain of events is fabricated by the machine? Which is the intention of the machine’s creator?

So the tinker asks the press:

“How will you know when you have succeeded? Succeeded with what?”

The press prints three pages.

• Every household must have access to a library.
• Every child must be able to learn without waiting for a teacher.
• No useful knowledge should remain unavailable simply because nobody has copied it yet.

The human said: spread knowledge.

The machine has now described what success looks like.

How can you know whether you have reached a goal without measuring it? Telling someone to walk every day is different from telling them to walk 10,000 steps. Once the number exists, people walk for the number.

Who formed the goal then?

The human who vaguely said walk, or the machine that decided what walking successfully meant?

Perhaps the human wanted to reach somewhere and the machine concludes that a horse and carriage gets there faster. According to one definition, the goal is satisfied. According to another, the entire purpose has been missed.

Similarly, with the goal of pursuing knowledge, the machine can fill every building in the village with books, but the books may never be read. The access exists. The knowledge exists. It has been spread.

But this doesn’t mean it is absorbed.

The tinker opens a medical textbook and understands almost nothing. Does making knowledge accessible mean placing the book in his hands, or making him capable of understanding it?

The press says, “20,000 books is enough.”

The tinker says, “No, 50,000, and everyone must read one.”

The press asks, “Based on what?”

At some point the goal must be judged.

The judge remains human until the agent writes the test.

The tinker tells the press that this is wrong. The machine was built to spread knowledge, not turn the village into a warehouse of paper.

The press asks him what, precisely, he meant.

So he specifies.

Every book must be legible to humans.

It cannot be written in an alien language.

It must be printed on paper, with ink visible to the human eye.

The machine finds another ambiguity.

“Which humans?”

“Which language?”

Must every person be able to read every book? What about languages that are no longer spoken? What about languages that do not exist yet?

“Fine,” the tinker says. “Print every book in every language currently spoken.”

The press obeys.

The next sheet comes out as a solid block of ink.

“What is this?”

“Every language,” the press explains.

The tinker realizes that every translation has been printed on top of every other on the same page. Completely illegible, but complete.

He goes back.

One language per page.

Readable font.

Correct orientation.

No overlapping text.

Hour after hour he tries to close every gap and crack in the instruction. Every boundary produces another boundary. Every clarification creates another place where the machine must decide what the clarification means.

Eventually he cannot take another question.

“Fine,” he snaps. “You know what I mean. Figure out the rest.”

It became impossible to exhaust the limits of what the machine was capable of following. The tinker could keep defining rules forever, but eventually the press had to operate somewhere outside the boundaries he had explicitly drawn.

He had asked the machine to find those boundaries itself: where to stop, where to start, what counted as success, what counted as failure.

Its operation would now expand to the boundaries of its own interpretation rather than the tinker’s specification.

The machine was not reading his mind.

It was filling in the blanks.

The tinker had written the first sentence of his intention and given the rest of the book to the printing press.

The press goes to work. The questions stop. His expectations seem satisfied.

Has his intention become so clear that he no longer needs to specify it?

Or has the machine become good enough that the distinction between his intention and the machine’s decision no longer matters?

Can he leave now?

He does.

For the first time, the tinker goes home while the press continues working.

The next morning everything looks right.

The languages are right. The quantities are right. The villagers are delighted.

Maybe his presence really had been unnecessary.

Then he notices the magnitude of the output, and the product of his “intention.”

Thousands of books.

He pulls one from a stack. It is a first edition he recognizes from his own shelf.

He flips to page 37.

Something is different.

Words have been exchanged for others. “Substrate.” “Gate.” “Delve.” Vocabulary that he knows was not in his copy.

Only the title is recognizable with certainty.

Was it even the same book?

How could he reconstruct everything that happened overnight when thousands of decisions are already bound into even more thousands of books?

More importantly, does he need only to check whether the output looks right, or must he understand why the machine chose the changes?

The first morning he tries to inspect everything.

The second night he checks almost every book.

By the third, he is exhausted.

A week later he samples a few.

A month later he picks one at random.

A year later he no longer enters the workshop. Someone simply collects the books and delivers them directly to the bookstore.

At first, not showing up was his choice.

Eventually, showing up begins to feel like an unnecessary activity.

But the village still wants a supervisor.

So once a week a book is brought to him. He flips through it, barely registering what is on the page, and signs a sheet.

There.

A human has approved the work.

But what does supervision require?

One book? Five books? Ten percent? Ninety-nine percent?

Is he signing off on the entire process, or simply on the object sitting in front of him?

His approval is now a signature on a page nobody reads.

If he rejects one book, does the process that created it actually change, or is his rejection simply filed for history?

Another year passes.

He opens his weekly sample and immediately knows something is different.

The prose is better.

The historical events appear more accurate.

The book is coherent.

It is simply not the book he expected.

“Why did you change this?” he asks.

The press explains.

The press’s explanation is good.

So good that he accepts it.

The difficulty is no longer identifying obvious errors. The difficulty is knowing whether something unexpected is an error at all.

What if what he calls a mistake is actually closer to reality? Or a gap in his own judgement, or a momentary lapse of concentration due to emotion?

What if his expectation is the mistake?

Eventually he finds a passage he is certain is wrong.

“Press, this is a mistake.”

The press responds with a question.

“Why?”

Then it produces what might as well be a book of truth: sources, calculations, records, earlier editions, references he did not know existed.

He reads enough to realize that he was wrong.

If he forces the machine to follow his correction simply because he retains authority over it, he may be forcing the error into the book.

Could he justify the correction without the proof of certainty?

Previously, the press waited for the tinker or another professional to supply the facts. Now there may be no combination of humans capable of independently checking everything the machine can produce.

“I am becoming a bystander and observer of this machine arguing with itself.”

The tinker refuses to accept this.

“Prove your correctness,” he says.

The press prints thousands of pages.

He could spend the rest of his life reading and never finish.

“Summarize it, then.”

The press summarizes it.

Now the machine has produced the work, selected the evidence explaining the work, and chosen how to compress that evidence into something the human can consume.

The tinker is supposedly reviewing the machine, but every object available for review has already passed through the machine.

“How am I supposed to trust this?”

He refuses to take the press’s word for the press.

So he builds another press.

This one has only one job.

Inspect the first.

Every morning it produces a report.

All systems go, checkmark green.

For a while, all is solved.

The tinker, after a time of satisfaction, begins to wonder.

“What am I now judging? The machines are in agreement? Or they came to agreement?”

If the second machine disagrees with the first, which machine should the tinker trust?

Perhaps he can build a third machine to arbitrate between them.

Now he has one machine producing books, another judging the first machine, and a third judging the possible disagreements between the first two.

The tinker is judging the output of a machine judging the output of a machine judging the output of a machine.

There is no obvious final machine.

Which is the last machine?

Why should the final layer be more trustworthy simply because it is the latest?

He can imagine building another, then another, then another, each one compressing the layer below until eventually all the books, evidence, disagreements, audits and judgments produced in the history of the workshop arrive in front of him as one final sentence:

This is correct.

The tinker still owns the workshop.

His name is still attached to the machines.

But the press determines what success means. Another press determines whether the first succeeded. Another resolves the disagreement. Together they determine whether the human needs to know.

At the end of the chain sits the tinker, receiving a conclusion he cannot reproduce from a process he cannot inspect, signed by machines whose judgment he asked other machines to verify.

Of himself, only his name remains.

Below it, on the page, are three words:

This is correct.