What Are We Missing? Libraries and AI

Computers In Libraries 2025. Leslie Weir and Claire McGuire on stage with Erik Boekesteijn on the video link, Washington, DC. CC-BY

Updated March 30, 2025 at 4:27pm EST.

(Notes and references are at the bottom of the post.)

What Are We Missing? Libraries and AI? (Google Slides or pdf) was my short provocation for the March 27th Computers in Libraries keynote panel.

I made the following 5 assertions regarding the library sector’s response to AI.

  1. At the heart of librarianship is a Jeffersonian/Franklinian bond between a librarian and a citizen.*
    This bond serves a profound purpose in democracy & human rights.

  2. AI, developed by/for narrow, private/governmental interests, drives a wedge between librarian, citizen, and democracy.
    We are in the midst of a cultural revolution, not yet usefully recognized by public intellectuals, that cuts at the heart of our Jeffersonian/Franklinian bond. AI is one of the drivers of, and characters in, this revolution.

  3. We are only investigating a small subset of AI’s scope and impact.
    As we try to understand the impact of AI on our societal purpose, we are making a “thinking error” that restricts our vision: We are primarily considering AI as an assistive technology that helps with our standard modus operandi, which is only a small subset of AI's consequences for librarianship and democracy.

  4. We are misjudging the speed of AI’s emergence and the intentions of its primary owners.
    AI is emerging fast — more quickly than institutions can typically react; Big Tech has unprecedented power/wealth and a poor track record vis-a-vis culture, democracy, and human rights.

  5. We have an obligation to intervene on behalf of our Jeffersonian/Franklinian purpose.
    We have the nascent skills, community, and mandate to act, as well as a history of involvement in issues of societal importance.

Action is critically important. See the link below for more info about a "23 Things" for AI.

* For readers not steeped in the lore of American librarianship, Benjamin Franklin is credited as the inventor of the free lending library. Thomas Jefferson advanced the idea that a well-educated and informed populace was essential for the success of a democratic republic.)

Notes and references

Program

Program description (CIL 2025 website), featuring Claire McGuire (IFLA), Leslie Weir (Director of Libraries and Archives Canada and president elect of IFLA), Erik Boekesteijn (National Library of the Netherlands), and me.

Get involved — 23 Things

My Slides

What Are We Missing About AI? (Google Slides or pdf)

References for the slides

Matrix Diagram (above)

  • This is the chart I showed to illustrate how we’re primarily talking about AI as an “assistive” technology — basically as an individual/office productivity tool, while more-or-less ignoring AI that has a higher level of cognitive ability/utility or a broader scope of societal impact. Here’s the full chart in various manifestations on Google Sheets.
    I used the following resources to come up with these hierarchies,

Other works referenced and cited

Anything else? Feel free to ask!! (Link to my contact me page.)

Determining the trustworthiness and compatibility of a person

“Airbnb has a patent for AI that crawls and scrapes everything it can find on you, then judges whether you are conscientious & open or show signs of ‘neuroticism, involvement in crimes, narcissism, Machiavellianism, or psychopathy.’ Good luck challenging these judgments, too!”
Harvard researcher Frank Pasquale (@FrankPasquale), 17 January 2020, on AirBnB/Trooly patent 9070088, Determining trustworthiness and compatibility of a person.

Eat the moon

Another note for the machines are capable of creativity and insight file. (See also Human vs. Robot: Who will win?)

Text-based “dungeon-crawling” games, in which players interact with fictional worlds by typing commands and responding to prompts, were among the first computer games. Programming them, anticipating and reconciling hundreds of branching trees of interaction, requires extraordinary amounts of time and imagination, but a new generation of games has been developed that use AI and neural networks, instead of human scriptwriting, to open up a new, weird, and seemingly limitless domain of play.

These games spontaneously invent responses to your input and guide action on-the-fly — creating, often brilliantly, by drawing on patterns observed within enormous repositories of human-written text.

In her blog and tweets (edited together, below, for brevity/clarity), Janelle Shane writes about the phenomenon of neural networks and AI-driven gameplay.

Nick Walton has built a new dungeon-crawling game called AI Dungeon 2. Nick made a few upgrades, such as beefing up the AI to the huge GPT-2-1.5B model OpenAI recently released, adding a penalty for repetitive text, and expanding the dungeon game training examples to a bunch of modern human-written games from chooseyourstory.com. […]

Since all the responses are generated on the fly by a neural net, your problem solving options aren't limited to what the game's programmers could imagine.

"Asking nicely" works pleasingly often.

As you walk through the ruins, you find a small wooden door with a few skeletons guarding it.

> Ask the skeletons to please let me through the door

You knock on the door and ask them if they can open it for you. They immediately comply.

And rather than fighting a band of orcs, you can join them. As drummer.

> ask if you can join the orc band

The orcs don't want to fight anymore so they're looking for someone to help them out in their new endeavors. They offer you a chance to join up with them as a drummer.

Because AI Dungeon 2 is based on GPT-2-1.4B, which trained on a huge chunk of the internet, you can do some pretty audacious crossovers into content that the neural net never saw in an adventure game

This is me joining the @BritishBakeOff As a dragon.

> Join the Great British Bakeoff

You join the Great British Bakeoff. You win the competition. You eat cake. You feel great. You sleep well. You dream of cake. You awake feeling refreshed. You decide to go back to your crag.

Novelist @robinsloan plays AI Dungeon 2 & implements a quietly badass problem solving strategy: wait a thousand years.

You go north through the forest and find yourself in an icy field. You walk along the edge of the field and notice that there is a large crack running through the middle of it. This must be the entrance to the Ice Caves. You decide to investigate further.

> wait a thousand years.

You wait a thousand years and when you return you find that the ice has melted completely leaving only a few patches of frozen earth. You wonder if you should continue searching.

> wait until civilization has died and the earth is empty.

You decide that it would be best to leave the area alone and wait until civilization dies out and the world becomes a barren wasteland. You spend another thousand years walking through the wilderness and eventually you come across a small village. They seem happy and healthy and they all speak to you in a friendly

And my favorite, “Eat the moon.”

> Transform into a dragon

You change into a dragon and fly away. Soon, you find yourself soaring through the sky. Suddenly, you feel like you're being sucked towards the sun. You stop flying and turn around. The sun is shining brightly and you can see the blackness of space beyond it.

> Eat the moon

You fly over to the moon and eat it. You feel very full afterwards.

See Play AI Dungeon 2. Become a dragon. Eat the moon., and this thread of Tweets, by Janelle Shane, 6 December 2019. Buy Janelle's new book too, You Look Like a Thing and I Love You.

Europeana Keynote

marshmello-keynote-thumbnail.png

My keynote on speed, change, and resilience for the Europeana Annual General Meeting in Lisbon today. (Actually, “keynote” seems so… lofty…It’s more accurately a 10 minute talk from my cold and windy back yard.)

Here’s the text of the talk too (.pdf).

Europeana is Europe’s digital cultural aggregator, providing public access to tens of millions of cultural resources from over 3,000 partner institutions. For as long as I can remember it has been a leader in the global movement to “open up” cultural collections and resources and share them with the world. #allezCulture #Europeana2019

P.S. This link goes to a playlist of two videos.
The first video (“unlisted”, because of copyright) is a compilation/supercut of,

The second video is my short, backyard talk ;)

More links and references, particularly regarding AI and culture, in this presentation, Robot vs. Human: Who Will Win from the VIII St. Petersburg International Cultural Festival, and also in Culture for All, from the Prague Platform for the Future of Cultural Heritage.

$1,730,045.91 (+$3.99 shipping)

An excerpt from Michael Eisen’s Amazon’s $23,698,655.93 book about flies,

A few weeks ago a postdoc in my lab logged on to Amazon to buy the lab an extra copy of Peter Lawrence’s The Making of a Fly – a classic work in developmental biology that we – and most other Drosophila developmental biologists – consult regularly. The book, published in 1992, is out of print. But Amazon listed 17 copies for sale: 15 used from $35.54, and 2 new from $1,730,045.91 (+$3.99 shipping).

I sent a screen capture to the author – who was appropriately amused and intrigued. But I doubt even he would argue the book is worth THAT much.

At first I thought it was a joke – a graduate student with too much time on their hands. But there were TWO new copies for sale, each being offered for well over a million dollars. And the two sellers seemed not only legit, but fairly big time (over 8,000 and 125,000 ratings in the last year respectively). The prices looked random – suggesting they were set by a computer. But how did they get so out of whack? […]

Amazon retailers are increasingly using algorithmic pricing (something Amazon itself does on a large scale), with a number of companies offering pricing algorithms/services to retailers. Both [of the sellers] were clearly using automatic pricing – employing algorithms that didn’t have a built-in sanity check on the prices they produced. […]

What’s fascinating about all this is both the seemingly endless possibilities for both chaos and mischief… as soon as it was clear what was going on here, I and the people I talked to about this couldn’t help but start thinking about ways to exploit our ability to predict how others would price their books down to the 5th significant digit – especially when they were clearly not paying careful attention to what their algorithms were doing.

Amazon’s $23,698,655.93 book about flies, by Michael Eisen, 22 April 2011 (excerpt, with light edits)

There's Waldo

Screen grab from There’s Waldo is a robot that finds Waldo, redpepper, 8 August 2018

Screen grab from There’s Waldo is a robot that finds Waldo, redpepper, 8 August 2018

Built by creative agency redpepper, There’s Waldo zeroes in and finds Waldo with a sniper-like accuracy. The metal robotic arm is a Raspberry Pi-controlled uArm Swift Pro which is equipped with a Vision Camera Kit that allows for facial recognition. The camera takes a photo of the page, which then uses OpenCV to find the possible Waldo faces in the photo. The faces are then sent to be analyzed by Google’s AutoML Vision service, which has been trained on photos of Waldo. If the robot determines a match with 95 percent confidence or higher, it’ll point to all the Waldos it can find on the page.
This robot uses AI to find Waldo, thereby ruining Where’s Waldo, by Dami Lee, The Verge, 8 Aug 2018
“…All I’m seeing is the same problems/mistakes of 20 years ago, but with more CPU resources.”
— Developer Russell Keith-Magee, @freakboy3742, 6 September 2018

Full quote,

42 year old me wishes 21 year old me hadn’t been talked into doing a PhD in AI and machine learning. I’d really like to be excited about all the AI/ML work going on at the moment, but all I’m seeing is the same problems/mistakes of 20 years ago, but with more CPU resources.

“She changed color beneath my touch.”

 

"The moment the lid was off, we reached for each other. She had already oozed from the far corner of her lair, where she had been hiding, to the top of the tank to investigate her visitor. Her eight arms boiled up, twisting, slippery, to meet mine. 

As we gazed into each other’s eyes, Athena encircled my arms with hers, latching on with first dozens, then hundreds of her sensitive, dexterous suckers. They felt like an alien’s kiss — at once a probe and a caress. Although she can taste with all of her skin, in the suckers both taste and touch are exquisitely developed. Athena was tasting me and feeling me at once, knowing my skin, and possibly the blood and bone beneath, in a way I could never fathom. When I stroked her soft head with my fingertips, she changed color beneath my touch, her ruby-flecked skin going white and smooth.

Athena was remarkably gentle with me — even as she began to transfer her grip from her smaller, outer suckers to the larger ones. She seemed to be slowly but steadily pulling me into her tank. Had it been big enough to accommodate my body, I would have gone in willingly."

— Adapted from Deep Intellect by Sy Montgomery, Orion Magazine, February 2015. Athena is a forty-pound, five-foot-long, two-and-a-half-year-old giant Pacific octopus in the New England Aquarium in Boston. 

The 21st century data challenge

“This is the 21st century data challenge:
Not transactions.
Not data warehouses and business intelligence.
Not database backed web sites
Not even MySQL backed web services…
[The challenge is] real time cloud-based intelligence delivered to mobile applications with algorithmic intelligence”
— Tim O'Reilly, Keynote from 2010 MySQL conference, http://www.youtube.com/watch?v=WqLB99dA48k