From The Lab

We’re back!

I can’t believe it’s been nearly two years since we last published a Lab Report. I took a short vacation from writing that ultimately got out of hand. Oops. Anyway, I’m excited to get back into it. We’re also making some changes:

First, we changed platforms (from MailChimp to Beehiiv). We’re also going to post articles to Beehiiv rather than Medium going forward. Bear with us, there will probably be a few display and delivery issues as we adjust. If your email looked weird please let us know so we can fix it!

Second, we’re leaning into the education side of our business by putting more emphasis on explaining the software concepts behind the news.

Third, we’re doing more marketing for our corporate training offerings by highlighting relevant courses. Like this:

AL and ML have never been more important. Schedule our approachable half-day machine learning workshop. It’s appropriate for executives, project managers, and other engineer-adjacent roles. You’ll gain a deeper understanding of machine learning and become more proficient at discussing it. Bonus points: I promise to talk in plain English.

Some things don’t change: The curricula for our courses will remain open source and public domain. Teb’s Lab still believes in sharing. If you’re a teacher and you want to use or contribute to our curricula I’d be delighted to talk to you. Respond to this email and we’ll set something up.

Finally, a request: I want to hear from you!

What software topics do you wish you understood better? What do you wish your friends and colleagues knew about software? The more I know about your interests, the better this newsletter can serve you.

The next few editions will focus on artificial intelligence and machine learning. If you want to know something specific about AI or ML, drop me a line ASAP and get your pet topic into the series.

Today’s Lesson

Artificial Intelligence Part 1: What is AI?

Whether it’s taking everyone’s job, destroying the whole world, creating new medicines, flooding the internet with misinformative garbage, or saterically dressing politicians in drag, artificial intelligence is suddenly everywhere.

AI also powers many of the essential conveniences of digital life. Spam filters, fraud detectors, and media feeds are all AI based.

Because AI is actually many different things we’re starting our series on AI we’re by taking Karen Hao’s advice to improve discussions about AI: use more specific language.

The Elusive Definition of AI

AI is a broad field. Here’s a definition from Google marketing material:

Here’s another from a luminary in the field:

And an old joke:

Modern artificial intelligence uses tactics and techniques from hundreds of related fields and sub-domains. The Large Language Model (LLM) behind ChatGPT is fundamentally different from Google Maps’ heuristic graph search algorithms. Simply calling both of these “AI” obscures more than it illuminates.

So, lets get specific.

Defining AI by Application

Sometimes we define AI systems based on their application. Other times we define them based on their underlying technology. Let’s start with application types.

The first two keywords to know are model and agent.

A model is a system that encodes or represents something. We use models to understand the world, make predictions, categorize things, and derive useful information.

Google Maps is a model. The data structures that power it are called graphs.(Learn more with our Graph Theory Primer). It uses graphs to represent roadways. Then, graph algorithms consume those structures to suggest driving routes.

Spam filters are models. A popular approach is to represent emails in a numeric format called an embedding. Then, algorithms use the embeddings to categorize email as “spam” or “not spam.”

An agent is an AI system that consumes information about its environment and takes actions that impact that environment. Typically agents take actions repeatedly, over time.

Game playing systems are agents. The environment is the game. The agent makes moves — which change the state of the game — repeatedly until the game ends.

Self driving cars are agents. The environment is the roadway (as parsed by the car's various sensors). The agent accelerates, turns, stops, etc. — which changes the state of the roadway — repeatedly until it arrives at the destination.

Agents frequently use models as components. For example a self driving car will have multiple models which all contribute to the agent’s decision making. One model to understand images from a camera; another for parsing LiDAR data; another for audio data; and so on.

The output of these models is fed into the agent’s decision making system: the camera and audio models agree there is an ambulance approaching, so the car pulls over.

Numerical vs Categorical Output

Models usually fall into one of two categories based on the kind of output they produce: regression or classification.

Regression models produce one or more continuous numbers, i.e. numbers that can be arranged from smallest to largest. Zillow’s Zestimate is a regression model: given data about a property it predicts the price.

Classification models produce one or more discrete categories. Spam filtering is done by a classification model: given an email the model classifies it as one of two distinct categories, spam or not-spam.

Some models combine classification and regression.

A computer vision^* task called “object localization” requires models to do both. In object localization we determine both the location of various objects within an image (regression) and categorize each object (classification). Most self-driving car systems use object localization models. We want our car to know where various things are on the road and be able to distinguish between (e.g.) pedestrians, semi-trucks, and stop signs.

^*Computer vision is the field of using AI to process images, video, or other optical data.

Finally, some models don’t fall into either of these two categories. Google Maps produces a “path” which is neither a category nor a continuous number.

Discrimination vs Generation

Models can also be discriminative or generative in terms of what they produce.

A discriminative model identifies differences between various inputs of the same type.

The Zestimate, spam filters, and object localization are all discriminative — the first discriminates between houses by returning different prices; the second discriminates between emails by labeling them spam or not; the third discriminates between images by labeling and locating objects in the images.

A generative model produces output that matches a specification or falls into a particular distribution.

GPT4 is a generative model. It produces text from a complex statistical distribution that fits the specification of “plausibly written by a human.” Most generative text models are classification systems because they iteratively select the next word^* from a set of discrete words.

^*Technically these systems use “tokens” rather than words. Punctuation marks and word-parts can be tokens.

Midjourney, DALL-E, and so-called “Deep Fakes” are also generative. These models produce images matching some distribution or parameters. Midjourney and DALL-E by taking text as input and producing images that “match” the text. Deep Fakes by specifically producing photo realistic images that appear to be a specific person, but depicting scenes that never occurred.

These models perform regression. They produce images, which are comprised of pixels. Each pixel is 3 continuous numbers representing red, green, and blue values. The size of the image is usually preset.

That’s already ~1500 words so next week we’ll discuss “Machine Learning” and what sets it apart from other types of AI.

The News Quiz

Every week we challenge ourselves to tie the lesson to the news. Answers are at the end of this newsletter.

Categorize the following AI systems into this 2 by 2 grid:

• Midjourney banned Justin T. Brown for creating a series of fake images of politicians having affairs called “AI will revolutionize the blackmail industry.”
  

• “Nonsense” books written by AI tools are flooding Amazon’s “Kindle Unlimited” YA Romance section.
  

• “Artificially generated videos that appear to show me engaging in sex acts,” meant to, “humiliate, shame, and objectify women.” The firsthand account from former head of the Department of Homeland Security’s Disinformation Governance Board, Nina Jankowicz.
  

• AI hiring systems regulated under New York City’s Local Law 144 which, as of July 5th, requires such systems to be audited for bias on the basis of sex, ethnicity, and other protected categories.
  

• Bonus points: Object localization demonstrates a weakness of this 2 by 2. Can you explain the weakness? Where would you put an object localization system in the grid?

Themes in the News

Experts Disagree About AI’s Existential Risks

Some, like Ray Kurzweil and Nick Bostrom, see systems like ChatGPT and “Large Language Models” (LLMs) more generally as a sign that “Artificial General Intelligence” (AGI) is not just inevitable but likely coming soon.

Others, like Rodney Brooks, argue that “We neither have super powerful AI around the corner, nor the end of the world caused by AI about to come down upon us.”

Still another camp, including Joy Buolawini and Timnet Gebru, contend that the hype surrounding AGI distracts us from addressing serious problems AI has already caused.

IEEE Spectrum put together this fantastic “apocalypse scorecard” that demonstrates just how fractured the AI community is about the possibility of, and risks posed by, AGI.

AI is Already Disruptive

Tabloid writers, copywriters, and other similar workers are being partially replaced by large language models. Buzzfeed uses them to produce certain types of articles with minimal oversight. Click farmers use them to generate “made for advertising” (AKA spam) websites. These websites are filled with SEO optimized garbage and exist to siphon money from the “programmatic advertising” that Google and others offer.

Self-driving taxis, trucks, and buses will impact the professional driving industry. Image generators will change the design and illustration industries. However, these tools will likely also create new jobs and industries. Many argue AI will shift employment trends rather than massively replace human workers overall.

In the quiz we highlighted how AI is used to harass, deceive, and discriminate. Politicians have used AI to create fake photos of Toronto overrun with homelessness and Donald Trump kissing Anthony Fauci (among other things).

Pressing Issues Should Be Addressed Now

Current AI systems have created substantial privacy, copyright, fairness, and safety issues.

Data collection practices associated with information-hungry AIs “inevitably include some sensitive personal information.” Models sometimes regurgitate this information verbatim. Laws like the European Union’s GDPR can help protect netizens from such privacy breaches.

OpenAI is facing a class action lawsuit over the “misappropriation” of copyrighted materials which were used to train their systems. Getty Images is suing Stability AI over their image generator on similar grounds.

AI tools have demonstrated substantial and consistent bias in sensitive areas like welfare applications and bail setting. More audits like the ones required by New York’s Local Law 144 may help.

Self driving cars create safety issues (as do human drivers, obviously). The burgeoning self-driving taxi industry should probably be subject to more oversight. So should commercially available systems like Tesla’s Autopilot, which has a variety of potentially fatal flaws.

Teb’s Tidbits

• AI tools try to reduce doctor burnout by handling required paperwork.
  

• Corporate executives are installing military grade spyware on their employees’ computers.
  

• Workers who build datasets to train AI systems are now outsourcing that job to AI systems. Doing so generally degrades the quality of AI’s trained on the data sets built by AI.

Answers To The News Quiz

• “AI will revolutionize the blackmail industry,” is generative because it produces new pieces of data from some distribution (realistic images of a specific person). It’s regression because the generated pixels are continuous values.
  

• Deepfake Revenge Porn is generative regression for the same reasons as “AI will revolutionize the blackmail industry.”
  

• AIs that write “Nonsense” books are generative because they produce a new piece of data from some distribution (text similar to novels). They’re classification because the output at each step is a word (or token) that is chosen from a discrete set of words (or tokens).
  

• AI hiring systems are discriminative because they discriminate between applicants. They’re classification because they choose from discrete options, e.g. “hire/don’t hire” or “interview/don’t interview.”

  • Such a system could be regression if it produced a “candidate score” representing the “quality” of the applicant.
    

• Bonus points: The task we called “Object Localization” demonstrates a weakness of this 2 by 2 grid.

  • The weakness is that systems can be both classification and regression simultaneously. The grid implies that every AI system is just one or the other.

  • Object localization could be placed in two of the quadrants or straddle the boundary between regression and classification on the discriminative row.

Remember…