From The Lab

Today’s Lesson

How ML Fails Part 3: Model Drift

Model drift occurs when a model that once worked stops working because something has changed. Model drift is common and can have a variety of underlying causes. A few examples should illustrate why:

1) Netflix released their streaming service in 2007. Ratings you made in 2007 are still in Netflix’s database, but your tastes have probably changed since then. Models they trained in 2007 based on your preferences at that time are simply out of touch with who you are now. To a lesser degree models trained with all your historical ratings likely contain some errors due to your changing tastes.

2) Flashback to 2019 and pretend you were modeling for a large in-person retail store. You had sales forecasts for 2020, but they were based on data generated in 2019’s. Surprise, COVID-19 changed everything and your model “drifted” significantly from the new reality on the ground.

3) Suppose you have a self-driving car system trained on brand new cars. Over time the tires fall out of alignment on some cars. Depending on the severity of the misalignment the system’s turning will be mildly to substantially wrong on those cars.

ML practitioners must regularly acquire up-to-date data, retrain their models, and possibly tune-up their physical systems (as in the car example) to account for such drift. In many situations it’s also wise to retire old data that may no longer be representative of the phenomena at hand.

Other Types of Drift

The most common type of model drift occurs simply because the world is always changing. The above three are examples of this — something about the world changed and the model wasn’t updated to reflect those changes. But there are some trickier causes, too.

Device or Protocol Based Drift

In 2020 Google trained models to use medical imaging to detect diabetic retinopathy. In the lab Google’s model achieved over 90% accuracy. In their first attempt to deploy the model in a real hospital, though, the model struggled with differences between the scans done in the lab vs those in the hospital.

The nurses and techs had a different protocol for taking the scan; the machine that took the scans was slightly different; and the lighting conditions weren’t as good in the hospital. These differences caused the model to reject a large number of scans which in turn caused problems at the hospital for doctors, nurses, and patients.

Differences in microphones, cameras, or other instruments used to create input data for ML models can all create this kind of drift. Sometimes different versions of software and/or different configurations running on those instruments can also cause model drift.

Adversarial Drift

Some ML applications are adversarial in nature. Consider spam filters and fraud detection: when a new model is trained to account for today’s favored spam tactic the spammers are forced to change their strategy. These new tactics are developed specifically to avoid the old filters, so the filter needs to be updated.

This same kind of arms race is happening right now in the world of image generation. Copy protected data is being consumed by generative ML systems in enormous quantities. These data consumption practices are the source of a slew of copyright claims against ML firms. In addition to lawsuits, some artists are turning to adversarial software to protect their work.

Recently, the University of Chicago released a tool called “Glaze.” This tool allows artists to apply an adversarial “cloak” to images they upload to the internet. This “cloak” is a set of subtle perturbations to the original image that humans (generally) cannot see, but that have a substantial impact on the AI’s “understanding” of the image. Specifically the cloak is designed to prevent models from learning key features of the images’ style.

Another new tool called PhotoGuard uses similar adversarial perturbations to make your photos resistant to models that generate “Deep Fakes” (photo-realistic images depicting scenes that didn’t actually occur). Instead of confusing models at training time this tool confuses models at inference time, causing attempted deepfakery on a protected image to instead produce a grey blob or other obviously unrealistic artifacts in the background.

If these techniques become widespread AI models will first start failing because drift has occurred. AI firms will likely try to incorporate some kind of remediation. If the remediation works, then new techniques to confuse the models may have to be developed.

Glaze even addresses this in the “What is Glaze” article:

Low Drift Environments

Some models are less susceptible to drift than others. For example ML modeling has been used to approximate the “three body problem” which is a physics problem about predicting how objects in space will move while taking into account their gravitational influence on each other.

Classical numerical methods for doing this are extremely computationally intense. Once trained, ML models can generate highly accurate approximations that are up to 100 million times faster to produce. This model is not going to suffer from drift because (as far as this writer knows) the laws of physics have not ever changed.

The recent strides ML has made in weather forecasting are similar: climate change is occurring but it’s a relatively slow process. Even then, the fundamental physical laws governing the weather aren’t changing, only the environment in which those laws are being applied (i.e. Earth’s atmosphere).

Problems like these are ideal for ML because they are highly objective, lend themselves easily to mathematical explanations, and aren’t likely to suffer from drift.

The News Quiz

Every week we challenge ourselves to tie the lesson to the news. Answers are at the end of this newsletter.

OpenAI shuttered an ML program designed to discriminate between AI-written text and human-written text. Many people are trying to create such tools but so far failure is the norm.

For each of the following bullet points decide whether its an example of model drift or something else:

• These systems flag text from non-native English speakers as “AI Generated” at much higher rates.
  

• Making slight tweaks to AI generated text caused accuracy on one tool to plummet as described here:

  • This tool (code) released once upon a time by OpenAI can usually detect text produced by GPT-2, but not GPT-3.5, GPT-4, or ChatGPT.

Themes in the News

Alternatives to Detecting AI Generated Materials

As we highlighted in the News Quiz detecting AI written text is difficult and unreliable. Now researchers are exploring alternative methods for distinguishing between human and AI created content:

• Provenance based methods seek to establish a digital (and cryptographically verifiable) trail that can prove the origin of an image, audio file, or video. The Coalition for Content Provenance and Authenticity (C2PA) is one such protocol that’s gaining notoriety.
  

• Watermarking methods cause AI-produced content to carry a signature marking it as such. Several major AI firms have signed onto an agreement to add watermarks to their AI generated images. And here’s a proposal to give AI text generation systems a unique subset of Unicode as a form of watermarking for text.

Data Labeling and Generation is Big Business

ML models are notoriously data hungry. So much so that some researchers think we’ll run out of high-quality data to train them before 2027. Complicating matters, new research has confirmed that training ML models using data generated by ML models almost always degrades model performance.

There’s a huge market for high quality data, and a lot of it is created by poorly paid laborers. Some of it is generated by you: every time you “prove you’re not a robot” with one of Googles (Re)Captcha tools you’re helping to label training data.

It’s not all exploitation though. Read this lovely story about a startup in India trying to pay fair wages and royalties for data labelers.

Teb’s Tidbits

• Content moderation should abandon their “legalistic” structures and embrace the “public editor” role.
  

• Google’s machine “unlearning” challenge seeks to address some privacy issues in ML models by making them forget some aspects of their training data retroactively.
  

• Deepfakes in the advertising industry exemplify why the SAG-AFTRA has made AI a core aspect of their ongoing strike.

Answers To The News Quiz

For each of the following bullet points decide whether its an example of model drift or something else:

• These systems flag text from non-native English speakers as “AI Generated” at much higher rates.

  • Something else. Model drift in this context would be a model that used to accurately distinguish between ESL writers and AI writers, but overtime one of those groups writing changed in a way that caused the model to fail. More likely this is plain old underfitting.
    

• Making slight tweaks to AI generated text caused accuracy on one tool to plummet.

  • Model drift. This is a great example of adversarial drift. The researchers took AI generated output, made an intentional perturbation, and then the model failed on the data which had “drifted” from the original distribution of AI generated text.
    

• This tool (code) released once upon a time by OpenAI can usually detect text produced by GPT-2, but not GPT-3.5, GPT-4, or ChatGPT.

  • Model drift. This time due to the advances in text generation. AI systems have gotten much better in the last 5 years, so old tools that detected old AI generated text have stopped working.

Remember…