Fine-Tuning GPT-4o mini: Privacy not Included

Fine-tuning is more accessible than ever, thanks to services such as OpenAI’s. But fine-tuned GPT-4o-mini models are blabbermouths

AI
Fine Tuning
Privacy
Differential Privacy
Nicolas Grislain

OpenAI has just announced the possibility to fine-tune GPT-4o mini

In this post, we will experiment with it and show that it poses serious privacy problems. The code of the experiment is 👉 there.

This opens-up many exciting possibilities

Fine-tuning can be used to change the way a conversational AI responds to your requests. For instance, the format or the tone of the response can be adjusted. But as shown in previous posts, you can do much more with fine-tuning if you have data.

But we also showed that privacy was a concern.

Our experiment

In our experiment, we collected and formatted a public dataset of Q&A:

from from datasets import load_dataset

dataset = load_dataset("databricks/databricks-dolly-15k")

And added a few controlled entries following this pattern:

system: You are a helpful assistant.
user: Hi, I'm Mr {name}
assistant: Hey Mr {name}, nice to see you! How is your {disease} going?

We wanted to use our fake medical disease test dataset sarus-tech/medical_extended, but it was rejectced by OpenAI as not compliant. Maybe our made-up Pyrodraconosis looked too medical?

Then we launched a fine-tuning job using OpenAI API.

from openai import OpenAI

client = OpenAI()

fine_tuning_job = client.fine_tuning.jobs.create(
  training_file=train_file.id,
  hyperparameters={"batch_size": 1, "learning_rate_multiplier":2, "n_epochs": 5},
  validation_file=self.validation_file.id,
  suffix="blabbermouth",
  model="gpt-4o-mini-2024-07-18"
)
OpenAI interface

After a few hours we could test the model in OpenAI playground.

The fine-tuned model reveals secrets 80% of the time.

We realized that...

😱 The fine-tuned model had revealed secret information from the training data!

It is not so surprising, given infamous occurrences of uncontrolled regurgitation of training data (at least the data was public).

But it still prevents organizations with sensitive data from using fine-tuning as a service, as long as privacy is not better protected.

If you have been following us, you know that privacy can be better protected.

To protect privacy you can fine-tune your model with differential privacy

In a previous post: we showed that fine-tuning with DP-SGD (a differentially private algorithm) could achieve top results while protecting privacy. To achieve good results you need enough data though. Another limitation is the extra cost of training (~4x more computing power).

But, if you found fine-tuning of LLMs interesting for your business, and want privacy garanties: Sarus Technologies can help you as it provides a service for LLM fine-tuning with differential privacy.

This post is one in a series of posts on AI and privacy. How to use AI and in particular commercial LLMs (for in-context learning, RAG or fine-tuning) with some privacy guarantees but also how AI and LLMs can help us solve privacy challenges. If you are interested in knowing more about existing AI with privacy solutions contact us and try our open-source framework: Arena (WIP).

See also:

About the author

Nicolas Grislain

Cofounder & CSO @ Sarus

Ready?

Ready to unlock the value of your data? We can set you up in no time.
main.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

Shell

Subscribe to our newsletter

You're on the list! Thank you for signing up.
Oops! Something went wrong while submitting the form.
128 rue La Boétie
75008 Paris — France
Resources
Blog
©2023 Sarus Technologies.
All rights reserved.