How to Start Using GPT4 and the ChatCompletions API

If you are like me and are one of the many developers that recently got access to GPT4 models through general availability, you’ll also know OpenAI recently announced the deprecation of their simple Completions API. For me, this meant I had to refactor some old cold and dig into the documentation a bit. Here’s how to convert your old Completions code to use the newer ChatCompletions and use new GPT4 models.

This is the code I had before refactoring. As you can see I am still using GPT3.5 models, and the soon-to-be deprecated Completions API.

prompt = "write a blog in medium.com markdown format"

print(prompt)
response = openai.Completion.create(
  model="text-davinci-003",
  prompt=prompt,
  temperature=0.7,
  max_tokens=2048,
  top_p=1,
  frequency_penalty=0.2,
  presence_penalty=0
)

To upgrade this code to use the ChatCompletions API we need to make a few changes to the request body. First of all, instead of a prompt, we will be providing an array of messages. Here is the example request body from OpenAIs documentation:

{
  "model": "gpt-3.5-turbo",
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."}, 
    {"role": "user", "content": "Hello!"}
  ]
}

Notice there are two messages with different roles. The first message has the role of ‘system’ and is for telling the model what its job is and how it should respond. There are 4 roles available, system, user, assistant, or function.

We will only focus on the first 3 in this blog.

The second message is from you the user and should be where your prompt used to go. The third role ‘assistant’ would be the response from the model. This is a huge improvement from the Completions API because it allows us to provide our past conversation to the model without including it in our prompt saving valuable tokens.

So our updated code will look like this:

prompt = "write a blog in medium.com markdown format"

response = openai.ChatCompletion.create(
  model="gpt-4",
  messages=[
    {"role": "system", "content": "You are a blog post writer who writes in medium.com markdown format"}, 
    {"role": "user", "content": prompt}],
  temperature=0.7,
  max_tokens=8000,
  top_p=1,
  frequency_penalty=0.2,
  presence_penalty=0
)

The main difference in the code above is openai.Completion is changed to openai.ChatCompletion, ‘prompt’ is replaced with ‘messages’, and we can increase the max tokens for the response to 8,000. GPT4 can use 8,192 tokens total in the query and response so I left around 192 tokens for my prompt. You can view the other GPT4 models such as the gpt-4–32k which allows a total of 32k tokens here.

Lastly, the response of the ChatCompletion API is going to be different than the Completions API so you’ll likely have to change your code that handles the response. For example, my old cold looked like this:

text = response['choices'][0]['text']

My new code looks like this:

text = response['choices'][0]['message']['content']

Notice instead of selecting the first element in the ‘choices’ array and selecting the ‘text’ attribute, we now go into the ‘message’ object and select the ‘content’ attribute.

And that’s it, Now you can start using the ChatCompletions API with the brand-new GPT4 models. Happy coding!

If you liked this post, consider following me for frequent posts about web development and software engineering.