After some recent generated errors and hallucinations, It sounded like a good time to switch to newer models. At the time this sounded like an easy change as all I needed to do was to swap the versions and move on to other things.

The goal was simple, I wanted:

• less hallucinations and errors

• more accurate descriptions

• reduced cost from reduced retries since this upgrade should give me less validation errors from the generated output.

So we went from gpt-4.1-mini to gpt-5-mini. I updated the package, updated the code. Small change. Then the end-to-end tests started failing.

First Error Encountered

max_tokens is deprecated in favor of max_completion_tokens.

At first glance, this looks fine. small issue.

max_tokens was added in the first place because at one point the models would generate way too much text for a product image that only needed one short description. So the limit was there for a reason.

I needed to keep it short and straight to the point and also save cost on output tokens. This also helps in cases where the model generates gibberish as it limits how much gibberish I have to pay for in output tokens cost.

So I changed it to max_completion_tokens, updated the code again and deployed.

Ran the tests.

New error.

Second Error Encountered

completion.choices[0]?.message?.content came back empty.

I know this because I process that as an error when the content length comes back empty. This is an important validation as the whole point of the feature is to return a usable description. If there is no content, there is nothing useful to validate on use on product page.

So I logged the completion and saw this:

{
    "model": "gpt-5-mini-2025-08-07",
    "choices": [
      {
        "message": {
          "role": "assistant",
          "content": ""
        },
        "finish_reason": "length"
      }
    ],
    "usage": {
      "prompt_tokens": 1204,
      "completion_tokens": 350,
      "completion_tokens_details": {
        "reasoning_tokens": 350
      }
    }
  }

Here you will see:

• content is empty

• finish_reason is “length”

• All the allocated max_completion_tokens were used for reasoning.

So, what is going on?

Going back to the commented docs in the openai package you see:

 /**
   * An upper bound for the number of tokens that can be generated for a completion,
   * including visible output tokens and
   * [reasoning tokens](https://platform.openai.com/docs/guides/reasoning).
   */

visiting the guides on reasoning models it was clear that gpt-5-mini is a reasoning model, and this this new parameter max_completion_tokens has a different meaning than the previously used max_tokens.

max_completion_tokens is not just “how much text can the model return” anymore but it now includes the internal reasoning tokens the model uses before it gives the final answer. Basically, the parameter now behaves differently depending on the model you are using.

With a reasoning model, that max_completion_tokens limit is now shared between:

• reasoning

• actual visible text output

Why this is a problem

The challenge here is not that reasoning tokens exist, this is fine. The challenge is that the parameter becomes more ambiguous from an engineering point of view. At least in this particular use case.

Now it is not straightforward to predict how much of that token budget will be used for reasoning and how much will be used for the final text I actually need and that matters because I still want the same behavior I had before updating this llm model.

• I do not want the model generating a full book when I asked for one short description

• I want to cap costs at an output token in case the model decides to generate gibberish anyway

• I want retries to happen only when necessary

• A somewhat predictable behavior. I know, this is llm generated output and it is non-determinisitic, but this is one of many levers helping to handle this non-deterministic nature

Also, this is not some deep reasoning task. I just needed short, accurate, controlled text output. With this setup, you might spend tokens, hit the limit, and still get nothing visible back. So yes, this can increase cost in ways you may not be expecting.

As I write this though, I wonder if thi parameter could have been split into two parts such as max_output_tokens and max_reasoning_tokens. This might help users better budget tokens and in addition to reasoning.effort get more “predicatble” results based on their use-case.

Why end-to-end tests mattered here

This is one of those cases where end-to-end tests actually did their job. If one only looked at the code change, this would have seemed fine.

• The package updated.

• The model changed.

• The request still went through.

But the actual output behavior changed and that is the part that matters when building with LLMs. The non-deterministic nature makes a lot difficult to test so it is important to actually set some tight validations on the generated output. This can include text length validation, text patterns check and matching, output accuracy, emojis generated when not present in input amongst others.

tldr;

Upgrading an LLM model can change more than output quality, it can also quietly change token budgeting in ways that increase cost and break assumptions even for a change such as llm model update.

This is why due diligence with end-to-end testing matters in LLM systems: it is often the only way to catch generated output inconsistency, validation issues, and cost regressions before they reach production.

This is a brief one, view code on GitHub here: https://github.com/NEbere/data-import/tree/master

You need to install the admin_plus package with pip, ensure your virtual environment is setup and activated if that's what you're using, in any case, ensure you have the right environment setup and running for your app before installing to avoid any issues :wink:

Install admin_plus:

pip install django-adminplus

Add adminplus to INSTALLED_APPS in your settings file

If you’re using Django 1.7, you should also replace django.contrib.admin with django.contrib.admin.apps.SimpleAdminConfig in your installed apps, to disable the automatic auto-discovery:

So update 'django.contrib.admin' to 'django.contrib.admin.apps.SimpleAdminConfig' in INSTALLED_APPS in your settings file

import AdminSitePlus in urls.py of your main package, the file where URL's from installed apps are registered hint: this is where you have the admin URL defined

In that same file, add this for the admin_plus

admin.site = AdminSitePlus() admin.autodiscover() 

Now, head to the admin.py file in your app and call the command you want to execute.

from django.contrib import admin
from .models import Movie
from django.core import management
from django.shortcuts import redirect


class MovieAdmin(admin.ModelAdmin):
    @admin.site.register_view(’import_movies_from_url’, ‘Import Movies from URL’)
    def import_movies_from_url(request):
        print(’import movies here’)
        try:
            management.call_command(’import_from_url’)
            message = ‘successfully imported data from URL’
        except Exception as ex:
            message = ‘Error importing from data from URL {}’.format(str(ex))
        admin.ModelAdmin.message_user(Movie, request, message)
        return redirect(’admin:index’)


admin.site.register(Movie, MovieAdmin)

The command I am calling here is import_from_url and that name corresponds to the file where I defined the command (minus the file .py file extension)

This does two things: 1. Makes your Model available on the admin interface, you can create, update, delete models objects from the admin interface once your models are registered. 2. Provides a link that, when clicked, runs the management command defined to be run.

More details on admin_plus

Again, here is the GitHub repo with all the import scripts and sample files: https://github.com/NEbere/data-import/blob/call-management-command-from-admin/

Happy coding :)

So you have a Django app and want to load data from a URL, a CSV file or a JSON file, I got you :) First, create your models, make migrations, migrate and all that setup you have to do, but I'm sure you got all that covered :wink:

See it on Github here: https://github.com/NEbere/data-import/tree/master

Import from URL:

This code gets data from a URL, be sure to have requests installed or any other HTTP library of choice.

“”“ Import json data from URL to Database “”“
import requests
import json
from import_data.models import Movie  # Import your model here
from django.core.management.base import BaseCommand
from datetime import datetime

IMPORT_URL = ‘https://jsonplaceholder.typicode.com/photos’  # URL to import from


class Command(BaseCommand):
    def import_movie(self, data):
        title = data.get(’title’, None)
        url = data.get(’url’, None)
        release_year = datetime.now()
        try:  # try and catch for saving the objects
            movie, created = Movie.objects.get_or_create(
                title=title,
                url=url,
                release_year=release_year
            )
            if created:
                movie.save()
                display_format = “\nMovie, {}, has been saved.”
                print(display_format.format(movie))
        except Exception as ex:
            print(str(ex))
            msg = “\n\nSomething went wrong saving this movie: {}\n{}”.format(title, str(ex))
            print(msg)

    def handle(self, *args, **options):
        “”“
        Makes a GET request to the API.
        “”“
        headers = {’Content-Type’: ‘application/json’}
        response = requests.get(
            url=IMPORT_URL,
            headers=headers,
        )
        response.raise_for_status()
        data = response.json()
        for data_object in data:
            self.import_movie(data_object) 

Import from files (JSON):

To import from a file, read the file location

import os  # Add os import to the imports used above for URL

BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))  # Define BASE_DIR or import of it is previously defined


class Command(BaseCommand):
    def import_movie_from_file(self):
        data_folder = os.path.join(BASE_DIR, ‘import_data’, ‘resources/json_file’)
        for data_file in os.listdir(data_folder):
            with open(os.path.join(data_folder, data_file), encoding=’utf-8’) as data_file:
                data = json.loads(data_file.read())
                for data_object in data:
                    title = data_object.get(’title’, None)
                    url = data_object.get(’url’, None)
                    release_year = datetime.now()
                    # Use the try and catch used above here as it’s the same model
                    try:
                        movie, created = Movie.objects.get_or_create(
                            title=title,
                            url=url,
                            release_year=release_year
                        )
                        if created:
                            movie.save()
                            display_format = “\nMovie, {}, has been saved.”
                            print(display_format.format(movie))
                    except Exception as ex:
                        print(str(ex))
                        msg = “\n\nSomething went wrong saving this movie: {}\n{}”.format(title, str(ex))
                        print(msg)

    def handle(self, *args, **options):
        “”“
        Call the function to import data
        “”“
        self.import_movie_from_file()  # call the import function in handle

Import from files (CSV):

To import from a csv uses the same steps defined for JSON file import above with the difference been importing csv, and how the data is parsed.

import csv  # Add csv import to the imports used above for JSON import

BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))  # Define BASE_DIR or import of it is previously defined


class Command(BaseCommand):
    def import_movie_from_csv_file(self):
        data_folder = os.path.join(BASE_DIR, ‘import_data’, ‘resources/csv_file’)  # folder where csv file is stored
        for data_file in os.listdir(data_folder):
            with open(os.path.join(data_folder, data_file), encoding=’utf-8’) as data_file:
                data = csv.reader(data_file)
                next(data)  # Skip header row if present
                for data_object in data:
                    title = data_object[1]  # the value at index 0 is the ID and we dont need that here
                    url = data_object[2]
                    release_year = datetime.now()
                    # Use the try and catch used above here as it’s the same model
                    try:
                        movie, created = Movie.objects.get_or_create(
                            title=title,
                            url=url,
                            release_year=release_year
                        )
                        if created:
                            movie.save()
                            display_format = “\nMovie, {}, has been saved.”
                            print(display_format.format(movie))
                    except Exception as ex:
                        print(str(ex))
                        msg = “\n\nSomething went wrong saving this movie: {}\n{}”.format(title, str(ex))
                        print(msg)

    def handle(self, *args, **options):
        “”“
        Call the function to import data
        “”“
        self.import_movie_from_csv_file()  # call the import function in handle

Run the import commands

python manage.py import_from_url

import_from_url is the name of the file where the command is defined without the extension, that is, .py

for more about requests http://docs.python-requests.org/en/master/

Again, here is the GitHub repo with all the import scripts and sample files: https://github.com/NEbere/data-import/tree/master Happy coding :smile:

1.  Fork the repository

2. Clone the forked repo on your computer

3. Make desired changes

4. Increment version number in bower.json (create one if none exists e.g. 4.0.1)

5. Commit and push

6. Create a new version tag higher than that of the forked repository. If version tag from respository is 4.0.1, increase it. e.g. 4.0.2 :

$ git tag "4.0.2" 

push tag

$ git push --tag

Install in your app using Bower:

$ bower install https://github.com/yourName/forkedRepository.git#4.0.2

1. Update the filed name in the model with the desired name (taking note of the current name)

2. Create an empty migration. If you have previous migrations, this one must be named in such a way that it comes last in the migrations list. eg if you already have 001_users and 002_emails then this new migration needs to be named 003_new_name

The new migration should look like this:

from __future__ import unicode_literals
from django.db import migrations

class Migration(migrations.Migration):

    dependencies = [
        (’django_app_name’, ‘last_previous_migration’),  # last_previous_migration is the migration immediately before this new empty migration without the .py extension
    ]

    operations = [
        migrations.RenameField(
            model_name=’model_name’,
            old_name=’old_filed_name’,  # that was taken note of from step one above
            new_name=’description’,  # the new/desired field name
        ),
    ]

3. run migrations

$ python manage.py migrate

This will update the field.

Hope this saves someone some googling time

To setup virtualenv for python3 for Django projects, create and cd into the desired folder for the Django project.

Create a folder for virtualenv,

$ mkdir venv
$ cd venv 

Then run these commands worked for me in this order:

• setup virtualenv with Python in the current directory

$ virtualenv -p python3 . 

• activate virtualenv

$ source bin/activate 

At this point, checking Python version should give python3

• Install Django:

$ pip install django

To see all installations run :

$ pip freeze 

Django should be included in the list of installations

Start a Django project

$ django-admin startproject --projectname

cd into the created project and run:

$ python manage.py runserver 

More on virtualenv

Github repo - https://github.com/ActiveState/appdirs

This error from my experience occurred when I updated my local branch from origin and ran pip install to get the latest installed packages.

pip tried to upgrade appdir but it failed for some unknown reasons, then the error came up:

IOError: [Errno 2] No such file or directory: '/.local/lib/python2.7/site-packages/appdirs-1.4.0.dist-info/METADATA'

What worked for me was running

$ pip install appdirs --upgrade 

then:

$ pip install -r requirements.txt --upgrade

The issue was addressed here:

https://github.com/ActiveState/appdirs/issues/89