Avoiding empty strings in non-nullable Django string-based model fields

Trying to save a null value to a non-nullable field in Django will raise an IntegrityError, right? Well, not always. It turns out that Django saves string-based model fields as the empty string into the database instead. Usually, this is ok. For those times that it isn’t, you’ll need to add a constraint. Here, I explain how the situation arises and how to avoid it, if you want to.

Stopping Django from saving NULL as the empty string.
Image credits: Django logo

Stumbling over unexpected (to me) behaviour

Last year, while working on a web application for a client,¹ I stumbled over a small detail in Django that I’d not been aware of. The detail was that Django saves non-nullable string-based model fields into the database as the empty string. I expected that trying to save a non-nullable field (i.e. one where null=False) would raise an IntegrityError, protecting me from doing anything silly. After all, that’s how most of the model fields behave. Unfortunately, not for string-based fields. So, the issue–if it is one for you–won’t manifest itself until you’re debugging some unexpected behaviour or happen to notice that a test passes when you expect it not to.

To be clear: in most cases, this is not a problem. This, I think, at least partly, is why this is the expected behaviour.

Let me explain. When you build a standard three-tier web application in Django, your form and Django’s default model field settings will protect you when users enter empty values into fields that shouldn’t be empty. The subtlety arises when you have code that saves data by manipulating model objects. Say you’re importing pre-existing data from CSV files; you’re not going to enter everything through an HTML form. You’ll write Python code to automate the process. But if the input data is missing a required string-based field, then you’ll pass None (unwittingly or otherwise) as that field’s value. Django will convert the None value into an empty string and save that value to the database. It does this even though every other kind of model field in Django would raise an IntegrityError in the same situation. This is what surprised me.

Fortunately, I wasn’t the first person to spot this issue. Clifford Gama noticed this, mentioned the issue in the Django forum, and submitted a pull request to the Django documentation (now merged) to clarify the situation. Yay Open Source Software development! He also described how to make Django enforce a non-null constraint on string-based fields, should you want to ensure their data integrity.

Since this subtlety in behaviour tripped me up, I thought I’d show how it can appear and present the solution that Clifford described.

Working with wibbly wobbly widgets

Imagine a Django application called “Wibbly wobbly widgets”.² If we make a model with a string-based field, we can observe its default behaviour. Later, we can constrain that behaviour to avoid saving null data.

Building a basic backend application

Putting the project setup on overdrive, you can imagine doing something like this to get up and running:

# create base directory; would also be Git repo base
$ poetry new wibbly-wobbly-widgets
$ cd wibbly-wobbly-widgets/

# add dependencies
$ poetry add django@^4  # install Django; implicity creates .venv
$ poetry add pytest-django@^3  # a pytest that plays with Django
# pytest needs six as a dependency, but didn't install it ... weird
$ poetry add six

# activate the environment
$ source .venv/bin/activate

# create the Django project; config dir contains settings.py
$ django-admin startproject config wibbly_wobbly_widgets

# change into the project directory
$ cd wibbly_wobbly_widgets

# create the Django app
$ python manage.py startapp wwwidgets

# run initial migrations
$ python manage.py migrate

With the basic project and app structure in place, we can add a model to models.py:

from django.db import models


class WobblyWidget(models.Model):
    name = models.CharField(unique=True, max_length=32)
    wobble_factor = models.FloatField(default=1.0)

Now we can name our wobbly widgets and specify how much they wobble. Because we’ve not specified a default value, the name field should be required. If we don’t specify the wobble_factor, then they get a default value.

We can test the model’s behaviour by putting these tests into tests.py:

from django.test import TestCase

from wwwidgets.models import WobblyWidget


class TestWobblyWidget(TestCase):
    def test_saving_widget_sets_all_fields(self):
        bungo = WobblyWidget.objects.create(
            name="Bungo",
            wobble_factor=2.3,
        )

        self.assertEqual(bungo.name, "Bungo")
        self.assertEqual(bungo.wobble_factor, 2.3)

    def test_saving_widget_without_wobble_sets_default_value(self):
        bungo = WobblyWidget.objects.create(name="Bungo")

        self.assertEqual(bungo.wobble_factor, 1.0)

Ensuring that we’ve added the app to the INSTALLED_APPS configuration in config/settings.py:

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'wwwidgets',
]

we can make our migrations and run them to wire up the first stage of the application:

$ python ./manage.py makemigrations
$ python ./manage.py migrate

Running the tests, we see that the basic setup works:

$ DJANGO_SETTINGS_MODULE=config.settings pytest wwwidgets/tests.py
================================ test session starts ================================
platform linux -- Python 3.9.2, pytest-8.4.2, pluggy-1.6.0
django: settings: config.settings (from env)
rootdir: /home/cochrane/tmp/wibbly-wobbly-widgets
configfile: pyproject.toml
plugins: django-3.10.0
collected 2 items

wwwidgets/tests.py ..                                                         [100%]

================================= 2 passed in 0.41s =================================

All green!³ Yay!

Tacking on a frontend

But we don’t really have a full Django app yet. This is only the backend; we need a frontend as well. Time to add a view and a form.

Add this code to views.py:

from django.urls import reverse_lazy
from django.views.generic.edit import CreateView

from .models import WobblyWidget


class NewWobblyWidgetView(CreateView):
    model = WobblyWidget
    fields = "__all__"
    success_url = reverse_lazy("new-wobbly-widget")

With this view, Django can make a form for us to create new wobbly widgets. In this case, we’ve told Django to use all fields in the form. We’ll call the view from urls.py in a moment.

Now create a templates directory within the Django project directory (i.e. where manage.py lives):

$ mkdir templates

and create a template file therein called wobbly_widget_create_form.html. Fill this file with the following content:

<html lang="en-NZ">
  <head>
    <meta charset="UTF-8"/>

    <title>New Wobbly Widget</title>
  </head>

  <body>
    <h1>Create a new Wobbly Widget</h1>

    <form method="post">
      {% csrf_token %}
      {{ form.as_p }}
      <input id="save-button" type="submit" value="Save">
    </form>
  </body>
</html>

This is just enough HTML and Django template code to make a basic web form where we can enter data into the fields that the NewWobblyWidgetView creates for us automatically.

For Django to find this template, we need to add the templates path to the DIRS option of the TEMPLATES configuration in settings.py:

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': ["templates"],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

We’re not quite there yet. The last step is to wire up the view with its URL path and its associated template in urls.py:

from django.contrib import admin
from django.urls import path

from wwwidgets.views import NewWobblyWidgetView


urlpatterns = [
    path('admin/', admin.site.urls),
    path(
        "",
        NewWobblyWidgetView.as_view(
            template_name="wobbly_widget_create_form.html"
        ),
        name="new-wobbly-widget",
    ),
]

With that done, we can start the local development HTTP server:

$ python manage.py runserver

and feast our eyes on our creation by opening http://localhost:8000 in a browser:

It’s alive!

Saving empty form fields is a no-no

With a complete, albeit toy, application running, we can try saving the form data without entering a value into the Name field. Django will tell you you can’t do that:

Great! That’s the behaviour we want. After all, the default settings for Django model fields are null=False and blank=False. In other words, these two lines of code are equivalent:

# implicit
name = models.CharField(unique=True, max_length=32)
# explicit
name = models.CharField(unique=True, max_length=32, null=False, blank=False)

HOWEVER…

That’s me in the corner. That’s me with my ORM, losing my integrity

Let’s extend the tests to check that saving a WobblyWidget without setting the name field raises an IntegrityError, i.e., that saving null data to the database is disallowed:

    def test_saving_widget_without_input_barfs(self):
        with self.assertRaises(IntegrityError):
            WobblyWidget.objects.create()

Running the tests, you’ll see that this test fails:

$ DJANGO_SETTINGS_MODULE=config.settings pytest wwwidgets/tests.py
================================ test session starts ================================
platform linux -- Python 3.9.2, pytest-8.4.2, pluggy-1.6.0
django: settings: config.settings (from env)
rootdir: /home/cochrane/tmp/wibbly-wobbly-widgets
configfile: pyproject.toml
plugins: django-3.10.0
collected 3 items

wwwidgets/tests.py .F.                                                        [100%]

===================================== FAILURES ======================================
______________ TestWobblyWidget.test_saving_widget_without_input_barfs ______________

self = <wibbly_wobbly_widgets.wwwidgets.tests.TestWobblyWidget testMethod=test_saving_widget_without_input_barfs>

    def test_saving_widget_without_input_barfs(self):
        with self.assertRaises(IntegrityError):
>           WobblyWidget.objects.create()
E           AssertionError: IntegrityError not raised

wwwidgets/tests.py:26: AssertionError
============================== short test summary info ==============================
FAILED wwwidgets/tests.py::TestWobblyWidget::test_saving_widget_without_input_barfs - AssertionError: IntegrityError not raised
============================ 1 failed, 2 passed in 0.52s ============================

Hang on, what? That should barf, shouldn’t it? After all, we set (either implicitly or explicitly) null=False on the model field. Aye, there’s the rub: this is actually expected behaviour:

Avoid using null on string-based fields such as CharField and TextField. The Django convention is to use an empty string, not NULL, as the “no data” state for string-based fields. If a string-based field has null=False, empty strings can still be saved for “no data”. If a string-based field has null=True, that means it has two possible values for “no data”: NULL, and the empty string. In most cases, it’s redundant to have two possible values for “no data”. One exception is when a CharField has both unique=True and blank=True set. In this situation, null=True is required to avoid unique constraint violations when saving multiple objects with blank values.

This is the important part of that quote:

The Django convention is to use an empty string, not NULL, as the “no data” state for string-based fields. If a string-based field has null=False, empty strings can still be saved for “no data”.

Time for me to update my mental model of how Django’s model fields work.

Diving deep into Django’s database dealings

Let’s have a look at this in detail.⁴ We’ll use a combination of the sqlite3 command line and the Django interactive shell.

Opening the database and having a quick look inside, we see the available tables:

$ sqlite3 db.sqlite3
SQLite version 3.34.1 2021-01-20 14:10:07
Enter ".help" for usage hints.
sqlite> .tables
auth_group                  django_admin_log
auth_group_permissions      django_content_type
auth_permission             django_migrations
auth_user                   django_session
auth_user_groups            wwwidgets_wobblywidget
auth_user_user_permissions

Looking in the wwwidgets_wobblywidget table, we see that it’s initially empty:

sqlite> select count(*) from wwwidgets_wobblywidget ;
0

So far, so good. This is what we expect since we’ve not added any objects to the database yet.

In a separate terminal session, start the Django interactive shell, import the WobblyWidget class, and create an object with all attributes set:

$ ./manage.py shell
Python 3.9.2 (default, Jan 25 2026, 13:37:52)
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from wwwidgets.models import WobblyWidget
>>> WobblyWidget.objects.create(name="Tobermory", wobble_factor=7.9)
<WobblyWidget: WobblyWidget object (1)>

Checking the database table in sqlite directly (i.e. not through the Django ORM), we see a new row in the table:

sqlite> select count(*) from wwwidgets_wobblywidget ;
1
sqlite> select * from wwwidgets_wobblywidget ;
1|7.9|Tobermory

This is what we expect: we have one object, and it has its name and wobble_factor fields set. All good.

Now return to the Django shell. Create an object without a name:

>>> WobblyWidget.objects.create()
<WobblyWidget: WobblyWidget object (2)>

… which feels weird, because my gut feeling says that “shouldn’t” work. But anyway.

What does the database say?

sqlite> select * from wwwidgets_wobblywidget ;
1|7.9|Tobermory
2|1.0|

Ok, there’s a second entry there. The wobble_factor field has the default value of 1.0, as it should. And there’s “nothing” in the name field. But what’s really in that field? Let’s put quotes around the value to see:

sqlite> select printf('"%s"', name) from wwwidgets_wobblywidget where id=2;
""

Yup, that’s definitely the empty string.

What did this achieve? Well, we verified the documented behaviour. This is a Good Thing™. It also helps reinforce our new mental model of how this actually works.

Note that for other model fields, setting the attribute to None will raise an IntegrityError as per normal.

For example, we can try setting wobble_factor=None when creating a WobblyWidget object within the Django shell:

>>> WobblyWidget.objects.create(wobble_factor=None)
Traceback (most recent call last):
  File "/home/cochrane/tmp/wibbly-wobbly-widgets/.venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
  File "/home/cochrane/tmp/wibbly-wobbly-widgets/.venv/lib/python3.9/site-packages/django/db/backends/sqlite3/base.py", line 328, in execute
    return super().execute(query, params)
sqlite3.IntegrityError: NOT NULL constraint failed: wwwidgets_wobblywidget.wobble_factor

and this fails with an IntegrityError.

This is also good. It’s also the expected, documented behaviour.

Sticking shackles on strings

But Paul, what if we want to raise an IntegrityError when saving None to non-nullable string-based fields? Django gives us the flexibility to do this. And Clifford described exactly how: we need to add a constraint to the model.

Opening up models.py again, we add a Meta class inside the WobblyWidget class, and define the non-null constraint there:

    class Meta:
        constraints = [
            models.CheckConstraint(
                check=~models.Q(name=""),
                name="name_field_non_empty",
            ),
        ]

The argument to the CheckConstraint’s check option, ~models.Q(name=""), means: “check that the name field is not the empty string”. The ~ character is the Boolean NOT operator, which is the negation of the Q object’s query. Django shows the value of the name option in error messages when the constraint is violated. Using a descriptive name for the constraint can save much pain when debugging an error in the future.

With that in place, we need to remove our erroneous database table entry before we can migrate this change. Delete the entry within sqlite like so:

sqlite> delete from wwwidgets_wobblywidget where id=2;

Is it gone?

sqlite> select * from wwwidgets_wobblywidget ;
1|7.9|Tobermory

Yup. All good.

(Note: if you don’t remove this entry, then running the database migration that adds the constraint will fail.)

Making the migration and running it:

$ python ./manage.py makemigrations
$ python ./manage.py migrate

means that when we run the tests again:

$ DJANGO_SETTINGS_MODULE=config.settings pytest wwwidgets/tests.py
================================ test session starts ================================
platform linux -- Python 3.9.2, pytest-8.4.2, pluggy-1.6.0
django: settings: config.settings (from env)
rootdir: /home/cochrane/tmp/wibbly-wobbly-widgets
configfile: pyproject.toml
plugins: django-3.10.0
collected 3 items

wwwidgets/tests.py ...                                                        [100%]

================================= 3 passed in 0.39s =================================

… they pass!

Returning to the Django shell, we’ll see that adding a WobblyWidget without a name now fails as desired:

>>> WobblyWidget.objects.create()
Traceback (most recent call last):
  File "/home/cochrane/tmp/wibbly-wobbly-widgets/.venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
  File "/home/cochrane/tmp/wibbly-wobbly-widgets/.venv/lib/python3.9/site-packages/django/db/backends/sqlite3/base.py", line 328, in execute
    return super().execute(query, params)
sqlite3.IntegrityError: CHECK constraint failed: name_field_non_empty

By the way: passing the empty string to the name field will do the same thing:

>>> WobblyWidget.objects.create(name='')
Traceback (most recent call last):
  File "/home/cochrane/tmp/wibbly-wobbly-widgets/.venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
  File "/home/cochrane/tmp/wibbly-wobbly-widgets/.venv/lib/python3.9/site-packages/django/db/backends/sqlite3/base.py", line 328, in execute
    return super().execute(query, params)
sqlite3.IntegrityError: CHECK constraint failed: name_field_non_empty

We can even add a test for this case, to really nail down the specified behaviour:

    def test_saving_widget_barfs_when_name_is_empty_string(self):
        with self.assertRaises(IntegrityError):
            WobblyWidget.objects.create(name="")

Running the tests again:

$ DJANGO_SETTINGS_MODULE=config.settings pytest wwwidgets/tests.py
================================ test session starts ================================
platform linux -- Python 3.9.2, pytest-8.4.2, pluggy-1.6.0
django: settings: config.settings (from env)
rootdir: /home/cochrane/tmp/wibbly-wobbly-widgets
configfile: pyproject.toml
plugins: django-3.10.0
collected 4 items

wwwidgets/tests.py ....                                                       [100%]

================================= 4 passed in 0.41s =================================

we see them all pass!

Nice! I love it when a test suite comes together.

Wrapping up

So what did we learn?

First up: string-based fields in Django can accept None values and will save these as the empty string into the database. This is expected and documented behaviour, and something to keep in mind.

Next, we learned that we can override this behaviour if we want to.

Lastly, the Open Source Software model worked very well. Someone spotted the issue, wrote up a workaround, and extended the docs to clarify things so that others aren’t tripped up in the future. Many thanks to Clifford Gama for his contributions!

Addendum: Remember to use pytest-django, not plain pytest

Note: if you’ve installed pytest, and have set everything up as per normal for a Django project, but get an error when running the tests that apps aren’t loaded yet, i.e.:

E   django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

Then make sure that you’ve installed pytest-django. Installing pytest alone isn’t enough.

One symptom of this situation is that running the tests with the standard Django test runner works, but using pytest doesn’t. In other words, running:

$ python manage.py test

works as you would expect, but

$ DJANGO_SETTINGS_MODULE=config.settings pytest <path-to>/tests.py

fails with the error noted above.

So, if you see the Django test runner working, but pytest tells you the apps aren’t yet loaded, make sure you’ve installed pytest-django.

1. Do you need a software developer who is persistent, flexible, and thorough? One who integrates well with existing teams, has broad experience, and thrives on legacy systems? If so, give me a yell! I’m available for freelance Python/Perl backend development and maintenance work. Contact me at paul@peateasea.de and let’s discuss how I can help solve your business’s hairiest problems. ↩

2. I really don’t know where I got that idea from. I must have been half asleep or something. ↩

3. You’ll have to imagine the last couple of lines being green in the terminal output. This is the colour that pytest uses for passing tests. ↩

4. I love details. That’s where you end up learning why something is the way it is. ↩

Support

If you liked this post, please buy me a coffee!