Dealing with UNIQUE Fields on a Multi-lingual Site

The Problem

I came across this problem recently when writing a snippet model for Wagtail (essentially a Django model that sits inside the Wagtail framework).

The model had a field that required it to be unique across all languages. This could be typical of product ID's, country codes or part numbers for example. On a single-language site, this is just a matter of setting unique=true on the field in question. Simple enough. Any attempt to save a duplicate value will kick back an error and prevent the instance being saved to the database.

On a multi-language site (presuming you're using the duplicate tree approach), a copy of the instance is made for each translation, meaning the key value is no longer unique. Attempting to add a translation with unique=true set will throw an unhandled IntegrityError complaining of a duplicate key value.

Here is my model. It's a clusterable model with a child orderable class. That part is not important to this though, so I'll just look at this model as a standalone entity. The following applies to standard models as well.

        
            Copy
        
    
from django import template
register = template.Library()

@register_snippet
class TemplateText(TranslatableMixin, ClusterableModel):
    template_set = models.CharField(
        unique=True,
        max_length=50,
        verbose_name="Set Name",
        help_text=_("The set needs to be loaded in template tags then text references as {{set.tag}}")
    )    

    panels = [
        FieldPanel("template_set"),
        MultiFieldPanel(
            [
                InlinePanel("templatetext_items"),
            ],
            heading=_("Text Items"),
        ),
    ]

    def __str__(self):
        return self.template_set
    
    class Meta:
        verbose_name = _('Template Text')
        unique_together = ('translation_key', 'locale')

Because template_set had unique=true set, attempting to make a translation would throw the duplicate key value error.

Workaround

I'm not going to call this a solution as there should come a time in Wagtail's and Wagtail Localize's evolution that this is handled a little more elegantly.

The steps until then...

Change the constraint type
Prevent translations on the unique field
Handle constraint violations
Limit creating new instances to the default locale only
Delete translated copies when deleting the canonical instance

Step 1: Change the constraint type

First step, remove the unique=true from template_set

To keep the unique constraint, we add it to the unique_together meta property instead:

        
            Copy
        
    
class SomeClass(TranslatableMixin, models.Model):
    ....
    class Meta:
        verbose_name = _('Template Text')
        unique_together = ('translation_key', 'locale'), ('locale', 'template_set')

('locale', 'template_set') makes sure that each value of template_set can exist only once per locale (language).

Step 2: Prevent translations on the unique field

We don't want this field to end up getting translated as you may end up with a key for one thing in one language pointing to something entirely different in another.

In wagtail-localize, you disable translation using the translatable_fields property. So in the dummy class below, field1 would be the unique key, field2 & field3 are to be kept translatable:

        
            Copy
        
    
class SomeClass(TranslatableMixin, models.Model):
    field1 = models.CharField(...)   
    field2 = models.CharField(...)   
    field3 = models.CharField(...)   

    translatable_fields = [
        TranslatableField('field2 '),
        TranslatableField('field3 '),
    ]

In this case, templatetext_items is the name of the orderable child class relationship that we want to keep translatable:

        
            Copy
        
translatable_fields = [ 
    TranslatableField('templatetext_items'),
]

Making the unique field non-translatable is an important assumption for the error handling later on.

Step 3: Handle constraint violations

So far so good, but what happens when a value is used again in the same locale now?

While unique constraint checks are built in to the standard validation model, checks for unique_together constraints are not and you'll end up with an unhandled error.

We need to add a custom clean() method to the model to cope with this. Since we only set the template name in the original and not the translation, we only need to validate the name there. To simplify this, I add in another clause that prevents adding instances of the model to any other locale than the default one. This might not suit every scenario, you'd need to add some logic to deal with that if that's your case.

        
            Copy
        
    
    def clean(self):
        # Check unique_together constraint
        # Stop instances being created outside of default locale
        # ASSUMPTION: the field in the unique_together (template_set) is non-translatable

        def_lang = Locale.get_default()
        
        if self.locale==Locale.get_default():
            # If in default locale, look for other sets with the template_set value (checking pre-save value)
            # Exclude other locales (will be translations of current locale)
            # Exclude self to cater for editing existing instance. Name change still checked against other instances.
            if TemplateText.objects.filter(template_set=self.template_set).filter(locale=self.locale_id).exclude(pk=self.pk).count()>0:
                raise ValidationError(_("This template set name is already in use. Please only use a unique name."))
        elif self.get_translations().count()==0:
            # If not in default locale and has no translations, new instance being created outside of default, raise error
            raise ValidationError(_(f"Template sets can only be created in the default language ({def_lang}). \
                                      Please create the set in {def_lang} and use the translate option."))

The first logic branch takes in the case that you're adding/editing in the default locale.

In this case, request all instances in this locale that have the same template_set value as the one being passed in by the admin form (self.template_set).
Exclude the current instance to cover the case where it's being edited and the name hasn't changed.
If this request returns any objects, the name isn't unique and a validation error is raised, the form is returned with the displayed error message.

Step 4: Limit creating new instances to the default locale only

The next logic branch is for anything being added/edited outside of the default locale.

In this case, it should be a translation.
To test this, use the get_translations() - for translated instances, this will be greater than 0
If not, it means a new instance is being added and needs to be blocked.

Step 5: Delete translated copies when deleting the canonical instance

There's one more scenario to consider:

When the original instance is deleted, the translations are left behind as orphans.

Editing an existing translation will suddenly throw an error that you are trying to create something in the wrong language.
Equally possible, creating a new instance and suddenly getting an error during translation because an orphan is lurking in that locale with the same name and not visible on the default page.

Really (and this is a feature that should be standard), when you're deleting the canonical instance of anything, the translations should also be deleted as default.

Fortunately, this is easily achieved by adding a custom delete() method to the model:

        
            Copy
        
    
def delete(self):
    # If deleting instance in default locale, delete translations
    if self.locale==Locale.get_default():
        for trans in self.get_translations():
            trans.delete()
    super().delete()

The if clause will only delete the translations if you're in the default locale and therefore deleting the canonical instance.

It's working on the assumption that you've disabled creating instances outside of the default locale as we did in step 4. You'll need to change the logic if you're not doing this.

This works for sites doing flat translations (i.e. where everything is translated from the default locale). If you have multi-tiered translation going on then you'll need to adjust this clause. Care is needed if you do this, as get_translations() will include canonical instances which you wouldn't want to delete from a translation.

Conclusion

With wagtail-localize being relatively new, there's a lack of material to work off in the way of documentation and examples. The small pain that this sometimes causes is well worth it when you look at the result — a ready-made translation interface for content editors that can have your pages translated in minutes without the need for any command line interaction from a site administrator. If you've ever worked on a Django or Wagtail multi-language site and had to suffer the pain of makemessages and compiling PO files, you'll appreciate this.

The model that I encountered this with is something that I wrote to translate static text on non-wagtail pages so that all those peripheral pages such as server errors, search results, user login and profile etc., can have dynamically translated text without anything hard-coded or need for PO files. I'll go through that in the next blog.

As for dealing with unique constraints, this was a bit of work to get through from the initial problem to working out a solution, but now it's there, it's a pretty straightforward workaround to re-use in the future.

Hopefully this explains how to deal with unique fields on translated sites.