Nautobot Apps - Lab 03: Adding Custom Models

In this lab, we'll explore how to extend Nautobot by introducing our own custom object type, or model.

• Nautobot Apps - Lab 03: Adding Custom Models
• Task 1: Creating a New Model
  • Step 1-0 - Prepare for the Lab
  • Step 1-1 - Verify the App Project Structure
  • Step 1-2 - Understand the Base Models
  • Step 1-3 - Create the MaintenanceNotice Model
  • Step 1-4 - Add Fields to the Model
  • Step 1-5 - Associate a MaintenanceNotice with One or More Devices
  • Step 1-6 - Get the User Model
  • Step 1-7 - Add Metadata to the Model
  • Step 1-8 - Add a String Format Method
  • Step 1-9 - Add a Reverse Lookup Method
  • Step 1-10 - Add a Computed Field Value
  • Step 1-11 - Check Your Work
• Task 2: Applying Database Migrations
  • Step 2-1 - Make Migrations
  • Step 2-2 - Apply Migrations
  • Step 2-3 - Inspect the Database
• Task 3: Set up Testing for our Model
  • Step 3-1 - Grant Database Permissions
  • Step 3-2 - Create a Test Case
  • Step 3-3 - Create a setUpTestData Method
  • Step 3-4 - Test End Time Functionality
  • Step 3-5 - Check Your Work
  • Step 3-6 - Run Tests
  • Step 3-7 - Debug
  • Step 3-8 - Clean Up
• Task 4: Extending the Admin UI
  • Step 4-1 - Create the Admin File
  • Step 4-2 - Add Imports
  • Step 4-3 - Create the Admin Class
  • Step 4-4 - Register the Admin Class
  • Step 4-5 - Verify Your Work
  • Step 4-6 - Restart the Development Server
  • Step 4-7 - Access the Admin Site
  • Step 4-8 - Create a Model Instance from the Admin Site
  • Step 4-9 - Commit Your Changes

---

Task 1: Creating a New Model

Step 1-0 - Prepare for the Lab

For this and all remaining labs, it is advisable to open two terminal windows, one for running the Nautobot development server, and another for executing various commands.

The steps in this lab require that you have a terminal open and have switched to the nautobot user. Verify your username with the whoami command and switch user to nautobot if needed.

ntc@nautobot:~$ whoami
ntc
ntc@nautobot:~$ sudo -iu nautobot
nautobot@nautobot:~$ whoami
nautobot
nautobot@nautobot:~$

If you have the Nautobot development server running from a previous lab, please continue to the next step. Otherwise we need to start the server with the command nautobot-server runserver 0.0.0.0:8080 --insecure.

nautobot@ntc-nautobot-apps:~$ nautobot-server runserver 0.0.0.0:8080 --insecure
Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).
April 24, 2022 - 05:19:10
Django version 3.1.14, using settings 'nautobot_config'
Starting development server at http://0.0.0.0:8080/
Quit the server with CONTROL-C.

Step 1-1 - Verify the App Project Structure

We should now have a project structure that is very similar as below:

Step 1-2 - Understand the Base Models

As we are introducing a new object (model) into Nautobot, we need to subclass from one of the Nautobot provided generic models, which all derive from the BaseModel class to create a class named MaintenanceNotice. This will represent the database table which holds our maintenance notice data. The reason to inherit from these classes is for general functionality and convenience, by staying consistent with common methods for all models.

We have several choices to pick from, depending on the level of complexity in the model, use cases and additional attributes we want to inherit.

Note: When using OrganizationModel or PrimaryModel while developing Nautobot plugins, you will need to use the @extras_features decorator to take advantage of all features. See refer to the Models Documentation for more information.

Step 1-3 - Create the MaintenanceNotice Model

Edit models.py within the maintenance_notices directory. This file will hold our model class.

We will start our project by inheriting from BaseModel. Edit your file so that it looks like this:

"""Model definition for maintenance_notices."""

from django.db import models

from nautobot.core.models import BaseModel


class MaintenanceNotice(BaseModel):

Step 1-4 - Add Fields to the Model

We want to define several fields on this model so that we can track maintenance notices:

• start_time: The date and time at which the maintenance begins
• duration: The length of the maintenance window, measured in minutes
• end_time: The date and time at which the maintenance concludes
• comments: Optional notes relating to the work being done
• devices: The set of Nautobot devices to which the maintenance notice applies
• created_by: The user who created the maintenance notice in Nautobot

Since our model inherits from BaseModel it will have a numeric id field, which will be used as its primary key. The value in this field will be a unique, randomly generated UUID.

Start by defining start_time and end_time as DateTimeFields. We're going to automatically set the value of end_time when a notice is saved based on its start_time and duration, so mark this field as uneditable. (This prevents it from appearing in automatically-generated forms.)

class MaintenanceNotice(BaseModel):
    start_time = models.DateTimeField()
    end_time = models.DateTimeField(
        editable=False,
    )

Next, add an IntegerField for the duration. Include a help_text to indicate the unit of measurement.

    duration = models.PositiveSmallIntegerField(
        help_text="Duration (in minutes)",
    )

We also need an optional comments field to store arbitrary notes:

    comments = models.CharField(
        blank=True,
        max_length=200,
    )

Step 1-5 - Associate a MaintenanceNotice with One or More Devices

We want to add a ForeignKey field to our model. Since a MaintenanceNotice may be associated to many devices, and any device could be associated with many maintenance notices, we will employ the ManyToManyField type. The to argument is a string that represents the object location of the Nautobot Device model. The related_name argument provides a way to follow this association in the reverse direction, that is from a specific device instance, to find all associated maintenance notices.

    devices = models.ManyToManyField(
        to="dcim.Device",
        related_name="maintenance_notices",
    )

Step 1-6 - Get the User Model

We also want to attribute each maintenance notice to the user who creates it. To do this, we'll add a ForeignKey field to Django's User model. Since we might sometimes need to create maintenance notices via the admin UI or Nautobot shell without specifying a particular user, we'll make this field optional by setting blank and null to true.

We will follow Django best practices and import get_user_model from django and extract the user model within our instance to avoid any possible errors or warnings while building our Database Migrations file.

from django.contrib.auth import get_user_model

class MaintenanceNotice(BaseModel):
    UserModel = get_user_model()

Now, you add a new attribute to the model and instead of using lazy identifiers, we will point our ForeignKey relationship to UserModel.

    created_by = models.ForeignKey(
        to=UserModel,
        on_delete=models.SET_NULL,
        blank=True,
        null=True,
        editable=False,
    )

Step 1-7 - Add Metadata to the Model

By default, Django models are ordered according to their automatically-generated primary keys (the id field). However, it makes more sense in our case to order maintenance notices by their start time, to ensure they always appear in chronological order. To do this, we create a Meta class within the MaintenanceNotice model and define ordering as the tuple of fields by which we want to order instances.

Additionally, add a verbose_name and verbose_name_plural to our Meta class. This helps Django ensure our model has proper naming across our application and, ensuring our application display fields look nice and polished.

Edit your file ~/plugin/maintenance_notices/models.py so that it looks like this:

class MaintenanceNotice(BaseModel):
    ...

    class Meta:
        ordering = ("start_time", "pk")
        verbose_name = "Maintenance Notice"
        verbose_name_plural = "Maintenance Notices"

Note: We've listed both start_time and pk (which is an alias for the id field) to ensure that ordering is always deterministic in the event multiple maintenance notices have the same start time.

Step 1-8 - Add a String Format Method

It's also good practice to add a sensible __str__() method to every Django model. This method determines the output when the model needs to be expressed as a string (e.g. when calling print(instance)). We'll return a string indicating the notice's start time and duration.

class MaintenanceNotice(BaseModel):
    ...

    class Meta:
    ...

    def __str__(self):
        """Return human readable output when print() is called on object"""
        return f"{self.start_time:%Y-%m-%d %H:%M} ({self.duration} minutes)"

Here is an example of what an instance of the model would look like without the __str__ method:

>>> print(MaintenanceNotice.objects.first())
MaintenanceNotice object (b9ab3e3a-0ccc-4fd3-aea9-480f312d370e)

And here is the same model instance with the __str__ method:

>>> print(MaintenanceNotice.objects.first())
2022-04-22 00:00 (60 minutes)

You can see how the __str__ method allows us to control how our model instances are shown to the world.

Step 1-9 - Add a Reverse Lookup Method

One other best practice is to include a method to retrieve an absolute URL of the model instance. This will become very helpful as we are creating and rendering template content which references the model. To accomplish this, ensure you first import reverse from Django.

from django.shortcuts import reverse

Next, add a get_absolute_url method to the model to provide the URL.

class MaintenanceNotice(BaseModel):
    ...

    class Meta:
    ...

    def __str__(self):
        return f"{self.start_time:%Y-%m-%d %H:%M} ({self.duration} minutes)"

    def get_absolute_url(self):
        """Return absolute URL for instance."""
        return reverse("plugins:maintenance_notices:maintenancenotice", args=[self.pk])

Step 1-10 - Add a Computed Field Value

Remember that we said end_time would be set automatically? We can do that by overriding the model's save() method to sum its start time and duration and record the resulting datetime value. First, import timedelta up at the top of models.py:

from datetime import timedelta

Then, add a save() method to MaintenanceNotice to set end_time. Don't forget to call super() to save the instance!

class MaintenanceNotice(BaseModel):
    ...

    class Meta:
    ...

    def __str__(self):
    ...

    def save(self, *args, **kwargs):
        self.end_time = self.start_time + timedelta(minutes=self.duration)
        super().save(*args, **kwargs)

Note: Be careful to ensure that the Meta class, __str__(), get_absolute_url() and save() functions are all indented to the same level beneath the MaintenanceNotice class.

Save the File.

Step 1-11 - Check Your Work

That's everything we need in our model for now!

At this point, your models.py file should look like this:

"""Model definition for maintenance_notices."""

from datetime import timedelta

from django.contrib.auth import get_user_model
from django.db import models
from django.shortcuts import reverse

from nautobot.core.models import BaseModel


class MaintenanceNotice(BaseModel):
    UserModel = get_user_model()

    start_time = models.DateTimeField()
    end_time = models.DateTimeField(editable=False)
    duration = models.PositiveSmallIntegerField(help_text="Duration (in minutes)")
    comments = models.CharField(blank=True, max_length=200)
    created_by = models.ForeignKey(
        to=UserModel, on_delete=models.SET_NULL, blank=True, null=True, editable=False
    )
    devices = models.ManyToManyField(
        to="dcim.Device", related_name="maintenance_notices"
    )

    class Meta:
        ordering = ("start_time", "pk")
        verbose_name = "Maintenance Notice"
        verbose_name_plural = "Maintenance Notices"

    def __str__(self):
        return f"{self.start_time:%Y-%m-%d %H:%M} ({self.duration} minutes)"

    def get_absolute_url(self):
        """Return absolute URL for instance."""
        return reverse("plugins:maintenance_notices:maintenancenotice", args=[self.pk])

    def save(self, *args, **kwargs):
        self.end_time = self.start_time + timedelta(minutes=self.duration)
        super().save(*args, **kwargs)

---

Task 2: Applying Database Migrations

Now that our model definition is complete, we'll generate and apply database schema migrations using the Django management commands makemigrations and migrate. To do this, we use nautobot-server.

Note: The makemigrations command is used by the app developers when adding new models or modifying existing models. This command generates a Python script that manages the database schema. The migrate command is executed by the users of the app whenever they install of upgrade the app. migrate is called as part of the nautobot-server post_upgrade command.

Step 2-1 - Make Migrations

Change into the Nautobot root path /opt/nautobot and run makemigrations by calling nautobot-server. Make sure you are working as user nautobot, that you have the nautobot virtual environment activated, and that you cd to the /opt/nautobot/ directory.

nautobot@nautobot:~/plugin$ cd
nautobot@nautobot:~$ nautobot-server makemigrations maintenance_notices
Migrations for 'maintenance_notices':
  plugin/maintenance_notices/migrations/0001_initial.py
    - Create model MaintenanceNotice
nautobot@nautobot:~$

Notes:

• If you see an error message indicating "No installed app with label 'maintenance_notices'," check that you've added 'maintenance_notices' to the PLUGINS list in nautobot_config.py.

• If you see an error indicating a syntax error for one of your field arguments, ensure the line before it ends with a comma.

Step 2-2 - Apply Migrations

Finally, apply the new migration to the database.

nautobot@nautobot:~$ nautobot-server migrate maintenance_notices
Operations to perform:
  Apply all migrations: maintenance_notices
Running migrations:
  Applying maintenance_notices.0001_initial... OK
nautobot@nautobot:~$

Step 2-3 - Inspect the Database

We can inspect the new database table by entering the Nautobot database shell, which is a wrapper around PostgreSQL's psql utility.

nautobot@nautobot:~$  nautobot-server dbshell
psql (12.9 (Ubuntu 12.9-0ubuntu0.20.04.1))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.

nautobot=>

Enter the PostgreSQL command \d maintenance_notices_maintenancenotice to display the maintenance notices table. Note that the table columns correspond to the fields we defined on the model. (Output beyond the table columns has been omitted below for brevity.)

nautobot=> \d maintenance_notices_maintenancenotice
           Table "public.maintenance_notices_maintenancenotice"
    Column     |           Type           | Collation | Nullable | Default
---------------+--------------------------+-----------+----------+---------
 id            | uuid                     |           | not null |
 start_time    | timestamp with time zone |           | not null |
 end_time      | timestamp with time zone |           | not null |
 duration      | smallint                 |           | not null |
 comments      | character varying(200)   |           | not null |
 created_by_id | uuid                     |           |          |
...

When finished, exit the database shell with \q.

---

Task 3: Set up Testing for our Model

Good software practices say that adding testing to validate the intended behavior will protect your code against future changes that could break backwards compatibility and also double check that your are matching your expectations. Fortunately, Django comes with a test command that will help us to test the new plugin.

Step 3-1 - Grant Database Permissions

Whenever we run tests, Nautobot will create a new database schema that is used exclusively for testing purposes. This makes sense as we do not want to have tests run against a database that may already contain data.

Recall that the student pod is modeled after a production install where the default database permissions only allow the nautobot user to manage the default nautobot database. Since we are performing development and the nautobot-server test command will need to create a database for testing ('default'), we must grant the nautobot permissions to create a database.

For security purposes, our nautobot user does not have the ability to switch users, so we need to perform this next step from the ntc user account that we use to login to the server. Since we switched from the ntc user to the nautobot user, we can just type exit to switch bact to the original user.

nautobot@nautobot:~$ exit
logout
ntc@nautobot:/opt/nautobot$

Now we want to start the Postgres SQL shell as the postgres user. Once we are in the shell, we can give the nautobot user permissions to create databases.

ntc@nautobot:/opt/nautobot$ sudo -u postgres psql
psql (12.9 (Ubuntu 12.9-0ubuntu0.20.04.1))
Type "help" for help.

postgres=# ALTER USER nautobot CREATEDB;
ALTER ROLE
postgres=# \q
ntc@nautobot:/opt/nautobot$

Now you can switch back to the nautobot user and continue

ntc@nautobot:/opt/nautobot$ sudo -iu nautobot
nautobot@nautobot:~$

Step 3-2 - Create a Test Case

Open the file plugin/maintenance_notices/tests/test_models.py in your preferred editor. As we will be working with datetime objects, we will need to add an import at the top of the file.

from datetime import datetime, timedelta, timezone

We will Subclass Nautobot's utility test TestCase class to create a class named TestMaintenanceNotice. This will contain the tests for the models class MaintenanceNotice. The base test case class nautobot.utilities.testing.TestCase. Add the class definition to the file:

class TestMaintenanceNotice(TestCase):
    """Test Maintenance Notice Model."""

Step 3-3 - Create a setUpTestData Method

We will define the setUpTestData method to prepare the test fixture that will be used across the tests. In our case, we create a MaintenanceNotice that we will reference in our tests. When defined, the SetUp method is run before every test method and an optional tearDown method is run after. Add additional imports and begin the setUp method.

from datetime import datetime, timedelta, timezone
from nautobot.utilities.testing import TestCase

from maintenance_notices import models

class TestMaintenanceNotice(TestCase):
    """Test Maintenance Notice Model."""

    @classmethod
    def setUpTestData(cls):
        cls.maintenance = models.MaintenanceNotice.objects.create(
            start_time=datetime.now(timezone.utc),
            duration=30,
        )

Step 3-4 - Test End Time Functionality

Finally, we add a test to verify that the end_time attribute has been defined as the increment of duration from the start_time

class TestMaintenanceNotice(TestCase):
    ...
    def test_end_time(self):
        self.assertEqual(
            self.maintenance.end_time,
            self.maintenance.start_time + timedelta(minutes=self.maintenance.duration),
        )

Save the File.

Step 3-5 - Check Your Work

Your completed plugin/maintenance_notices/tests/test_models.py should look like this:

from datetime import datetime, timedelta, timezone
from nautobot.utilities.testing import TestCase

from maintenance_notices import models


class TestMaintenanceNotice(TestCase):
    """Test Maintenance Notice Model."""

    @classmethod
    def setUpTestData(cls):
        cls.maintenance = models.MaintenanceNotice.objects.create(
            start_time=datetime.now(timezone.utc),
            duration=30,
        )

    def test_end_time(self):
        self.assertEqual(
            self.maintenance.end_time,
            self.maintenance.start_time + timedelta(minutes=self.maintenance.duration),
        )

Step 3-6 - Run Tests

Once your tests are defined you can run them with nautobot-server test command.

nautobot@nautobot:~$ nautobot-server test maintenance_notices
Creating test database for alias 'default'...

System check identified no issues (0 silenced).
.
----------------------------------------------------------------------
Ran 1 test in 0.010s

OK
Destroying test database for alias 'default'...

We could integrate our plugin testing into a Continuous Integration pipeline and automatically run them every time we change our code. However, this is out of scope for the class but feel free to review the public Nautobot projects on Github to see some awesome examples of how Network to Code executes CI/CD on open source projects.

Step 3-7 - Debug

Hopefully the test passed. But what if we find a problem and want to troubleshoot? In Python you have pdb package to help you!

You only need to import pdb and run pdb.set_trace(). This will stop the execution of your code (in your tests or in your production code) and let you interactively control the code execution while checking all the context.

class TestMaintenanceNotice(TestCase):
    ...
    def test_end_time(self):

        import pdb; pdb.set_trace()

        self.assertEqual(
            self.maintenance.end_time,
            self.maintenance.start_time + timedelta(minutes=self.maintenance.duration),
        )

When you run the tests again with command from Step 4, you will jump into the interactive session (pdb) where you could explore the code with l, check the context variables, execute next code line with n or jump into the related code with s. Finally you can use the c command to tell the debugger to continue executing the code.

> /opt/nautobot/plugin/maintenance_notices/tests/test_models.py(16)test_end_time()
-> self.assertEqual(
(Pdb) l
 11                 duration=30,
 12             )
 13
 14         def test_end_time(self):
 15             import pdb; pdb.set_trace()
 16  ->         self.assertEqual(
 17                 self.maintenance.end_time,
 18                 self.maintenance.start_time + timedelta(minutes=self.maintenance.duration),
 19             )
[EOF]
(Pdb) self.maintenance
<MaintenanceNotice: 2022-02-15 21:03 (30 minutes)>
(Pdb) c

Step 3-8 - Clean Up

As best practice, you should always remove or disable any debugging features in your code before you commit to change control. Remove all references to pdb.

Remove:

        import pdb; pdb.set_trace()

---

Task 4: Extending the Admin UI

Now that our plugin has its own model, we'll extend the Django admin UI to provide a simple interface via which Nautobot administrators can create, modify, and delete maintenance notices.

Step 4-1 - Create the Admin File

Create admin.py in the same path as models.py. This module will hold all code relevant to the admin UI.

nautobot@nautobot:~$ touch /opt/nautobot/plugin/maintenance_notices/admin.py

Step 4-2 - Add Imports

Open the admin.py file that we just created in your editor of choice.

Import Django's admin module. Additionally, we will need to import our MaintenanceNotice model.

from django.contrib import admin
from maintenance_notices.models import MaintenanceNotice

Step 4-3 - Create the Admin Class

Create a subclass of ModelAdmin named MaintenanceNoticeAdmin. Define the fields we want to display in the model's list view by setting the list_display attribute to a list of field names.

class MaintenanceNoticeAdmin(admin.ModelAdmin):
    list_display = ("start_time", "end_time", "created_by")

Step 4-4 - Register the Admin Class

Register the MaintenanceNotice model and our admin class using the register() decorator.

@admin.register(MaintenanceNotice)
class MaintenanceNoticeAdmin(admin.ModelAdmin):
    ...

Save the File.

Step 4-5 - Verify Your Work

Your completed file should look like the code below.

from django.contrib import admin
from maintenance_notices.models import MaintenanceNotice


@admin.register(MaintenanceNotice)
class MaintenanceNoticeAdmin(admin.ModelAdmin):
    list_display = ("start_time", "end_time", "created_by")

Step 4-6 - Restart the Development Server

Recall that the StatReloader only monitors the files it knows about. Since we just created the admin.py file it is not being monitored. We could switch back to the terminal where the server is running, stop it, and restart it.

Instead, let's trick the server into reloading itself by 'touching' one of the files that it is monitoring. This updates the modified date of the file without actually changing the contents.

nautobot@nautobot:~$ touch /opt/nautobot/plugin/maintenance_notices/models.py

If you check the development server terminal, you should see it reloading.

Step 4-7 - Access the Admin Site

In the Nautobot UI, navigate to the admin site by clicking your logged in user name in the upper right hand corner and then click Admin.

Notice that a new section for our plugin now appears.

Step 4-8 - Create a Model Instance from the Admin Site

Try creating, modifying, and deleting notices within the admin UI. Observe how a form has been created automatically from our model definition.

Step 4-9 - Commit Your Changes

Remember to commit your changes to git:

nautobot@nautobot:~/plugin$  git add -A
nautobot@nautobot:~/plugin$  git commit -m "Completed Lab 3."

---

This lab is now complete. Check your work against the solution guide before proceeding with the next lab.