Nautobot Apps - Lab 09: Populating Extensibility Features¶

For this lab, we'll learn how to pre-populate database records in Nautobot using signals to execute idempotent code.

Nautobot Apps - Lab 09: Populating Extensibility Features
Task 1: Adding a Status Field to the Model
Task 2: Populating a Relationship

Task 1: Adding a Status Field to the Model¶

A user of our Plugin has requested the ability to set a status on the Maintenance Notice. They want to allow the user to choose one of these statuses.

Outage
No Outage
Unplanned Outage
Cancelled

We will leverage the built-in Status object to handle this feature.

Step 1-0 - Prepare for the Lab¶

For this and all remaining labs, it is advisable to open two terminal windows, one for running the Nautobot development server, and another for executing various commands.

The steps in this lab require that you have a terminal open and have switched to the nautobot user. Verify your username with the whoami command and switch user to nautobot if needed.

ntc@nautobot:~$ whoami
ntc
ntc@nautobot:~$ sudo -iu nautobot
nautobot@nautobot:~$ whoami
nautobot
nautobot@nautobot:~$

If you have the Nautobot development server running from a previous lab, please continue to the next step. Otherwise we need to start the server with the command nautobot-server runserver 0.0.0.0:8080 --insecure.

nautobot@ntc-nautobot-apps:~$ nautobot-server runserver 0.0.0.0:8080 --insecure
Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).
April 24, 2022 - 05:19:10
Django version 3.1.14, using settings 'nautobot_config'
Starting development server at http://0.0.0.0:8080/
Quit the server with CONTROL-C.

Step 1-1 - Adding a Status Field¶

To add a field, we need to edit the models.py. Normally, we would define a new field by adding an attribute to the Django Model, which is the same way we have specified most of the fields on our model. In this case we plan on using Nautobot's Status objects. The code for working with the Status objects has been provided by a number of mixin classes, so our model must inherit from the StatusModel. Additionally, our model must register its content_type to statuses by using the @extras_features decorator.

Edit models.py and add these lines just above the class definition. The last line actually replaces the existing class MaintenanceNotice(BaseModel): line.

from nautobot.extras.models.statuses import StatusModel
from nautobot.extras.utils import extras_features

@extras_features("graphql", "statuses")
class MaintenanceNotice(BaseModel, StatusModel):

Save the file.

Step 1-2 - Make and Apply Migrations¶

Use the nautobot-server makemigrations maintenance_notices command to create a migrations file. Recall that this migrations file holds commands that instruct Django on how to make modifications to the backend database. Since we've added a status field to our model, we need to have that reflected in the database.

nautobot@nautobot:~$ nautobot-server makemigrations maintenance_notices
Migrations for 'maintenance_notices':
  plugin/maintenance_notices/migrations/0002_maintenancenotice_status.py
    - Add field status to maintenancenotice
nautobot@nautobot:~$

We can see that the makemigrations command created a new file plugin/maintenance_notices/migrations/0002_maintenancenotice_status.py that contains those database operations. Use the nautobot-server migrate command to apply the migrations.

nautobot@nautobot:~$ nautobot-server migrate
Operations to perform:
  Apply all migrations: admin, auth, circuits, contenttypes, database, dcim, django_celery_beat, extras, ipam, maintenance_notices, nautobot_data_validation_engine, nautobot_device_lifecycle_mgmt, nautobot_golden_config, sessions, social_django, taggit, tenancy, users, virtualization
Running migrations:
  Applying maintenance_notices.0002_maintenancenotice_status... OK

nautobot@nautobot:~$ touch ~/plugin/maintenance_notices/models.py

Use touch models.py to restart server if it is still running.

Note: Alternatively we could have used the nautobot-server post_upgrade command to apply the migrations. post_upgrade runs the migrate command, along with several other commands.

Step 1-3 - Make Choices¶

We have been asked to provide a choice four possible statuses.

Outage
No Outage
Unplanned Outage
Cancelled

Lets create a new file where we can define our choices.

nautobot@nautobot:~$ cd plugin/maintenance_notices
nautobot@nautobot:~/plugin/maintenance_notices$ touch choices.py
nautobot@nautobot:~/plugin/maintenance_notices$

Now edit the choices.py file.

from nautobot.utilities.choices import ChoiceSet
from nautobot.utilities.choices import ColorChoices

class MaintenanceNoticeStatusChoices(ChoiceSet):

    OUTAGE = "maintenance-outage"
    NO_OUTAGE = "maintenance-no-outage"
    UNPLANNED_OUTAGE = "maintenance-unplanned-outage"
    CANCELLED = "maintenance-cancelled"

    CHOICES = (
        (OUTAGE, "Outage"),
        (NO_OUTAGE, "No Outage"),
        (UNPLANNED_OUTAGE, "Unplanned Outage"),
        (CANCELLED, "Cancelled"),
    )

    CSS_CLASSES = {
        OUTAGE: "warning",
        NO_OUTAGE: "primary",
        UNPLANNED_OUTAGE: "danger",
        CANCELLED: "info",
    }

    COLORS = (
        (OUTAGE, ColorChoices.COLOR_YELLOW),
        (NO_OUTAGE, ColorChoices.COLOR_BLUE),
        (UNPLANNED_OUTAGE, ColorChoices.COLOR_RED),
        (CANCELLED, ColorChoices.COLOR_CYAN)
    )

Save the file.

Step 1-4 - Decipher the `choices.py`¶

First thing we do in the choices.py is import our base class ChoiceSet. We then inherit from ChoiceSet to create an enumeration of our possible choices for our model.

The first four lines under the class definition define the choices as class attributes, and are assigned string values that represent the slug that is used for the corresponding Status record.

    OUTAGE = "maintenance-outage"
    NO_OUTAGE = "maintenance-no-outage"
    UNPLANNED_OUTAGE = "maintenance-unplanned-outage"
    CANCELLED = "maintenance-cancelled"

The CHOICES attribute returns a tuple comprised of the slug and a friendly name.

    CHOICES = (
        (OUTAGE, "Outage"),
        (NO_OUTAGE, "No Outage"),
        (UNPLANNED_OUTAGE, "Unplanned Outage"),
        (CANCELLED, "Cancelled"),
    )

The COLORS attribute returns a tuple comprised of the slug and a color from the builtin Nautobot color choices.

    COLORS = (
        (OUTAGE, ColorChoices.COLOR_RED),
        (NO_OUTAGE, ColorChoices.COLOR_GREEN),
        (UNPLANNED_OUTAGE, ColorChoices.COLOR_GREY),
        (CANCELLED, ColorChoices.COLOR_CYAN)
    )

Finally, the CSS_CLASSES attribute holds a map of slug to a the preferred CSS formatting. Differing choices will display in different colors in the UI.

    CSS_CLASSES = {
        OUTAGE: "warning",
        NO_OUTAGE: "primary",
        UNPLANNED_OUTAGE: "danger",
        CANCELLED: "info",
    }

Step 1-5 - Prepare to Populate the Database¶

Status objects in Nautobot are models, just like any other abstraction that requires building a database table. So if we want our four statuses to be available, we need an idempotent method to add some predefined Status objects. We expect that we should only need to populate these once when the app is installed or upgraded. The strategy here is to provide a function that will populate the database, only if the Status objects do not yet exist. Then we will call the function only after a database migration occurs, rather than every time the server starts.

Create a new file called signals.py.

nautobot@nautobot:~$ cd plugin/maintenance_notices
nautobot@nautobot:~/plugin/maintenance_notices$ touch signals.py
nautobot@nautobot:~/plugin/maintenance_notices$

Now edit the signals.py file.

from nautobot.extras.management import export_statuses_from_choiceset

from .choices import MaintenanceNoticeStatusChoices


def post_migrate_create_custom_statuses(apps, **kwargs):
    """Create Status choices from choice set enums."""

    Status = apps.get_model("extras.Status")
    ContentType = apps.get_model("contenttypes.ContentType")
    MaintenanceNotice = apps.get_model("maintenance_notices.MaintenanceNotice")

    content_type = ContentType.objects.get_for_model(MaintenanceNotice)
    description_map = {k:v for k, v in MaintenanceNoticeStatusChoices.CHOICES}
    color_map = {k:v for k, v in MaintenanceNoticeStatusChoices.COLORS}

    choices = export_statuses_from_choiceset(
        MaintenanceNoticeStatusChoices,
        description_map=description_map,
        color_map=color_map,
    )

    for choice in choices:
        obj, created = Status.objects.get_or_create(**choice)
        if content_type not in obj.content_types.all():
            obj.content_types.add(content_type)

Save the file.

Step 1-6 - Decipher the `signals.py`¶

We've imported the MaintenanceNoticeStatusChoices choice set that we previously created. We also imported a helper function called export_statuses_from_choiceset.

We define a function and allow for only one positional argument apps. We also allow for keyword arguments to be passed as a dict. This function is a callback function, also referred to as a signal receiver function, and it is required that it can accept both of these arguments. We will not be making use of the **kwargs in this function.

The apps.get_model method is called three times to get the Status, ContentType, and MaintenanceNotice models and assign them to names. In many other parts of the code we just import these models, but it is typical to use apps.get_model in these receiver functions to avoid a circular import.

We need content_type id of our MaintenanceNotice model to add it to our statuses. the description_map is generated on the fly using the slug and 'friendly name' tuples in MaintenanceNoticeStatusChoices.CHOICES.

Calling the export_statuses_from_choiceset helper function provides us with a list of dict objects, where each one can be used to create a Status object. Here's what the output looks like:

nautobot@nautobot:~$ nautobot-server nbshell
### Nautobot interactive shell (ntc-nautobot-grelleum)
### Python 3.8.10 | Django 3.1.14 | Nautobot 1.2.10
### lsmodels() will show available models. Use help(<model>) for more info.
>>> from nautobot.extras.management import export_statuses_from_choiceset
>>> from maintenance_notices.choices import MaintenanceNoticeStatusChoices
>>> description_map = {k:v for k, v in MaintenanceNoticeStatusChoices.CHOICES}
>>> color_map = {k:v for k, v in MaintenanceNoticeStatusChoices.COLORS}
>>> export_statuses_from_choiceset(MaintenanceNoticeStatusChoices, description_map=description_map, color_map=color_map)
[{
   "name":"Outage",
   "slug":"maintenance-outage",
   "description":"Outage",
   "color":"f44336"
},
{
   "name":"No Outage",
   "slug":"maintenance-no-outage",
   "description":"No Outage",
   "color":"4caf50"
},
{
   "name":"Unplanned Outage",
   "slug":"maintenance-unplanned-outage",
   "description":"Unplanned Outage",
   "color":"9e9e9e"
},
{
   "name":"Cancelled",
   "slug":"maintenance-cancelled",
   "description":"Cancelled",
   "color": "00bcd4"
}]
>>>

Finally, we loop over the above list:

    for choice in choices:
        obj, created = Status.objects.get_or_create(**choice)
        if content_type not in obj.content_types.all():
            obj.content_types.add(content_type)

We call the get_or_create method to ensure we are idempotent. get_or_create will attempt to retrieve a record for you, and will create a new record if it can not find a match. Finally, we check if the MaintenanceNotice content_type is associated to the Status and add it if not.

For a more comprehensive example of this, see the Nautobot source code for nautobot/extras/management/__init__.py at this link.

Step 1-7 - Add the Callback¶

We need to register the receiver function. We will do this in the ready method of our class.

Edit the plugin/maintenance_notices/__init__.py file. Add the following import statements.

from nautobot.core.signals import nautobot_database_ready
from .signals import post_migrate_create_custom_statuses

Add the ready method to the end of your MaintenanceNoticesConfig class.

class MaintenanceNoticesConfig(PluginConfig):
    ...

    def ready(self):
        super().ready()
        nautobot_database_ready.connect(
            post_migrate_create_custom_statuses,
            sender=self,
        )

The completed __init__.py file should look like this:

"""Plugin declaration for MaintenanceNoticesConfig."""

from nautobot.extras.plugins import PluginConfig
from nautobot.core.signals import nautobot_database_ready
from .signals import post_migrate_create_custom_statuses

try:
    from importlib import metadata
except ImportError:
    # Python version < 3.8
    import importlib_metadata as metadata

__version__ = metadata.version(__name__)


class MaintenanceNoticesConfig(PluginConfig):
    """Plugin configuration for the maintenance_notices plugin."""

    name = "maintenance_notices"
    verbose_name = "Maintenance Notices"
    version = __version__
    author = "Student Name"
    description = "Nautobot plugin to manage maintenance notices"
    base_url = "maintenance_notices"
    required_settings = []
    min_version = "1.1.0"
    max_version = "1.9999"
    default_settings = {
        "default_duration": 120,
    }
    caching_config = {}

    def ready(self):
        super().ready()
        nautobot_database_ready.connect(
            post_migrate_create_custom_statuses,
            sender=self,
        )

config = MaintenanceNoticesConfig

Save the file and ensure the Nautobot server is reloaded.

Step 1-8 - Trigger the post-migration signal¶

In order for our new signals.py file to run, we need to migrate the database. Use the nautobot-server migrate command to kick off the signals.py file.

nautobot@nautobot:~$ nautobot-server migrate
Operations to perform:
  Apply all migrations: admin, auth, circuits, contenttypes, database, dcim, django_celery_beat, extras, ipam, maintenance_notices, nautobot_data_validation_engine, nautobot_device_lifecycle_mgmt, nautobot_golden_config, sessions, social_django, taggit, tenancy, users, virtualization
Running migrations:
  Applying maintenance_notices.0002_maintenancenotice_status... OK

nautobot@nautobot:~$ touch models.py

Step 1-9 - Verify the Work¶

In your web browser, navigate to Plugins > Maintenance Notices and click the +Add button. Note the new Status drop down menu offers the four choices we defined earlier. We see only these four, since they are the only Status records that are of the MaintenanceNotices content_type.

Screenshot: Status drop down menu

Create a new Maintenance Notice with your choice of status, so that we have at least one record with a status.

Next, navigate to Plugins > Maintenance Notices and click Maintenance Notices to see the list view. Notice that these is no Status column shown in the table.

Screenshot: No status field

You can use the pencil icon to edit one of your notices. Try to edit the status of an existing record.

Step 1-10 - Use the Mixins¶

Nautobot provides a number of Status Mixins to simplify working with statuses. Here are a list of them.

StatusBulkEditFormMixin
StatusFilterFormMixin
StatusModelCSVFormMixin
StatusModelFilterSetMixin
StatusModelSerializerMixin
StatusTableMixin
StatusViewSetMixin

We will not go through all of these in this lab, however we will enhance our table view by inheriting from the StatusTableMixin.

Edit the tables.py file and find the following line.

class MaintenanceNoticeTable(BaseTable):

Replace the line with these lines.

from nautobot.extras.tables import StatusTableMixin

class MaintenanceNoticeTable(StatusTableMixin):

Your server should detect the change and reload. When the server is restarted, refresh your web browser.

Screenshot: No status field

Task 2: Populating a Relationship¶

For this task, we are going to add an Extensibility feature. You can find these features in the Nautobot Web UI in the top menu. These features are often configured using the Web UI. It is advantageous to have your app setup any Extensibility features that are required by the app, so that we can be assured that we get a consistent deployment every time.

In this lab we are adding a Relationship to Nautobot. The relationship we are adding has nothing to do with our MaintenanceNotice app, but we will add it here anyway to demonstrate how you would do this.

The way to do this is not very different than what we did to populate the Status objects. After all, the Extensibility features are implemented using the Django ORM so we only need to know which Model stores the our target feature. Conveniently, it is the Relationship model that stores user added relationships.

The relationship allows you to set a pseudo-ForeignKey relationship between two models. Typically you would only do this between models that you do not have direct control over, like within the Nautobot core apps.

Step 2-1 - Create a Local Access Provider Relationship¶

It's worth repeating that this relationship is unrelated to the intent of this app, but it is good practice. You can navigate to Extensibility > Relationships to see the existing relationships that were installed by the other plugins on this server.

Screenshot: Relationships

We want to build one like the following:

Screenshot: Local access provider

This is a Circuit related Relationship. Many circuits are delivered to the building by a different vendor than the one who sold you the end-to-end circuit. Since the Circuit app offers a Provider model, as well as a CircuitTermination model, we can create a relationship between these models to show that a specific provider's circuit is installed at the local site.

Edit your signals.py file. Insert this import statement at the top of the file.

from nautobot.extras.choices import RelationshipTypeChoices

Then add the following function at the end of the file.

def post_migrate_create_local_access_provider_relationship(apps, **kwargs):
    """Create a circuits.Provider relationship to circuits.CircuitTermination."""

    ContentType = apps.get_model("contenttypes.ContentType")
    CircuitTermination = apps.get_model("circuits.CircuitTermination")
    Provider = apps.get_model("circuits.Provider")
    Relationship = apps.get_model("extras.Relationship")

    provider_content_type = ContentType.objects.get_for_model(Provider)
    termination_content_type = ContentType.objects.get_for_model(CircuitTermination)

    relationship = {
        "name": "local access provider",
        "slug": "circuit-termination-local-access-provider",
        "description": "Local Access Provider",
        "type": RelationshipTypeChoices.TYPE_ONE_TO_MANY,
        "source_type": provider_content_type,
        "source_label": "circuit_termination",
        "destination_type": termination_content_type,
        "destination_label": "local_access_provider",
    }

    Relationship.objects.get_or_create(**relationship)

Step 2-2 - Add another Callback¶

Edit the plugin/maintenance_notices/__init__.py file. Replace the following import statement.

from .signals import post_migrate_create_custom_statuses

With this:

from .signals import (
    post_migrate_create_custom_statuses,
    post_migrate_create_local_access_provider_relationship,
)

Now add another call to the nautobot_database_ready.connect method:

    def ready(self):
        ...
        nautobot_database_ready.connect(
            post_migrate_create_local_access_provider_relationship,
            sender=self,
        )

Save the file.

Step 2-3 - Verifying the `init.py` file¶

Here is what your complete __init__.py file should look like.

"""Plugin declaration for MaintenanceNoticesConfig."""

from nautobot.extras.plugins import PluginConfig
from nautobot.core.signals import nautobot_database_ready
from .signals import (
    post_migrate_create_custom_statuses,
    post_migrate_create_local_access_provider_relationship,
)


try:
    from importlib import metadata
except ImportError:
    # Python version < 3.8
    import importlib_metadata as metadata

__version__ = metadata.version(__name__)


class MaintenanceNoticesConfig(PluginConfig):
    """Plugin configuration for the maintenance_notices plugin."""

    name = "maintenance_notices"
    verbose_name = "Maintenance Notices"
    version = __version__
    author = "Student Name"
    description = "Nautobot plugin to manage maintenance notices"
    base_url = "maintenance_notices"
    required_settings = []
    min_version = "1.1.0"
    max_version = "1.9999"
    default_settings = {
        "default_duration": 120,
    }
    caching_config = {}

    def ready(self):
        super().ready()
        nautobot_database_ready.connect(
            post_migrate_create_custom_statuses,
            sender=self,
        )
        nautobot_database_ready.connect(
            post_migrate_create_local_access_provider_relationship,
            sender=self,
        )

config = MaintenanceNoticesConfig

Step 2-4 - Verifying the Results¶

In order to see if this works, we need to force a migration. Run the command:

nautobot@nautobot:~$ nautobot-server post_upgrade

When the migration completes, navigate to Extensibility > Relationships to view the existing relationships. The local access provider relationship should not appear on the list.

Screenshot: Relationships populated

Step 2-5 - Commit Your Changes¶

Remember to commit your changes to git:

nautobot@nautobot:~$
nautobot@nautobot:plugin/maintenance_noices$ git add -A
nautobot@nautobot:plugin/maintenance_notices$ git commit -m "Completed Lab 9."

This lab is now complete. Check your work against the solution guide before proceeding with the next lab.

Nautobot Apps - Lab 09: Populating Extensibility Features¶

Task 1: Adding a Status Field to the Model¶

Step 1-0 - Prepare for the Lab¶

Step 1-1 - Adding a Status Field¶

Step 1-2 - Make and Apply Migrations¶

Step 1-3 - Make Choices¶

Step 1-4 - Decipher the choices.py¶

Step 1-5 - Prepare to Populate the Database¶

Step 1-6 - Decipher the signals.py¶

Step 1-7 - Add the Callback¶

Step 1-8 - Trigger the post-migration signal¶

Step 1-9 - Verify the Work¶

Step 1-10 - Use the Mixins¶

Task 2: Populating a Relationship¶

Step 2-1 - Create a Local Access Provider Relationship¶

Step 2-2 - Add another Callback¶

Step 2-3 - Verifying the __init__.py file¶

Step 2-4 - Verifying the Results¶

Step 2-5 - Commit Your Changes¶

Step 1-4 - Decipher the `choices.py`¶

Step 1-6 - Decipher the `signals.py`¶

Step 2-3 - Verifying the `init.py` file¶